2024-09-21T03:22:00.066 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T03:22:00.091 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T03:22:00.182 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913469 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iogen}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913469' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJFnViquWdVspaoJqEWJQsc32l9Eh0LvmHljPbkTLDcAU9tk20Th6oXXI/sb+1V7Xux2zUYx0COMyIg/AJhgfHU= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPrMPun9VE18M+GfC3lswD6U6As9u0jJS0qhPlADrXZ+o73UgpD+8Zl5UHPcTmTuAlfAcbdtGiFxAMKT8irE0hQ= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPDYVBKZiLokEbevBJa0jlZdAv/fC5Rr/MShdq7SKRgF1/Vfr4FaRMlLrWvhdh2HrRn2BrrKW/NMFkJNjrzUfG0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T03:22:00.182 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T03:22:00.183 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T03:22:00.183 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T03:22:00.185 INFO:teuthology.task.internal:Checking packages... 2024-09-21T03:22:00.220 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T03:22:00.220 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T03:22:00.220 INFO:teuthology.packaging:ref: None 2024-09-21T03:22:00.220 INFO:teuthology.packaging:tag: None 2024-09-21T03:22:00.220 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:22:00.220 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:22:00.221 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:22:00.369 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T03:22:00.371 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T03:22:00.383 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T03:22:00.383 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T03:22:00.497 INFO:teuthology.task.internal:Saving configuration 2024-09-21T03:22:00.517 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T03:22:00.577 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T03:22:00.600 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913469', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:17:10.603809', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJFnViquWdVspaoJqEWJQsc32l9Eh0LvmHljPbkTLDcAU9tk20Th6oXXI/sb+1V7Xux2zUYx0COMyIg/AJhgfHU='} 2024-09-21T03:22:00.621 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913469', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:17:10.605488', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPrMPun9VE18M+GfC3lswD6U6As9u0jJS0qhPlADrXZ+o73UgpD+8Zl5UHPcTmTuAlfAcbdtGiFxAMKT8irE0hQ='} 2024-09-21T03:22:00.641 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913469', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:17:10.606678', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPDYVBKZiLokEbevBJa0jlZdAv/fC5Rr/MShdq7SKRgF1/Vfr4FaRMlLrWvhdh2HrRn2BrrKW/NMFkJNjrzUfG0='} 2024-09-21T03:22:00.641 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T03:22:00.652 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T03:22:00.652 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T03:22:00.653 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T03:22:00.653 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T03:22:00.919 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe040bc8160>, signals=[15]) 2024-09-21T03:22:00.919 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T03:22:01.051 INFO:teuthology.task.internal:Opening connections... 2024-09-21T03:22:01.051 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-09-21T03:22:01.052 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:22:01.128 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-09-21T03:22:01.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:22:01.204 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-09-21T03:22:01.204 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:22:01.281 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T03:22:01.349 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-09-21T03:22:01.371 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-09-21T03:22:01.371 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-09-21T03:22:01.429 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-09-21T03:22:01.429 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-09-21T03:22:01.429 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-09-21T03:22:01.429 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:22:01.429 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-09-21T03:22:01.429 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:22:01.430 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:22:01.430 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-09-21T03:22:01.430 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:22:01.430 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:22:01.430 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:22:01.430 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:22:01.430 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:22:01.430 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:22:01.430 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-09-21T03:22:01.452 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-09-21T03:22:01.470 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-09-21T03:22:01.470 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-09-21T03:22:01.527 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-09-21T03:22:01.527 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-09-21T03:22:01.527 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-09-21T03:22:01.527 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:22:01.527 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-09-21T03:22:01.528 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:22:01.528 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:22:01.528 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-09-21T03:22:01.528 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:22:01.528 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:22:01.528 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:22:01.528 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:22:01.528 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:22:01.528 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:22:01.528 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-09-21T03:22:01.552 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-09-21T03:22:01.570 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-09-21T03:22:01.570 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-09-21T03:22:01.628 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-09-21T03:22:01.628 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-09-21T03:22:01.628 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-09-21T03:22:01.628 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:22:01.628 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-09-21T03:22:01.628 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:22:01.628 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:22:01.628 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-09-21T03:22:01.628 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:22:01.628 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:22:01.628 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:22:01.628 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:22:01.629 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:22:01.629 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:22:01.629 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-09-21T03:22:01.652 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T03:22:01.664 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T03:22:01.731 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T03:22:01.731 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:22:01.733 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:22:01.735 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:22:01.752 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T03:22:01.852 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T03:22:01.853 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:22:01.855 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:22:01.857 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:22:01.936 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T03:22:02.014 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T03:22:02.033 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T03:22:02.033 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-21T03:22:02.033 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-21T03:22:02.034 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:22:02.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:22:02.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-21T03:22:02.214 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T03:22:02.214 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-21T03:22:02.214 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-21T03:22:02.232 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-503.el9.x86_64 2024-09-21T03:22:02.232 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T03:22:02.232 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-21T03:22:02.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum upgrade -y linux-firmware 2024-09-21T03:22:04.698 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-21T03:22:06.832 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-21T03:22:12.349 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.5 MB 00:01 2024-09-21T03:22:13.875 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 19 kB 00:00 2024-09-21T03:22:14.996 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-21T03:22:20.660 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-21T03:22:22.753 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-21T03:22:22.764 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-21T03:22:22.764 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-09-21T03:22:22.764 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-21T03:22:22.765 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-09-21T03:22:22.765 INFO:teuthology.orchestra.run.smithi016.stdout: linux-firmware noarch 20240905-146.el9 baseos 388 M 2024-09-21T03:22:22.765 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:22:22.765 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-21T03:22:22.765 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-21T03:22:22.765 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 1 Package 2024-09-21T03:22:22.765 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:22:22.765 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 388 M 2024-09-21T03:22:22.765 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-21T03:22:40.810 INFO:teuthology.orchestra.run.smithi016.stdout:linux-firmware-20240905-146.el9.noarch.rpm 22 MB/s | 388 MB 00:17 2024-09-21T03:22:40.810 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:22:40.811 INFO:teuthology.orchestra.run.smithi016.stdout:Total 22 MB/s | 388 MB 00:18 2024-09-21T03:22:44.082 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-21T03:22:44.164 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-21T03:22:44.164 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-21T03:22:47.576 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-21T03:22:47.576 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-21T03:22:57.213 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-21T03:22:58.069 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T03:22:58.163 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T03:22:58.677 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T03:22:58.677 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T03:22:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T03:22:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:22:59.402 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-09-21T03:22:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: linux-firmware-20240905-146.el9.noarch 2024-09-21T03:22:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:22:59.402 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-21T03:22:59.734 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-21T03:22:59.735 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T03:22:59.875 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-21T03:22:59.875 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-21T03:22:59.877 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi016.front.sepia.ceph.com to complete... 2024-09-21T03:23:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21 03:23:06 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-21T03:23:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:FINISHED --2024-09-21 03:23:06-- 2024-09-21T03:23:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:Total wall clock time: 6.3s 2024-09-21T03:23:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:Downloaded: 1 files, 343M in 6.0s (57.1 MB/s) 2024-09-21T03:23:06.181 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-21T03:23:06.181 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-21T03:23:08.345 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying... ######################################## 2024-09-21T03:23:08.403 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing... ######################################## 2024-09-21T03:23:08.466 INFO:teuthology.orchestra.run.smithi016.stdout:Updating / installing... 2024-09-21T03:23:16.786 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-21T03:23:57.846 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-21T03:23:57.846 DEBUG:teuthology.orchestra.run.smithi016:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-21T03:23:57.932 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T03:23:57.932 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T03:23:57.932 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T03:23:57.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-21T03:23:57.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-21T03:23:57.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-21T03:23:57.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-21T03:23:57.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-21T03:23:57.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-21T03:23:57.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-21T03:23:57.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-21T03:23:57.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-21T03:23:57.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-21T03:23:57.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-21T03:23:57.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-21T03:23:57.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-21T03:23:57.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-21T03:23:57.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-21T03:23:57.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-21T03:23:57.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-21T03:23:57.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-21T03:23:57.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-21T03:23:57.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-21T03:23:57.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-21T03:23:57.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-21T03:23:57.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-21T03:23:57.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-21T03:23:57.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-21T03:23:57.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-21T03:23:57.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-21T03:23:57.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-21T03:23:57.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-21T03:23:57.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-21T03:23:57.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-21T03:23:57.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-21T03:23:57.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-21T03:23:57.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-21T03:23:57.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-21T03:23:57.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-21T03:23:57.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-21T03:23:57.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-21T03:23:57.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-21T03:23:57.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-21T03:23:57.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-21T03:23:57.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-21T03:23:57.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-21T03:23:57.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-21T03:23:57.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-21T03:23:57.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-21T03:23:57.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-21T03:23:57.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-21T03:23:57.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-21T03:23:57.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-21T03:23:57.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-21T03:23:57.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-21T03:23:57.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-21T03:23:57.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-21T03:23:57.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-21T03:23:57.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-21T03:23:57.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-21T03:23:57.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-21T03:23:57.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-21T03:23:57.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-21T03:23:57.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-21T03:23:57.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-21T03:23:57.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-21T03:23:57.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-21T03:23:57.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-21T03:23:57.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-21T03:23:57.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-21T03:23:57.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-21T03:23:57.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-21T03:23:57.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-21T03:23:57.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-21T03:23:57.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-21T03:23:57.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-21T03:23:57.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-21T03:23:57.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-21T03:23:57.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-21T03:23:57.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-21T03:23:57.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-21T03:23:57.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-21T03:23:57.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-21T03:23:57.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-21T03:23:57.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-21T03:23:57.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-21T03:23:57.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-21T03:23:57.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-21T03:23:57.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-21T03:23:57.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-21T03:23:57.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-21T03:23:57.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-21T03:23:57.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-21T03:23:57.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-21T03:23:57.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-21T03:23:57.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-21T03:23:57.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-21T03:23:57.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-21T03:23:57.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-21T03:23:57.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-21T03:23:57.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-21T03:23:57.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-21T03:23:57.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-21T03:23:57.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-21T03:23:57.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-21T03:23:57.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-21T03:23:57.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-21T03:23:57.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-21T03:23:57.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-21T03:23:57.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-21T03:23:57.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-21T03:23:57.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-21T03:23:57.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-21T03:23:57.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-21T03:23:57.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-21T03:23:57.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-21T03:23:57.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-21T03:23:57.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-21T03:23:57.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-21T03:23:57.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-21T03:23:57.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-21T03:23:57.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-21T03:23:57.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-21T03:23:57.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-21T03:23:57.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-21T03:23:57.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-21T03:23:57.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-21T03:23:57.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-21T03:23:57.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-21T03:23:57.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-21T03:23:57.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-21T03:23:57.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-21T03:23:57.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-21T03:23:57.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-21T03:23:57.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-21T03:23:57.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-21T03:23:57.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-21T03:23:57.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-21T03:23:57.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-21T03:23:57.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-21T03:23:57.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-21T03:23:57.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-21T03:23:57.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-21T03:23:57.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-21T03:23:57.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-21T03:23:57.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-21T03:23:57.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-21T03:23:57.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-21T03:23:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-21T03:23:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-21T03:23:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-21T03:23:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-21T03:23:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-21T03:23:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-21T03:23:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-21T03:23:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-21T03:23:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-21T03:23:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-21T03:23:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-21T03:23:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-21T03:23:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-21T03:23:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-21T03:23:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-21T03:23:57.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-21T03:23:57.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-21T03:23:57.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-21T03:23:57.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-21T03:23:57.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-21T03:23:57.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-21T03:23:57.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-21T03:23:57.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-21T03:23:57.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-21T03:23:57.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-21T03:23:57.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-21T03:23:57.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-21T03:23:57.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-21T03:23:57.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-21T03:23:57.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-21T03:23:57.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-21T03:23:57.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-21T03:23:57.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-21T03:23:57.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-21T03:23:57.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-21T03:23:57.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-21T03:23:57.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-21T03:23:57.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-21T03:23:57.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-21T03:23:57.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-21T03:23:57.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-21T03:23:57.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-21T03:23:57.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-21T03:23:57.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-21T03:23:57.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-21T03:23:57.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-21T03:23:57.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-21T03:23:57.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-21T03:23:57.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-21T03:23:57.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-21T03:23:57.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-21T03:23:57.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-21T03:23:57.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-21T03:23:57.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-21T03:23:57.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-21T03:23:57.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-21T03:23:57.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-21T03:23:57.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-21T03:23:57.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-21T03:23:57.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-21T03:23:57.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-21T03:23:57.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-21T03:23:57.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-21T03:23:57.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-21T03:23:57.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-21T03:23:57.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-21T03:23:57.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-21T03:23:57.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-21T03:23:57.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-21T03:23:57.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-21T03:23:57.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-21T03:23:57.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-21T03:23:57.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-21T03:23:57.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-21T03:23:57.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-21T03:23:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-21T03:23:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-21T03:23:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-21T03:23:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-21T03:23:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-21T03:23:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-21T03:23:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-21T03:23:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-21T03:23:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-21T03:23:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-21T03:23:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-21T03:23:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-21T03:23:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-21T03:23:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-21T03:23:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-21T03:23:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-21T03:23:57.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-21T03:23:57.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-21T03:23:57.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-21T03:23:57.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-21T03:23:57.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-21T03:23:57.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-21T03:23:57.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-21T03:23:57.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-21T03:23:57.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-21T03:23:57.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-21T03:23:57.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-21T03:23:57.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-21T03:23:57.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-21T03:23:57.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-21T03:23:57.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-21T03:23:57.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-21T03:23:57.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-21T03:23:57.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-21T03:23:57.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-21T03:23:57.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-21T03:23:57.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-21T03:23:57.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-21T03:23:57.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-21T03:23:57.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-21T03:23:57.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-21T03:23:57.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-21T03:23:57.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-21T03:23:57.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-21T03:23:57.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-21T03:23:57.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-21T03:23:57.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-21T03:23:57.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-21T03:23:57.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-21T03:23:57.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-21T03:23:57.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-21T03:23:57.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-21T03:23:57.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-21T03:23:57.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-21T03:23:57.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-21T03:23:57.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-21T03:23:57.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-21T03:23:57.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-21T03:23:57.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-21T03:23:57.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-21T03:23:57.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-21T03:23:57.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-21T03:23:57.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-21T03:23:57.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-21T03:23:57.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-21T03:23:57.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-21T03:23:57.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-21T03:23:57.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-21T03:23:57.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-21T03:23:57.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-21T03:23:57.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-21T03:23:57.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-21T03:23:57.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-21T03:23:57.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-21T03:23:57.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-21T03:23:57.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-21T03:23:57.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-21T03:23:57.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-21T03:23:57.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-21T03:23:57.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-21T03:23:57.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-21T03:23:57.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-21T03:23:57.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-21T03:23:57.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-21T03:23:57.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-21T03:23:57.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-21T03:23:57.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-21T03:23:57.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-21T03:23:57.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-21T03:23:57.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-21T03:23:57.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-21T03:23:57.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-21T03:23:57.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-21T03:23:57.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-21T03:23:57.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-21T03:23:57.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-21T03:23:57.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-21T03:23:57.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-21T03:23:57.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-21T03:23:57.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-21T03:23:57.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-21T03:23:57.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-21T03:23:57.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-21T03:23:57.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-21T03:23:57.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-21T03:23:57.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-21T03:23:57.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-21T03:23:57.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-21T03:23:57.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-21T03:23:57.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-21T03:23:57.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-21T03:23:57.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-21T03:23:57.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-21T03:23:57.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-21T03:23:57.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-21T03:23:57.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-21T03:23:57.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-21T03:23:57.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-21T03:23:57.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-21T03:23:57.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-21T03:23:57.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-21T03:23:57.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-21T03:23:57.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-21T03:23:57.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-21T03:23:57.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-21T03:23:57.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-21T03:23:57.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-21T03:23:57.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-21T03:23:57.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-21T03:23:57.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-21T03:23:57.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-21T03:23:57.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-21T03:23:57.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-21T03:23:57.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-21T03:23:57.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-21T03:23:57.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-21T03:23:57.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-21T03:23:57.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-21T03:23:57.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-21T03:23:57.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-21T03:23:57.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-21T03:23:57.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-21T03:23:57.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-21T03:23:57.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-21T03:23:57.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-21T03:23:57.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-21T03:23:57.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-21T03:23:57.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-21T03:23:57.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-21T03:23:57.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-21T03:23:57.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-21T03:23:57.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-21T03:23:57.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-21T03:23:57.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-21T03:23:57.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-21T03:23:57.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-21T03:23:57.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-21T03:23:57.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-21T03:23:57.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-21T03:23:57.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-21T03:23:57.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-21T03:23:57.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-21T03:23:57.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-21T03:23:57.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-21T03:23:57.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-21T03:23:57.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-21T03:23:57.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-21T03:23:57.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-21T03:23:57.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-21T03:23:57.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-21T03:23:57.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-21T03:23:57.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-21T03:23:57.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-21T03:23:57.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-21T03:23:57.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-21T03:23:57.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-21T03:23:57.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-21T03:23:57.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-21T03:23:57.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-21T03:23:57.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-21T03:23:57.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-21T03:23:57.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-21T03:23:57.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-21T03:23:57.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-21T03:23:57.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-21T03:23:57.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-21T03:23:57.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-21T03:23:57.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-21T03:23:57.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-21T03:23:57.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-21T03:23:57.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-21T03:23:57.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-21T03:23:57.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-21T03:23:57.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-21T03:23:57.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-21T03:23:57.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-21T03:23:57.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-21T03:23:57.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-21T03:23:57.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-21T03:23:57.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-21T03:23:57.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-21T03:23:57.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-21T03:23:57.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-21T03:23:57.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-21T03:23:57.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-21T03:23:57.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-21T03:23:57.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-21T03:23:57.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-21T03:23:57.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-21T03:23:57.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-21T03:23:57.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-21T03:23:57.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-21T03:23:57.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-21T03:23:57.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-21T03:23:57.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-21T03:23:57.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-21T03:23:57.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-21T03:23:57.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-21T03:23:57.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-21T03:23:57.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-21T03:23:57.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-21T03:23:57.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-21T03:23:57.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-21T03:23:57.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-21T03:23:57.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-21T03:23:57.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-21T03:23:57.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-21T03:23:57.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-21T03:23:57.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-21T03:23:57.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-21T03:23:57.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-21T03:23:57.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-21T03:23:57.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-21T03:23:57.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-21T03:23:57.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-21T03:23:57.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-21T03:23:57.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-21T03:23:57.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-21T03:23:57.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-21T03:23:57.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-21T03:23:57.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-21T03:23:57.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-21T03:23:57.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-21T03:23:57.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-21T03:23:57.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-21T03:23:57.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-21T03:23:57.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-21T03:23:57.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-21T03:23:57.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-21T03:23:57.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-21T03:23:57.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-21T03:23:57.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-21T03:23:57.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-21T03:23:57.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-21T03:23:57.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-21T03:23:57.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-21T03:23:57.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-21T03:23:57.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-21T03:23:57.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-21T03:23:57.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-21T03:23:57.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-21T03:23:57.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-21T03:23:57.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-21T03:23:57.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-21T03:23:57.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-21T03:23:57.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-21T03:23:57.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-21T03:23:57.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-21T03:23:57.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-21T03:23:57.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-21T03:23:57.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-21T03:23:57.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-21T03:23:57.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-21T03:23:57.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-21T03:23:57.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-21T03:23:57.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-21T03:23:57.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-21T03:23:57.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-21T03:23:57.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-21T03:23:57.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-21T03:23:57.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-21T03:23:57.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-21T03:23:57.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-21T03:23:57.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-21T03:23:57.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-21T03:23:57.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-21T03:23:57.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-21T03:23:57.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-21T03:23:57.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-21T03:23:57.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-21T03:23:57.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-21T03:23:57.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-21T03:23:57.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-21T03:23:57.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-21T03:23:57.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-21T03:23:57.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-21T03:23:57.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-21T03:23:57.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-21T03:23:57.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-21T03:23:57.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-21T03:23:57.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-21T03:23:57.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-21T03:23:57.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-21T03:23:57.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-21T03:23:57.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-21T03:23:57.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-21T03:23:57.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-21T03:23:57.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-21T03:23:57.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-21T03:23:57.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-21T03:23:57.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-21T03:23:57.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-21T03:23:57.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-21T03:23:57.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-21T03:23:57.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-21T03:23:57.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-21T03:23:57.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-21T03:23:57.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-21T03:23:57.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-21T03:23:57.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-21T03:23:57.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-21T03:23:57.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-21T03:23:57.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-21T03:23:57.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-21T03:23:57.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-21T03:23:57.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-21T03:23:57.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-21T03:23:57.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-21T03:23:57.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-21T03:23:57.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-21T03:23:57.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-21T03:23:57.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-21T03:23:57.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-21T03:23:57.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-21T03:23:57.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-21T03:23:57.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-21T03:23:57.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-21T03:23:57.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-21T03:23:57.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-21T03:23:57.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-21T03:23:57.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-21T03:23:57.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-21T03:23:57.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-21T03:23:57.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-21T03:23:57.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-21T03:23:57.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-21T03:23:57.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-21T03:23:57.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-21T03:23:57.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-21T03:23:57.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-21T03:23:57.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-21T03:23:57.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-21T03:23:57.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-21T03:23:57.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-21T03:23:57.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-21T03:23:57.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-21T03:23:57.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-21T03:23:57.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-21T03:23:57.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-21T03:23:57.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-21T03:23:57.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-21T03:23:57.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-21T03:23:57.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-21T03:23:57.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-21T03:23:57.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-21T03:23:57.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-21T03:23:57.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-21T03:23:57.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-21T03:23:57.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-21T03:23:57.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-21T03:23:57.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-21T03:23:57.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-21T03:23:57.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-21T03:23:57.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-21T03:23:57.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-21T03:23:57.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-21T03:23:57.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-21T03:23:57.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-21T03:23:57.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-21T03:23:57.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-21T03:23:57.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-21T03:23:57.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-21T03:23:57.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-21T03:23:57.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-21T03:23:57.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-21T03:23:57.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-21T03:23:57.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-21T03:23:57.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-21T03:23:57.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-21T03:23:57.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-21T03:23:57.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-21T03:23:57.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-21T03:23:57.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-21T03:23:57.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-21T03:23:57.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-21T03:23:57.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-21T03:23:57.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-21T03:23:57.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-21T03:23:57.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-21T03:23:57.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-21T03:23:57.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-21T03:23:57.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-21T03:23:57.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-21T03:23:57.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-21T03:23:57.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-21T03:23:57.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-21T03:23:57.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-21T03:23:57.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-21T03:23:57.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-21T03:23:57.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-21T03:23:57.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-21T03:23:57.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-21T03:23:57.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-21T03:23:57.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-21T03:23:57.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-21T03:23:57.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-21T03:23:57.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-21T03:23:57.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-21T03:23:57.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-21T03:23:57.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-21T03:23:57.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-21T03:23:57.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-21T03:23:57.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-21T03:23:57.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-21T03:23:57.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-21T03:23:57.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-21T03:23:57.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-21T03:23:57.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-21T03:23:57.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-21T03:23:57.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-21T03:23:57.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-21T03:23:57.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-21T03:23:57.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-21T03:23:57.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-21T03:23:57.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-21T03:23:57.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-21T03:23:57.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-21T03:23:57.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-21T03:23:57.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-21T03:23:57.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-21T03:23:57.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-21T03:23:57.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-21T03:23:57.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-21T03:23:57.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-21T03:23:57.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-21T03:23:57.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-21T03:23:57.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-21T03:23:57.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-21T03:23:57.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-21T03:23:57.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-21T03:23:57.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-21T03:23:57.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-21T03:23:57.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-21T03:23:57.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-21T03:23:57.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-21T03:23:57.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-21T03:23:57.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-21T03:23:57.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-21T03:23:57.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-21T03:23:57.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-21T03:23:57.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-21T03:23:57.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-21T03:23:57.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-21T03:23:57.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-21T03:23:57.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-21T03:23:57.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-21T03:23:57.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-21T03:23:57.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-21T03:23:57.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-21T03:23:57.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-21T03:23:57.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-21T03:23:57.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-21T03:23:57.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-21T03:23:57.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-21T03:23:57.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-21T03:23:57.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-21T03:23:57.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-21T03:23:57.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-21T03:23:57.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-21T03:23:57.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-21T03:23:57.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-21T03:23:57.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-21T03:23:57.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-21T03:23:57.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-21T03:23:57.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-21T03:23:57.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-21T03:23:57.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-21T03:23:57.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-21T03:23:57.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-21T03:23:57.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-21T03:23:57.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-21T03:23:57.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-21T03:23:57.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-21T03:23:57.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-21T03:23:57.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-21T03:23:57.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-21T03:23:57.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-21T03:23:57.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-21T03:23:57.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-21T03:23:57.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-21T03:23:57.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-21T03:23:57.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-21T03:23:57.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-21T03:23:57.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-21T03:23:57.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-21T03:23:57.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-21T03:23:57.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-21T03:23:57.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-21T03:23:57.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-21T03:23:57.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-21T03:23:57.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-21T03:23:57.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-21T03:23:57.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-21T03:23:57.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-21T03:23:57.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-21T03:23:57.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-21T03:23:57.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-21T03:23:57.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-21T03:23:57.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-21T03:23:57.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-21T03:23:57.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-21T03:23:57.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-21T03:23:57.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-21T03:23:57.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-21T03:23:57.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-21T03:23:57.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-21T03:23:57.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-21T03:23:57.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-21T03:23:57.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-21T03:23:57.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-21T03:23:57.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-21T03:23:57.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-21T03:23:57.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-21T03:23:57.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-21T03:23:57.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-21T03:23:57.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-21T03:23:57.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-21T03:23:57.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-21T03:23:57.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-21T03:23:57.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-21T03:23:57.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-21T03:23:57.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-21T03:23:57.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-21T03:23:57.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-21T03:23:57.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-21T03:23:57.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-21T03:23:57.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-21T03:23:57.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-21T03:23:57.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-21T03:23:57.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-21T03:23:57.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-21T03:23:57.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-21T03:23:57.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-21T03:23:57.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-21T03:23:57.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-21T03:23:57.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-21T03:23:57.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-21T03:23:57.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-21T03:23:57.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-21T03:23:57.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-21T03:23:57.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-21T03:23:57.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-21T03:23:57.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-21T03:23:57.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-21T03:23:57.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-21T03:23:57.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-21T03:23:57.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-21T03:23:57.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-21T03:23:57.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-21T03:23:57.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-21T03:23:57.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-21T03:23:57.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-21T03:23:57.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-21T03:23:57.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-21T03:23:57.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-21T03:23:57.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-21T03:23:57.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-21T03:23:57.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-21T03:23:57.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-21T03:23:57.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-21T03:23:57.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-21T03:23:57.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-21T03:23:57.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-21T03:23:57.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-21T03:23:57.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-21T03:23:57.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-21T03:23:57.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-21T03:23:57.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-21T03:23:57.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-21T03:23:57.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-21T03:23:57.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-21T03:23:57.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-21T03:23:57.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-21T03:23:57.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-21T03:23:57.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-21T03:23:57.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-21T03:23:57.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-21T03:23:57.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-21T03:23:57.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-21T03:23:57.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-21T03:23:57.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-21T03:23:57.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-21T03:23:57.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-21T03:23:57.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-21T03:23:57.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-21T03:23:57.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-21T03:23:57.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-21T03:23:57.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-21T03:23:57.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-21T03:23:57.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-21T03:23:57.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-21T03:23:57.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-21T03:23:57.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-21T03:23:57.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-21T03:23:57.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-21T03:23:57.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-21T03:23:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-21T03:23:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-21T03:23:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-21T03:23:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-21T03:23:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-21T03:23:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-21T03:23:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-21T03:23:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-21T03:23:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-21T03:23:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-21T03:23:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-21T03:23:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-21T03:23:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-21T03:23:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-21T03:23:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-21T03:23:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-21T03:23:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-21T03:23:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-21T03:23:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-21T03:23:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-21T03:23:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-21T03:23:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-21T03:23:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-21T03:23:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-21T03:23:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-21T03:23:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-21T03:23:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-21T03:23:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-21T03:23:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-21T03:23:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-21T03:23:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-21T03:23:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-21T03:23:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-21T03:23:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-21T03:23:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-21T03:23:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-21T03:23:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-21T03:23:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-21T03:23:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-21T03:23:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-21T03:23:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-21T03:23:57.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-21T03:23:57.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-21T03:23:57.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-21T03:23:57.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-21T03:23:57.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-21T03:23:57.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-21T03:23:57.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-21T03:23:57.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-21T03:23:57.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-21T03:23:57.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-21T03:23:57.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-21T03:23:57.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-21T03:23:57.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-21T03:23:57.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-21T03:23:57.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-21T03:23:57.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-21T03:23:57.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-21T03:23:57.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-21T03:23:57.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-21T03:23:57.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-21T03:23:57.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-21T03:23:57.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-21T03:23:57.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-21T03:23:57.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-21T03:23:57.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-21T03:23:57.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-21T03:23:57.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-21T03:23:57.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-21T03:23:57.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-21T03:23:57.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-21T03:23:57.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-21T03:23:57.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-21T03:23:57.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-21T03:23:57.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-21T03:23:57.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-21T03:23:57.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-21T03:23:57.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-21T03:23:57.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-21T03:23:57.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-21T03:23:57.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-21T03:23:57.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-21T03:23:57.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-21T03:23:57.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-21T03:23:57.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-21T03:23:57.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-21T03:23:57.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-21T03:23:57.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-21T03:23:57.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-21T03:23:57.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-21T03:23:57.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-21T03:23:57.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-21T03:23:57.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-21T03:23:57.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-21T03:23:57.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-21T03:23:57.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-21T03:23:57.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-21T03:23:57.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-21T03:23:57.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-21T03:23:57.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-21T03:23:57.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-21T03:23:57.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-21T03:23:57.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-21T03:23:57.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-21T03:23:57.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-21T03:23:57.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-21T03:23:57.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-21T03:23:57.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-21T03:23:57.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-21T03:23:57.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-21T03:23:57.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-21T03:23:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-21T03:23:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-21T03:23:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-21T03:23:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-21T03:23:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-21T03:23:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-21T03:23:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-21T03:23:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-21T03:23:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-21T03:23:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-21T03:23:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-21T03:23:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-21T03:23:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-21T03:23:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-21T03:23:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-21T03:23:58.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-21T03:23:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-21T03:23:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-21T03:23:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-21T03:23:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-21T03:23:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-21T03:23:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-21T03:23:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-21T03:23:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-21T03:23:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-21T03:23:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-21T03:23:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-21T03:23:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-21T03:23:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-21T03:23:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-21T03:23:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-21T03:23:58.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-21T03:23:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-21T03:23:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-21T03:23:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-21T03:23:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-21T03:23:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-21T03:23:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-21T03:23:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-21T03:23:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-21T03:23:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-21T03:23:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-21T03:23:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-21T03:23:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-21T03:23:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-21T03:23:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-21T03:23:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-21T03:23:58.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-21T03:23:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-21T03:23:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-21T03:23:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-21T03:23:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-21T03:23:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-21T03:23:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-21T03:23:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-21T03:23:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-21T03:23:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-21T03:23:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-21T03:23:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-21T03:23:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-21T03:23:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-21T03:23:58.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-21T03:23:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-21T03:23:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-21T03:23:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-21T03:23:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-21T03:23:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-21T03:23:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-21T03:23:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-21T03:23:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-21T03:23:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-21T03:23:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-21T03:23:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-21T03:23:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-21T03:23:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-21T03:23:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-21T03:23:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-21T03:23:58.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-21T03:23:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-21T03:23:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-21T03:23:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-21T03:23:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-21T03:23:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-21T03:23:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-21T03:23:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-21T03:23:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-21T03:23:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-21T03:23:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-21T03:23:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-21T03:23:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-21T03:23:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-21T03:23:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-21T03:23:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-21T03:23:58.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-21T03:23:58.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-21T03:23:58.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-21T03:23:58.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-21T03:23:58.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-21T03:23:58.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-21T03:23:58.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-21T03:23:58.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-21T03:23:58.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-21T03:23:58.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-21T03:23:58.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-21T03:23:58.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-21T03:23:58.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-21T03:23:58.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-21T03:23:58.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-21T03:23:58.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-21T03:23:58.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-21T03:23:58.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-21T03:23:58.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-21T03:23:58.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-21T03:23:58.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-21T03:23:58.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-21T03:23:58.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-21T03:23:58.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-21T03:23:58.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-21T03:23:58.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-21T03:23:58.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-21T03:23:58.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-21T03:23:58.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-21T03:23:58.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-21T03:23:58.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-21T03:23:58.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-21T03:23:58.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-21T03:23:58.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-21T03:23:58.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-21T03:23:58.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-21T03:23:58.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-21T03:23:58.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-21T03:23:58.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-21T03:23:58.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-21T03:23:58.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-21T03:23:58.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-21T03:23:58.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-21T03:23:58.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-21T03:23:58.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-21T03:23:58.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-21T03:23:58.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-21T03:23:58.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-21T03:23:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-21T03:23:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-21T03:23:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-21T03:23:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-21T03:23:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-21T03:23:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-21T03:23:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-21T03:23:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-21T03:23:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-21T03:23:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-21T03:23:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-21T03:23:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-21T03:23:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-21T03:23:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-21T03:23:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-21T03:23:58.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-21T03:23:58.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-21T03:23:58.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-21T03:23:58.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-21T03:23:58.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-21T03:23:58.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-21T03:23:58.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-21T03:23:58.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-21T03:23:58.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-21T03:23:58.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-21T03:23:58.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-21T03:23:58.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-21T03:23:58.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-21T03:23:58.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-21T03:23:58.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-21T03:23:58.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-21T03:23:58.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-21T03:23:58.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-21T03:23:58.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-21T03:23:58.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-21T03:23:58.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-21T03:23:58.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-21T03:23:58.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-21T03:23:58.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-21T03:23:58.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-21T03:23:58.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-21T03:23:58.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-21T03:23:58.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-21T03:23:58.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-21T03:23:58.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-21T03:23:58.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-21T03:23:58.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-21T03:23:58.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-21T03:23:58.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-21T03:23:58.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-21T03:23:58.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-21T03:23:58.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-21T03:23:58.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-21T03:23:58.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-21T03:23:58.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-21T03:23:58.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-21T03:23:58.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-21T03:23:58.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-21T03:23:58.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-21T03:23:58.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-21T03:23:58.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-21T03:23:58.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-21T03:23:58.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-21T03:23:58.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-21T03:23:58.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-21T03:23:58.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-21T03:23:58.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-21T03:23:58.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-21T03:23:58.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-21T03:23:58.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-21T03:23:58.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-21T03:23:58.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-21T03:23:58.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-21T03:23:58.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-21T03:23:58.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-21T03:23:58.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-21T03:23:58.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-21T03:23:58.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-21T03:23:58.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-21T03:23:58.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-21T03:23:58.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-21T03:23:58.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-21T03:23:58.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-21T03:23:58.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-21T03:23:58.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-21T03:23:58.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-21T03:23:58.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-21T03:23:58.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-21T03:23:58.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-21T03:23:58.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-21T03:23:58.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-21T03:23:58.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-21T03:23:58.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-21T03:23:58.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-21T03:23:58.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-21T03:23:58.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-21T03:23:58.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-21T03:23:58.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-21T03:23:58.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-21T03:23:58.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-21T03:23:58.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-21T03:23:58.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-21T03:23:58.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-21T03:23:58.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-21T03:23:58.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-21T03:23:58.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-21T03:23:58.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-21T03:23:58.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-21T03:23:58.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-21T03:23:58.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-21T03:23:58.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-21T03:23:58.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-21T03:23:58.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-21T03:23:58.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-21T03:23:58.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-21T03:23:58.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-21T03:23:58.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-21T03:23:58.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-21T03:23:58.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-21T03:23:58.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-21T03:23:58.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-21T03:23:58.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-21T03:23:58.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-21T03:23:58.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-21T03:23:58.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-21T03:23:58.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-21T03:23:58.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-21T03:23:58.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-21T03:23:58.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-21T03:23:58.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-21T03:23:58.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-21T03:23:58.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-21T03:23:58.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-21T03:23:58.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-21T03:23:58.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-21T03:23:58.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-21T03:23:58.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-21T03:23:58.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-21T03:23:58.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-21T03:23:58.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-21T03:23:58.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-21T03:23:58.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-21T03:23:58.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-21T03:23:58.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-21T03:23:58.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-21T03:23:58.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-21T03:23:58.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-21T03:23:58.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-21T03:23:58.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-21T03:23:58.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-21T03:23:58.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-21T03:23:58.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-21T03:23:58.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-21T03:23:58.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-21T03:23:58.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-21T03:23:58.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-21T03:23:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-21T03:23:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-21T03:23:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-21T03:23:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-21T03:23:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-21T03:23:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-21T03:23:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-21T03:23:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-21T03:23:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-21T03:23:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-21T03:23:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-21T03:23:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-21T03:23:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-21T03:23:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-21T03:23:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-21T03:23:58.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-21T03:23:58.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-21T03:23:58.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-21T03:23:58.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-21T03:23:58.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-21T03:23:58.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-21T03:23:58.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-21T03:23:58.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-21T03:23:58.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-21T03:23:58.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-21T03:23:58.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-21T03:23:58.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-21T03:23:58.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-21T03:23:58.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-21T03:23:58.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-21T03:23:58.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-21T03:23:58.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-21T03:23:58.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-21T03:23:58.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-21T03:23:58.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-21T03:23:58.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-21T03:23:58.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-21T03:23:58.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-21T03:23:58.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-21T03:23:58.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-21T03:23:58.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-21T03:23:58.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-21T03:23:58.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-21T03:23:58.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-21T03:23:58.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-21T03:23:58.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-21T03:23:58.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-21T03:23:58.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-21T03:23:58.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-21T03:23:58.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-21T03:23:58.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-21T03:23:58.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-21T03:23:58.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-21T03:23:58.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-21T03:23:58.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-21T03:23:58.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-21T03:23:58.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-21T03:23:58.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-21T03:23:58.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-21T03:23:58.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-21T03:23:58.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-21T03:23:58.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-21T03:23:58.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-21T03:23:58.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-21T03:23:58.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-21T03:23:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-21T03:23:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-21T03:23:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-21T03:23:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-21T03:23:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-21T03:23:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-21T03:23:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-21T03:23:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-21T03:23:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-21T03:23:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-21T03:23:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-21T03:23:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-21T03:23:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-21T03:23:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-21T03:23:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-21T03:23:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-21T03:23:58.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-21T03:23:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-21T03:23:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-21T03:23:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-21T03:23:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-21T03:23:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-21T03:23:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-21T03:23:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-21T03:23:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-21T03:23:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-21T03:23:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-21T03:23:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-21T03:23:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-21T03:23:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-21T03:23:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-21T03:23:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-21T03:23:58.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-21T03:23:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-21T03:23:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-21T03:23:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-21T03:23:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-21T03:23:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-21T03:23:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-21T03:23:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-21T03:23:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-21T03:23:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-21T03:23:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-21T03:23:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-21T03:23:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-21T03:23:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-21T03:23:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-21T03:23:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-21T03:23:58.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-21T03:23:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-21T03:23:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-21T03:23:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-21T03:23:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-21T03:23:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-21T03:23:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-21T03:23:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-21T03:23:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-21T03:23:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-21T03:23:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-21T03:23:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-21T03:23:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-21T03:23:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-21T03:23:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-21T03:23:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-21T03:23:58.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-21T03:23:58.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-21T03:23:58.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-21T03:23:58.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-21T03:23:58.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-21T03:23:58.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-21T03:23:58.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-21T03:23:58.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-21T03:23:58.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-21T03:23:58.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-21T03:23:58.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-21T03:23:58.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-21T03:23:58.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-21T03:23:58.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-21T03:23:58.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-21T03:23:58.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-21T03:23:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-21T03:23:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-21T03:23:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-21T03:23:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-21T03:23:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-21T03:23:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-21T03:23:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-21T03:23:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-21T03:23:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-21T03:23:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-21T03:23:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-21T03:23:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-21T03:23:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-21T03:23:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-21T03:23:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-21T03:23:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-21T03:23:58.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-21T03:23:58.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-21T03:23:58.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-21T03:23:58.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-21T03:23:58.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-21T03:23:58.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-21T03:23:58.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-21T03:23:58.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-21T03:23:58.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-21T03:23:58.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-21T03:23:58.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-21T03:23:58.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-21T03:23:58.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-21T03:23:58.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-21T03:23:58.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-21T03:23:58.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-21T03:23:58.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-21T03:23:58.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-21T03:23:58.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-21T03:23:58.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-21T03:23:58.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-21T03:23:58.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-21T03:23:58.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-21T03:23:58.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-21T03:23:58.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-21T03:23:58.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-21T03:23:58.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-21T03:23:58.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-21T03:23:58.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-21T03:23:58.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-21T03:23:58.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-21T03:23:58.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-21T03:23:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-21T03:23:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-21T03:23:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-21T03:23:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-21T03:23:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-21T03:23:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-21T03:23:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-21T03:23:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-21T03:23:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-21T03:23:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-21T03:23:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-21T03:23:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-21T03:23:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-21T03:23:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-21T03:23:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-21T03:23:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-21T03:23:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-21T03:23:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-21T03:23:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-21T03:23:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-21T03:23:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-21T03:23:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-21T03:23:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-21T03:23:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-21T03:23:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-21T03:23:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-21T03:23:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-21T03:23:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-21T03:23:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-21T03:23:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-21T03:23:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-21T03:23:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-21T03:23:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-21T03:23:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-21T03:23:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-21T03:23:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-21T03:23:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-21T03:23:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-21T03:23:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-21T03:23:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-21T03:23:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-21T03:23:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-21T03:23:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-21T03:23:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-21T03:23:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-21T03:23:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-21T03:23:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-21T03:23:58.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-21T03:23:58.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-21T03:23:58.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-21T03:23:58.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-21T03:23:58.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-21T03:23:58.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-21T03:23:58.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-21T03:23:58.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-21T03:23:58.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-21T03:23:58.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-21T03:23:58.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-21T03:23:58.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-21T03:23:58.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-21T03:23:58.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-21T03:23:58.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-21T03:23:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-21T03:23:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-21T03:23:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-21T03:23:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-21T03:23:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-21T03:23:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-21T03:23:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-21T03:23:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-21T03:23:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-21T03:23:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-21T03:23:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-21T03:23:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-21T03:23:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-21T03:23:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-21T03:23:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-21T03:23:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-21T03:23:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-21T03:23:58.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-21T03:23:58.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-21T03:23:58.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-21T03:23:58.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-21T03:23:58.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-21T03:23:58.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-21T03:23:58.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-21T03:23:58.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-21T03:23:58.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-21T03:23:58.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-21T03:23:58.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-21T03:23:58.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-21T03:23:58.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-21T03:23:58.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-21T03:23:58.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-21T03:23:58.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-21T03:23:58.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-21T03:23:58.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-21T03:23:58.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-21T03:23:58.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-21T03:23:58.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-21T03:23:58.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-21T03:23:58.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-21T03:23:58.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-21T03:23:58.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-21T03:23:58.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-21T03:23:58.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-21T03:23:58.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-21T03:23:58.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-21T03:23:58.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-21T03:23:58.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-21T03:23:58.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-21T03:23:58.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-21T03:23:58.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-21T03:23:58.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-21T03:23:58.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-21T03:23:58.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-21T03:23:58.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-21T03:23:58.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-21T03:23:58.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-21T03:23:58.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-21T03:23:58.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-21T03:23:58.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-21T03:23:58.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-21T03:23:58.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-21T03:23:58.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-21T03:23:58.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-21T03:23:58.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-21T03:23:58.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-21T03:23:58.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-21T03:23:58.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-21T03:23:58.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-21T03:23:58.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-21T03:23:58.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-21T03:23:58.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-21T03:23:58.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-21T03:23:58.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-21T03:23:58.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-21T03:23:58.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-21T03:23:58.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-21T03:23:58.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-21T03:23:58.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-21T03:23:58.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-21T03:23:58.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-21T03:23:58.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-21T03:23:58.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-21T03:23:58.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-21T03:23:58.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-21T03:23:58.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-21T03:23:58.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-21T03:23:58.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-21T03:23:58.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-21T03:23:58.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-21T03:23:58.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-21T03:23:58.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-21T03:23:58.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-21T03:23:58.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-21T03:23:58.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-21T03:23:58.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-21T03:23:58.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-21T03:23:58.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-21T03:23:58.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-21T03:23:58.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-21T03:23:58.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-21T03:23:58.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-21T03:23:58.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-21T03:23:58.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-21T03:23:58.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-21T03:23:58.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-21T03:23:58.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-21T03:23:58.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-21T03:23:58.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-21T03:23:58.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-21T03:23:58.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-21T03:23:58.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-21T03:23:58.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-21T03:23:58.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-21T03:23:58.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-21T03:23:58.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-21T03:23:58.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-21T03:23:58.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-21T03:23:58.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-21T03:23:58.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-21T03:23:58.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-21T03:23:58.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-21T03:23:58.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-21T03:23:58.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-21T03:23:58.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-21T03:23:58.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-21T03:23:58.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-21T03:23:58.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-21T03:23:58.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-21T03:23:58.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-21T03:23:58.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-21T03:23:58.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-21T03:23:58.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-21T03:23:58.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-21T03:23:58.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-21T03:23:58.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-21T03:23:58.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-21T03:23:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-21T03:23:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-21T03:23:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-21T03:23:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-21T03:23:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-21T03:23:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-21T03:23:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-21T03:23:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-21T03:23:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-21T03:23:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-21T03:23:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-21T03:23:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-21T03:23:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-21T03:23:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-21T03:23:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-21T03:23:58.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-21T03:23:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-21T03:23:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-21T03:23:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-21T03:23:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-21T03:23:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-21T03:23:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-21T03:23:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-21T03:23:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-21T03:23:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-21T03:23:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-21T03:23:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-21T03:23:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-21T03:23:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-21T03:23:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-21T03:23:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-21T03:23:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-21T03:23:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-21T03:23:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-21T03:23:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-21T03:23:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-21T03:23:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-21T03:23:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-21T03:23:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-21T03:23:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-21T03:23:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-21T03:23:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-21T03:23:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-21T03:23:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-21T03:23:58.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-21T03:23:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-21T03:23:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-21T03:23:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-21T03:23:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-21T03:23:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-21T03:23:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-21T03:23:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-21T03:23:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-21T03:23:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-21T03:23:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-21T03:23:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-21T03:23:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-21T03:23:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-21T03:23:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-21T03:23:58.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-21T03:23:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-21T03:23:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-21T03:23:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-21T03:23:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-21T03:23:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-21T03:23:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-21T03:23:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-21T03:23:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-21T03:23:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-21T03:23:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-21T03:23:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-21T03:23:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-21T03:23:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-21T03:23:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-21T03:23:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-21T03:23:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-21T03:23:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-21T03:23:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-21T03:23:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-21T03:23:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-21T03:23:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-21T03:23:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-21T03:23:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-21T03:23:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-21T03:23:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-21T03:23:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-21T03:23:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-21T03:23:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-21T03:23:58.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-21T03:23:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-21T03:23:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-21T03:23:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-21T03:23:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-21T03:23:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-21T03:23:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-21T03:23:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-21T03:23:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-21T03:23:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-21T03:23:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-21T03:23:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-21T03:23:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-21T03:23:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-21T03:23:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-21T03:23:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-21T03:23:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-21T03:23:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-21T03:23:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-21T03:23:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-21T03:23:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-21T03:23:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-21T03:23:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-21T03:23:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-21T03:23:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-21T03:23:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-21T03:23:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-21T03:23:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-21T03:23:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-21T03:23:58.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-21T03:23:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-21T03:23:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-21T03:23:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-21T03:23:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-21T03:23:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-21T03:23:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-21T03:23:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-21T03:23:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-21T03:23:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-21T03:23:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-21T03:23:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-21T03:23:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-21T03:23:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-21T03:23:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-21T03:23:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-21T03:23:58.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-21T03:23:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-21T03:23:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-21T03:23:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-21T03:23:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-21T03:23:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-21T03:23:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-21T03:23:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-21T03:23:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-21T03:23:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-21T03:23:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-21T03:23:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-21T03:23:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-21T03:23:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-21T03:23:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-21T03:23:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-21T03:23:58.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-21T03:23:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-21T03:23:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-21T03:23:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-21T03:23:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-21T03:23:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-21T03:23:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-21T03:23:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-21T03:23:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-21T03:23:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-21T03:23:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-21T03:23:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-21T03:23:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-21T03:23:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-21T03:23:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-21T03:23:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-21T03:23:58.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-21T03:23:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-21T03:23:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-21T03:23:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-21T03:23:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-21T03:23:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-21T03:23:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-21T03:23:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-21T03:23:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-21T03:23:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-21T03:23:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-21T03:23:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-21T03:23:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-21T03:23:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-21T03:23:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-21T03:23:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-21T03:23:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-21T03:23:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-21T03:23:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-21T03:23:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-21T03:23:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-21T03:23:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-21T03:23:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-21T03:23:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-21T03:23:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-21T03:23:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-21T03:23:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-21T03:23:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-21T03:23:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-21T03:23:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-21T03:23:58.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-21T03:23:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-21T03:23:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-21T03:23:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-21T03:23:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-21T03:23:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-21T03:23:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-21T03:23:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-21T03:23:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-21T03:23:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-21T03:23:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-21T03:23:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-21T03:23:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-21T03:23:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-21T03:23:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-21T03:23:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-21T03:23:58.058 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-21T03:23:58.058 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-21T03:23:58.058 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-21T03:23:58.058 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-21T03:23:58.058 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-21T03:23:58.058 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-21T03:23:58.058 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-21T03:23:58.058 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-21T03:23:58.058 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-21T03:23:58.058 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-21T03:23:58.058 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-21T03:23:58.058 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-21T03:23:58.058 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-21T03:23:58.058 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-21T03:23:58.058 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-21T03:23:58.059 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-21T03:23:58.059 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-21T03:23:58.059 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-21T03:23:58.059 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-21T03:23:58.059 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-21T03:23:58.059 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-21T03:23:58.059 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-21T03:23:58.059 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-21T03:23:58.059 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-21T03:23:58.059 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-21T03:23:58.059 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-21T03:23:58.059 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-21T03:23:58.059 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-21T03:23:58.059 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-21T03:23:58.059 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-21T03:23:58.060 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-21T03:23:58.060 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-21T03:23:58.060 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-21T03:23:58.060 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-21T03:23:58.060 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-21T03:23:58.060 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-21T03:23:58.060 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-21T03:23:58.060 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-21T03:23:58.060 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-21T03:23:58.060 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-21T03:23:58.060 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-21T03:23:58.060 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-21T03:23:58.060 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-21T03:23:58.060 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-21T03:23:58.060 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-21T03:23:58.060 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-21T03:23:58.061 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-21T03:23:58.061 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-21T03:23:58.061 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-21T03:23:58.061 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-21T03:23:58.061 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-21T03:23:58.061 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-21T03:23:58.061 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-21T03:23:58.061 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-21T03:23:58.061 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-21T03:23:58.061 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-21T03:23:58.061 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-21T03:23:58.061 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-21T03:23:58.061 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-21T03:23:58.061 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-21T03:23:58.061 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-21T03:23:58.061 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-21T03:23:58.062 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-21T03:23:58.062 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-21T03:23:58.062 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-21T03:23:58.062 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-21T03:23:58.062 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-21T03:23:58.062 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-21T03:23:58.062 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-21T03:23:58.062 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-21T03:23:58.062 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-21T03:23:58.062 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-21T03:23:58.062 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-21T03:23:58.062 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-21T03:23:58.062 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-21T03:23:58.062 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-21T03:23:58.062 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-21T03:23:58.063 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-21T03:23:58.063 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-21T03:23:58.063 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-21T03:23:58.063 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-21T03:23:58.063 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-21T03:23:58.063 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-21T03:23:58.063 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-21T03:23:58.063 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-21T03:23:58.063 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-21T03:23:58.063 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-21T03:23:58.063 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-21T03:23:58.063 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-21T03:23:58.063 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-21T03:23:58.063 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-21T03:23:58.063 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-21T03:23:58.064 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-21T03:23:58.064 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-21T03:23:58.064 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-21T03:23:58.064 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-21T03:23:58.064 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-21T03:23:58.064 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-21T03:23:58.064 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-21T03:23:58.064 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-21T03:23:58.064 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-21T03:23:58.064 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-21T03:23:58.064 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-21T03:23:58.064 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-21T03:23:58.064 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T03:23:58.068 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T03:23:58.068 DEBUG:teuthology.orchestra.run.smithi016:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-21T03:23:58.091 INFO:teuthology.orchestra.run.smithi016.stdout:postinstall scriptlet (using /bin/sh): 2024-09-21T03:23:58.092 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T03:23:58.092 INFO:teuthology.orchestra.run.smithi016.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T03:23:58.092 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2024-09-21T03:23:58.092 INFO:teuthology.orchestra.run.smithi016.stdout:for file in vmlinuz System.map config; do 2024-09-21T03:23:58.092 INFO:teuthology.orchestra.run.smithi016.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-21T03:23:58.092 INFO:teuthology.orchestra.run.smithi016.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T03:23:58.092 INFO:teuthology.orchestra.run.smithi016.stdout: fi 2024-09-21T03:23:58.092 INFO:teuthology.orchestra.run.smithi016.stdout:done 2024-09-21T03:23:58.092 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-21T03:23:58.092 INFO:teuthology.orchestra.run.smithi016.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-21T03:23:58.092 INFO:teuthology.orchestra.run.smithi016.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T03:23:58.092 INFO:teuthology.orchestra.run.smithi016.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T03:23:58.092 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2024-09-21T03:23:58.093 INFO:teuthology.orchestra.run.smithi016.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-21T03:23:58.093 INFO:teuthology.orchestra.run.smithi016.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T03:23:58.093 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2024-09-21T03:23:58.093 INFO:teuthology.orchestra.run.smithi016.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-21T03:23:58.093 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T03:23:58.093 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T03:23:58.093 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2024-09-21T03:23:58.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2024-09-21T03:23:58.167 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2024-09-21T03:23:58.167 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2024-09-21T03:23:58.167 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2024-09-21T03:23:58.167 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 82.el9 2024-09-21T03:23:58.168 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2024-09-21T03:23:58.168 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T03:23:58.168 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2024-09-21T03:23:58.168 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8274670 2024-09-21T03:23:58.168 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2024-09-21T03:23:58.168 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T03:23:58.168 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T03:23:58.168 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T03:23:58.168 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T03:23:58.168 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2024-09-21T03:23:58.168 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2024-09-21T03:23:58.168 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T03:23:58.168 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2024-09-21T03:23:58.168 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2024-09-21T03:23:58.168 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:23:58.169 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T03:23:58.169 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T03:23:58.169 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T03:23:58.169 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2024-09-21T03:23:58.169 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:23:58.169 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T03:23:58.170 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T03:23:58.170 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T03:23:58.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T03:23:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-09-21T03:24:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T03:24:00.059 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-09-21T03:24:00.061 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-09-21T03:24:00.076 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.3ri12q4E2E 2024-09-21T03:24:00.076 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3ri12q4E2E 2024-09-21T03:24:00.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.3ri12q4E2E 2024-09-21T03:24:00.289 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.3ri12q4E2E is 6KB 2024-09-21T03:24:00.309 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.3ri12q4E2E 2024-09-21T03:24:00.324 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T03:24:00.386 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T03:24:00.386 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T03:24:00.386 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T03:24:00.386 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-21T03:24:00.388 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T03:24:00.539 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2024-09-21T03:24:30.541 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T03:24:30.541 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-21T03:24:30.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:25:30.544 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T03:25:39.545 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-21T03:25:39.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:25:42.607 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2024-09-21T03:25:54.608 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-21T03:25:54.608 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:25:55.035 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-21T03:25:55.664 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-21T03:25:55.664 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-21T03:25:55.665 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-21T03:25:55.732 INFO:teuthology.orchestra.run.smithi016.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-21T03:25:55.732 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T03:25:55.733 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T03:25:55.733 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T03:25:56.733 DEBUG:teuthology.parallel:result is None 2024-09-21T03:25:56.733 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T03:25:56.740 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T03:25:56.741 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:25:56.752 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:25:56.755 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:25:56.830 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T03:25:56.846 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T03:25:57.032 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T03:25:57.032 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:25:57.043 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:25:57.045 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:25:57.122 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T03:25:57.183 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T03:25:57.183 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:25:57.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:25:57.202 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:25:57.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:25:57.219 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:25:57.226 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:25:57.245 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:25:57.281 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:25:57.294 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:25:57.315 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:25:57.334 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:25:57.416 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:25:57.442 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:25:57.447 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T03:25:57.454 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T03:25:57.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:25:57.459 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:25:57.461 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:25:57.512 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T03:25:57.547 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T03:25:57.547 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:25:57.551 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:25:57.553 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:25:57.584 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:25:57.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:25:57.778 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:25:57.843 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:25:57.915 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:25:57.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:25:57.971 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:25:58.012 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:25:58.072 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:25:58.154 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:25:58.199 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T03:25:58.200 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:25:58.266 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:25:58.310 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:25:58.361 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:25:58.445 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:25:58.488 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T03:25:58.488 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:25:58.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-09-21T03:25:58.558 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-09-21T03:25:58.561 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-09-21T03:25:58.594 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:25:58.638 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:25:58.840 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:25:59.136 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T03:25:59.143 INFO:teuthology.task.internal:Starting timer... 2024-09-21T03:25:59.143 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T03:25:59.237 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T03:25:59.407 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T03:25:59.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-09-21T03:25:59.503 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-09-21T03:25:59.506 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-09-21T03:25:59.558 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-09-21T03:25:59.560 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2024-09-21T03:25:59.609 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2024-09-21T03:25:59.611 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T03:25:59.611 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-09-21T03:25:59.655 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-09-21T03:25:59.655 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-09-21T03:25:59.675 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-09-21T03:25:59.675 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2024-09-21T03:25:59.695 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2024-09-21T03:25:59.695 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive'} 2024-09-21T03:25:59.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:25:59.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:25:59.752 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:25:59.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:25:59.780 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:25:59.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:25:59.808 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T03:25:59.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:25:59.868 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:25:59.903 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:25:59.939 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T03:25:59.948 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T03:25:59.950 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T03:26:00.056 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T03:26:00.065 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T03:26:00.066 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi123.front.sepia.ceph.com,smithi134.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T03:30:38.903 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com')] 2024-09-21T03:30:38.904 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-21T03:30:38.905 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:30:38.988 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-21T03:30:39.092 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-21T03:30:39.092 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-21T03:30:39.092 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:30:39.166 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-09-21T03:30:39.246 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-21T03:30:39.246 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-21T03:30:39.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:30:39.320 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-09-21T03:30:39.398 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-21T03:30:39.398 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T03:30:39.408 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T03:30:39.408 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:30:39.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:30:39.412 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:30:39.412 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:30:39.415 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:30:39.415 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:30:39.444 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:30:39.459 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:30:39.468 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:30:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:30:39.482 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:30:39.483 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-09-21T03:30:39.494 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-09-21T03:30:39.499 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:30:39.506 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2024-09-21T03:30:39.509 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:30:39.518 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2024-09-21T03:30:39.523 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:30:39.532 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:30:39.546 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:30:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-09-21T03:30:39.575 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-09-21T03:30:39.585 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:30:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:30:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-09-21T03:30:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:39.602 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:30:39.604 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:30:39.605 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:30:39.605 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-09-21T03:30:39.605 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:39.605 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:39.605 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:39.605 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:39.634 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:30:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:30:39.741 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:30:39.741 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-21T03:30:39.742 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:39.742 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:39.742 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:39.742 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:30:39.743 INFO:teuthology.run_tasks:Running task install... 2024-09-21T03:30:39.750 DEBUG:teuthology.task.install:project ceph 2024-09-21T03:30:39.750 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T03:30:39.750 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T03:30:39.750 INFO:teuthology.task.install:Using flavor: default 2024-09-21T03:30:39.755 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T03:30:39.755 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T03:30:39.756 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:30:39.756 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:30:39.758 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:30:39.758 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:30:39.759 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:30:39.759 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:30:39.897 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:30:39.897 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:30:39.929 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:30:39.929 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:30:39.930 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:30:39.930 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:30:40.008 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:30:40.008 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:30:40.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:30:40.037 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:30:40.038 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T03:30:40.038 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:30:40.043 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:30:40.043 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T03:30:40.043 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:30:40.064 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:30:40.064 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:30:40.066 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:30:40.066 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:30:40.070 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:30:40.070 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:30:40.131 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:30:40.137 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:30:40.164 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:30:40.206 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:30:40.213 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:30:40.244 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:30:40.270 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-09-21T03:30:40.272 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-09-21T03:30:40.278 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2024-09-21T03:30:40.280 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2024-09-21T03:30:40.299 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-09-21T03:30:40.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-09-21T03:30:40.495 INFO:teuthology.orchestra.run.smithi123.stdout:45 files removed 2024-09-21T03:30:40.506 INFO:teuthology.orchestra.run.smithi134.stdout:45 files removed 2024-09-21T03:30:40.521 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:30:40.533 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:30:40.585 INFO:teuthology.orchestra.run.smithi016.stdout:45 files removed 2024-09-21T03:30:40.618 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:30:41.077 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-21T03:30:41.081 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-21T03:30:41.327 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 286 kB/s | 84 kB 00:00 2024-09-21T03:30:41.356 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 104 kB/s | 24 kB 00:00 2024-09-21T03:30:41.369 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-21T03:30:41.615 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-21T03:30:41.653 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-21T03:30:41.777 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 60 kB/s | 24 kB 00:00 2024-09-21T03:30:42.070 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 7.5 kB/s | 1.9 kB 00:00 2024-09-21T03:30:42.553 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.3 MB 00:00 2024-09-21T03:30:43.132 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.3 MB 00:01 2024-09-21T03:30:43.436 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.3 MB 00:01 2024-09-21T03:30:45.296 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-21T03:30:49.863 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 6.5 MB 00:00 2024-09-21T03:30:51.522 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 19 kB 00:00 2024-09-21T03:30:52.074 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-21T03:30:52.401 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-21T03:30:56.558 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 9.0 MB/s | 6.5 MB 00:00 2024-09-21T03:30:57.857 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-09-21T03:30:58.250 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-21T03:30:59.238 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-21T03:31:00.010 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:31:00.011 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:31:00.025 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:31:00.026 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:31:00.027 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:31:00.028 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:31:00.030 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:31:00.033 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:31:00.034 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:31:00.099 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-21T03:31:00.117 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-09-21T03:31:00.117 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-21T03:31:00.117 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-09-21T03:31:00.117 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-21T03:31:00.117 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:31:00.117 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:31:00.117 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:31:00.118 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:31:00.118 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:31:00.118 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:31:00.118 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:31:00.118 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:31:00.118 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:31:00.118 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:31:00.118 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:31:00.118 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:31:00.118 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:31:00.118 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:31:00.118 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:31:00.118 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:31:00.119 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:31:00.119 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:31:00.119 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:31:00.119 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:31:00.119 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:31:00.119 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:31:00.119 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:31:00.119 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:31:00.119 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:31:00.119 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:31:00.119 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:31:00.119 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:31:00.119 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:31:00.119 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:31:00.119 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:31:00.120 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:31:00.120 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:31:00.120 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:31:00.120 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:31:00.120 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:31:00.120 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:31:00.120 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-09-21T03:31:00.120 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:31:00.120 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:31:00.120 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:31:00.120 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:31:00.120 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:31:00.120 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:31:00.120 INFO:teuthology.orchestra.run.smithi123.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:31:00.120 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:31:00.121 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-21T03:31:00.121 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:31:00.121 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:31:00.121 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:31:00.121 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:31:00.121 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:31:00.121 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:31:00.121 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:31:00.121 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:31:00.121 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:31:00.121 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:31:00.121 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:31:00.121 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:31:00.122 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:31:00.122 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:31:00.122 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:31:00.122 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:31:00.122 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:31:00.122 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:31:00.122 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:31:00.122 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:31:00.122 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:31:00.122 INFO:teuthology.orchestra.run.smithi123.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:31:00.122 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:31:00.122 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:31:00.122 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:31:00.123 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:31:00.123 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:31:00.123 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:31:00.123 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:31:00.123 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:31:00.123 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:31:00.123 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:31:00.123 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:31:00.123 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:31:00.123 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:31:00.123 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:31:00.123 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:31:00.123 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:31:00.123 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:31:00.124 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:31:00.124 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:31:00.124 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:31:00.124 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:31:00.124 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:31:00.124 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:31:00.124 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:31:00.124 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:31:00.124 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:31:00.124 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:31:00.124 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:31:00.124 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:31:00.124 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:31:00.124 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:31:00.125 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:31:00.125 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:31:00.125 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:31:00.125 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:31:00.125 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:31:00.125 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:31:00.125 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:31:00.125 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:31:00.125 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:31:00.125 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:31:00.125 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:31:00.125 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:31:00.125 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:31:00.125 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:31:00.126 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:31:00.126 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:31:00.126 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:31:00.126 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:31:00.126 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:31:00.126 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:31:00.126 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:31:00.126 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:31:00.126 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:31:00.126 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:31:00.126 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:31:00.126 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:31:00.126 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:31:00.126 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:31:00.126 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:31:00.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:31:00.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:31:00.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:31:00.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:31:00.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:31:00.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:31:00.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:31:00.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:31:00.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:31:00.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:31:00.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:31:00.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:31:00.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:31:00.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:31:00.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:31:00.128 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:31:00.128 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:31:00.128 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:31:00.128 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:31:00.128 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:31:00.128 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:31:00.128 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:31:00.128 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:31:00.128 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:31:00.128 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:31:00.128 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:31:00.128 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:31:00.128 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:31:00.128 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:31:00.128 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-09-21T03:31:00.129 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:31:00.129 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:31:00.129 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-21T03:31:00.129 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-09-21T03:31:00.129 INFO:teuthology.orchestra.run.smithi123.stdout:Install 149 Packages 2024-09-21T03:31:00.129 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 8 Packages 2024-09-21T03:31:00.129 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:31:00.129 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 228 M 2024-09-21T03:31:00.129 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-21T03:31:00.650 INFO:teuthology.orchestra.run.smithi123.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 57 kB/s | 6.6 kB 00:00 2024-09-21T03:31:00.825 INFO:teuthology.orchestra.run.smithi123.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 4.9 MB/s | 873 kB 00:00 2024-09-21T03:31:00.925 INFO:teuthology.orchestra.run.smithi123.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.5 MB/s | 148 kB 00:00 2024-09-21T03:31:01.001 INFO:teuthology.orchestra.run.smithi123.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 12 MB/s | 5.5 MB 00:00 2024-09-21T03:31:01.101 INFO:teuthology.orchestra.run.smithi123.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 15 MB/s | 1.5 MB 00:00 2024-09-21T03:31:01.160 INFO:teuthology.orchestra.run.smithi123.stdout:(6/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 9.8 MB/s | 2.3 MB 00:00 2024-09-21T03:31:01.644 INFO:teuthology.orchestra.run.smithi123.stdout:(7/157): ceph-common-19.3.0-5120.g18432218.el9. 18 MB/s | 20 MB 00:01 2024-09-21T03:31:01.727 INFO:teuthology.orchestra.run.smithi123.stdout:(8/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 7.9 MB/s | 5.0 MB 00:00 2024-09-21T03:31:01.769 INFO:teuthology.orchestra.run.smithi123.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 604 kB/s | 25 kB 00:00 2024-09-21T03:31:02.186 INFO:teuthology.orchestra.run.smithi123.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 19 MB/s | 10 MB 00:00 2024-09-21T03:31:02.361 INFO:teuthology.orchestra.run.smithi123.stdout:(11/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 15 MB/s | 18 MB 00:01 2024-09-21T03:31:02.386 INFO:teuthology.orchestra.run.smithi123.stdout:(12/157): cephfs-mirror-19.3.0-5120.g18432218.e 1.1 MB/s | 226 kB 00:00 2024-09-21T03:31:02.411 INFO:teuthology.orchestra.run.smithi123.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 541 kB/s | 27 kB 00:00 2024-09-21T03:31:02.437 INFO:teuthology.orchestra.run.smithi123.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 633 kB/s | 32 kB 00:00 2024-09-21T03:31:02.495 INFO:teuthology.orchestra.run.smithi123.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 9.0 MB/s | 766 kB 00:00 2024-09-21T03:31:02.521 INFO:teuthology.orchestra.run.smithi123.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 1.9 MB/s | 158 kB 00:00 2024-09-21T03:31:02.571 INFO:teuthology.orchestra.run.smithi123.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T03:31:02.596 INFO:teuthology.orchestra.run.smithi123.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 6.6 MB/s | 507 kB 00:00 2024-09-21T03:31:02.663 INFO:teuthology.orchestra.run.smithi123.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 678 kB/s | 45 kB 00:00 2024-09-21T03:31:02.738 INFO:teuthology.orchestra.run.smithi123.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.9 MB/s | 148 kB 00:00 2024-09-21T03:31:02.847 INFO:teuthology.orchestra.run.smithi123.stdout:(21/157): librgw2-19.3.0-5120.g18432218.el9.x86 19 MB/s | 5.3 MB 00:00 2024-09-21T03:31:02.872 INFO:teuthology.orchestra.run.smithi123.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 1.2 MB/s | 161 kB 00:00 2024-09-21T03:31:02.904 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 20 MB 00:19 2024-09-21T03:31:02.906 INFO:teuthology.orchestra.run.smithi123.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 5.4 MB/s | 321 kB 00:00 2024-09-21T03:31:02.939 INFO:teuthology.orchestra.run.smithi123.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 4.5 MB/s | 307 kB 00:00 2024-09-21T03:31:02.965 INFO:teuthology.orchestra.run.smithi123.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.7 MB/s | 99 kB 00:00 2024-09-21T03:31:02.998 INFO:teuthology.orchestra.run.smithi123.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.4 MB/s | 86 kB 00:00 2024-09-21T03:31:03.082 INFO:teuthology.orchestra.run.smithi123.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.0 MB/s | 170 kB 00:00 2024-09-21T03:31:03.174 INFO:teuthology.orchestra.run.smithi123.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 15 MB/s | 3.1 MB 00:00 2024-09-21T03:31:03.199 INFO:teuthology.orchestra.run.smithi123.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 293 kB/s | 34 kB 00:00 2024-09-21T03:31:03.224 INFO:teuthology.orchestra.run.smithi123.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T03:31:03.841 INFO:teuthology.orchestra.run.smithi123.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 12 MB/s | 7.4 MB 00:00 2024-09-21T03:31:03.933 INFO:teuthology.orchestra.run.smithi123.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 3.1 MB/s | 292 kB 00:00 2024-09-21T03:31:04.009 INFO:teuthology.orchestra.run.smithi123.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 660 kB/s | 49 kB 00:00 2024-09-21T03:31:04.184 INFO:teuthology.orchestra.run.smithi123.stdout:(34/157): ceph-mgr-dashboard-19.3.0-5120.g18432 14 MB/s | 14 MB 00:00 2024-09-21T03:31:04.209 INFO:teuthology.orchestra.run.smithi123.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 82 kB/s | 16 kB 00:00 2024-09-21T03:31:04.234 INFO:teuthology.orchestra.run.smithi123.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.6 MB/s | 285 kB 00:00 2024-09-21T03:31:04.685 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-21T03:31:04.727 INFO:teuthology.orchestra.run.smithi123.stdout:(37/157): ceph-test-19.3.0-5120.g18432218.el9.x 13 MB/s | 40 MB 00:02 2024-09-21T03:31:04.752 INFO:teuthology.orchestra.run.smithi123.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 36 kB/s | 18 kB 00:00 2024-09-21T03:31:04.777 INFO:teuthology.orchestra.run.smithi123.stdout:(39/157): cephadm-19.3.0-5120.g18432218.el9.noa 1.3 MB/s | 784 kB 00:00 2024-09-21T03:31:04.894 INFO:teuthology.orchestra.run.smithi123.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 479 kB/s | 80 kB 00:00 2024-09-21T03:31:04.919 INFO:teuthology.orchestra.run.smithi123.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-21T03:31:04.953 INFO:teuthology.orchestra.run.smithi123.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-21T03:31:04.995 INFO:teuthology.orchestra.run.smithi123.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-21T03:31:05.028 INFO:teuthology.orchestra.run.smithi123.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-21T03:31:05.079 INFO:teuthology.orchestra.run.smithi123.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 7.7 MB/s | 1.2 MB 00:00 2024-09-21T03:31:05.113 INFO:teuthology.orchestra.run.smithi123.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-21T03:31:05.145 INFO:teuthology.orchestra.run.smithi123.stdout:(47/157): python3-pycparser-2.20-6.el9.noarch.r 900 kB/s | 135 kB 00:00 2024-09-21T03:31:05.179 INFO:teuthology.orchestra.run.smithi123.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-09-21T03:31:05.304 INFO:teuthology.orchestra.run.smithi123.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 855 kB/s | 107 kB 00:00 2024-09-21T03:31:05.338 INFO:teuthology.orchestra.run.smithi123.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-09-21T03:31:05.355 INFO:teuthology.orchestra.run.smithi123.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-21T03:31:05.397 INFO:teuthology.orchestra.run.smithi123.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 3.7 MB/s | 943 kB 00:00 2024-09-21T03:31:05.422 INFO:teuthology.orchestra.run.smithi123.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-09-21T03:31:05.456 INFO:teuthology.orchestra.run.smithi123.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-21T03:31:05.481 INFO:teuthology.orchestra.run.smithi123.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-09-21T03:31:05.506 INFO:teuthology.orchestra.run.smithi123.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-21T03:31:05.540 INFO:teuthology.orchestra.run.smithi123.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.8 MB/s | 171 kB 00:00 2024-09-21T03:31:05.607 INFO:teuthology.orchestra.run.smithi123.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-09-21T03:31:05.632 INFO:teuthology.orchestra.run.smithi123.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 492 kB/s | 45 kB 00:00 2024-09-21T03:31:05.657 INFO:teuthology.orchestra.run.smithi123.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-21T03:31:05.691 INFO:teuthology.orchestra.run.smithi123.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 9.7 MB/s | 581 kB 00:00 2024-09-21T03:31:05.716 INFO:teuthology.orchestra.run.smithi123.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 419 kB/s | 24 kB 00:00 2024-09-21T03:31:05.750 INFO:teuthology.orchestra.run.smithi123.stdout:(63/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-21T03:31:05.775 INFO:teuthology.orchestra.run.smithi123.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-09-21T03:31:05.800 INFO:teuthology.orchestra.run.smithi123.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-21T03:31:05.825 INFO:teuthology.orchestra.run.smithi123.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-21T03:31:05.851 INFO:teuthology.orchestra.run.smithi123.stdout:(67/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-21T03:31:05.926 INFO:teuthology.orchestra.run.smithi123.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-09-21T03:31:05.993 INFO:teuthology.orchestra.run.smithi123.stdout:(69/157): openblas-openmp-0.3.26-2.el9.x86_64.r 26 MB/s | 4.9 MB 00:00 2024-09-21T03:31:06.044 INFO:teuthology.orchestra.run.smithi123.stdout:(70/157): postgresql-13.16-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-09-21T03:31:06.069 INFO:teuthology.orchestra.run.smithi123.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.8 MB/s | 137 kB 00:00 2024-09-21T03:31:06.094 INFO:teuthology.orchestra.run.smithi123.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 5.3 MB/s | 910 kB 00:00 2024-09-21T03:31:06.136 INFO:teuthology.orchestra.run.smithi123.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-21T03:31:06.170 INFO:teuthology.orchestra.run.smithi123.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-21T03:31:06.212 INFO:teuthology.orchestra.run.smithi123.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-21T03:31:06.295 INFO:teuthology.orchestra.run.smithi123.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-09-21T03:31:06.320 INFO:teuthology.orchestra.run.smithi123.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-21T03:31:06.354 INFO:teuthology.orchestra.run.smithi123.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-09-21T03:31:06.379 INFO:teuthology.orchestra.run.smithi123.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-09-21T03:31:06.404 INFO:teuthology.orchestra.run.smithi123.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-21T03:31:06.430 INFO:teuthology.orchestra.run.smithi123.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-21T03:31:06.455 INFO:teuthology.orchestra.run.smithi123.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-21T03:31:06.531 INFO:teuthology.orchestra.run.smithi123.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-21T03:31:06.556 INFO:teuthology.orchestra.run.smithi123.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-21T03:31:06.581 INFO:teuthology.orchestra.run.smithi123.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-21T03:31:06.606 INFO:teuthology.orchestra.run.smithi123.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T03:31:06.623 INFO:teuthology.orchestra.run.smithi123.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-21T03:31:06.648 INFO:teuthology.orchestra.run.smithi123.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-09-21T03:31:06.674 INFO:teuthology.orchestra.run.smithi123.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T03:31:06.699 INFO:teuthology.orchestra.run.smithi123.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-21T03:31:06.716 INFO:teuthology.orchestra.run.smithi123.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-21T03:31:06.733 INFO:teuthology.orchestra.run.smithi123.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-21T03:31:06.816 INFO:teuthology.orchestra.run.smithi123.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-21T03:31:06.841 INFO:teuthology.orchestra.run.smithi123.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-09-21T03:31:06.867 INFO:teuthology.orchestra.run.smithi123.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-09-21T03:31:06.880 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:31:06.881 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:31:06.895 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:31:06.896 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:31:06.897 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:31:06.898 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:31:06.900 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:31:06.900 INFO:teuthology.orchestra.run.smithi123.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-21T03:31:06.903 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:31:06.904 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:31:06.926 INFO:teuthology.orchestra.run.smithi123.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-09-21T03:31:06.951 INFO:teuthology.orchestra.run.smithi123.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-21T03:31:06.968 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-21T03:31:06.976 INFO:teuthology.orchestra.run.smithi123.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T03:31:06.982 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-09-21T03:31:06.982 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-09-21T03:31:06.982 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-09-21T03:31:06.982 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-21T03:31:06.982 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:31:06.983 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:31:06.983 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:31:06.983 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:31:06.983 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:31:06.983 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:31:06.983 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:31:06.983 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:31:06.983 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:31:06.983 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:31:06.983 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:31:06.983 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:31:06.983 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:31:06.983 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:31:06.984 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:31:06.984 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:31:06.984 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:31:06.984 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:31:06.984 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:31:06.984 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:31:06.984 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:31:06.984 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:31:06.984 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:31:06.984 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:31:06.984 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:31:06.984 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:31:06.984 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:31:06.985 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:31:06.985 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:31:06.985 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:31:06.985 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:31:06.985 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:31:06.985 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:31:06.985 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:31:06.985 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:31:06.985 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:31:06.985 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:31:06.985 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-09-21T03:31:06.985 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:31:06.985 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:31:06.986 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:31:06.986 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:31:06.986 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:31:06.986 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:31:06.986 INFO:teuthology.orchestra.run.smithi134.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:31:06.986 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:31:06.986 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-21T03:31:06.986 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:31:06.986 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:31:06.986 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:31:06.986 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:31:06.986 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:31:06.986 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:31:06.986 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:31:06.987 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:31:06.987 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:31:06.987 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:31:06.987 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:31:06.987 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:31:06.987 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:31:06.987 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:31:06.987 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:31:06.987 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:31:06.987 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:31:06.987 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:31:06.987 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:31:06.987 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:31:06.988 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:31:06.988 INFO:teuthology.orchestra.run.smithi134.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:31:06.988 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:31:06.988 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:31:06.988 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:31:06.988 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:31:06.988 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:31:06.988 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:31:06.988 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:31:06.988 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:31:06.988 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:31:06.988 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:31:06.989 INFO:teuthology.orchestra.run.smithi134.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:31:06.989 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:31:06.989 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:31:06.989 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:31:06.989 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:31:06.989 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:31:06.989 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:31:06.989 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:31:06.989 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:31:06.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:31:06.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:31:06.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:31:06.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:31:06.989 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:31:06.990 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:31:06.990 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:31:06.990 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:31:06.990 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:31:06.990 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:31:06.990 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:31:06.990 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:31:06.990 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:31:06.990 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:31:06.990 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:31:06.990 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:31:06.990 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:31:06.990 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:31:06.990 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:31:06.991 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:31:06.991 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:31:06.991 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:31:06.991 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:31:06.991 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:31:06.991 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:31:06.991 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:31:06.991 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:31:06.991 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:31:06.991 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:31:06.991 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:31:06.991 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:31:06.991 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:31:06.992 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:31:06.992 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:31:06.992 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:31:06.992 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:31:06.992 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:31:06.992 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:31:06.992 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:31:06.992 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:31:06.992 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:31:06.992 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:31:06.992 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:31:06.992 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:31:06.992 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:31:06.993 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:31:06.993 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:31:06.993 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:31:06.993 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:31:06.993 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:31:06.993 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:31:06.993 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:31:06.993 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:31:06.993 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:31:06.993 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:31:06.993 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:31:06.993 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:31:06.993 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:31:06.993 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:31:06.994 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:31:06.994 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:31:06.994 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:31:06.994 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:31:06.994 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:31:06.994 INFO:teuthology.orchestra.run.smithi134.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:31:06.994 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:31:06.994 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:31:06.994 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:31:06.994 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:31:06.994 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:31:06.994 INFO:teuthology.orchestra.run.smithi134.stdout:Installing weak dependencies: 2024-09-21T03:31:06.994 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:31:06.994 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:31:06.995 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-21T03:31:06.995 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-09-21T03:31:06.995 INFO:teuthology.orchestra.run.smithi134.stdout:Install 149 Packages 2024-09-21T03:31:06.995 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 8 Packages 2024-09-21T03:31:06.995 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:31:06.995 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 228 M 2024-09-21T03:31:06.995 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-21T03:31:07.001 INFO:teuthology.orchestra.run.smithi123.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T03:31:07.027 INFO:teuthology.orchestra.run.smithi123.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T03:31:07.052 INFO:teuthology.orchestra.run.smithi123.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T03:31:07.077 INFO:teuthology.orchestra.run.smithi123.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-21T03:31:07.103 INFO:teuthology.orchestra.run.smithi123.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 442 kB/s | 22 kB 00:00 2024-09-21T03:31:07.145 INFO:teuthology.orchestra.run.smithi123.stdout:(105/157): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-09-21T03:31:07.195 INFO:teuthology.orchestra.run.smithi123.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 5.8 MB/s | 548 kB 00:00 2024-09-21T03:31:07.220 INFO:teuthology.orchestra.run.smithi123.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 840 kB/s | 19 kB 00:00 2024-09-21T03:31:07.254 INFO:teuthology.orchestra.run.smithi123.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 2.8 MB/s | 308 kB 00:00 2024-09-21T03:31:07.279 INFO:teuthology.orchestra.run.smithi123.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T03:31:07.305 INFO:teuthology.orchestra.run.smithi123.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T03:31:07.338 INFO:teuthology.orchestra.run.smithi123.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-21T03:31:07.372 INFO:teuthology.orchestra.run.smithi123.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-21T03:31:07.430 INFO:teuthology.orchestra.run.smithi123.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-21T03:31:07.581 INFO:teuthology.orchestra.run.smithi123.stdout:(114/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-09-21T03:31:07.656 INFO:teuthology.orchestra.run.smithi123.stdout:(115/157): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-21T03:31:07.690 INFO:teuthology.orchestra.run.smithi123.stdout:(116/157): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-09-21T03:31:07.715 INFO:teuthology.orchestra.run.smithi123.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 1.4 MB/s | 32 kB 00:00 2024-09-21T03:31:07.722 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-21T03:31:07.740 INFO:teuthology.orchestra.run.smithi123.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 294 kB/s | 14 kB 00:00 2024-09-21T03:31:07.765 INFO:teuthology.orchestra.run.smithi123.stdout:(119/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 236 kB/s | 43 kB 00:00 2024-09-21T03:31:07.791 INFO:teuthology.orchestra.run.smithi123.stdout:(120/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-21T03:31:07.825 INFO:teuthology.orchestra.run.smithi123.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-21T03:31:07.858 INFO:teuthology.orchestra.run.smithi123.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-09-21T03:31:07.878 INFO:teuthology.orchestra.run.smithi134.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 50 kB/s | 6.6 kB 00:00 2024-09-21T03:31:07.883 INFO:teuthology.orchestra.run.smithi123.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 2.3 MB/s | 220 kB 00:00 2024-09-21T03:31:07.908 INFO:teuthology.orchestra.run.smithi123.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-21T03:31:07.933 INFO:teuthology.orchestra.run.smithi123.stdout:(125/157): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-21T03:31:07.959 INFO:teuthology.orchestra.run.smithi123.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-21T03:31:07.984 INFO:teuthology.orchestra.run.smithi123.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-21T03:31:08.009 INFO:teuthology.orchestra.run.smithi123.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-21T03:31:08.026 INFO:teuthology.orchestra.run.smithi123.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 471 kB/s | 20 kB 00:00 2024-09-21T03:31:08.043 INFO:teuthology.orchestra.run.smithi123.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-21T03:31:08.068 INFO:teuthology.orchestra.run.smithi123.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T03:31:08.078 INFO:teuthology.orchestra.run.smithi134.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 4.3 MB/s | 873 kB 00:00 2024-09-21T03:31:08.085 INFO:teuthology.orchestra.run.smithi123.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-21T03:31:08.119 INFO:teuthology.orchestra.run.smithi123.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-21T03:31:08.136 INFO:teuthology.orchestra.run.smithi123.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 996 kB/s | 16 kB 00:00 2024-09-21T03:31:08.137 INFO:teuthology.orchestra.run.smithi134.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.5 MB/s | 148 kB 00:00 2024-09-21T03:31:08.178 INFO:teuthology.orchestra.run.smithi123.stdout:(135/157): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-21T03:31:08.203 INFO:teuthology.orchestra.run.smithi123.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T03:31:08.237 INFO:teuthology.orchestra.run.smithi123.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 892 kB/s | 90 kB 00:00 2024-09-21T03:31:08.262 INFO:teuthology.orchestra.run.smithi123.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-09-21T03:31:08.287 INFO:teuthology.orchestra.run.smithi123.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T03:31:08.313 INFO:teuthology.orchestra.run.smithi123.stdout:(140/157): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-21T03:31:08.329 INFO:teuthology.orchestra.run.smithi134.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 9.5 MB/s | 5.5 MB 00:00 2024-09-21T03:31:08.338 INFO:teuthology.orchestra.run.smithi123.stdout:(141/157): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-21T03:31:08.363 INFO:teuthology.orchestra.run.smithi123.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T03:31:08.379 INFO:teuthology.orchestra.run.smithi134.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 9.4 MB/s | 2.3 MB 00:00 2024-09-21T03:31:08.397 INFO:teuthology.orchestra.run.smithi123.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 2.6 MB/s | 90 kB 00:00 2024-09-21T03:31:08.422 INFO:teuthology.orchestra.run.smithi123.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-21T03:31:08.472 INFO:teuthology.orchestra.run.smithi123.stdout:(145/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-09-21T03:31:08.487 INFO:teuthology.orchestra.run.smithi134.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 9.6 MB/s | 1.5 MB 00:00 2024-09-21T03:31:08.498 INFO:teuthology.orchestra.run.smithi123.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-09-21T03:31:08.523 INFO:teuthology.orchestra.run.smithi123.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T03:31:08.557 INFO:teuthology.orchestra.run.smithi123.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-09-21T03:31:08.607 INFO:teuthology.orchestra.run.smithi123.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T03:31:08.732 INFO:teuthology.orchestra.run.smithi123.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 19 MB/s | 2.3 MB 00:00 2024-09-21T03:31:08.804 INFO:teuthology.orchestra.run.smithi134.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 12 MB/s | 5.0 MB 00:00 2024-09-21T03:31:08.857 INFO:teuthology.orchestra.run.smithi123.stdout:(151/157): NetworkManager-libnm-1.51.0-1.el9.x8 14 MB/s | 1.8 MB 00:00 2024-09-21T03:31:08.924 INFO:teuthology.orchestra.run.smithi123.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 614 kB/s | 41 kB 00:00 2024-09-21T03:31:09.000 INFO:teuthology.orchestra.run.smithi123.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 3.2 MB/s | 249 kB 00:00 2024-09-21T03:31:09.108 INFO:teuthology.orchestra.run.smithi123.stdout:(154/157): librados2-19.3.0-5120.g18432218.el9. 5.8 MB/s | 3.4 MB 00:00 2024-09-21T03:31:09.150 INFO:teuthology.orchestra.run.smithi123.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 9.0 MB/s | 1.4 MB 00:00 2024-09-21T03:31:09.226 INFO:teuthology.orchestra.run.smithi123.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 20 MB/s | 2.4 MB 00:00 2024-09-21T03:31:09.238 INFO:teuthology.orchestra.run.smithi134.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 13 MB/s | 20 MB 00:01 2024-09-21T03:31:09.276 INFO:teuthology.orchestra.run.smithi123.stdout:(157/157): librbd1-19.3.0-5120.g18432218.el9.x8 4.4 MB/s | 3.2 MB 00:00 2024-09-21T03:31:09.282 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:31:09.282 INFO:teuthology.orchestra.run.smithi123.stdout:Total 25 MB/s | 228 MB 00:09 2024-09-21T03:31:09.296 INFO:teuthology.orchestra.run.smithi134.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 433 kB/s | 25 kB 00:00 2024-09-21T03:31:09.542 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-21T03:31:09.672 INFO:teuthology.orchestra.run.smithi134.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 15 MB/s | 18 MB 00:01 2024-09-21T03:31:09.789 INFO:teuthology.orchestra.run.smithi134.stdout:(11/157): ceph-radosgw-19.3.0-5120.g18432218.el 11 MB/s | 10 MB 00:00 2024-09-21T03:31:09.822 INFO:teuthology.orchestra.run.smithi134.stdout:(12/157): cephfs-mirror-19.3.0-5120.g18432218.e 1.5 MB/s | 226 kB 00:00 2024-09-21T03:31:09.848 INFO:teuthology.orchestra.run.smithi134.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 462 kB/s | 27 kB 00:00 2024-09-21T03:31:09.873 INFO:teuthology.orchestra.run.smithi134.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 634 kB/s | 32 kB 00:00 2024-09-21T03:31:09.931 INFO:teuthology.orchestra.run.smithi134.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 9.0 MB/s | 766 kB 00:00 2024-09-21T03:31:09.957 INFO:teuthology.orchestra.run.smithi134.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 1.8 MB/s | 158 kB 00:00 2024-09-21T03:31:09.982 INFO:teuthology.orchestra.run.smithi134.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T03:31:10.057 INFO:teuthology.orchestra.run.smithi134.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 5.0 MB/s | 507 kB 00:00 2024-09-21T03:31:10.124 INFO:teuthology.orchestra.run.smithi134.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 674 kB/s | 45 kB 00:00 2024-09-21T03:31:10.183 INFO:teuthology.orchestra.run.smithi134.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.5 MB/s | 148 kB 00:00 2024-09-21T03:31:10.241 INFO:teuthology.orchestra.run.smithi134.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.7 MB/s | 161 kB 00:00 2024-09-21T03:31:10.309 INFO:teuthology.orchestra.run.smithi134.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 4.7 MB/s | 321 kB 00:00 2024-09-21T03:31:10.384 INFO:teuthology.orchestra.run.smithi134.stdout:(23/157): librgw2-19.3.0-5120.g18432218.el9.x86 13 MB/s | 5.3 MB 00:00 2024-09-21T03:31:10.417 INFO:teuthology.orchestra.run.smithi134.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 2.8 MB/s | 307 kB 00:00 2024-09-21T03:31:10.443 INFO:teuthology.orchestra.run.smithi134.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.7 MB/s | 99 kB 00:00 2024-09-21T03:31:10.468 INFO:teuthology.orchestra.run.smithi134.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T03:31:10.477 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-21T03:31:10.518 INFO:teuthology.orchestra.run.smithi134.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T03:31:10.577 INFO:teuthology.orchestra.run.smithi134.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 590 kB/s | 34 kB 00:00 2024-09-21T03:31:10.629 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-21T03:31:10.629 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-21T03:31:10.660 INFO:teuthology.orchestra.run.smithi134.stdout:(29/157): rbd-mirror-19.3.0-5120.g18432218.el9. 14 MB/s | 3.1 MB 00:00 2024-09-21T03:31:10.684 INFO:teuthology.orchestra.run.smithi134.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.4 MB/s | 154 kB 00:00 2024-09-21T03:31:10.863 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-21T03:31:11.127 INFO:teuthology.orchestra.run.smithi134.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 17 MB/s | 7.4 MB 00:00 2024-09-21T03:31:11.194 INFO:teuthology.orchestra.run.smithi134.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.3 MB/s | 292 kB 00:00 2024-09-21T03:31:11.252 INFO:teuthology.orchestra.run.smithi134.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 850 kB/s | 49 kB 00:00 2024-09-21T03:31:11.319 INFO:teuthology.orchestra.run.smithi134.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 247 kB/s | 16 kB 00:00 2024-09-21T03:31:11.403 INFO:teuthology.orchestra.run.smithi134.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 3.3 MB/s | 285 kB 00:00 2024-09-21T03:31:11.495 INFO:teuthology.orchestra.run.smithi134.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 8.4 MB/s | 784 kB 00:00 2024-09-21T03:31:11.662 INFO:teuthology.orchestra.run.smithi134.stdout:(37/157): ceph-mgr-dashboard-19.3.0-5120.g18432 14 MB/s | 14 MB 00:01 2024-09-21T03:31:11.678 INFO:teuthology.orchestra.run.smithi134.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 100 kB/s | 18 kB 00:00 2024-09-21T03:31:12.029 INFO:teuthology.orchestra.run.smithi134.stdout:(39/157): ceph-test-19.3.0-5120.g18432218.el9.x 15 MB/s | 40 MB 00:02 2024-09-21T03:31:12.129 INFO:teuthology.orchestra.run.smithi134.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 171 kB/s | 80 kB 00:00 2024-09-21T03:31:12.154 INFO:teuthology.orchestra.run.smithi134.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-21T03:31:12.204 INFO:teuthology.orchestra.run.smithi134.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-09-21T03:31:12.238 INFO:teuthology.orchestra.run.smithi134.stdout:(43/157): libgfortran-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 801 kB 00:00 2024-09-21T03:31:12.263 INFO:teuthology.orchestra.run.smithi134.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-21T03:31:12.289 INFO:teuthology.orchestra.run.smithi134.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-21T03:31:12.314 INFO:teuthology.orchestra.run.smithi134.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T03:31:12.339 INFO:teuthology.orchestra.run.smithi134.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-21T03:31:12.381 INFO:teuthology.orchestra.run.smithi134.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 5.4 MB/s | 1.2 MB 00:00 2024-09-21T03:31:12.445 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-21T03:31:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-21T03:31:12.556 INFO:teuthology.orchestra.run.smithi134.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 611 kB/s | 107 kB 00:00 2024-09-21T03:31:12.607 INFO:teuthology.orchestra.run.smithi134.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-21T03:31:12.649 INFO:teuthology.orchestra.run.smithi134.stdout:(51/157): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 689 kB 00:00 2024-09-21T03:31:12.682 INFO:teuthology.orchestra.run.smithi134.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-09-21T03:31:12.708 INFO:teuthology.orchestra.run.smithi134.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-09-21T03:31:12.733 INFO:teuthology.orchestra.run.smithi134.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-09-21T03:31:12.758 INFO:teuthology.orchestra.run.smithi134.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-21T03:31:12.784 INFO:teuthology.orchestra.run.smithi134.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-21T03:31:12.817 INFO:teuthology.orchestra.run.smithi134.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.8 MB/s | 171 kB 00:00 2024-09-21T03:31:12.842 INFO:teuthology.orchestra.run.smithi134.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-21T03:31:12.867 INFO:teuthology.orchestra.run.smithi134.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-09-21T03:31:12.960 INFO:teuthology.orchestra.run.smithi134.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-21T03:31:12.985 INFO:teuthology.orchestra.run.smithi134.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 4.9 MB/s | 581 kB 00:00 2024-09-21T03:31:13.010 INFO:teuthology.orchestra.run.smithi134.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 488 kB/s | 24 kB 00:00 2024-09-21T03:31:13.035 INFO:teuthology.orchestra.run.smithi134.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-09-21T03:31:13.077 INFO:teuthology.orchestra.run.smithi134.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-09-21T03:31:13.111 INFO:teuthology.orchestra.run.smithi134.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-21T03:31:13.136 INFO:teuthology.orchestra.run.smithi134.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 659 kB/s | 38 kB 00:00 2024-09-21T03:31:13.878 INFO:teuthology.orchestra.run.smithi134.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.5 MB/s | 3.0 MB 00:01 2024-09-21T03:31:14.653 INFO:teuthology.orchestra.run.smithi134.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-09-21T03:31:14.954 INFO:teuthology.orchestra.run.smithi134.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.4 MB/s | 4.4 MB 00:01 2024-09-21T03:31:15.062 INFO:teuthology.orchestra.run.smithi134.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-21T03:31:15.237 INFO:teuthology.orchestra.run.smithi134.stdout:(71/157): openblas-openmp-0.3.26-2.el9.x86_64.r 2.3 MB/s | 4.9 MB 00:02 2024-09-21T03:31:15.262 INFO:teuthology.orchestra.run.smithi134.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 1.5 MB/s | 910 kB 00:00 2024-09-21T03:31:15.346 INFO:teuthology.orchestra.run.smithi134.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-21T03:31:15.779 INFO:teuthology.orchestra.run.smithi134.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 1.9 MB/s | 1.0 MB 00:00 2024-09-21T03:31:15.830 INFO:teuthology.orchestra.run.smithi134.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-21T03:31:15.988 INFO:teuthology.orchestra.run.smithi134.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-09-21T03:31:16.139 INFO:teuthology.orchestra.run.smithi134.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-09-21T03:31:16.189 INFO:teuthology.orchestra.run.smithi134.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-09-21T03:31:16.239 INFO:teuthology.orchestra.run.smithi134.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-21T03:31:16.356 INFO:teuthology.orchestra.run.smithi134.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-21T03:31:16.423 INFO:teuthology.orchestra.run.smithi134.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 521 kB/s | 35 kB 00:00 2024-09-21T03:31:17.525 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T03:31:17.648 INFO:teuthology.orchestra.run.smithi134.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 2.2 MB/s | 5.8 MB 00:02 2024-09-21T03:31:18.049 INFO:teuthology.orchestra.run.smithi134.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 2.2 MB/s | 6.0 MB 00:02 2024-09-21T03:31:18.074 INFO:teuthology.orchestra.run.smithi134.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 590 kB/s | 250 kB 00:00 2024-09-21T03:31:18.124 INFO:teuthology.orchestra.run.smithi134.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T03:31:18.174 INFO:teuthology.orchestra.run.smithi134.stdout:(86/157): python3-prettytable-0.7.2-27.el9.noar 848 kB/s | 42 kB 00:00 2024-09-21T03:31:18.208 INFO:teuthology.orchestra.run.smithi134.stdout:(87/157): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-09-21T03:31:18.233 INFO:teuthology.orchestra.run.smithi134.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-21T03:31:18.292 INFO:teuthology.orchestra.run.smithi134.stdout:(89/157): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-21T03:31:18.342 INFO:teuthology.orchestra.run.smithi134.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-21T03:31:18.367 INFO:teuthology.orchestra.run.smithi134.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 678 kB/s | 51 kB 00:00 2024-09-21T03:31:18.392 INFO:teuthology.orchestra.run.smithi134.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T03:31:18.468 INFO:teuthology.orchestra.run.smithi134.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.5 MB/s | 5.1 MB 00:02 2024-09-21T03:31:18.493 INFO:teuthology.orchestra.run.smithi134.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-09-21T03:31:18.527 INFO:teuthology.orchestra.run.smithi134.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 756 kB/s | 44 kB 00:00 2024-09-21T03:31:18.585 INFO:teuthology.orchestra.run.smithi134.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 616 kB/s | 36 kB 00:00 2024-09-21T03:31:18.644 INFO:teuthology.orchestra.run.smithi134.stdout:(97/157): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-09-21T03:31:18.669 INFO:teuthology.orchestra.run.smithi134.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-09-21T03:31:18.704 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-21T03:31:18.719 INFO:teuthology.orchestra.run.smithi134.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T03:31:18.820 INFO:teuthology.orchestra.run.smithi134.stdout:(100/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 961 kB/s | 96 kB 00:00 2024-09-21T03:31:18.829 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:31:18.845 INFO:teuthology.orchestra.run.smithi134.stdout:(101/157): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-09-21T03:31:18.895 INFO:teuthology.orchestra.run.smithi134.stdout:(102/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-09-21T03:31:18.921 INFO:teuthology.orchestra.run.smithi134.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 634 kB/s | 64 kB 00:00 2024-09-21T03:31:18.926 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:31:18.996 INFO:teuthology.orchestra.run.smithi134.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-09-21T03:31:19.046 INFO:teuthology.orchestra.run.smithi134.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-21T03:31:19.118 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:31:19.155 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:31:19.181 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:31:19.205 INFO:teuthology.orchestra.run.smithi134.stdout:(106/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-09-21T03:31:19.224 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:31:19.238 INFO:teuthology.orchestra.run.smithi134.stdout:(107/157): gperftools-libs-2.9.1-3.el9.x86_64.r 1.6 MB/s | 308 kB 00:00 2024-09-21T03:31:19.263 INFO:teuthology.orchestra.run.smithi134.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-09-21T03:31:19.306 INFO:teuthology.orchestra.run.smithi134.stdout:(109/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-09-21T03:31:19.322 INFO:teuthology.orchestra.run.smithi134.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 442 kB/s | 26 kB 00:00 2024-09-21T03:31:19.365 INFO:teuthology.orchestra.run.smithi134.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.9 MB/s | 67 kB 00:00 2024-09-21T03:31:19.390 INFO:teuthology.orchestra.run.smithi134.stdout:(112/157): liboath-2.6.7-2.el9.x86_64.rpm 581 kB/s | 49 kB 00:00 2024-09-21T03:31:19.423 INFO:teuthology.orchestra.run.smithi134.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-21T03:31:19.482 INFO:teuthology.orchestra.run.smithi134.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-21T03:31:19.515 INFO:teuthology.orchestra.run.smithi134.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-09-21T03:31:19.549 INFO:teuthology.orchestra.run.smithi134.stdout:(116/157): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-09-21T03:31:19.562 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:31:19.574 INFO:teuthology.orchestra.run.smithi134.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-09-21T03:31:19.599 INFO:teuthology.orchestra.run.smithi134.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-21T03:31:19.666 INFO:teuthology.orchestra.run.smithi134.stdout:(119/157): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-21T03:31:19.681 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:31:19.692 INFO:teuthology.orchestra.run.smithi134.stdout:(120/157): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-09-21T03:31:19.717 INFO:teuthology.orchestra.run.smithi134.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-21T03:31:19.751 INFO:teuthology.orchestra.run.smithi134.stdout:(122/157): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2024-09-21T03:31:19.776 INFO:teuthology.orchestra.run.smithi134.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-09-21T03:31:19.807 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:31:19.809 INFO:teuthology.orchestra.run.smithi134.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-09-21T03:31:19.835 INFO:teuthology.orchestra.run.smithi134.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-09-21T03:31:19.841 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:31:19.860 INFO:teuthology.orchestra.run.smithi134.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-21T03:31:19.885 INFO:teuthology.orchestra.run.smithi134.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-21T03:31:19.911 INFO:teuthology.orchestra.run.smithi134.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-09-21T03:31:19.923 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:31:19.936 INFO:teuthology.orchestra.run.smithi134.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-21T03:31:19.953 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:31:19.961 INFO:teuthology.orchestra.run.smithi134.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T03:31:19.982 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:31:19.995 INFO:teuthology.orchestra.run.smithi134.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-21T03:31:20.041 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:31:20.042 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:31:20.045 INFO:teuthology.orchestra.run.smithi134.stdout:(132/157): python3-grpcio-1.46.7-10.el9.x86_64. 6.2 MB/s | 2.0 MB 00:00 2024-09-21T03:31:20.057 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:31:20.058 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:31:20.059 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:31:20.060 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:31:20.062 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:31:20.065 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:31:20.066 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:31:20.070 INFO:teuthology.orchestra.run.smithi134.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 768 kB/s | 58 kB 00:00 2024-09-21T03:31:20.104 INFO:teuthology.orchestra.run.smithi134.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 495 kB/s | 16 kB 00:00 2024-09-21T03:31:20.133 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:31:20.139 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-21T03:31:20.145 INFO:teuthology.orchestra.run.smithi134.stdout:(135/157): python3-kubernetes-26.1.0-2.el9.noar 4.3 MB/s | 1.0 MB 00:00 2024-09-21T03:31:20.155 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-09-21T03:31:20.155 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-09-21T03:31:20.155 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-09-21T03:31:20.155 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-09-21T03:31:20.156 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:31:20.156 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:31:20.156 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:31:20.156 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:31:20.156 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:31:20.156 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:31:20.156 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:31:20.156 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:31:20.156 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:31:20.156 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:31:20.156 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:31:20.156 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:31:20.156 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:31:20.156 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:31:20.157 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:31:20.157 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:31:20.157 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:31:20.157 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:31:20.157 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:31:20.157 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:31:20.157 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:31:20.157 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:31:20.157 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:31:20.157 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:31:20.157 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:31:20.157 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:31:20.157 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:31:20.157 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:31:20.158 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:31:20.158 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:31:20.158 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:31:20.158 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:31:20.158 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:31:20.158 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:31:20.158 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:31:20.158 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:31:20.158 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:31:20.158 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-09-21T03:31:20.158 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:31:20.158 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:31:20.158 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:31:20.158 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:31:20.159 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:31:20.159 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:31:20.159 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:31:20.159 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:31:20.159 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-09-21T03:31:20.159 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:31:20.159 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:31:20.159 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:31:20.159 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:31:20.159 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:31:20.159 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:31:20.159 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:31:20.159 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:31:20.159 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:31:20.160 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:31:20.160 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:31:20.160 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:31:20.160 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:31:20.160 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:31:20.160 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:31:20.160 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:31:20.160 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:31:20.160 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:31:20.160 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:31:20.160 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:31:20.160 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:31:20.160 INFO:teuthology.orchestra.run.smithi016.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:31:20.160 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:31:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:31:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:31:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:31:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:31:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:31:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:31:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:31:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:31:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:31:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:31:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:31:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:31:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:31:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:31:20.162 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:31:20.162 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:31:20.162 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:31:20.162 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:31:20.162 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:31:20.162 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:31:20.162 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:31:20.162 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:31:20.162 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:31:20.162 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:31:20.162 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:31:20.162 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:31:20.162 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:31:20.162 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:31:20.163 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:31:20.163 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:31:20.163 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:31:20.163 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:31:20.163 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:31:20.163 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:31:20.163 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:31:20.163 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:31:20.163 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:31:20.163 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:31:20.163 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:31:20.163 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:31:20.163 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:31:20.163 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:31:20.163 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:31:20.164 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:31:20.164 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:31:20.164 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:31:20.164 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:31:20.164 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:31:20.164 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:31:20.164 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:31:20.164 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:31:20.164 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:31:20.164 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:31:20.164 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:31:20.165 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:31:20.165 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:31:20.165 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:31:20.165 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:31:20.165 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:31:20.165 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:31:20.165 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:31:20.165 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:31:20.165 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:31:20.165 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:31:20.165 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:31:20.165 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:31:20.166 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:31:20.166 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:31:20.166 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:31:20.166 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:31:20.166 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:31:20.166 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:31:20.166 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:31:20.166 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:31:20.166 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:31:20.166 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:31:20.166 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:31:20.166 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:31:20.166 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:31:20.167 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:31:20.167 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:31:20.167 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:31:20.167 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:31:20.167 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:31:20.167 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:31:20.167 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:31:20.167 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:31:20.167 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:31:20.167 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-09-21T03:31:20.167 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:31:20.167 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:31:20.167 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-21T03:31:20.168 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-09-21T03:31:20.168 INFO:teuthology.orchestra.run.smithi016.stdout:Install 149 Packages 2024-09-21T03:31:20.168 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 8 Packages 2024-09-21T03:31:20.168 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:31:20.168 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 228 M 2024-09-21T03:31:20.168 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-21T03:31:20.179 INFO:teuthology.orchestra.run.smithi134.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-21T03:31:20.184 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:31:20.204 INFO:teuthology.orchestra.run.smithi134.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 894 kB/s | 90 kB 00:00 2024-09-21T03:31:20.220 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:31:20.230 INFO:teuthology.orchestra.run.smithi134.stdout:(138/157): python3-pyperclip-1.8.2-2.el9.noarch 336 kB/s | 28 kB 00:00 2024-09-21T03:31:20.255 INFO:teuthology.orchestra.run.smithi134.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-09-21T03:31:20.280 INFO:teuthology.orchestra.run.smithi134.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-21T03:31:20.306 INFO:teuthology.orchestra.run.smithi134.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-21T03:31:20.325 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:31:20.331 INFO:teuthology.orchestra.run.smithi134.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T03:31:20.355 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:31:20.356 INFO:teuthology.orchestra.run.smithi134.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-21T03:31:20.381 INFO:teuthology.orchestra.run.smithi134.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T03:31:20.407 INFO:teuthology.orchestra.run.smithi134.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T03:31:20.417 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:31:20.432 INFO:teuthology.orchestra.run.smithi134.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-21T03:31:20.439 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:31:20.457 INFO:teuthology.orchestra.run.smithi134.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T03:31:20.460 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:31:20.482 INFO:teuthology.orchestra.run.smithi134.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-21T03:31:20.566 INFO:teuthology.orchestra.run.smithi134.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-21T03:31:20.640 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:31:20.693 INFO:teuthology.orchestra.run.smithi016.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 49 kB/s | 6.6 kB 00:00 2024-09-21T03:31:20.741 INFO:teuthology.orchestra.run.smithi134.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 12 MB/s | 3.4 MB 00:00 2024-09-21T03:31:20.783 INFO:teuthology.orchestra.run.smithi134.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-09-21T03:31:20.784 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:31:20.817 INFO:teuthology.orchestra.run.smithi134.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-09-21T03:31:20.852 INFO:teuthology.orchestra.run.smithi016.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 5.4 MB/s | 873 kB 00:00 2024-09-21T03:31:20.859 INFO:teuthology.orchestra.run.smithi134.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 5.8 MB/s | 249 kB 00:00 2024-09-21T03:31:20.904 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:31:20.927 INFO:teuthology.orchestra.run.smithi016.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.9 MB/s | 148 kB 00:00 2024-09-21T03:31:20.934 INFO:teuthology.orchestra.run.smithi134.stdout:(154/157): librbd1-19.3.0-5120.g18432218.el9.x8 7.0 MB/s | 3.2 MB 00:00 2024-09-21T03:31:20.976 INFO:teuthology.orchestra.run.smithi134.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 7.7 MB/s | 1.8 MB 00:00 2024-09-21T03:31:20.995 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:31:21.009 INFO:teuthology.orchestra.run.smithi134.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 9.0 MB/s | 1.4 MB 00:00 2024-09-21T03:31:21.063 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:31:21.136 INFO:teuthology.orchestra.run.smithi016.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 9.6 MB/s | 5.5 MB 00:00 2024-09-21T03:31:21.168 INFO:teuthology.orchestra.run.smithi134.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 10 MB/s | 2.4 MB 00:00 2024-09-21T03:31:21.174 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:31:21.174 INFO:teuthology.orchestra.run.smithi134.stdout:Total 16 MB/s | 228 MB 00:14 2024-09-21T03:31:21.186 INFO:teuthology.orchestra.run.smithi016.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 8.8 MB/s | 2.3 MB 00:00 2024-09-21T03:31:21.272 INFO:teuthology.orchestra.run.smithi016.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 11 MB/s | 1.5 MB 00:00 2024-09-21T03:31:21.291 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:31:21.409 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:31:21.443 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:31:21.477 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:31:21.520 INFO:teuthology.orchestra.run.smithi016.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 15 MB/s | 5.0 MB 00:00 2024-09-21T03:31:21.540 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:31:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:31:21.626 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:31:21.670 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:31:21.703 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:31:21.734 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:31:21.779 INFO:teuthology.orchestra.run.smithi016.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 16 MB/s | 20 MB 00:01 2024-09-21T03:31:21.809 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:31:21.821 INFO:teuthology.orchestra.run.smithi016.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 610 kB/s | 25 kB 00:00 2024-09-21T03:31:21.872 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:31:21.911 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:31:21.947 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:31:22.027 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:31:22.058 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:31:22.102 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:31:22.130 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:31:22.206 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:31:22.261 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:31:22.279 INFO:teuthology.orchestra.run.smithi016.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 14 MB/s | 10 MB 00:00 2024-09-21T03:31:22.343 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:31:22.416 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-21T03:31:22.471 INFO:teuthology.orchestra.run.smithi016.stdout:(11/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 15 MB/s | 18 MB 00:01 2024-09-21T03:31:22.474 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:31:22.496 INFO:teuthology.orchestra.run.smithi016.stdout:(12/157): cephfs-mirror-19.3.0-5120.g18432218.e 1.0 MB/s | 226 kB 00:00 2024-09-21T03:31:22.522 INFO:teuthology.orchestra.run.smithi016.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 539 kB/s | 27 kB 00:00 2024-09-21T03:31:22.529 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:31:22.547 INFO:teuthology.orchestra.run.smithi016.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 631 kB/s | 32 kB 00:00 2024-09-21T03:31:22.567 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-21T03:31:22.568 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-21T03:31:22.606 INFO:teuthology.orchestra.run.smithi016.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 9.0 MB/s | 766 kB 00:00 2024-09-21T03:31:22.631 INFO:teuthology.orchestra.run.smithi016.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 1.9 MB/s | 158 kB 00:00 2024-09-21T03:31:22.681 INFO:teuthology.orchestra.run.smithi016.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 1.6 MB/s | 127 kB 00:00 2024-09-21T03:31:22.715 INFO:teuthology.orchestra.run.smithi016.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 5.9 MB/s | 507 kB 00:00 2024-09-21T03:31:22.774 INFO:teuthology.orchestra.run.smithi016.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 776 kB/s | 45 kB 00:00 2024-09-21T03:31:22.857 INFO:teuthology.orchestra.run.smithi016.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.7 MB/s | 148 kB 00:00 2024-09-21T03:31:22.958 INFO:teuthology.orchestra.run.smithi016.stdout:(21/157): librgw2-19.3.0-5120.g18432218.el9.x86 19 MB/s | 5.3 MB 00:00 2024-09-21T03:31:22.983 INFO:teuthology.orchestra.run.smithi016.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 1.3 MB/s | 161 kB 00:00 2024-09-21T03:31:23.016 INFO:teuthology.orchestra.run.smithi016.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 5.4 MB/s | 321 kB 00:00 2024-09-21T03:31:23.041 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:31:23.042 INFO:teuthology.orchestra.run.smithi016.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 5.1 MB/s | 307 kB 00:00 2024-09-21T03:31:23.097 INFO:teuthology.orchestra.run.smithi016.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.2 MB/s | 99 kB 00:00 2024-09-21T03:31:23.126 INFO:teuthology.orchestra.run.smithi016.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.0 MB/s | 86 kB 00:00 2024-09-21T03:31:23.126 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:31:23.155 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:31:23.186 INFO:teuthology.orchestra.run.smithi016.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.8 MB/s | 170 kB 00:00 2024-09-21T03:31:23.197 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:31:23.235 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:31:23.260 INFO:teuthology.orchestra.run.smithi016.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 19 MB/s | 3.1 MB 00:00 2024-09-21T03:31:23.285 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:31:23.286 INFO:teuthology.orchestra.run.smithi016.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 342 kB/s | 34 kB 00:00 2024-09-21T03:31:23.310 INFO:teuthology.orchestra.run.smithi016.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T03:31:23.444 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:31:23.505 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:31:23.603 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:31:23.670 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:31:23.741 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:31:23.769 INFO:teuthology.orchestra.run.smithi016.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 16 MB/s | 7.4 MB 00:00 2024-09-21T03:31:23.781 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:31:23.817 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:31:23.836 INFO:teuthology.orchestra.run.smithi016.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.3 MB/s | 292 kB 00:00 2024-09-21T03:31:23.863 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:31:23.900 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:31:23.951 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:31:24.001 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:31:24.137 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:31:24.179 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:31:24.219 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:31:24.295 INFO:teuthology.orchestra.run.smithi016.stdout:(33/157): ceph-test-19.3.0-5120.g18432218.el9.x 16 MB/s | 40 MB 00:02 2024-09-21T03:31:24.320 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:31:24.321 INFO:teuthology.orchestra.run.smithi016.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 102 kB/s | 49 kB 00:00 2024-09-21T03:31:24.346 INFO:teuthology.orchestra.run.smithi016.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 330 kB/s | 16 kB 00:00 2024-09-21T03:31:24.353 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:31:24.371 INFO:teuthology.orchestra.run.smithi016.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.6 MB/s | 285 kB 00:00 2024-09-21T03:31:24.383 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-21T03:31:24.383 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-21T03:31:24.502 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:31:24.536 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:31:24.555 INFO:teuthology.orchestra.run.smithi016.stdout:(37/157): ceph-mgr-dashboard-19.3.0-5120.g18432 11 MB/s | 14 MB 00:01 2024-09-21T03:31:24.569 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:31:24.580 INFO:teuthology.orchestra.run.smithi016.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 88 kB/s | 18 kB 00:00 2024-09-21T03:31:24.613 INFO:teuthology.orchestra.run.smithi016.stdout:(39/157): cephadm-19.3.0-5120.g18432218.el9.noa 2.9 MB/s | 784 kB 00:00 2024-09-21T03:31:24.651 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:31:24.697 INFO:teuthology.orchestra.run.smithi016.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 563 kB/s | 80 kB 00:00 2024-09-21T03:31:24.781 INFO:teuthology.orchestra.run.smithi016.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-21T03:31:24.785 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:31:24.826 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:31:24.835 INFO:teuthology.orchestra.run.smithi016.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.8 MB/s | 253 kB 00:00 2024-09-21T03:31:24.854 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:31:24.873 INFO:teuthology.orchestra.run.smithi016.stdout:(43/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.7 MB/s | 801 kB 00:00 2024-09-21T03:31:24.888 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:31:24.899 INFO:teuthology.orchestra.run.smithi016.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-21T03:31:24.927 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:31:24.949 INFO:teuthology.orchestra.run.smithi016.stdout:(45/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T03:31:24.958 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:31:24.982 INFO:teuthology.orchestra.run.smithi016.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-21T03:31:24.988 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:31:25.023 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:31:25.041 INFO:teuthology.orchestra.run.smithi016.stdout:(47/157): python3-cryptography-36.0.1-4.el9.x86 4.7 MB/s | 1.2 MB 00:00 2024-09-21T03:31:25.056 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:31:25.066 INFO:teuthology.orchestra.run.smithi016.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-21T03:31:25.106 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:31:25.149 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:31:25.175 INFO:teuthology.orchestra.run.smithi016.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-09-21T03:31:25.200 INFO:teuthology.orchestra.run.smithi016.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 799 kB/s | 107 kB 00:00 2024-09-21T03:31:25.217 INFO:teuthology.orchestra.run.smithi016.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 520 kB/s | 22 kB 00:00 2024-09-21T03:31:25.235 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:31:25.284 INFO:teuthology.orchestra.run.smithi016.stdout:(52/157): flexiblas-3.0.4-8.el9.x86_64.rpm 492 kB/s | 33 kB 00:00 2024-09-21T03:31:25.289 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:31:25.321 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:31:25.326 INFO:teuthology.orchestra.run.smithi016.stdout:(53/157): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-09-21T03:31:25.351 INFO:teuthology.orchestra.run.smithi016.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.0 MB/s | 313 kB 00:00 2024-09-21T03:31:25.385 INFO:teuthology.orchestra.run.smithi016.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-09-21T03:31:25.410 INFO:teuthology.orchestra.run.smithi016.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-09-21T03:31:25.423 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:31:25.436 INFO:teuthology.orchestra.run.smithi016.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T03:31:25.461 INFO:teuthology.orchestra.run.smithi016.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-21T03:31:25.471 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:31:25.486 INFO:teuthology.orchestra.run.smithi016.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-09-21T03:31:25.537 INFO:teuthology.orchestra.run.smithi016.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-09-21T03:31:25.570 INFO:teuthology.orchestra.run.smithi016.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-21T03:31:25.596 INFO:teuthology.orchestra.run.smithi016.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 420 kB/s | 24 kB 00:00 2024-09-21T03:31:25.621 INFO:teuthology.orchestra.run.smithi016.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-09-21T03:31:25.646 INFO:teuthology.orchestra.run.smithi016.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-21T03:31:25.672 INFO:teuthology.orchestra.run.smithi016.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-21T03:31:25.722 INFO:teuthology.orchestra.run.smithi016.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 512 kB/s | 38 kB 00:00 2024-09-21T03:31:25.772 INFO:teuthology.orchestra.run.smithi016.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.1 MB/s | 3.0 MB 00:00 2024-09-21T03:31:25.848 INFO:teuthology.orchestra.run.smithi016.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-21T03:31:25.931 INFO:teuthology.orchestra.run.smithi016.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-09-21T03:31:26.015 INFO:teuthology.orchestra.run.smithi016.stdout:(70/157): openblas-openmp-0.3.26-2.el9.x86_64.r 14 MB/s | 4.9 MB 00:00 2024-09-21T03:31:26.040 INFO:teuthology.orchestra.run.smithi016.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-21T03:31:26.074 INFO:teuthology.orchestra.run.smithi016.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 3.9 MB/s | 910 kB 00:00 2024-09-21T03:31:26.145 INFO:teuthology.orchestra.run.smithi016.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 9.7 MB/s | 1.0 MB 00:00 2024-09-21T03:31:26.175 INFO:teuthology.orchestra.run.smithi016.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 874 kB/s | 87 kB 00:00 2024-09-21T03:31:26.180 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:31:26.208 INFO:teuthology.orchestra.run.smithi016.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-09-21T03:31:26.227 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:31:26.260 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:31:26.267 INFO:teuthology.orchestra.run.smithi016.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-21T03:31:26.357 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:31:26.375 INFO:teuthology.orchestra.run.smithi016.stdout:(77/157): postgresql-server-13.16-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-09-21T03:31:26.400 INFO:teuthology.orchestra.run.smithi016.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-09-21T03:31:26.415 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:31:26.418 INFO:teuthology.orchestra.run.smithi016.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 612 kB/s | 26 kB 00:00 2024-09-21T03:31:26.434 INFO:teuthology.orchestra.run.smithi016.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 569 kB/s | 19 kB 00:00 2024-09-21T03:31:26.468 INFO:teuthology.orchestra.run.smithi016.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-21T03:31:26.468 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:31:26.494 INFO:teuthology.orchestra.run.smithi016.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 592 kB/s | 35 kB 00:00 2024-09-21T03:31:26.548 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:31:26.596 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:31:26.623 INFO:teuthology.orchestra.run.smithi016.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-09-21T03:31:26.653 INFO:teuthology.orchestra.run.smithi016.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-09-21T03:31:26.678 INFO:teuthology.orchestra.run.smithi016.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.0 MB/s | 222 kB 00:00 2024-09-21T03:31:26.703 INFO:teuthology.orchestra.run.smithi016.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T03:31:26.728 INFO:teuthology.orchestra.run.smithi016.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-21T03:31:26.754 INFO:teuthology.orchestra.run.smithi016.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-21T03:31:26.779 INFO:teuthology.orchestra.run.smithi016.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T03:31:26.787 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:31:26.804 INFO:teuthology.orchestra.run.smithi016.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-21T03:31:26.819 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:31:26.830 INFO:teuthology.orchestra.run.smithi016.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T03:31:26.855 INFO:teuthology.orchestra.run.smithi016.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T03:31:26.898 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:31:26.898 INFO:teuthology.orchestra.run.smithi016.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 998 kB/s | 42 kB 00:00 2024-09-21T03:31:26.972 INFO:teuthology.orchestra.run.smithi016.stdout:(94/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-09-21T03:31:26.997 INFO:teuthology.orchestra.run.smithi016.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-09-21T03:31:27.056 INFO:teuthology.orchestra.run.smithi016.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-21T03:31:27.090 INFO:teuthology.orchestra.run.smithi016.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 391 kB/s | 36 kB 00:00 2024-09-21T03:31:27.115 INFO:teuthology.orchestra.run.smithi016.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-09-21T03:31:27.140 INFO:teuthology.orchestra.run.smithi016.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T03:31:27.166 INFO:teuthology.orchestra.run.smithi016.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T03:31:27.191 INFO:teuthology.orchestra.run.smithi016.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T03:31:27.216 INFO:teuthology.orchestra.run.smithi016.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T03:31:27.241 INFO:teuthology.orchestra.run.smithi016.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-21T03:31:27.325 INFO:teuthology.orchestra.run.smithi016.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-09-21T03:31:27.527 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:31:27.551 INFO:teuthology.orchestra.run.smithi016.stdout:(105/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-21T03:31:27.601 INFO:teuthology.orchestra.run.smithi016.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-09-21T03:31:27.618 INFO:teuthology.orchestra.run.smithi016.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 1.3 MB/s | 19 kB 00:00 2024-09-21T03:31:27.641 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:31:27.651 INFO:teuthology.orchestra.run.smithi016.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 3.0 MB/s | 308 kB 00:00 2024-09-21T03:31:27.668 INFO:teuthology.orchestra.run.smithi016.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-21T03:31:27.686 INFO:teuthology.orchestra.run.smithi016.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 2.8 MB/s | 49 kB 00:00 2024-09-21T03:31:27.719 INFO:teuthology.orchestra.run.smithi016.stdout:(111/157): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-09-21T03:31:27.744 INFO:teuthology.orchestra.run.smithi016.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-21T03:31:27.803 INFO:teuthology.orchestra.run.smithi016.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-09-21T03:31:27.837 INFO:teuthology.orchestra.run.smithi016.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 10 MB/s | 839 kB 00:00 2024-09-21T03:31:27.854 INFO:teuthology.orchestra.run.smithi016.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-21T03:31:27.871 INFO:teuthology.orchestra.run.smithi016.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-21T03:31:27.887 INFO:teuthology.orchestra.run.smithi016.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 865 kB/s | 14 kB 00:00 2024-09-21T03:31:27.913 INFO:teuthology.orchestra.run.smithi016.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-21T03:31:27.963 INFO:teuthology.orchestra.run.smithi016.stdout:(119/157): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-09-21T03:31:28.005 INFO:teuthology.orchestra.run.smithi016.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-21T03:31:28.080 INFO:teuthology.orchestra.run.smithi016.stdout:(121/157): libarrow-9.0.0-11.el9.x86_64.rpm 9.5 MB/s | 4.4 MB 00:00 2024-09-21T03:31:28.114 INFO:teuthology.orchestra.run.smithi016.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-09-21T03:31:28.139 INFO:teuthology.orchestra.run.smithi016.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 1.6 MB/s | 220 kB 00:00 2024-09-21T03:31:28.165 INFO:teuthology.orchestra.run.smithi016.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-21T03:31:28.190 INFO:teuthology.orchestra.run.smithi016.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-09-21T03:31:28.207 INFO:teuthology.orchestra.run.smithi016.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-21T03:31:28.224 INFO:teuthology.orchestra.run.smithi016.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 694 kB/s | 23 kB 00:00 2024-09-21T03:31:28.249 INFO:teuthology.orchestra.run.smithi016.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 466 kB/s | 19 kB 00:00 2024-09-21T03:31:28.274 INFO:teuthology.orchestra.run.smithi016.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-09-21T03:31:28.300 INFO:teuthology.orchestra.run.smithi016.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T03:31:28.333 INFO:teuthology.orchestra.run.smithi016.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-21T03:31:28.359 INFO:teuthology.orchestra.run.smithi016.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-09-21T03:31:28.417 INFO:teuthology.orchestra.run.smithi016.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-09-21T03:31:28.451 INFO:teuthology.orchestra.run.smithi016.stdout:(134/157): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-09-21T03:31:28.476 INFO:teuthology.orchestra.run.smithi016.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-09-21T03:31:28.501 INFO:teuthology.orchestra.run.smithi016.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T03:31:28.527 INFO:teuthology.orchestra.run.smithi016.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-21T03:31:28.560 INFO:teuthology.orchestra.run.smithi016.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 521 kB/s | 31 kB 00:00 2024-09-21T03:31:28.586 INFO:teuthology.orchestra.run.smithi016.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T03:31:28.611 INFO:teuthology.orchestra.run.smithi016.stdout:(140/157): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-21T03:31:28.648 INFO:teuthology.orchestra.run.smithi016.stdout:(141/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-21T03:31:28.673 INFO:teuthology.orchestra.run.smithi016.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-21T03:31:28.715 INFO:teuthology.orchestra.run.smithi016.stdout:(143/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-09-21T03:31:28.741 INFO:teuthology.orchestra.run.smithi016.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-21T03:31:28.766 INFO:teuthology.orchestra.run.smithi016.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-21T03:31:28.791 INFO:teuthology.orchestra.run.smithi016.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-21T03:31:28.841 INFO:teuthology.orchestra.run.smithi016.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-21T03:31:28.852 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:31:28.892 INFO:teuthology.orchestra.run.smithi016.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-09-21T03:31:28.942 INFO:teuthology.orchestra.run.smithi016.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-21T03:31:28.947 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:31:28.975 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:31:29.008 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:31:29.092 INFO:teuthology.orchestra.run.smithi016.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 14 MB/s | 3.4 MB 00:00 2024-09-21T03:31:29.301 INFO:teuthology.orchestra.run.smithi016.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 7.7 MB/s | 3.2 MB 00:00 2024-09-21T03:31:29.334 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:31:29.379 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:31:29.426 INFO:teuthology.orchestra.run.smithi016.stdout:(152/157): NetworkManager-1.51.0-1.el9.x86_64.r 4.9 MB/s | 2.3 MB 00:00 2024-09-21T03:31:29.452 INFO:teuthology.orchestra.run.smithi016.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 272 kB/s | 41 kB 00:00 2024-09-21T03:31:29.470 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:31:29.493 INFO:teuthology.orchestra.run.smithi016.stdout:(154/157): NetworkManager-libnm-1.51.0-1.el9.x8 4.5 MB/s | 1.8 MB 00:00 2024-09-21T03:31:29.527 INFO:teuthology.orchestra.run.smithi016.stdout:(155/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.4 MB/s | 249 kB 00:00 2024-09-21T03:31:29.550 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:31:29.780 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:31:29.869 INFO:teuthology.orchestra.run.smithi016.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 3.2 MB/s | 1.4 MB 00:00 2024-09-21T03:31:29.953 INFO:teuthology.orchestra.run.smithi016.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 5.2 MB/s | 2.4 MB 00:00 2024-09-21T03:31:29.959 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:31:29.959 INFO:teuthology.orchestra.run.smithi016.stdout:Total 23 MB/s | 228 MB 00:09 2024-09-21T03:31:30.309 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:31:30.341 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:31:30.412 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:31:30.442 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:31:30.810 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-21T03:31:30.938 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:31:30.976 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:31:31.038 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:31:31.072 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:31:31.098 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:31:31.149 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:31:31.480 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:31:31.590 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:31:31.716 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:31:31.750 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:31:31.841 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:31:31.871 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:31:31.900 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:31:32.052 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:31:32.093 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-21T03:31:32.118 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:31:32.146 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:31:32.243 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:31:32.272 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:31:32.295 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-21T03:31:32.295 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-21T03:31:32.318 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:31:32.340 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:31:32.363 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:31:32.541 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:31:32.686 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:31:32.732 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:31:32.771 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:31:32.821 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:31:32.917 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:31:33.006 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:31:33.218 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:31:33.323 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:31:33.339 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:31:33.363 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:31:33.383 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:31:33.397 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:31:33.462 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:31:33.464 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:31:33.464 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:31:33.464 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:31:33.464 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:31:33.492 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:31:33.506 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:31:33.539 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:31:33.582 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:31:33.615 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:31:33.651 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:31:33.719 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:31:33.781 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:31:33.812 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:31:33.847 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:31:33.928 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:31:33.967 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:31:34.010 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:31:34.039 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:31:34.107 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:31:34.162 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:31:34.244 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:31:34.377 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:31:34.446 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:31:34.821 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-21T03:31:34.822 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-21T03:31:34.968 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:31:35.051 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:31:35.080 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:31:35.123 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:31:35.161 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:31:35.202 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:31:35.362 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:31:35.429 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:31:35.520 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:31:35.584 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:31:35.657 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:31:35.689 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:31:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:31:35.772 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:31:35.817 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:31:35.859 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:31:35.901 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:31:36.037 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:31:36.078 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:31:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:31:36.435 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:31:36.480 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:31:36.520 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:31:36.546 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:31:36.570 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:31:36.654 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:31:36.780 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:31:36.836 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:31:36.863 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:31:36.898 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:31:36.937 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:31:36.968 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:31:36.998 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:31:37.033 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:31:37.066 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:31:37.107 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:31:37.150 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:31:37.245 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:31:37.297 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:31:37.328 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:31:37.433 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:31:37.488 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:31:38.213 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:31:38.260 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:31:38.293 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:31:38.409 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:31:38.456 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:31:38.502 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:31:38.598 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:31:38.646 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:31:38.829 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:31:38.860 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:31:38.938 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:31:39.580 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:31:39.691 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:31:40.900 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:31:41.013 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:31:41.041 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:31:41.074 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:31:41.399 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:31:41.445 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:31:41.537 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:31:41.566 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:31:41.611 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:31:42.025 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-21T03:31:42.106 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:31:42.139 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:31:42.166 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:31:42.209 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:31:42.210 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:31:42.258 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:31:42.278 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:31:42.336 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:31:42.367 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:31:42.428 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:31:42.829 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:31:42.943 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:31:43.096 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:31:43.127 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:31:43.242 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:31:43.274 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:31:43.316 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:31:43.535 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:31:43.629 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:31:43.657 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:31:43.788 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:31:43.825 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:31:43.907 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:31:43.934 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:31:43.980 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:31:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:31:44.484 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:31:44.487 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:31:44.671 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:31:44.671 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:31:44.797 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:31:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:31:45.101 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:31:45.101 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-09-21T03:31:45.101 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-09-21T03:31:45.101 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-09-21T03:31:45.102 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-09-21T03:31:45.102 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-09-21T03:31:45.102 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-09-21T03:31:45.102 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-09-21T03:31:45.102 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:31:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:31:45.237 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:31:45.278 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:31:45.282 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:31:45.293 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:31:45.293 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:45.294 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:31:45.294 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:31:45.366 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:31:45.366 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:31:45.366 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:31:45.366 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:31:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:31:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:31:45.408 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:31:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:31:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:31:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:31:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:31:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:31:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:31:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:31:45.866 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:31:45.941 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:31:45.974 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:31:46.021 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:31:46.121 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:31:46.161 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:31:46.207 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:31:46.258 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:31:46.353 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:31:46.404 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:31:46.497 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:31:46.671 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:31:46.889 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:31:46.915 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:31:46.915 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:46.915 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:31:46.915 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:31:46.916 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:31:46.916 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:31:46.982 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:31:47.140 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:31:47.162 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:31:47.162 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:47.162 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:31:47.163 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:31:47.163 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:31:47.163 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:31:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:31:47.659 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:31:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:31:47.664 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:47.664 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:31:47.664 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:31:47.664 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:31:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:31:47.762 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:31:47.818 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:31:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:31:48.024 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:31:48.067 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:31:48.085 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:31:48.115 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:31:48.133 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:31:48.176 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:31:48.186 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:31:48.222 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:31:48.281 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:31:48.408 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:31:48.423 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:31:48.466 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:31:48.498 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:31:48.509 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:31:48.526 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:31:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:31:48.717 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:31:48.802 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:31:48.839 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:31:48.875 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:31:48.918 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:31:48.948 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:31:48.955 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:31:48.981 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:31:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:31:49.045 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:31:49.047 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:31:49.072 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:31:49.095 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:31:49.278 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:31:49.327 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:31:49.397 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:31:49.519 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:31:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:31:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:31:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:31:49.653 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:31:49.769 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:31:49.949 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:31:49.995 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:31:50.032 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:31:50.066 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:31:50.107 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:31:50.147 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:31:50.183 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:31:50.243 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:31:50.284 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:31:50.329 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:31:50.360 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:31:50.465 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:31:50.528 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:31:50.564 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:31:50.683 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:31:50.754 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:31:51.695 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:31:51.750 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:31:51.783 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:31:51.903 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:31:51.966 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:31:52.020 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:31:52.130 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:31:52.177 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:31:52.247 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:31:52.311 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:31:52.426 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:31:52.465 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:31:52.563 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:31:53.272 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:31:53.303 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:31:53.416 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:31:53.433 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:31:53.576 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:31:53.586 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:31:53.613 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:31:53.636 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:31:53.637 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:53.637 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:31:53.637 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:31:53.637 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:31:53.637 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:31:53.680 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:31:53.691 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:31:53.791 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:31:53.845 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:31:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:31:54.849 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:31:54.872 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:31:54.873 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:54.873 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:31:54.873 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:31:54.873 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:31:54.873 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:31:54.916 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:31:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:31:54.939 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:54.939 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:31:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:31:54.984 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:31:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:31:55.007 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:55.007 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:31:55.007 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:31:55.007 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:31:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:31:55.222 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:31:55.326 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:31:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:31:55.349 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:31:55.349 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:55.349 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:31:55.349 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:31:55.349 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:31:55.349 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:31:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:31:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:31:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:31:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:31:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:31:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:31:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:31:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:31:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:31:56.763 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:31:56.790 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:31:57.223 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:31:57.223 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2024-09-21T03:31:57.223 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2024-09-21T03:31:57.224 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2024-09-21T03:31:57.224 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2024-09-21T03:31:57.224 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2024-09-21T03:31:57.224 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /root 2024-09-21T03:31:57.224 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2024-09-21T03:31:57.224 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:31:57.401 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:31:57.417 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:31:57.417 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:57.417 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:31:57.417 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:31:59.020 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:31:59.048 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:31:59.048 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:59.048 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:31:59.048 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:31:59.048 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:31:59.048 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:31:59.280 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:31:59.303 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:31:59.303 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:59.304 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:31:59.304 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:31:59.304 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:31:59.304 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:31:59.331 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:31:59.371 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:31:59.412 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:31:59.442 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:31:59.444 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:31:59.471 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:31:59.478 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:31:59.652 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:31:59.701 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:31:59.731 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:31:59.761 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:31:59.771 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:31:59.796 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:31:59.796 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:31:59.796 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:31:59.796 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:31:59.796 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:31:59.796 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:31:59.815 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:31:59.901 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:31:59.958 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:32:00.014 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:32:00.180 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:32:00.198 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:32:00.242 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:32:00.247 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:32:00.325 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:32:00.370 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:32:00.370 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:32:00.370 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:32:00.370 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:32:00.401 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:32:00.413 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:32:00.435 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:32:00.471 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:32:00.491 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:32:00.551 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:32:00.597 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:32:00.629 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:32:00.657 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:32:01.070 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:32:01.105 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:32:01.137 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:32:01.203 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:32:01.228 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:32:01.325 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:32:01.409 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:32:01.450 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:32:01.539 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:32:01.570 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:32:01.595 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:32:01.621 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:32:01.621 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:32:01.706 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:32:01.706 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:32:01.798 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:32:01.823 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:32:01.828 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:32:01.828 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:32:02.016 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:32:02.016 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:32:02.100 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:32:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:32:04.401 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:32:04.459 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:32:05.416 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:32:05.442 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:32:05.554 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:32:05.725 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:32:05.753 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:32:05.777 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:32:05.777 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:05.777 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:32:05.777 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:32:05.777 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:32:05.777 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:32:05.828 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:32:05.840 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:32:05.938 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:32:05.993 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:32:06.957 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:32:06.989 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:32:07.013 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:32:07.013 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:07.013 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:32:07.013 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:32:07.013 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:32:07.014 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:32:07.056 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:32:07.079 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:32:07.079 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:07.080 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:32:07.080 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:32:07.132 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:32:07.156 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:32:07.156 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:07.156 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:32:07.156 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:32:07.156 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:32:07.156 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:32:07.475 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:32:07.498 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:32:07.498 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:07.498 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:32:07.498 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:32:07.498 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:32:07.499 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:32:08.401 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:32:08.401 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:32:08.402 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:32:08.402 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:32:08.402 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:32:08.402 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:32:08.402 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:32:08.402 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:32:08.402 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:32:08.402 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:32:08.402 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:32:08.402 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:32:08.402 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:32:08.402 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:32:08.403 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:32:08.403 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:32:08.403 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:32:08.403 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:32:08.403 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:32:08.403 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:32:08.403 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:32:08.403 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:32:08.403 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:32:08.403 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:32:08.403 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:32:08.403 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:32:08.403 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:32:08.403 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:32:08.404 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:32:08.404 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:32:08.404 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:32:08.404 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:32:08.404 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:32:08.404 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:32:08.404 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:32:08.404 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:32:08.404 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:32:08.404 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:32:08.404 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:32:08.404 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:32:08.404 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:32:08.404 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:32:08.404 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:32:08.405 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:32:08.405 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:32:08.405 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:32:08.405 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:32:08.405 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:32:08.405 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:32:08.405 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:32:08.405 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:32:08.405 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:32:08.405 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:32:08.405 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:32:08.405 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:32:08.405 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:32:08.405 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:32:08.405 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:32:08.406 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:32:08.406 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:32:08.406 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:32:08.406 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:32:08.406 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:32:08.406 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:32:08.406 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:32:08.406 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:32:08.406 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:32:08.406 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:32:08.406 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:32:08.406 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:32:08.406 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:32:08.406 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:32:08.406 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:32:08.407 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:32:08.407 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:32:08.407 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:32:08.407 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:32:08.407 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:32:08.407 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:32:08.407 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:32:08.407 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:32:08.407 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:32:08.407 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:32:08.407 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:32:08.407 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:32:08.407 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:32:08.407 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:32:08.407 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:32:08.408 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:32:08.408 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:32:08.408 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:32:08.408 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:32:08.408 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:32:08.408 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:32:08.408 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:32:08.408 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:32:08.408 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:32:08.408 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:32:08.408 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:32:08.408 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:32:08.408 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:32:08.408 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:32:08.408 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:32:08.409 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:32:08.409 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:32:08.409 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:32:08.409 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:32:08.409 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:32:08.409 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:32:08.409 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:32:08.409 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:32:08.409 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:32:08.409 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:32:08.409 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:32:08.409 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:32:08.409 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:32:08.410 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:32:08.410 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:32:08.410 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:32:08.410 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:32:08.410 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:32:08.410 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:32:08.410 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:32:08.410 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:32:08.410 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:32:08.410 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:32:08.410 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:32:08.410 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:32:08.410 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:32:08.410 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:32:08.410 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:32:08.410 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:32:08.411 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:32:08.411 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:32:08.411 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:32:08.411 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:32:08.411 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:32:08.411 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:32:08.411 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:32:08.411 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:32:08.411 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:32:08.411 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:32:08.411 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:32:08.411 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:32:08.411 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:32:08.412 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:32:08.412 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:32:08.412 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:32:08.412 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:32:08.412 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:32:08.412 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:32:08.412 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:32:08.412 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:32:08.412 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:32:08.412 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:32:08.412 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:32:08.412 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:32:08.412 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:32:08.412 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:32:08.412 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:32:08.412 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:32:08.413 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:32:08.413 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:32:08.413 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:32:08.413 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:32:10.965 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:32:10.965 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:32:10.965 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-09-21T03:32:10.965 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:10.966 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:10.966 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:10.966 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:10.966 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.966 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.966 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:32:10.966 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:32:10.966 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-21T03:32:10.966 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:32:10.966 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:32:10.966 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:32:10.966 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:32:10.966 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.966 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.966 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.968 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.968 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:10.968 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:10.968 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.968 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.968 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:10.968 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:32:10.968 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:32:10.968 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:32:10.968 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:32:10.968 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:32:10.968 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:32:10.968 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:32:10.968 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:32:10.968 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:32:10.969 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:32:10.969 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:32:10.969 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.969 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.969 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.969 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:32:10.969 INFO:teuthology.orchestra.run.smithi123.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:32:10.969 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:32:10.969 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:32:10.969 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:32:10.969 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:32:10.969 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.969 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.969 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:32:10.969 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.970 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:32:10.970 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:32:10.970 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:32:10.970 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:32:10.970 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:32:10.970 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:32:10.970 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:32:10.970 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:32:10.970 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:32:10.970 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:32:10.970 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:32:10.970 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:32:10.970 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:32:10.970 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:32:10.970 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:32:10.971 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:32:10.971 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:32:10.971 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:32:10.971 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:32:10.971 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:32:10.971 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:32:10.971 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:32:10.971 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:32:10.971 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.971 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.971 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.971 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:32:10.972 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:32:10.972 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:32:10.972 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:32:10.972 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:32:10.972 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:32:10.972 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:32:10.972 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:32:10.972 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:32:10.972 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:32:10.972 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:32:10.972 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:32:10.972 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:32:10.972 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:32:10.973 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:32:10.973 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:32:10.973 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:32:10.973 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:32:10.973 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:32:10.973 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:32:10.973 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:32:10.973 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:32:10.973 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:32:10.973 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:32:10.973 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:32:10.973 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:32:10.973 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:32:10.973 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:32:10.974 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:32:10.974 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:32:10.974 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:32:10.974 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:32:10.974 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:32:10.974 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:32:10.974 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:32:10.974 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:32:10.974 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:32:10.974 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:32:10.974 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:32:10.974 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:32:10.974 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:32:10.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:32:10.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:32:10.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:32:10.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:32:10.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:32:10.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:32:10.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:32:10.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:32:10.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:32:10.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:32:10.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:32:10.976 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:32:10.976 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:32:10.976 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:32:10.976 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:32:10.976 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:32:10.976 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:32:10.976 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.976 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.976 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:10.976 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:32:10.976 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:32:10.976 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:32:10.976 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:32:10.977 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:32:10.977 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:32:10.977 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:32:10.977 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:32:10.977 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-21T03:32:11.431 DEBUG:teuthology.parallel:result is None 2024-09-21T03:32:11.445 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:32:11.494 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:32:11.543 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:32:11.583 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:32:11.619 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:32:11.797 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:32:11.841 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:32:11.871 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:32:11.900 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:32:11.947 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:32:12.559 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:32:12.594 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:32:12.615 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:32:13.434 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:32:13.546 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:32:13.589 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:32:13.669 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:32:13.718 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:32:13.743 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:32:13.769 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:32:13.769 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:32:13.853 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:32:13.853 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:32:13.937 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:32:13.962 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:32:13.968 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:32:13.968 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:32:13.986 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:32:13.986 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-09-21T03:32:13.986 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-09-21T03:32:13.986 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-09-21T03:32:13.986 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-09-21T03:32:13.986 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-09-21T03:32:13.986 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-09-21T03:32:13.986 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-09-21T03:32:13.986 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:32:14.147 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:32:14.147 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:32:14.199 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:32:14.232 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:32:14.235 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:32:14.235 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:14.235 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:32:14.235 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:32:14.295 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:32:16.027 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:32:16.086 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:32:16.086 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:16.086 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:32:16.087 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:32:16.087 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:32:16.087 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:32:16.344 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:32:16.395 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:32:16.395 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:16.396 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:32:16.396 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:32:16.396 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:32:16.396 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:32:16.945 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:32:16.996 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:32:16.996 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:16.997 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:32:16.997 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:32:16.997 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:32:16.997 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:32:17.129 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:32:17.191 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:32:17.263 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:32:17.508 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:32:17.566 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:32:17.655 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:32:17.782 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:32:17.967 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:32:18.015 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:32:18.056 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:32:18.089 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:32:18.712 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:32:18.747 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:32:18.782 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:32:18.865 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:32:18.911 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:32:20.709 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:32:20.709 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:32:20.709 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:32:20.709 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:32:20.709 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:32:20.709 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:32:20.709 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:32:20.709 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:32:20.709 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:32:20.710 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:32:20.710 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:32:20.710 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:32:20.710 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:32:20.710 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:32:20.710 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:32:20.710 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:32:20.710 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:32:20.710 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:32:20.710 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:32:20.710 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:32:20.710 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:32:20.710 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:32:20.710 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:32:20.710 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:32:20.711 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:32:20.711 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:32:20.711 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:32:20.711 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:32:20.711 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:32:20.711 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:32:20.711 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:32:20.711 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:32:20.711 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:32:20.711 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:32:20.711 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:32:20.711 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:32:20.711 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:32:20.711 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:32:20.711 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:32:20.712 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:32:20.712 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:32:20.712 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:32:20.712 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:32:20.712 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:32:20.712 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:32:20.712 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:32:20.712 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:32:20.712 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:32:20.712 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:32:20.712 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:32:20.712 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:32:20.712 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:32:20.712 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:32:20.712 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:32:20.713 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:32:20.713 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:32:20.713 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:32:20.713 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:32:20.713 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:32:20.713 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:32:20.713 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:32:20.713 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:32:20.713 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:32:20.713 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:32:20.713 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:32:20.713 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:32:20.713 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:32:20.713 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:32:20.713 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:32:20.713 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:32:20.714 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:32:20.714 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:32:20.714 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:32:20.714 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:32:20.714 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:32:20.714 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:32:20.714 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:32:20.714 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:32:20.714 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:32:20.714 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:32:20.714 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:32:20.714 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:32:20.714 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:32:20.714 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:32:20.714 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:32:20.715 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:32:20.715 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:32:20.715 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:32:20.715 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:32:20.715 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:32:20.715 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:32:20.715 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:32:20.715 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:32:20.715 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:32:20.715 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:32:20.715 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:32:20.715 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:32:20.715 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:32:20.715 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:32:20.715 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:32:20.715 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:32:20.716 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:32:20.716 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:32:20.716 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:32:20.716 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:32:20.716 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:32:20.716 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:32:20.716 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:32:20.716 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:32:20.716 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:32:20.716 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:32:20.716 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:32:20.716 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:32:20.716 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:32:20.716 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:32:20.716 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:32:20.717 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:32:20.717 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:32:20.717 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:32:20.717 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:32:20.717 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:32:20.717 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:32:20.717 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:32:20.717 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:32:20.717 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:32:20.717 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:32:20.717 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:32:20.717 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:32:20.717 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:32:20.717 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:32:20.717 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:32:20.718 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:32:20.718 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:32:20.718 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:32:20.718 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:32:20.718 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:32:20.718 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:32:20.718 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:32:20.718 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:32:20.718 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:32:20.718 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:32:20.718 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:32:20.718 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:32:20.718 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:32:20.718 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:32:20.719 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:32:20.719 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:32:20.719 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:32:20.719 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:32:20.719 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:32:20.719 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:32:20.719 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:32:20.719 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:32:20.719 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:32:20.719 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:32:20.719 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:32:20.719 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:32:20.719 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:32:20.719 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:32:20.719 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:32:20.720 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:32:20.720 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:32:20.720 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:32:20.720 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:32:20.720 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:32:22.786 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:32:22.852 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:32:23.458 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:32:23.458 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:32:23.458 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-09-21T03:32:23.458 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:23.458 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:23.458 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:23.458 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:23.458 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.459 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.459 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:32:23.459 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:32:23.459 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-21T03:32:23.459 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:32:23.459 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:32:23.459 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:32:23.459 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:32:23.459 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.459 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.459 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.459 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.460 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:23.460 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.460 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.460 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.460 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:23.460 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:23.460 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:23.460 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:23.460 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:23.460 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.460 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.460 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:23.461 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.461 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.461 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.461 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:23.461 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:23.461 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.461 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.461 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:23.461 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:32:23.461 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:32:23.461 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:32:23.461 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:32:23.461 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:32:23.461 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:32:23.462 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:32:23.462 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:32:23.462 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:32:23.462 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:32:23.462 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:32:23.462 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.462 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.462 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.462 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:32:23.462 INFO:teuthology.orchestra.run.smithi134.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:32:23.462 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:32:23.462 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:32:23.463 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:32:23.463 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:32:23.463 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.463 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.463 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:32:23.463 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.463 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:32:23.463 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:32:23.463 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:32:23.463 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:32:23.463 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:32:23.463 INFO:teuthology.orchestra.run.smithi134.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:32:23.464 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:32:23.464 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:32:23.464 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:32:23.464 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:32:23.464 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:32:23.464 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:32:23.464 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:32:23.464 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:32:23.464 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:32:23.464 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:32:23.464 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:32:23.464 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:32:23.464 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:32:23.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:32:23.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:32:23.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:32:23.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:32:23.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:32:23.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:32:23.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:32:23.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:32:23.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:32:23.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:32:23.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:32:23.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:32:23.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:32:23.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:32:23.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:32:23.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:32:23.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:32:23.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:32:23.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:32:23.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:32:23.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:32:23.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:32:23.466 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:32:23.467 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:32:23.467 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:32:23.467 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:32:23.467 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:32:23.467 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:32:23.467 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:32:23.467 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:32:23.467 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:32:23.467 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:32:23.467 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:32:23.467 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:32:23.467 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:32:23.467 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:32:23.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:32:23.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:32:23.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:32:23.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:32:23.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:32:23.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:32:23.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:32:23.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:32:23.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:32:23.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:32:23.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:32:23.468 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.469 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:32:23.469 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:32:23.469 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:32:23.469 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:32:23.469 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:32:23.469 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:32:23.469 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:32:23.469 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:32:23.469 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:32:23.469 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:32:23.469 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:32:23.469 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:32:23.470 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:32:23.470 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:32:23.470 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:32:23.470 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.470 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.470 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:23.470 INFO:teuthology.orchestra.run.smithi134.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:32:23.470 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:32:23.470 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:32:23.470 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:32:23.470 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:32:23.470 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:32:23.471 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:32:23.471 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:32:23.471 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-21T03:32:23.904 DEBUG:teuthology.parallel:result is None 2024-09-21T03:32:24.004 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:32:24.043 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:32:24.204 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:32:24.422 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:32:24.445 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:32:24.498 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:32:24.498 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:24.498 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:32:24.498 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:32:24.498 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:32:24.498 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:32:24.554 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:32:24.581 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:32:24.686 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:32:24.755 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:32:25.875 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:32:25.907 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:32:25.959 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:32:25.959 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:25.959 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:32:25.960 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:32:25.960 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:32:25.960 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:32:26.024 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:32:26.075 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:32:26.076 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:26.076 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:32:26.076 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:32:26.133 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:32:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:32:26.185 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:26.185 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:32:26.186 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:32:26.186 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:32:26.186 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:32:26.559 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:32:26.611 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:32:26.611 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:32:26.612 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:32:26.612 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:32:26.612 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:32:26.612 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:32:31.240 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:32:31.290 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:32:31.342 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:32:31.378 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:32:31.414 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:32:31.695 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:32:31.744 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:32:31.773 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:32:31.805 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:32:31.849 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:32:32.612 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:32:32.663 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:32:32.706 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:32:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:32:34.196 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:32:34.273 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:32:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:32:34.441 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:32:34.474 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:32:34.511 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:32:34.511 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:32:34.626 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:32:34.627 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:32:34.744 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:32:34.768 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:32:34.785 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:32:34.785 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:32:35.046 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:32:35.046 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:32:35.172 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:32:35.249 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:32:35.457 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:32:35.457 INFO:teuthology.orchestra.run.smithi016.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-21T03:32:35.457 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:32:41.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:32:41.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:32:41.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:32:41.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:32:41.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:32:41.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:32:41.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:32:41.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:32:41.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:32:41.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:32:41.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:32:41.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:32:41.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:32:41.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:32:41.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:32:41.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:32:41.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:32:41.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:32:41.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:32:41.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:32:41.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:32:41.590 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:32:41.590 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:32:41.590 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:32:41.590 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:32:41.590 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:32:41.590 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:32:41.600 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:32:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:32:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:32:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:32:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:32:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:32:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:32:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:32:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:32:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:32:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:32:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:32:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:32:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:32:41.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:32:41.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:32:41.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:32:41.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:32:41.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:32:41.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:32:41.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:32:41.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:32:41.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:32:41.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:32:41.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:32:41.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:32:41.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:32:41.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:32:41.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:32:41.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:32:41.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:32:41.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:32:41.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:32:41.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:32:41.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:32:41.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:32:41.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:32:41.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:32:41.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:32:41.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:32:41.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:32:41.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:32:41.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:32:41.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:32:41.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:32:41.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:32:41.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:32:41.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:32:41.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:32:41.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:32:41.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:32:41.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:32:41.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:32:41.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:32:41.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:32:41.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:32:41.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:32:41.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:32:41.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:32:41.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:32:41.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:32:41.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:32:41.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:32:41.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:32:41.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:32:41.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:32:41.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:32:41.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:32:41.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:32:41.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:32:41.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:32:41.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:32:41.606 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:32:41.606 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:32:41.606 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:32:41.606 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:32:41.606 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:32:41.606 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:32:41.606 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:32:41.606 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:32:41.606 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:32:41.606 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:32:41.606 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:32:41.606 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:32:41.606 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:32:41.606 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:32:41.607 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:32:41.607 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:32:41.607 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:32:41.607 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:32:41.607 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:32:41.607 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:32:41.607 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:32:41.607 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:32:41.607 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:32:41.607 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:32:41.607 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:32:41.607 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:32:41.607 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:32:41.607 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:32:41.607 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:32:41.607 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:32:41.608 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:32:41.608 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:32:41.608 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:32:41.608 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:32:41.608 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:32:41.608 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:32:41.608 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:32:41.608 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:32:41.608 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:32:41.608 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:32:41.608 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:32:41.608 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:32:41.608 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:32:41.608 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:32:41.608 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:32:41.608 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:32:41.609 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:32:41.609 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:32:41.609 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:32:41.609 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:32:41.609 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:32:41.609 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:32:41.609 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:32:41.609 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:32:41.609 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:32:41.609 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:32:41.609 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:32:41.609 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:32:41.609 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:32:41.609 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:32:41.609 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:32:41.610 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:32:41.610 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:32:41.610 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:32:41.610 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:32:43.952 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:32:43.952 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:32:43.952 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:43.955 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.955 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.955 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:43.955 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.955 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.955 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.955 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:43.955 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:43.955 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.955 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.955 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:32:43.955 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:32:43.955 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:32:43.955 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi016.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi016.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:32:43.958 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:32:43.958 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:32:43.958 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:32:43.958 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:32:43.958 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:32:43.958 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:32:43.958 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:32:43.958 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:32:43.958 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:32:43.958 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:32:43.958 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:32:43.958 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:32:43.959 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:32:43.959 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:32:43.959 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:32:43.959 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:32:43.959 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.959 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.959 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.959 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:32:43.959 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:32:43.959 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:32:43.959 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:32:43.959 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:32:43.960 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:32:43.960 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:32:43.960 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:32:43.960 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:32:43.960 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:32:43.960 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:32:43.960 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:32:43.960 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:32:43.960 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:32:43.960 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:32:43.960 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:32:43.960 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:32:43.960 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:32:43.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:32:43.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:32:43.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:32:43.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:32:43.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:32:43.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:32:43.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:32:43.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:32:43.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:32:43.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:32:43.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:32:43.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:32:43.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:32:43.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:32:43.964 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:32:43.964 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:32:43.964 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:32:43.964 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.964 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.964 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:32:43.964 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:32:43.964 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:32:43.964 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:32:43.964 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:32:43.964 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:32:43.964 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:32:43.965 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:32:43.965 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:32:43.965 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-21T03:32:44.593 DEBUG:teuthology.parallel:result is None 2024-09-21T03:32:44.594 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:32:44.736 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:32:44.781 INFO:teuthology.orchestra.run.smithi016.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:32:44.782 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:32:44.782 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:32:44.784 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:32:44.925 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:32:44.952 INFO:teuthology.orchestra.run.smithi123.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:32:44.952 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:32:44.952 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:32:44.954 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:32:45.091 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:32:45.117 INFO:teuthology.orchestra.run.smithi134.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:32:45.117 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:32:45.117 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:32:45.119 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T03:32:45.119 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:32:45.119 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:32:45.183 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T03:32:45.183 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:32:45.216 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T03:32:45.216 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:32:45.248 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T03:32:45.249 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:32:45.249 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:32:45.309 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:32:45.412 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T03:32:45.412 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:32:45.437 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:32:45.502 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T03:32:45.502 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:32:45.533 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:32:45.600 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T03:32:45.601 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:32:45.601 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:32:45.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:32:45.760 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T03:32:45.760 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:32:45.785 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:32:45.850 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T03:32:45.850 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:32:45.875 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:32:45.941 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T03:32:45.941 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:32:45.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:32:46.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:32:46.098 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T03:32:46.098 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:32:46.122 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:32:46.187 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T03:32:46.187 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:32:46.212 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:32:46.276 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T03:32:46.358 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-21T03:32:46.358 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:32:46.359 INFO:tasks.cephadm:Cluster fsid is 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:32:46.359 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T03:32:46.359 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.16', 'mon.b': '172.21.15.123', 'mon.c': '172.21.15.134'} 2024-09-21T03:32:46.359 INFO:tasks.cephadm:First mon is mon.a on smithi016 2024-09-21T03:32:46.359 INFO:tasks.cephadm:First mgr is x 2024-09-21T03:32:46.359 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T03:32:46.359 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname $(hostname -s) 2024-09-21T03:32:46.416 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname $(hostname -s) 2024-09-21T03:32:46.440 DEBUG:teuthology.orchestra.run.smithi134:> sudo hostname $(hostname -s) 2024-09-21T03:32:46.465 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T03:32:46.465 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:32:46.606 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T03:32:46.745 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:32:46.746 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T03:32:46.746 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T03:32:46.746 DEBUG:teuthology.orchestra.run.smithi016:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:32:47.324 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:32 /home/ubuntu/cephtest/cephadm 2024-09-21T03:32:47.324 DEBUG:teuthology.orchestra.run.smithi123:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:32:47.853 INFO:teuthology.orchestra.run.smithi123.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:32 /home/ubuntu/cephtest/cephadm 2024-09-21T03:32:47.853 DEBUG:teuthology.orchestra.run.smithi134:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:32:48.383 INFO:teuthology.orchestra.run.smithi134.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:32 /home/ubuntu/cephtest/cephadm 2024-09-21T03:32:48.383 DEBUG:teuthology.orchestra.run.smithi016:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:32:48.425 DEBUG:teuthology.orchestra.run.smithi123:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:32:48.444 DEBUG:teuthology.orchestra.run.smithi134:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:32:48.477 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T03:32:48.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:32:48.481 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:32:48.486 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:32:50.018 INFO:teuthology.orchestra.run.smithi134.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:32:50.036 INFO:teuthology.orchestra.run.smithi123.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:32:50.460 INFO:teuthology.orchestra.run.smithi016.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:33:25.575 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:33:25.575 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:33:25.575 INFO:teuthology.orchestra.run.smithi016.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:33:25.576 INFO:teuthology.orchestra.run.smithi016.stdout: "repo_digests": [ 2024-09-21T03:33:25.576 INFO:teuthology.orchestra.run.smithi016.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:33:25.576 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:33:25.576 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:33:41.261 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-21T03:33:41.261 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:33:41.262 INFO:teuthology.orchestra.run.smithi123.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:33:41.262 INFO:teuthology.orchestra.run.smithi123.stdout: "repo_digests": [ 2024-09-21T03:33:41.262 INFO:teuthology.orchestra.run.smithi123.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:33:41.262 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-21T03:33:41.262 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-21T03:33:41.444 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-21T03:33:41.444 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:33:41.444 INFO:teuthology.orchestra.run.smithi134.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:33:41.444 INFO:teuthology.orchestra.run.smithi134.stdout: "repo_digests": [ 2024-09-21T03:33:41.444 INFO:teuthology.orchestra.run.smithi134.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:33:41.444 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-21T03:33:41.444 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-21T03:33:41.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph 2024-09-21T03:33:41.533 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph 2024-09-21T03:33:41.561 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph 2024-09-21T03:33:41.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /etc/ceph 2024-09-21T03:33:41.645 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /etc/ceph 2024-09-21T03:33:41.671 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /etc/ceph 2024-09-21T03:33:41.694 INFO:tasks.cephadm:Writing seed config... 2024-09-21T03:33:41.695 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-21T03:33:41.696 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T03:33:41.696 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T03:33:41.696 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T03:33:41.696 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T03:33:41.696 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T03:33:41.696 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T03:33:41.696 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T03:33:41.696 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T03:33:41.696 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T03:33:41.696 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-21T03:33:41.696 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-21T03:33:41.696 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T03:33:41.696 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T03:33:41.696 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-21T03:33:41.697 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T03:33:41.697 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T03:33:41.697 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T03:33:41.697 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T03:33:41.697 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T03:33:41.697 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T03:33:41.697 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-21T03:33:41.697 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:33:41.698 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T03:33:41.733 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2b324840-77ca-11ef-baf5-efdc52797490 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-21T03:33:41.733 DEBUG:teuthology.orchestra.run.smithi016:mon.a> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T03:33:41.777 DEBUG:teuthology.orchestra.run.smithi016:mgr.x> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T03:33:41.823 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T03:33:41.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid 2b324840-77ca-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.16 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:33:42.090 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:33:42.090 INFO:teuthology.orchestra.run.smithi016.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.16', '--skip-admin-label'] 2024-09-21T03:33:42.090 INFO:teuthology.orchestra.run.smithi016.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T03:33:42.090 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying podman|docker is present... 2024-09-21T03:33:42.202 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.2.2 2024-09-21T03:33:42.202 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying lvm2 is present... 2024-09-21T03:33:42.202 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying time synchronization is in place... 2024-09-21T03:33:42.216 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T03:33:42.216 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T03:33:42.229 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T03:33:42.229 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-21T03:33:42.243 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-09-21T03:33:42.255 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-09-21T03:33:42.255 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-09-21T03:33:42.256 INFO:teuthology.orchestra.run.smithi016.stdout:Repeating the final host check... 2024-09-21T03:33:42.309 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.2.2 2024-09-21T03:33:42.309 INFO:teuthology.orchestra.run.smithi016.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T03:33:42.309 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl is present 2024-09-21T03:33:42.309 INFO:teuthology.orchestra.run.smithi016.stdout:lvcreate is present 2024-09-21T03:33:42.322 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T03:33:42.322 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T03:33:42.334 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T03:33:42.334 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-21T03:33:42.348 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-09-21T03:33:42.361 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-09-21T03:33:42.361 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-09-21T03:33:42.361 INFO:teuthology.orchestra.run.smithi016.stdout:Host looks OK 2024-09-21T03:33:42.361 INFO:teuthology.orchestra.run.smithi016.stdout:Cluster fsid: 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:33:42.361 INFO:teuthology.orchestra.run.smithi016.stdout:Acquiring lock 140050321041776 on /run/cephadm/2b324840-77ca-11ef-baf5-efdc52797490.lock 2024-09-21T03:33:42.361 INFO:teuthology.orchestra.run.smithi016.stdout:Lock 140050321041776 acquired on /run/cephadm/2b324840-77ca-11ef-baf5-efdc52797490.lock 2024-09-21T03:33:42.361 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 3300 ... 2024-09-21T03:33:42.362 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 6789 ... 2024-09-21T03:33:42.362 INFO:teuthology.orchestra.run.smithi016.stdout:Base mon IP(s) is [172.21.15.16:3300, 172.21.15.16:6789], mon addrv is [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-21T03:33:42.369 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.16 metric 100 2024-09-21T03:33:42.369 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.16 metric 100 2024-09-21T03:33:42.375 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T03:33:42.375 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-21T03:33:42.375 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1793sec hoplimit 64 pref medium 2024-09-21T03:33:42.381 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T03:33:42.381 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T03:33:42.381 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T03:33:42.381 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-21T03:33:42.381 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-21T03:33:42.381 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T03:33:42.382 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-09-21T03:33:42.382 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-09-21T03:33:42.382 INFO:teuthology.orchestra.run.smithi016.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T03:33:42.382 INFO:teuthology.orchestra.run.smithi016.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T03:33:42.383 INFO:teuthology.orchestra.run.smithi016.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:33:42.823 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T03:33:42.823 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:33:42.823 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T03:33:42.823 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T03:33:42.823 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T03:33:42.823 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T03:33:42.823 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T03:33:44.470 INFO:teuthology.orchestra.run.smithi016.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:33:44.470 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:33:44.470 INFO:teuthology.orchestra.run.smithi016.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T03:33:45.493 INFO:teuthology.orchestra.run.smithi016.stdout:stat: stdout 167 167 2024-09-21T03:33:45.493 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial keys... 2024-09-21T03:33:46.783 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQCZPu5m78p7OBAA9g7kCpFU4u6W/fbTeCoQRA== 2024-09-21T03:33:48.038 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQCbPu5micYXGRAAt2L6wzfQ5DST+IVTTbgWng== 2024-09-21T03:33:49.075 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQCcPu5mLIrRGxAA5mb3z6ptAjJyB0aeSFPaDQ== 2024-09-21T03:33:49.076 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial monmap... 2024-09-21T03:33:50.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T03:33:50.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T03:33:50.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:33:50.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T03:33:50.032 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool for a [v2:172.21.15.16:3300,v1:172.21.15.16:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T03:33:50.032 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = quincy 2024-09-21T03:33:50.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: set fsid to 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:33:50.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T03:33:50.032 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:33:50.032 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mon... 2024-09-21T03:33:51.146 INFO:teuthology.orchestra.run.smithi016.stdout:create mon.a on 2024-09-21T03:33:51.444 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T03:33:51.667 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T03:33:51.930 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2b324840-77ca-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-2b324840-77ca-11ef-baf5-efdc52797490.target. 2024-09-21T03:33:51.930 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2b324840-77ca-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-2b324840-77ca-11ef-baf5-efdc52797490.target. 2024-09-21T03:33:52.260 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2b324840-77ca-11ef-baf5-efdc52797490@mon.a 2024-09-21T03:33:52.260 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2b324840-77ca-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-2b324840-77ca-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T03:33:52.527 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2b324840-77ca-11ef-baf5-efdc52797490.target.wants/ceph-2b324840-77ca-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-2b324840-77ca-11ef-baf5-efdc52797490@.service. 2024-09-21T03:33:53.161 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:53 smithi016 podman[29978]: 2024-09-21 03:33:53.084114261 +0000 UTC m=+0.324449304 container start c5746a01e2902c4b8260cc66d3be6a49f4787b716f3df21e30f8edfe17904fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-a, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T03:33:53.161 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:53 smithi016 bash[29978]: c5746a01e2902c4b8260cc66d3be6a49f4787b716f3df21e30f8edfe17904fb0 2024-09-21T03:33:53.193 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:33:53.193 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-21T03:33:53.208 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:33:53.208 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-21T03:33:53.208 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-21T03:33:53.208 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T03:33:53.209 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon to start... 2024-09-21T03:33:53.209 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon... 2024-09-21T03:33:53.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:53 smithi016 systemd[1]: Started Ceph mon.a for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:33:53.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:53 smithi016 ceph-mon[29997]: mkfs 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:33:53.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:53 smithi016 ceph-mon[29997]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:33:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:54 smithi016 ceph-mon[29997]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:33:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:54 smithi016 ceph-mon[29997]: monmap epoch 1 2024-09-21T03:33:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:54 smithi016 ceph-mon[29997]: fsid 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:33:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:54 smithi016 ceph-mon[29997]: last_changed 2024-09-21T03:33:49.528201+0000 2024-09-21T03:33:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:54 smithi016 ceph-mon[29997]: created 2024-09-21T03:33:49.528201+0000 2024-09-21T03:33:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:54 smithi016 ceph-mon[29997]: min_mon_release 19 (squid) 2024-09-21T03:33:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:54 smithi016 ceph-mon[29997]: election_strategy: 1 2024-09-21T03:33:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:54 smithi016 ceph-mon[29997]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-21T03:33:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:54 smithi016 ceph-mon[29997]: fsmap 2024-09-21T03:33:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:54 smithi016 ceph-mon[29997]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T03:33:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:54 smithi016 ceph-mon[29997]: mgrmap e1: no daemons active 2024-09-21T03:33:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:54 smithi016 ceph-mon[29997]: from='client.? 172.21.15.16:0/2167294284' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T03:33:54.482 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T03:33:54.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout id: 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:33:54.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T03:33:54.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T03:33:54.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout services: 2024-09-21T03:33:54.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.621565s) 2024-09-21T03:33:54.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T03:33:54.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T03:33:54.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T03:33:54.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout data: 2024-09-21T03:33:54.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T03:33:54.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T03:33:54.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T03:33:54.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T03:33:54.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T03:33:54.485 INFO:teuthology.orchestra.run.smithi016.stdout:mon is available 2024-09-21T03:33:54.485 INFO:teuthology.orchestra.run.smithi016.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T03:33:55.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:55 smithi016 ceph-mon[29997]: from='client.? 172.21.15.16:0/1060544546' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T03:33:55.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:55 smithi016 ceph-mon[29997]: from='client.? 172.21.15.16:0/1060544546' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T03:33:55.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T03:33:55.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T03:33:55.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:33:55.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T03:33:55.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-21T03:33:55.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T03:33:55.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T03:33:55.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T03:33:55.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T03:33:55.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T03:33:55.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T03:33:55.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T03:33:55.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T03:33:55.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T03:33:55.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T03:33:55.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T03:33:55.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T03:33:55.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T03:33:55.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T03:33:55.814 INFO:teuthology.orchestra.run.smithi016.stdout:Generating new minimal ceph.conf... 2024-09-21T03:33:56.723 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:56 smithi016 ceph-mon[29997]: from='client.? 172.21.15.16:0/3741266416' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:57.073 INFO:teuthology.orchestra.run.smithi016.stdout:Restarting the monitor... 2024-09-21T03:33:57.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:57 smithi016 systemd[1]: Stopping Ceph mon.a for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:33:57.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:57 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-a[29992]: 2024-09-21T03:33:57.409+0000 7fae3dc00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:33:57.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:57 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-a[29992]: 2024-09-21T03:33:57.409+0000 7fae3dc00640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T03:33:57.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:57 smithi016 podman[30250]: 2024-09-21 03:33:57.636129795 +0000 UTC m=+0.328088797 container died c5746a01e2902c4b8260cc66d3be6a49f4787b716f3df21e30f8edfe17904fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T03:33:58.318 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:58 smithi016 podman[30250]: 2024-09-21 03:33:58.031625837 +0000 UTC m=+0.723584837 container cleanup c5746a01e2902c4b8260cc66d3be6a49f4787b716f3df21e30f8edfe17904fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T03:33:58.318 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:58 smithi016 bash[30250]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-a 2024-09-21T03:33:58.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:58 smithi016 podman[30264]: 2024-09-21 03:33:58.315912414 +0000 UTC m=+0.673865014 container remove c5746a01e2902c4b8260cc66d3be6a49f4787b716f3df21e30f8edfe17904fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-21T03:33:58.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:58 smithi016 systemd[1]: ceph-2b324840-77ca-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T03:33:58.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:58 smithi016 systemd[1]: Stopped Ceph mon.a for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:33:58.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:58 smithi016 systemd[1]: ceph-2b324840-77ca-11ef-baf5-efdc52797490@mon.a.service: Consumed 1.318s CPU time. 2024-09-21T03:33:58.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:58 smithi016 systemd[1]: Starting Ceph mon.a for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:33:59.137 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:58 smithi016 podman[30356]: 2024-09-21 03:33:58.825107451 +0000 UTC m=+0.123103738 container create 01d63f4758cfefa6f0963cbf38a419697ee266258bf663a230fc8b1635436bb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-a, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T03:33:59.137 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:58 smithi016 podman[30356]: 2024-09-21 03:33:58.729365778 +0000 UTC m=+0.027362082 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:33:59.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 podman[30356]: 2024-09-21 03:33:59.05273863 +0000 UTC m=+0.350734922 container init 01d63f4758cfefa6f0963cbf38a419697ee266258bf663a230fc8b1635436bb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-a, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:33:59.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 podman[30356]: 2024-09-21 03:33:59.059390495 +0000 UTC m=+0.357386782 container start 01d63f4758cfefa6f0963cbf38a419697ee266258bf663a230fc8b1635436bb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T03:33:59.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 bash[30356]: 01d63f4758cfefa6f0963cbf38a419697ee266258bf663a230fc8b1635436bb7 2024-09-21T03:33:59.149 INFO:teuthology.orchestra.run.smithi016.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T03:33:59.387 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 systemd[1]: Started Ceph mon.a for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:33:59.387 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T03:33:59.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T03:33:59.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: pidfile_write: ignore empty --pid-file 2024-09-21T03:33:59.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: load: jerasure load: lrc 2024-09-21T03:33:59.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: RocksDB version: 7.9.2 2024-09-21T03:33:59.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Git sha 0 2024-09-21T03:33:59.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T03:33:59.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: DB SUMMARY 2024-09-21T03:33:59.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: DB Session ID: H14E5E7YG3HQ7ONKHFPG 2024-09-21T03:33:59.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: CURRENT file: CURRENT 2024-09-21T03:33:59.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T03:33:59.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T03:33:59.389 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T03:33:59.389 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 93716 ; 2024-09-21T03:33:59.389 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.error_if_exists: 0 2024-09-21T03:33:59.389 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.create_if_missing: 0 2024-09-21T03:33:59.389 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.paranoid_checks: 1 2024-09-21T03:33:59.389 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T03:33:59.389 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T03:33:59.389 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T03:33:59.389 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.env: 0x5577901e8d60 2024-09-21T03:33:59.389 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T03:33:59.390 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.info_log: 0x5577c8fb1320 2024-09-21T03:33:59.390 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T03:33:59.390 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.statistics: (nil) 2024-09-21T03:33:59.390 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.use_fsync: 0 2024-09-21T03:33:59.390 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_log_file_size: 0 2024-09-21T03:33:59.390 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T03:33:59.390 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T03:33:59.390 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T03:33:59.390 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T03:33:59.390 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.allow_fallocate: 1 2024-09-21T03:33:59.390 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T03:33:59.390 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T03:33:59.391 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.use_direct_reads: 0 2024-09-21T03:33:59.391 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T03:33:59.391 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T03:33:59.391 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.db_log_dir: 2024-09-21T03:33:59.391 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.wal_dir: 2024-09-21T03:33:59.392 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T03:33:59.392 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T03:33:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T03:33:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T03:33:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T03:33:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T03:33:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T03:33:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T03:33:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.write_buffer_manager: 0x5577c8fb5d60 2024-09-21T03:33:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T03:33:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T03:33:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T03:33:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T03:33:59.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T03:33:59.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T03:33:59.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T03:33:59.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T03:33:59.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.unordered_write: 0 2024-09-21T03:33:59.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T03:33:59.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T03:33:59.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T03:33:59.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T03:33:59.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.row_cache: None 2024-09-21T03:33:59.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.wal_filter: None 2024-09-21T03:33:59.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T03:33:59.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T03:33:59.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.two_write_queues: 0 2024-09-21T03:33:59.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T03:33:59.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.wal_compression: 0 2024-09-21T03:33:59.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.atomic_flush: 0 2024-09-21T03:33:59.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T03:33:59.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T03:33:59.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T03:33:59.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.log_readahead_size: 0 2024-09-21T03:33:59.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T03:33:59.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T03:33:59.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T03:33:59.396 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T03:33:59.396 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T03:33:59.396 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T03:33:59.396 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T03:33:59.396 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_background_jobs: 2 2024-09-21T03:33:59.396 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_background_compactions: -1 2024-09-21T03:33:59.396 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_subcompactions: 1 2024-09-21T03:33:59.396 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T03:33:59.396 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T03:33:59.396 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T03:33:59.396 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T03:33:59.396 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T03:33:59.396 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T03:33:59.397 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T03:33:59.397 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T03:33:59.397 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_open_files: -1 2024-09-21T03:33:59.397 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T03:33:59.397 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T03:33:59.397 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T03:33:59.397 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T03:33:59.397 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_background_flushes: -1 2024-09-21T03:33:59.397 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Compression algorithms supported: 2024-09-21T03:33:59.397 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: kZSTD supported: 0 2024-09-21T03:33:59.397 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: kXpressCompression supported: 0 2024-09-21T03:33:59.397 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: kBZip2Compression supported: 0 2024-09-21T03:33:59.398 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T03:33:59.398 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: kLZ4Compression supported: 1 2024-09-21T03:33:59.398 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: kZlibCompression supported: 1 2024-09-21T03:33:59.398 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T03:33:59.398 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: kSnappyCompression supported: 1 2024-09-21T03:33:59.398 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T03:33:59.398 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T03:33:59.398 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T03:33:59.398 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T03:33:59.398 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T03:33:59.398 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.merge_operator: 2024-09-21T03:33:59.398 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compaction_filter: None 2024-09-21T03:33:59.398 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T03:33:59.399 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T03:33:59.399 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T03:33:59.399 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T03:33:59.399 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5577c8fb0f60) 2024-09-21T03:33:59.399 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks: 1 2024-09-21T03:33:59.399 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T03:33:59.399 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T03:33:59.399 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_top_level_index_and_filter: 1 2024-09-21T03:33:59.399 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_type: 0 2024-09-21T03:33:59.399 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_index_type: 0 2024-09-21T03:33:59.399 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_shortening: 1 2024-09-21T03:33:59.399 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T03:33:59.400 INFO:journalctl@ceph.mon.a.smithi016.stdout: checksum: 4 2024-09-21T03:33:59.400 INFO:journalctl@ceph.mon.a.smithi016.stdout: no_block_cache: 0 2024-09-21T03:33:59.400 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache: 0x5577c4f75610 2024-09-21T03:33:59.400 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_name: BinnedLRUCache 2024-09-21T03:33:59.400 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_options: 2024-09-21T03:33:59.400 INFO:journalctl@ceph.mon.a.smithi016.stdout: capacity : 536870912 2024-09-21T03:33:59.400 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_shard_bits : 4 2024-09-21T03:33:59.400 INFO:journalctl@ceph.mon.a.smithi016.stdout: strict_capacity_limit : 0 2024-09-21T03:33:59.400 INFO:journalctl@ceph.mon.a.smithi016.stdout: high_pri_pool_ratio: 0.000 2024-09-21T03:33:59.400 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_compressed: (nil) 2024-09-21T03:33:59.400 INFO:journalctl@ceph.mon.a.smithi016.stdout: persistent_cache: (nil) 2024-09-21T03:33:59.400 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size: 4096 2024-09-21T03:33:59.400 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size_deviation: 10 2024-09-21T03:33:59.401 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_restart_interval: 16 2024-09-21T03:33:59.401 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_block_restart_interval: 1 2024-09-21T03:33:59.401 INFO:journalctl@ceph.mon.a.smithi016.stdout: metadata_block_size: 4096 2024-09-21T03:33:59.401 INFO:journalctl@ceph.mon.a.smithi016.stdout: partition_filters: 0 2024-09-21T03:33:59.401 INFO:journalctl@ceph.mon.a.smithi016.stdout: use_delta_encoding: 1 2024-09-21T03:33:59.401 INFO:journalctl@ceph.mon.a.smithi016.stdout: filter_policy: bloomfilter 2024-09-21T03:33:59.401 INFO:journalctl@ceph.mon.a.smithi016.stdout: whole_key_filtering: 1 2024-09-21T03:33:59.401 INFO:journalctl@ceph.mon.a.smithi016.stdout: verify_compression: 0 2024-09-21T03:33:59.401 INFO:journalctl@ceph.mon.a.smithi016.stdout: read_amp_bytes_per_bit: 0 2024-09-21T03:33:59.401 INFO:journalctl@ceph.mon.a.smithi016.stdout: format_version: 5 2024-09-21T03:33:59.401 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_index_compression: 1 2024-09-21T03:33:59.401 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_align: 0 2024-09-21T03:33:59.401 INFO:journalctl@ceph.mon.a.smithi016.stdout: max_auto_readahead_size: 262144 2024-09-21T03:33:59.402 INFO:journalctl@ceph.mon.a.smithi016.stdout: prepopulate_block_cache: 0 2024-09-21T03:33:59.402 INFO:journalctl@ceph.mon.a.smithi016.stdout: initial_auto_readahead_size: 8192 2024-09-21T03:33:59.402 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T03:33:59.402 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T03:33:59.402 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T03:33:59.402 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compression: NoCompression 2024-09-21T03:33:59.402 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T03:33:59.402 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T03:33:59.402 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T03:33:59.402 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.num_levels: 7 2024-09-21T03:33:59.402 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T03:33:59.402 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T03:33:59.402 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T03:33:59.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T03:33:59.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T03:33:59.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T03:33:59.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T03:33:59.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:33:59.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T03:33:59.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T03:33:59.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:33:59.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:33:59.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T03:33:59.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T03:33:59.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T03:33:59.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T03:33:59.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:33:59.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:33:59.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T03:33:59.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T03:33:59.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:33:59.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T03:33:59.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T03:33:59.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T03:33:59.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T03:33:59.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T03:33:59.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T03:33:59.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T03:33:59.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T03:33:59.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T03:33:59.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T03:33:59.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T03:33:59.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T03:33:59.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T03:33:59.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T03:33:59.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T03:33:59.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T03:33:59.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T03:33:59.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T03:33:59.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T03:33:59.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T03:33:59.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T03:33:59.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T03:33:59.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T03:33:59.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T03:33:59.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T03:33:59.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T03:33:59.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T03:33:59.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T03:33:59.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T03:33:59.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T03:33:59.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T03:33:59.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T03:33:59.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T03:33:59.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.inplace_update_support: 0 2024-09-21T03:33:59.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T03:33:59.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T03:33:59.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T03:33:59.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T03:33:59.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.bloom_locality: 0 2024-09-21T03:33:59.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.max_successive_merges: 0 2024-09-21T03:33:59.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T03:33:59.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T03:33:59.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T03:33:59.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T03:33:59.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.ttl: 2592000 2024-09-21T03:33:59.408 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T03:33:59.408 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T03:33:59.408 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T03:33:59.408 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.enable_blob_files: false 2024-09-21T03:33:59.408 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.min_blob_size: 0 2024-09-21T03:33:59.408 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T03:33:59.408 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T03:33:59.408 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T03:33:59.408 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T03:33:59.408 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T03:33:59.408 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T03:33:59.408 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T03:33:59.409 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T03:33:59.409 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T03:33:59.409 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T03:33:59.409 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 54533796-a125-45a4-8f7a-2298ddb6f756 2024-09-21T03:33:59.409 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889639172602, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T03:33:59.409 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T03:33:59.409 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889639174041, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 89985, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 279, "table_properties": {"data_size": 88055, "index_size": 250, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11333, "raw_average_key_size": 47, "raw_value_size": 81660, "raw_average_value_size": 343, "num_data_blocks": 11, "num_entries": 238, "num_filter_entries": 238, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726889639, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "54533796-a125-45a4-8f7a-2298ddb6f756", "db_session_id": "H14E5E7YG3HQ7ONKHFPG", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T03:33:59.409 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889639174281, "job": 1, "event": "recovery_finished"} 2024-09-21T03:33:59.409 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T03:33:59.409 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T03:33:59.409 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5577c8fd6a80 2024-09-21T03:33:59.409 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: DB pointer 0x5577c9d78000 2024-09-21T03:33:59.410 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T03:33:59.410 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T03:33:59.410 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** DB Stats ** 2024-09-21T03:33:59.410 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:33:59.410 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T03:33:59.410 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:33:59.410 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:33:59.410 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T03:33:59.410 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:33:59.410 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:33:59.410 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-21T03:33:59.410 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-09-21T03:33:59.410 INFO:journalctl@ceph.mon.a.smithi016.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:33:59.411 INFO:journalctl@ceph.mon.a.smithi016.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T03:33:59.411 INFO:journalctl@ceph.mon.a.smithi016.stdout: L0 2/0 89.74 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 104.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:33:59.411 INFO:journalctl@ceph.mon.a.smithi016.stdout: Sum 2/0 89.74 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 104.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:33:59.411 INFO:journalctl@ceph.mon.a.smithi016.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 104.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:33:59.411 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-21T03:33:59.411 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-09-21T03:33:59.411 INFO:journalctl@ceph.mon.a.smithi016.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:33:59.411 INFO:journalctl@ceph.mon.a.smithi016.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T03:33:59.411 INFO:journalctl@ceph.mon.a.smithi016.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 104.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:33:59.411 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-21T03:33:59.411 INFO:journalctl@ceph.mon.a.smithi016.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T03:33:59.411 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-21T03:33:59.411 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:33:59.412 INFO:journalctl@ceph.mon.a.smithi016.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T03:33:59.412 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T03:33:59.412 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T03:33:59.412 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T03:33:59.412 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T03:33:59.412 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative compaction: 0.00 GB write, 13.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:33:59.412 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval compaction: 0.00 GB write, 13.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:33:59.412 INFO:journalctl@ceph.mon.a.smithi016.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T03:33:59.412 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache BinnedLRUCache@0x5577c4f75610#2 capacity: 512.00 MB usage: 1.28 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-21T03:33:59.412 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2024-09-21T03:33:59.412 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-21T03:33:59.412 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T03:33:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: starting mon.a rank 0 at public addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] at bind addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:33:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: mon.a@-1(???) e1 preinit fsid 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:33:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: mon.a@-1(???).mds e1 new map 2024-09-21T03:33:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: mon.a@-1(???).mds e1 print_map 2024-09-21T03:33:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout: e1 2024-09-21T03:33:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout: btime 2024-09-21T03:33:53:220089+0000 2024-09-21T03:33:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:33:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:33:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout: legacy client fscid: -1 2024-09-21T03:33:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-21T03:33:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout: No filesystems configured 2024-09-21T03:33:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T03:33:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:33:59.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:33:59.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:33:59.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T03:33:59.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:33:59.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: monmap epoch 1 2024-09-21T03:33:59.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: fsid 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:33:59.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: last_changed 2024-09-21T03:33:49.528201+0000 2024-09-21T03:33:59.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: created 2024-09-21T03:33:49.528201+0000 2024-09-21T03:33:59.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: min_mon_release 19 (squid) 2024-09-21T03:33:59.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: election_strategy: 1 2024-09-21T03:33:59.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-21T03:33:59.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: fsmap 2024-09-21T03:33:59.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T03:33:59.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:33:59 smithi016 ceph-mon[30375]: mgrmap e1: no daemons active 2024-09-21T03:34:00.435 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T03:34:00.458 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:34:00.459 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mgr... 2024-09-21T03:34:00.459 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T03:34:00.460 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T03:34:00.807 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2b324840-77ca-11ef-baf5-efdc52797490@mgr.x 2024-09-21T03:34:00.807 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2b324840-77ca-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-2b324840-77ca-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T03:34:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2b324840-77ca-11ef-baf5-efdc52797490.target.wants/ceph-2b324840-77ca-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-2b324840-77ca-11ef-baf5-efdc52797490@.service. 2024-09-21T03:34:01.081 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:00 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2328633472' entity='client.admin' 2024-09-21T03:34:01.393 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:01 smithi016 systemd[1]: Starting Ceph mgr.x for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:34:01.663 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:01 smithi016 podman[30585]: 2024-09-21 03:34:01.296515971 +0000 UTC m=+0.027856220 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:34:01.663 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:01 smithi016 podman[30585]: 2024-09-21 03:34:01.501717588 +0000 UTC m=+0.233057835 container create 60b9b29bbfdb3e39283c364c4d4b0945b949580715779e9375dfe205519ee428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T03:34:01.800 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:34:01.800 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-21T03:34:01.815 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:34:01.815 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-21T03:34:01.815 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-21T03:34:01.815 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T03:34:01.833 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:34:01.833 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-21T03:34:01.847 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:34:01.847 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-21T03:34:01.847 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-21T03:34:01.847 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T03:34:01.847 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr to start... 2024-09-21T03:34:01.847 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr... 2024-09-21T03:34:01.981 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:01 smithi016 podman[30585]: 2024-09-21 03:34:01.704227109 +0000 UTC m=+0.435567381 container init 60b9b29bbfdb3e39283c364c4d4b0945b949580715779e9375dfe205519ee428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:34:01.981 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:01 smithi016 podman[30585]: 2024-09-21 03:34:01.710461745 +0000 UTC m=+0.441801993 container start 60b9b29bbfdb3e39283c364c4d4b0945b949580715779e9375dfe205519ee428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:34:01.982 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:01 smithi016 bash[30585]: 60b9b29bbfdb3e39283c364c4d4b0945b949580715779e9375dfe205519ee428 2024-09-21T03:34:01.982 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:01 smithi016 systemd[1]: Started Ceph mgr.x for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:34:02.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:02 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1566738535' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:34:03.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T03:34:03.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:34:03.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:34:03.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:34:03.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:34:03.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:34:03.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:34:03.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:03.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:34:03.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:34:03.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:34:03.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:34:03.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:34:03.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:34:03.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:34:03.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-21T03:34:03.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:34:03.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:34:03.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:34:03.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:34:03.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:03.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:34:03.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:34:03.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:34:03.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:34:03.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:34:03.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:34:03.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:34:03.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:34:03.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:03.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:34:03.122 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:34:03.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:34:03.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:34:03.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:34:03.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:34:03.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:34:03.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:34:03.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:34:03.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:03.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:34:03.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:34:03.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:33:53:220089+0000", 2024-09-21T03:34:03.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:34:03.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:34:03.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:03.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:34:03.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T03:34:03.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:34:03.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:34:03.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:34:03.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:34:03.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:34:03.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:34:03.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:34:03.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:03.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:34:03.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:34:03.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:33:53.221157+0000", 2024-09-21T03:34:03.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:34:03.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:03.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:34:03.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:34:03.127 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (1/15)... 2024-09-21T03:34:05.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:05 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1486216291' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:34:06.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T03:34:06.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:34:06.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:34:06.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:06.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:34:06.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:33:53:220089+0000", 2024-09-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:33:53.221157+0000", 2024-09-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:34:06.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:34:06.399 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (2/15)... 2024-09-21T03:34:08.526 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:08 smithi016 ceph-mon[30375]: Activating manager daemon x 2024-09-21T03:34:08.526 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:08 smithi016 ceph-mon[30375]: mgrmap e2: x(active, starting, since 0.00583292s) 2024-09-21T03:34:08.526 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:08 smithi016 ceph-mon[30375]: from='mgr.14100 172.21.15.16:0/2510815636' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:34:08.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:08 smithi016 ceph-mon[30375]: from='mgr.14100 172.21.15.16:0/2510815636' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:34:08.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:08 smithi016 ceph-mon[30375]: from='mgr.14100 172.21.15.16:0/2510815636' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:34:08.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:08 smithi016 ceph-mon[30375]: from='mgr.14100 172.21.15.16:0/2510815636' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:34:08.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:08 smithi016 ceph-mon[30375]: from='mgr.14100 172.21.15.16:0/2510815636' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:34:08.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:08 smithi016 ceph-mon[30375]: Manager daemon x is now available 2024-09-21T03:34:08.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:08 smithi016 ceph-mon[30375]: from='mgr.14100 172.21.15.16:0/2510815636' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:34:08.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:08 smithi016 ceph-mon[30375]: from='mgr.14100 172.21.15.16:0/2510815636' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:34:08.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:08 smithi016 ceph-mon[30375]: from='mgr.14100 172.21.15.16:0/2510815636' entity='mgr.x' 2024-09-21T03:34:08.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:08 smithi016 ceph-mon[30375]: from='mgr.14100 172.21.15.16:0/2510815636' entity='mgr.x' 2024-09-21T03:34:08.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:08 smithi016 ceph-mon[30375]: from='mgr.14100 172.21.15.16:0/2510815636' entity='mgr.x' 2024-09-21T03:34:09.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:09 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/4146466661' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:34:09.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:09 smithi016 ceph-mon[30375]: mgrmap e3: x(active, since 1.01008s) 2024-09-21T03:34:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T03:34:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:34:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:34:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:34:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:34:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:34:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:34:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:34:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:34:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:34:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:34:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:34:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:34:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:34:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-21T03:34:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:34:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:34:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:34:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:34:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:34:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:34:09.665 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:34:09.665 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:34:09.665 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:34:09.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:34:09.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:34:09.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:34:09.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:09.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:34:09.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:34:09.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:34:09.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:34:09.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:34:09.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:34:09.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:34:09.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:34:09.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:34:09.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:09.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:34:09.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:34:09.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:33:53:220089+0000", 2024-09-21T03:34:09.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:34:09.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:34:09.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:09.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:34:09.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T03:34:09.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:34:09.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:34:09.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:34:09.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:34:09.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:34:09.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:34:09.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:34:09.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:09.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:34:09.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:34:09.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:33:53.221157+0000", 2024-09-21T03:34:09.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:34:09.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:09.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:34:09.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:34:09.671 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (3/15)... 2024-09-21T03:34:11.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:11 smithi016 ceph-mon[30375]: mgrmap e4: x(active, since 2s) 2024-09-21T03:34:12.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:12 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3428206230' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:34:13.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T03:34:13.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:34:13.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:34:13.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:34:13.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:34:13.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:34:13.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:34:13.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:13.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:34:13.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:34:13.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:34:13.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:34:13.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:34:13.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:34:13.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:34:13.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-09-21T03:34:13.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:34:13.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:34:13.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:34:13.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:34:13.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:13.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:34:13.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:34:13.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:34:13.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:34:13.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:34:13.475 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:34:13.475 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:34:13.475 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:34:13.475 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:13.475 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:34:13.475 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:34:13.475 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:34:13.475 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:34:13.475 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:34:13.475 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:34:13.475 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:34:13.475 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:34:13.475 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:34:13.476 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:13.476 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:34:13.476 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:34:13.476 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:33:53:220089+0000", 2024-09-21T03:34:13.476 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:34:13.476 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:34:13.476 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:13.476 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:34:13.476 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:34:13.476 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:34:13.476 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:34:13.476 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:34:13.476 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:34:13.476 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:34:13.476 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:34:13.477 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:34:13.477 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:13.477 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:34:13.477 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:34:13.477 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:33:53.221157+0000", 2024-09-21T03:34:13.477 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:34:13.477 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:34:13.477 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:34:13.477 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:34:13.477 INFO:teuthology.orchestra.run.smithi016.stdout:mgr is available 2024-09-21T03:34:14.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:14 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/4108049276' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T03:34:15.201 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T03:34:15.201 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T03:34:15.201 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:34:15.201 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T03:34:15.201 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-21T03:34:15.201 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T03:34:15.201 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T03:34:15.201 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T03:34:15.201 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T03:34:15.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T03:34:15.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T03:34:15.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T03:34:15.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T03:34:15.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T03:34:15.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T03:34:15.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T03:34:15.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T03:34:15.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T03:34:15.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T03:34:15.202 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling cephadm module... 2024-09-21T03:34:16.390 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:16 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3371246460' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T03:34:16.391 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:16 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: ignoring --setuser ceph since I am not root 2024-09-21T03:34:16.391 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:16 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: ignoring --setgroup ceph since I am not root 2024-09-21T03:34:17.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:17 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3371246460' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T03:34:17.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:17 smithi016 ceph-mon[30375]: mgrmap e5: x(active, since 7s) 2024-09-21T03:34:18.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:18 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1461213721' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T03:34:18.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:34:18.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T03:34:18.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:34:18.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T03:34:18.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T03:34:18.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:34:18.223 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-09-21T03:34:18.223 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 5... 2024-09-21T03:34:22.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:22 smithi016 ceph-mon[30375]: Active manager daemon x restarted 2024-09-21T03:34:22.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:22 smithi016 ceph-mon[30375]: Activating manager daemon x 2024-09-21T03:34:22.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:22 smithi016 ceph-mon[30375]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T03:34:22.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:22 smithi016 ceph-mon[30375]: mgrmap e6: x(active, starting, since 0.00400472s) 2024-09-21T03:34:22.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:22 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:34:22.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:22 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:34:22.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:22 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:34:22.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:22 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:34:22.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:22 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:34:22.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:22 smithi016 ceph-mon[30375]: Manager daemon x is now available 2024-09-21T03:34:24.192 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:34:24.192 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T03:34:24.192 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T03:34:24.192 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:34:24.192 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 5 is available 2024-09-21T03:34:24.192 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying orchestrator module is enabled... 2024-09-21T03:34:24.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' 2024-09-21T03:34:24.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' 2024-09-21T03:34:24.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-mon[30375]: Found migration_current of "None". Setting to last migration. 2024-09-21T03:34:24.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' 2024-09-21T03:34:24.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' 2024-09-21T03:34:24.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:24.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:24.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:34:24.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:34:24.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-mon[30375]: mgrmap e7: x(active, since 1.0072s) 2024-09-21T03:34:25.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: 2024-09-21T03:34:24.733+0000 7f2125600640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:34:24] ENGINE Error in HTTPServer.serve 2024-09-21T03:34:25.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: Traceback (most recent call last): 2024-09-21T03:34:25.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:34:25.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: self._connections.run(self.expiration_interval) 2024-09-21T03:34:25.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:34:25.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: self._run(expiration_interval) 2024-09-21T03:34:25.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:34:25.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:34:25.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:34:25.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:34:25.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:34:25.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: s = self.context.wrap_socket( 2024-09-21T03:34:25.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:34:25.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: return self.sslsocket_class._create( 2024-09-21T03:34:25.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:34:25.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: self.do_handshake() 2024-09-21T03:34:25.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:34:25.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: self._sslobj.do_handshake() 2024-09-21T03:34:25.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:34:25.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:24 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: 2024-09-21T03:34:25.562 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:25 smithi016 ceph-mon[30375]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T03:34:25.563 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:25 smithi016 ceph-mon[30375]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T03:34:25.563 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:25 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:25.563 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:25 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1641855617' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T03:34:25.840 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T03:34:25.840 INFO:teuthology.orchestra.run.smithi016.stdout:Setting orchestrator backend to cephadm... 2024-09-21T03:34:26.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:26 smithi016 ceph-mon[30375]: [21/Sep/2024:03:34:24] ENGINE Bus STARTING 2024-09-21T03:34:26.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:26 smithi016 ceph-mon[30375]: [21/Sep/2024:03:34:24] ENGINE Serving on https://172.21.15.16:7150 2024-09-21T03:34:26.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:26 smithi016 ceph-mon[30375]: [21/Sep/2024:03:34:24] ENGINE Error in HTTPServer.serve 2024-09-21T03:34:26.413 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-09-21T03:34:26.413 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:34:26.413 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:34:26.413 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:34:26.413 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-09-21T03:34:26.413 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:34:26.413 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:34:26.413 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:34:26.413 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:34:26.414 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:34:26.414 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-09-21T03:34:26.414 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:34:26.414 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-09-21T03:34:26.414 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:34:26.414 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-09-21T03:34:26.414 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:34:26.414 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-09-21T03:34:26.414 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:34:26.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:26 smithi016 ceph-mon[30375]: [21/Sep/2024:03:34:24] ENGINE Serving on http://172.21.15.16:8765 2024-09-21T03:34:26.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:26 smithi016 ceph-mon[30375]: [21/Sep/2024:03:34:24] ENGINE Bus STARTED 2024-09-21T03:34:26.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:26 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1641855617' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T03:34:26.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:26 smithi016 ceph-mon[30375]: mgrmap e8: x(active, since 2s) 2024-09-21T03:34:27.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:27 smithi016 ceph-mon[30375]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:34:27.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:27 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' 2024-09-21T03:34:27.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:27 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:28.652 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T03:34:28.652 INFO:teuthology.orchestra.run.smithi016.stdout:Generating ssh key... 2024-09-21T03:34:28.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:28 smithi016 ceph-mon[30375]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:34:30.162 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: Generating public/private rsa key pair. 2024-09-21T03:34:30.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: Your identification has been saved in /tmp/tmpk14gukvj/key 2024-09-21T03:34:30.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: Your public key has been saved in /tmp/tmpk14gukvj/key.pub 2024-09-21T03:34:30.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: The key fingerprint is: 2024-09-21T03:34:30.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: SHA256:wl1AdVXwllHSkhagkfDRRtV+lAihyLBXh11GTRJWZ3E ceph-2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:34:30.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: The key's randomart image is: 2024-09-21T03:34:30.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: +---[RSA 3072]----+ 2024-09-21T03:34:30.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: | . .++*BO@%%E| 2024-09-21T03:34:30.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: | + o++*=.=*O| 2024-09-21T03:34:30.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: | . + .+. . =o| 2024-09-21T03:34:30.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: | ... . .o| 2024-09-21T03:34:30.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: | o S .| 2024-09-21T03:34:30.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: | . | 2024-09-21T03:34:30.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: | | 2024-09-21T03:34:30.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: | | 2024-09-21T03:34:30.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: | | 2024-09-21T03:34:30.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: +----[SHA256]-----+ 2024-09-21T03:34:31.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:30 smithi016 ceph-mon[30375]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:34:31.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:30 smithi016 ceph-mon[30375]: Generating ssh key... 2024-09-21T03:34:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:30 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' 2024-09-21T03:34:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:30 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' 2024-09-21T03:34:31.890 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClXqmCuOfNVvczBKOPGPNjVjx40lzizRG0XFza2hx8CkXHFKt8ghaPDQqSesW345rE7q0jSBL71TJVQr8ksN/IGbMJgJ3dpWOU7UiU1UbI0ZlbdpPrdlXfBQXvqGX9Bd6m1uiVgAGxa/PMKM09LSu9XqD33vHJzRlNFAv1nkei7cF4p6o1TPJnDgLKnEI7ntTjDgvU2BgXGe8ZfeS8JQPql44PYRWlHiD51FFdpAvV8avwV5MB70Re157Q14FrGE4K34GVqLwa7+QNP+hqgAUpgLS5Wn+kGyqX7gAcMTGwKpOP9dNodWYGYrjTWUbo7XMdE/KDWJqoySgp8JZIxusNgn4FQd+p/+9roUTaAxCZ6+/I4baEMisNFohOMc3mzbyxyX23jg5XDs47qviEeCJ4I2aGUCSN4hGBWwZIjSTgUnYH+Zs16cg9bN+Ls6kjkfZZ/XJk3j735FR8PAXP3Mp6D3MumqDl0hfghC9JCP15VVM+l8JtuVlnM5zXSvper80= ceph-2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:34:31.891 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:34:31.891 INFO:teuthology.orchestra.run.smithi016.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T03:34:31.896 INFO:teuthology.orchestra.run.smithi016.stdout:Adding host smithi016... 2024-09-21T03:34:32.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:31 smithi016 ceph-mon[30375]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:34:33.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:33 smithi016 ceph-mon[30375]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:34:34.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:34 smithi016 ceph-mon[30375]: Deploying cephadm binary to smithi016 2024-09-21T03:34:37.622 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Added host 'smithi016' with addr '172.21.15.16' 2024-09-21T03:34:37.622 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mon service... 2024-09-21T03:34:38.287 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:37 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' 2024-09-21T03:34:38.287 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:37 smithi016 ceph-mon[30375]: Added host smithi016 2024-09-21T03:34:38.287 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:37 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:39.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:39 smithi016 ceph-mon[30375]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:34:39.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:39 smithi016 ceph-mon[30375]: Saving service mon spec with placement count:5 2024-09-21T03:34:39.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:39 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' 2024-09-21T03:34:40.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T03:34:40.189 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mgr service... 2024-09-21T03:34:41.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:40 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' 2024-09-21T03:34:41.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:40 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' 2024-09-21T03:34:42.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:42 smithi016 ceph-mon[30375]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:34:42.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:42 smithi016 ceph-mon[30375]: Saving service mgr spec with placement count:2 2024-09-21T03:34:42.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:42 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' 2024-09-21T03:34:42.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:42 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' 2024-09-21T03:34:42.147 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T03:34:44.263 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:44 smithi016 ceph-mon[30375]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:34:44.263 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:44 smithi016 ceph-mon[30375]: from='mgr.14120 172.21.15.16:0/2189871257' entity='mgr.x' 2024-09-21T03:34:44.263 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:44 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2866477861' entity='client.admin' 2024-09-21T03:34:45.850 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling the dashboard module... 2024-09-21T03:34:46.147 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:45 smithi016 ceph-mon[30375]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:34:46.148 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:45 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2798241137' entity='client.admin' 2024-09-21T03:34:47.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:46 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3082386384' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T03:34:47.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:46 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: ignoring --setuser ceph since I am not root 2024-09-21T03:34:47.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:46 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: ignoring --setgroup ceph since I am not root 2024-09-21T03:34:48.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:47 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3082386384' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T03:34:48.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:47 smithi016 ceph-mon[30375]: mgrmap e9: x(active, since 24s) 2024-09-21T03:34:48.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:48 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3049502516' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T03:34:48.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:34:48.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-21T03:34:48.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:34:48.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T03:34:48.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T03:34:48.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:34:48.990 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-09-21T03:34:48.990 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 9... 2024-09-21T03:34:53.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:53 smithi016 ceph-mon[30375]: Active manager daemon x restarted 2024-09-21T03:34:53.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:53 smithi016 ceph-mon[30375]: Activating manager daemon x 2024-09-21T03:34:53.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:53 smithi016 ceph-mon[30375]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T03:34:53.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:53 smithi016 ceph-mon[30375]: mgrmap e10: x(active, starting, since 0.00466037s) 2024-09-21T03:34:53.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:34:53.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:34:53.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:34:53.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:34:53.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:34:53.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:53 smithi016 ceph-mon[30375]: Manager daemon x is now available 2024-09-21T03:34:54.589 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:54.589 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:34:54.589 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:34:54.590 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:54 smithi016 ceph-mon[30375]: mgrmap e11: x(active, since 1.01036s) 2024-09-21T03:34:54.936 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:34:54.936 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-21T03:34:54.936 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T03:34:54.937 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:34:54.937 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 9 is available 2024-09-21T03:34:54.937 INFO:teuthology.orchestra.run.smithi016.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T03:34:55.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-mon[30375]: from='client.14158 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T03:34:55.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-mon[30375]: from='client.14158 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T03:34:55.818 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: 2024-09-21T03:34:55.549+0000 7efca7a00640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:34:55] ENGINE Error in HTTPServer.serve 2024-09-21T03:34:55.819 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: Traceback (most recent call last): 2024-09-21T03:34:55.819 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:34:55.819 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: self._connections.run(self.expiration_interval) 2024-09-21T03:34:55.819 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:34:55.819 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: self._run(expiration_interval) 2024-09-21T03:34:55.819 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:34:55.819 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:34:55.819 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:34:55.819 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:34:55.819 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:34:55.819 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: s = self.context.wrap_socket( 2024-09-21T03:34:55.819 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:34:55.820 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: return self.sslsocket_class._create( 2024-09-21T03:34:55.820 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:34:55.820 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: self.do_handshake() 2024-09-21T03:34:55.820 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:34:55.820 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: self._sslobj.do_handshake() 2024-09-21T03:34:55.820 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:34:55.820 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:34:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: 2024-09-21T03:34:56.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:56 smithi016 ceph-mon[30375]: mgrmap e12: x(active, since 2s) 2024-09-21T03:34:56.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:56 smithi016 ceph-mon[30375]: [21/Sep/2024:03:34:55] ENGINE Bus STARTING 2024-09-21T03:34:56.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:56 smithi016 ceph-mon[30375]: [21/Sep/2024:03:34:55] ENGINE Serving on https://172.21.15.16:7150 2024-09-21T03:34:56.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:56 smithi016 ceph-mon[30375]: [21/Sep/2024:03:34:55] ENGINE Error in HTTPServer.serve 2024-09-21T03:34:56.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-09-21T03:34:56.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:34:56.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:34:56.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:34:56.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-09-21T03:34:56.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:34:56.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:34:56.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:34:56.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:34:56.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:34:56.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-09-21T03:34:56.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:34:56.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-09-21T03:34:56.664 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:34:56.664 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-09-21T03:34:56.664 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:34:56.664 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-09-21T03:34:56.664 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:34:56.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:56 smithi016 ceph-mon[30375]: [21/Sep/2024:03:34:55] ENGINE Serving on http://172.21.15.16:8765 2024-09-21T03:34:56.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:56 smithi016 ceph-mon[30375]: [21/Sep/2024:03:34:55] ENGINE Bus STARTED 2024-09-21T03:34:56.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:56 smithi016 ceph-mon[30375]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:34:56.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:56 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:34:58.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:34:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:34:59.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUe1sk8ASe4LYTmCE8L3iYaqpgHAEwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTAzMzQ1OFoXDTM0\nMDkyMjAzMzQ1OFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjE2MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEA4dAwevEQz0AXLRZK0y0yr68DikL457uxxR6G\nCROD9/OnyXCix1bZiCkbl3El6FlG5cO5hRkjWF2IFs7rI0hf72Q0GXJoaQLt1l31\nKQQIKo2aNwWKKJgVsRNf7LgMU6/WGoy0gUog4vOSdXUoMeJtjtxSzE2cBqoDX6PJ\nC2e/574cBS4OfLuC3TPeipRfCmmDiLupC73xg/ZWcbMFrQI9QrayDljR9oZx9WP+\n2JIzwc4gNF4O4mk1mfgO3IdKOgbZSCAvT6k5arpzkk6fkmkMX+ZA93eChQXZKovF\nfegSS7iMpN9PHe+c4z37whm4V4MX63RFgd7hfEJKuph2YAD9egemESeloVIlbhlw\n96x/lEwVnKAruq5LxspovFtfHRTh5lmCuSGs3/JzcxS0/J+pEaPb5oXJ4qcYx71y\nAcXkvw6tO4rJavmpB3XDoiT5IP7OaSNiwxG6FLr6kPXygb8aZbiHtla0tR2eAbal\nQIQI46Ra+fTVhQwTso+ry9m7wsg1r/pVik4PANPR6ePnFqMEaKEQNT8ySOjISi15\n33xqbRk+FhZ5cZtJMoAZXXnL1/GD+avCpAnIUj0B3j8sz6TKWvVp4EToJs717ktU\n5+CgO161DUQUUXTmCxdSIXP+cPeXH/3Zt6TLeqYKj1sAvfuFwWX6PSjqMqYXeZ7h\nApIL3KcCAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDE2ghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPEDAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQCp\nwsvnIm169DyD2648pEOzXcRQqPIVQ/tGEnwc2meP6MF45luyFtXUDl1WaXqHZUfM\nv5Ga19+WgR4oB7nSanWYMwafFd1DqrZMzfD1wgygO89Alo2BFoui42VRvfQtFjbT\n4L3UNjy1mwRhDNSH1ze0pX2wfUCulW1CyLDJEojjcp3wPtMTpboE8remHktSn/7T\n92WTFaIHJNLzPZP9TjpH7kRvMD3CNoVVDPsL1Dy1fb6P9lfeYJjii4I6dw8IDwgc\nqkzvWeXNvfoATza+vTy6Rkcrv6IhZlMI+A2nnLcBx7VMEkJXZuteuDh9vtAF8EKK\nbAKys53H6TbunWSP/vwL3LJ05qf320DCtLdh4+Re77qi0vZX0fKDwgoEH2LNJV2r\nzPz13LnBl7pvlaNboUBvMoMHjcPFlPfCMIuIU7pRerdoSCYHmwQi4DSe/4NI3UwV\nKwtlhs0zusU3weQawqwmQzEdWj884Q2Z3bTDQFyUYOOGZwjYpyO7D1lL7wI3MWWK\nQ+V0hvrILjXXpuXNYuYvAXMTtgo2F3fVoFt5wQhJv8iMJTTryLq+f94nS1F91TIW\n7gZ9HmZbFxpzYRY+cP0sWyZn1Sz+6FvaYIy7zxxXZRYTAwfdO4e2lxtiVfawYhV2\n1a3oV2ORZoriMf5wMnPIemdQg2WrN3c/ez7Hs79AoA==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKQIBAAKCAgEA4dAwevEQz0AXLRZK0y0yr68DikL457uxxR6GCROD9/OnyXCi\nx1bZiCkbl3El6FlG5cO5hRkjWF2IFs7rI0hf72Q0GXJoaQLt1l31KQQIKo2aNwWK\nKJgVsRNf7LgMU6/WGoy0gUog4vOSdXUoMeJtjtxSzE2cBqoDX6PJC2e/574cBS4O\nfLuC3TPeipRfCmmDiLupC73xg/ZWcbMFrQI9QrayDljR9oZx9WP+2JIzwc4gNF4O\n4mk1mfgO3IdKOgbZSCAvT6k5arpzkk6fkmkMX+ZA93eChQXZKovFfegSS7iMpN9P\nHe+c4z37whm4V4MX63RFgd7hfEJKuph2YAD9egemESeloVIlbhlw96x/lEwVnKAr\nuq5LxspovFtfHRTh5lmCuSGs3/JzcxS0/J+pEaPb5oXJ4qcYx71yAcXkvw6tO4rJ\navmpB3XDoiT5IP7OaSNiwxG6FLr6kPXygb8aZbiHtla0tR2eAbalQIQI46Ra+fTV\nhQwTso+ry9m7wsg1r/pVik4PANPR6ePnFqMEaKEQNT8ySOjISi1533xqbRk+FhZ5\ncZtJMoAZXXnL1/GD+avCpAnIUj0B3j8sz6TKWvVp4EToJs717ktU5+CgO161DUQU\nUXTmCxdSIXP+cPeXH/3Zt6TLeqYKj1sAvfuFwWX6PSjqMqYXeZ7hApIL3KcCAwEA\nAQKCAgBZ6EfdM00Tj3uKzpTlJtpG+8JHkunCbkdV6hTI2VXSa9dSy39sKca7wU5E\nil7i92TUh4y+kycxADB2kFUru53C1V0vAo/Su8FReCBr/7h2pOGRZanABumgEsw/\nFnqF+X0gHgSic47LfdpC3O4vim0/k7Z9tKraNmfYdQBPg7PzQ4ypfVB0Vj5A7218\nNJ3wqdkeoEUqsSpPdc2uQG6Sl1En1lDniHK0mowV/YXS2cc0s2q8HvUDLYuTbaIR\nKOajWuW5I2Ni7sg3F9x7FMXRB/VO24DjZth3OUBhgbwUqMlcD7Mj0+uOyj/ndF7g\nRdhRM5da/nTMRB74F/bj+18qU0qYQ6Do65WP20ME91WYTdRVLssAvODmrlw4huUQ\njCfywkDkuTQmOiqH+0kgmSYdnsyxgTjUrmGtgsinnTlP9gpY26AJWKrO+nGptKek\nPdN4CCMSCWvniv20TwoPpPc5ApXPYZ84n/9K43BGgjz2buPqn5F26g3B+03yuwse\nhat98CvN2Bi8hPP/v7h2drxOymx9dsJeozK7UGEPpIYiInt11eW0UT0Eo3wciEV6\nSlWFKgG5CDQoze3p5pp3sIgHlcZr2lnED5GkmT+ZztAsGX6yQimp6vLi4dCCy7GL\n68EzPFtRhqi+LYw+jQCXLKRxI6nOFKiY6lWas2FGk7jW/vaaKQKCAQEA9cPd3n2t\nRNsWOu/rKOu7AC+0TPUXAJyeMOuS8vYuTDAqqT5WBcfkJfKUfSdpakrlpPEoh5r2\nXwnh4Tuc2a29WZR2GUccBB56Ul8XIB28MvzdI0I+OtWGEc0xPtzgVXgxWDelq/++\nasdcpw5IEbRbKWQtGixFVXU6TImBtgUEds5SopfEICnqdt2SKeWAHWeTU7mpe943\nAraIfr6piV+DGQ90iOF5Mioe+YC86ybczv8O7KNqooyRtTuTARalJBE5gZCIPtrj\ntn4WvuZW8+QnCzSXJVR/9stvwmA5xFFsvo1467bNrLwe0BUTtA7fX/5ClKhWfnDA\njvefdt1IVYauyQKCAQEA6zedAf46b9T/gQpL9i1Pc9YY1MeEFHlDf/XKudIP3oCZ\nvLBfBkiSgqam/bdpLfUjedHCfTYAqLaZ3crxmOwlU3hFRGqus3LqW6jbbyGXiDS2\npgABiqKnmkQYFLD7I800uuaDLVvtHHDqleQbVyVQMgx+AkaXJHAN34uViwRsw3Ur\nGiwhbulN1ian6zC6nSVql3m+U9vJ2zwGyNrdLLC9PXDbdRo3PoywJc6TmVDNwR91\nHCqZkEtvuRNUtwXnQtJHky3JcaBC+9uQZWPFJXkUFlJsUCOe/7xtHFy+ipgu5khP\nl6jjTfb6TbB9wRgz6Xtrs5dbPAX9sHu5/InazPW37wKCAQAoO+WRrA92q0tLYMWl\n0Fmv4R9qTeFzJzC6n4ZSIj1YGbaSOHNKxPuzvq/SgWY/kX1g8F4LPTJCQtLo1aKK\n/Gd4R9bBsqOChR7PZ2PYzSVnT4BAanLXnOcy+N2sBerv8xpY79s+ekLJJlQjwTHJ\nPDaEYzMDCuZa5DMkHorOqqmdbY+XsLLZ9SK+TO3F0B2WMsOqvEVj0vA8kHduEHae\n17UVoDMayZbWwJCLlijXcmhJtvXRhbtKWyrreZJKZlOowO0iKfA0Ig+Pa7r7i6gG\ni35c1kI8SUYduBZWwvwSCKZ915lJNYccVTvSIvEz1A97npt1zM1EK99KSH2hrv5N\nihwJAoIBAQCJAI92xbkLwig/5Ntxu9p3QdCcvsjzk934/hTiPrP3iTgCa0gtxL9x\nUdgaF2Yv2oSbQ83gWD/yBMmCId6uj3z+LSNLQkgXRWuzj5qgFBTKxMlNztsXAL5W\nYCsWgi8FH1UDN5hYJx7oXeyE+aPjElyvig8rDCqkko/CEO89VYp0Hh7I08N1cnZf\nsYxEtNE9DroTGFn/RhlAlKnu+2uvyBGxY7avcmGTESbYHoZh2mq9fbEcgsOHBClC\nxzukjs+wYAJIe3bFs+7P+4cTRxVhSjCk/8UOuHC3y+xRhMOGvjUy9BNH5preomgQ\nMChO++XgrH6C0DnXsDWfQbGGolTK5KQJAoIBAQCYJjqfKAmZG2bQW031exiRAKK4\n0zGNRTY6Ki4kAHjVj3IBR/C4vQOa8bx88fUE0NQQxBq5NpX+q6aIcTwRYzUlQlIN\nsuqerdk/BJUlPGW7QAI1ilmSjHmuEIwHG6Fdm03le1+WPAGXbT14JKXnn0wP6T5h\n0jIhPeVIQypT27OPpwhjwi4FShp+JILKv9P3EMfF98LIJ2TjG4wIwEcnr4m8Bpzb\nyJBzMd1rfiz6hqS9BgWfZO1J75qLX9KR/9NbEyM34gXXCkTAzXNoyGTcp7t66dgO\nNPBGIFYihkwz7Ku5BVWFrczMaLkSY1JpCabRkEAj5tPnn7fCPuhhWefmu/jG\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T03:35:00.815 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T03:35:01.346 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:01 smithi016 ceph-mon[30375]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:01.347 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:01 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:02.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T03:35:02.349 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial admin user... 2024-09-21T03:35:02.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:02 smithi016 ceph-mon[30375]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:02.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:02 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:04.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$QosMNeNxfHz6MNUHqeXUIe2fl1.tKQe0dJbnBUMnJUP/Mjyg3idGq", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726889703, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T03:35:04.005 INFO:teuthology.orchestra.run.smithi016.stdout:Fetching dashboard port number... 2024-09-21T03:35:04.526 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:04 smithi016 ceph-mon[30375]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:04.526 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:04 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:05.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:05 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3678573544' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T03:35:05.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T03:35:05.512 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:35:05.512 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-21T03:35:05.526 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:35:05.526 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-21T03:35:05.526 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-21T03:35:05.526 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T03:35:05.528 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph Dashboard is now available at: 2024-09-21T03:35:05.528 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:35:05.528 INFO:teuthology.orchestra.run.smithi016.stdout: URL: https://smithi016.front.sepia.ceph.com:8443/ 2024-09-21T03:35:05.529 INFO:teuthology.orchestra.run.smithi016.stdout: User: admin 2024-09-21T03:35:05.529 INFO:teuthology.orchestra.run.smithi016.stdout: Password: 9a8j0qnkm4 2024-09-21T03:35:05.529 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:35:05.529 INFO:teuthology.orchestra.run.smithi016.stdout:Saving cluster configuration to /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/config directory 2024-09-21T03:35:06.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:06 smithi016 ceph-mon[30375]: mgrmap e13: x(active, since 12s) 2024-09-21T03:35:07.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T03:35:07.050 INFO:teuthology.orchestra.run.smithi016.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T03:35:07.051 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:35:07.051 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:35:07.051 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:35:07.051 INFO:teuthology.orchestra.run.smithi016.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T03:35:07.051 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:35:07.051 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T03:35:07.051 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:35:07.051 INFO:teuthology.orchestra.run.smithi016.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T03:35:07.051 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:35:07.051 INFO:teuthology.orchestra.run.smithi016.stdout: ceph telemetry on 2024-09-21T03:35:07.051 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:35:07.051 INFO:teuthology.orchestra.run.smithi016.stdout:For more information see: 2024-09-21T03:35:07.051 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:35:07.051 INFO:teuthology.orchestra.run.smithi016.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T03:35:07.052 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:35:07.052 INFO:teuthology.orchestra.run.smithi016.stdout:Bootstrap complete. 2024-09-21T03:35:07.110 INFO:tasks.cephadm:Fetching config... 2024-09-21T03:35:07.110 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:35:07.110 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T03:35:07.146 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T03:35:07.146 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:35:07.146 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T03:35:07.223 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T03:35:07.224 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:35:07.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T03:35:07.320 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T03:35:07.320 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:35:07.320 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T03:35:07.395 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T03:35:07.396 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClXqmCuOfNVvczBKOPGPNjVjx40lzizRG0XFza2hx8CkXHFKt8ghaPDQqSesW345rE7q0jSBL71TJVQr8ksN/IGbMJgJ3dpWOU7UiU1UbI0ZlbdpPrdlXfBQXvqGX9Bd6m1uiVgAGxa/PMKM09LSu9XqD33vHJzRlNFAv1nkei7cF4p6o1TPJnDgLKnEI7ntTjDgvU2BgXGe8ZfeS8JQPql44PYRWlHiD51FFdpAvV8avwV5MB70Re157Q14FrGE4K34GVqLwa7+QNP+hqgAUpgLS5Wn+kGyqX7gAcMTGwKpOP9dNodWYGYrjTWUbo7XMdE/KDWJqoySgp8JZIxusNgn4FQd+p/+9roUTaAxCZ6+/I4baEMisNFohOMc3mzbyxyX23jg5XDs47qviEeCJ4I2aGUCSN4hGBWwZIjSTgUnYH+Zs16cg9bN+Ls6kjkfZZ/XJk3j735FR8PAXP3Mp6D3MumqDl0hfghC9JCP15VVM+l8JtuVlnM5zXSvper80= ceph-2b324840-77ca-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:35:07.534 INFO:teuthology.orchestra.run.smithi016.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClXqmCuOfNVvczBKOPGPNjVjx40lzizRG0XFza2hx8CkXHFKt8ghaPDQqSesW345rE7q0jSBL71TJVQr8ksN/IGbMJgJ3dpWOU7UiU1UbI0ZlbdpPrdlXfBQXvqGX9Bd6m1uiVgAGxa/PMKM09LSu9XqD33vHJzRlNFAv1nkei7cF4p6o1TPJnDgLKnEI7ntTjDgvU2BgXGe8ZfeS8JQPql44PYRWlHiD51FFdpAvV8avwV5MB70Re157Q14FrGE4K34GVqLwa7+QNP+hqgAUpgLS5Wn+kGyqX7gAcMTGwKpOP9dNodWYGYrjTWUbo7XMdE/KDWJqoySgp8JZIxusNgn4FQd+p/+9roUTaAxCZ6+/I4baEMisNFohOMc3mzbyxyX23jg5XDs47qviEeCJ4I2aGUCSN4hGBWwZIjSTgUnYH+Zs16cg9bN+Ls6kjkfZZ/XJk3j735FR8PAXP3Mp6D3MumqDl0hfghC9JCP15VVM+l8JtuVlnM5zXSvper80= ceph-2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:07.559 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClXqmCuOfNVvczBKOPGPNjVjx40lzizRG0XFza2hx8CkXHFKt8ghaPDQqSesW345rE7q0jSBL71TJVQr8ksN/IGbMJgJ3dpWOU7UiU1UbI0ZlbdpPrdlXfBQXvqGX9Bd6m1uiVgAGxa/PMKM09LSu9XqD33vHJzRlNFAv1nkei7cF4p6o1TPJnDgLKnEI7ntTjDgvU2BgXGe8ZfeS8JQPql44PYRWlHiD51FFdpAvV8avwV5MB70Re157Q14FrGE4K34GVqLwa7+QNP+hqgAUpgLS5Wn+kGyqX7gAcMTGwKpOP9dNodWYGYrjTWUbo7XMdE/KDWJqoySgp8JZIxusNgn4FQd+p/+9roUTaAxCZ6+/I4baEMisNFohOMc3mzbyxyX23jg5XDs47qviEeCJ4I2aGUCSN4hGBWwZIjSTgUnYH+Zs16cg9bN+Ls6kjkfZZ/XJk3j735FR8PAXP3Mp6D3MumqDl0hfghC9JCP15VVM+l8JtuVlnM5zXSvper80= ceph-2b324840-77ca-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:35:07.600 INFO:teuthology.orchestra.run.smithi123.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClXqmCuOfNVvczBKOPGPNjVjx40lzizRG0XFza2hx8CkXHFKt8ghaPDQqSesW345rE7q0jSBL71TJVQr8ksN/IGbMJgJ3dpWOU7UiU1UbI0ZlbdpPrdlXfBQXvqGX9Bd6m1uiVgAGxa/PMKM09LSu9XqD33vHJzRlNFAv1nkei7cF4p6o1TPJnDgLKnEI7ntTjDgvU2BgXGe8ZfeS8JQPql44PYRWlHiD51FFdpAvV8avwV5MB70Re157Q14FrGE4K34GVqLwa7+QNP+hqgAUpgLS5Wn+kGyqX7gAcMTGwKpOP9dNodWYGYrjTWUbo7XMdE/KDWJqoySgp8JZIxusNgn4FQd+p/+9roUTaAxCZ6+/I4baEMisNFohOMc3mzbyxyX23jg5XDs47qviEeCJ4I2aGUCSN4hGBWwZIjSTgUnYH+Zs16cg9bN+Ls6kjkfZZ/XJk3j735FR8PAXP3Mp6D3MumqDl0hfghC9JCP15VVM+l8JtuVlnM5zXSvper80= ceph-2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:07.613 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClXqmCuOfNVvczBKOPGPNjVjx40lzizRG0XFza2hx8CkXHFKt8ghaPDQqSesW345rE7q0jSBL71TJVQr8ksN/IGbMJgJ3dpWOU7UiU1UbI0ZlbdpPrdlXfBQXvqGX9Bd6m1uiVgAGxa/PMKM09LSu9XqD33vHJzRlNFAv1nkei7cF4p6o1TPJnDgLKnEI7ntTjDgvU2BgXGe8ZfeS8JQPql44PYRWlHiD51FFdpAvV8avwV5MB70Re157Q14FrGE4K34GVqLwa7+QNP+hqgAUpgLS5Wn+kGyqX7gAcMTGwKpOP9dNodWYGYrjTWUbo7XMdE/KDWJqoySgp8JZIxusNgn4FQd+p/+9roUTaAxCZ6+/I4baEMisNFohOMc3mzbyxyX23jg5XDs47qviEeCJ4I2aGUCSN4hGBWwZIjSTgUnYH+Zs16cg9bN+Ls6kjkfZZ/XJk3j735FR8PAXP3Mp6D3MumqDl0hfghC9JCP15VVM+l8JtuVlnM5zXSvper80= ceph-2b324840-77ca-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:35:07.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:07 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3949374427' entity='client.admin' 2024-09-21T03:35:07.655 INFO:teuthology.orchestra.run.smithi134.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClXqmCuOfNVvczBKOPGPNjVjx40lzizRG0XFza2hx8CkXHFKt8ghaPDQqSesW345rE7q0jSBL71TJVQr8ksN/IGbMJgJ3dpWOU7UiU1UbI0ZlbdpPrdlXfBQXvqGX9Bd6m1uiVgAGxa/PMKM09LSu9XqD33vHJzRlNFAv1nkei7cF4p6o1TPJnDgLKnEI7ntTjDgvU2BgXGe8ZfeS8JQPql44PYRWlHiD51FFdpAvV8avwV5MB70Re157Q14FrGE4K34GVqLwa7+QNP+hqgAUpgLS5Wn+kGyqX7gAcMTGwKpOP9dNodWYGYrjTWUbo7XMdE/KDWJqoySgp8JZIxusNgn4FQd+p/+9roUTaAxCZ6+/I4baEMisNFohOMc3mzbyxyX23jg5XDs47qviEeCJ4I2aGUCSN4hGBWwZIjSTgUnYH+Zs16cg9bN+Ls6kjkfZZ/XJk3j735FR8PAXP3Mp6D3MumqDl0hfghC9JCP15VVM+l8JtuVlnM5zXSvper80= ceph-2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:07.668 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T03:35:08.034 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:09.976 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:09 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3729170135' entity='client.admin' 2024-09-21T03:35:09.976 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:09 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:09.976 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:09 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:09.976 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:09 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:09.976 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:09 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:09.976 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:09 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:09.976 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:09 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:10.011 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T03:35:10.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T03:35:10.612 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:12.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:12 smithi016 ceph-mon[30375]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:12.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:12 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:12.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:12 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:12.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:12 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:12.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:12 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:12.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:12 smithi016 ceph-mon[30375]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-21T03:35:12.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:12 smithi016 ceph-mon[30375]: Updating smithi016:/var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:35:12.872 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi123 2024-09-21T03:35:12.872 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T03:35:12.872 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.conf 2024-09-21T03:35:12.891 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T03:35:12.891 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:35:12.949 INFO:tasks.cephadm:Adding host smithi123 to orchestrator... 2024-09-21T03:35:12.949 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch host add smithi123 2024-09-21T03:35:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:13.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:13 smithi016 ceph-mon[30375]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:35:13.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:13 smithi016 ceph-mon[30375]: Updating smithi016:/var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:35:13.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:13.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:13.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:14.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:14 smithi016 ceph-mon[30375]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:15.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:15 smithi016 ceph-mon[30375]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:16.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:16 smithi016 ceph-mon[30375]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:16.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:16 smithi016 ceph-mon[30375]: Deploying cephadm binary to smithi123 2024-09-21T03:35:17.932 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi123' with addr '172.21.15.123' 2024-09-21T03:35:18.722 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T03:35:19.094 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:19.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:18 smithi016 ceph-mon[30375]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:19.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:18 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:19.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:18 smithi016 ceph-mon[30375]: Added host smithi123 2024-09-21T03:35:19.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:18 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:19.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:18 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:19.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:18 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:20.027 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:35:20.028 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}] 2024-09-21T03:35:20.335 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:20 smithi016 ceph-mon[30375]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:20.335 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:20 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:20.666 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi134 2024-09-21T03:35:20.666 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T03:35:20.667 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.conf 2024-09-21T03:35:20.687 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T03:35:20.687 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:35:20.745 INFO:tasks.cephadm:Adding host smithi134 to orchestrator... 2024-09-21T03:35:20.745 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch host add smithi134 2024-09-21T03:35:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:21.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:21 smithi016 ceph-mon[30375]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:35:22.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:22 smithi016 ceph-mon[30375]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:22.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:22 smithi016 ceph-mon[30375]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:22.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:22.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:22.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:22.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:22.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:22.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:22.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:23.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:23 smithi016 ceph-mon[30375]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-21T03:35:23.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:23 smithi016 ceph-mon[30375]: Updating smithi123:/var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:35:23.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:23 smithi016 ceph-mon[30375]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:35:23.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:23 smithi016 ceph-mon[30375]: Updating smithi123:/var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:35:23.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:23 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:23.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:23 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:23.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:23 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:24.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:24 smithi016 ceph-mon[30375]: Deploying cephadm binary to smithi134 2024-09-21T03:35:24.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:24 smithi016 ceph-mon[30375]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:25.921 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi134' with addr '172.21.15.134' 2024-09-21T03:35:26.686 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T03:35:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:27.228 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:26 smithi016 ceph-mon[30375]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:27.228 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:26 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:27.228 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:26 smithi016 ceph-mon[30375]: Added host smithi134 2024-09-21T03:35:27.228 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:26 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:27.228 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:26 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:27.228 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:26 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:27.898 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:35:27.898 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}] 2024-09-21T03:35:28.529 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:28 smithi016 ceph-mon[30375]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:28.529 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:28 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:28.529 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:28 smithi016 ceph-mon[30375]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:35:28.531 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T03:35:28.531 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T03:35:28.898 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-09-21T03:35:30.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:30 smithi016 ceph-mon[30375]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:30.839 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:30 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3466084945' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T03:35:31.014 INFO:tasks.cephadm:Adding mon.a on smithi016 2024-09-21T03:35:31.014 INFO:tasks.cephadm:Adding mon.b on smithi123 2024-09-21T03:35:31.014 INFO:tasks.cephadm:Adding mon.c on smithi134 2024-09-21T03:35:31.014 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi016:172.21.15.16=a;smithi123:172.21.15.123=b;smithi134:172.21.15.134=c' 2024-09-21T03:35:31.248 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:35:31.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:31 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3466084945' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T03:35:31.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:31 smithi016 ceph-mon[30375]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:35:31.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:31 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:31.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:31 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:31.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:31 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:31.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:31 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:31.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:31 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:31.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:31 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:31.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:31 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:31.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:31 smithi016 ceph-mon[30375]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-21T03:35:31.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:31 smithi016 ceph-mon[30375]: Updating smithi134:/var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:35:31.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:31 smithi016 ceph-mon[30375]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:35:31.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:31 smithi016 ceph-mon[30375]: Updating smithi134:/var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:35:31.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:31 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:31.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:31 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:31.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:31 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:31.869 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mon update... 2024-09-21T03:35:32.457 DEBUG:teuthology.orchestra.run.smithi123:mon.b> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T03:35:32.460 DEBUG:teuthology.orchestra.run.smithi134:mon.c> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T03:35:32.462 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:35:32.463 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:35:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:32 smithi016 ceph-mon[30375]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:32 smithi016 ceph-mon[30375]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi123:172.21.15.123=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:32 smithi016 ceph-mon[30375]: Saving service mon spec with placement smithi016:172.21.15.16=a;smithi123:172.21.15.123=b;smithi134:172.21.15.134=c;count:3 2024-09-21T03:35:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:32 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:32 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:32 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:32 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:32 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:32 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:35:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:32 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:32 smithi016 ceph-mon[30375]: Deploying daemon mon.c on smithi134 2024-09-21T03:35:33.560 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:35:35.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:34 smithi016 ceph-mon[30375]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:35.477 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:35:35.477 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":1,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","modified":"2024-09-21T03:33:49.528201Z","created":"2024-09-21T03:33:49.528201Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T03:35:35.478 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 1 2024-09-21T03:35:36.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:35 smithi016 ceph-mon[30375]: from='client.? 172.21.15.134:0/542462451' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:35:36.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:36 smithi134 systemd[1]: Started Ceph mon.c for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:35:37.053 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:35:37.053 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:35:37.306 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:35:41.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:35:41.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:35:41.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: mon.a calling monitor election 2024-09-21T03:35:41.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:41.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:35:41.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:35:41.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: mon.c calling monitor election 2024-09-21T03:35:41.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:41.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:35:41.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:35:41.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:35:41.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:35:41.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: monmap epoch 2 2024-09-21T03:35:41.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: fsid 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:41.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: last_changed 2024-09-21T03:35:36.504349+0000 2024-09-21T03:35:41.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: created 2024-09-21T03:33:49.528201+0000 2024-09-21T03:35:41.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: min_mon_release 19 (squid) 2024-09-21T03:35:41.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: election_strategy: 1 2024-09-21T03:35:41.854 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-21T03:35:41.854 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-09-21T03:35:41.854 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: fsmap 2024-09-21T03:35:41.854 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:35:41.854 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: mgrmap e13: x(active, since 48s) 2024-09-21T03:35:41.854 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: overall HEALTH_OK 2024-09-21T03:35:41.854 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:41.854 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:41.854 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:41.854 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:35:41.854 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:41.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:35:41.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:35:41.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: mon.a calling monitor election 2024-09-21T03:35:41.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:41.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:35:41.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:35:41.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: mon.c calling monitor election 2024-09-21T03:35:41.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:41.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:35:41.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:35:41.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:35:41.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:35:41.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: monmap epoch 2 2024-09-21T03:35:41.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: fsid 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:41.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: last_changed 2024-09-21T03:35:36.504349+0000 2024-09-21T03:35:41.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: created 2024-09-21T03:33:49.528201+0000 2024-09-21T03:35:41.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: min_mon_release 19 (squid) 2024-09-21T03:35:41.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: election_strategy: 1 2024-09-21T03:35:41.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-21T03:35:41.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-09-21T03:35:41.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: fsmap 2024-09-21T03:35:41.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:35:41.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: mgrmap e13: x(active, since 48s) 2024-09-21T03:35:41.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: overall HEALTH_OK 2024-09-21T03:35:41.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:41.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:41.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:41.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:35:41.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:42.417 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:35:42.417 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":2,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","modified":"2024-09-21T03:35:36.504349Z","created":"2024-09-21T03:33:49.528201Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T03:35:42.417 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 2 2024-09-21T03:35:42.715 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:42 smithi134 ceph-mon[35031]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:42.715 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:42 smithi134 ceph-mon[35031]: Deploying daemon mon.b on smithi123 2024-09-21T03:35:42.715 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:42 smithi134 ceph-mon[35031]: from='client.? 172.21.15.134:0/3515229273' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:35:42.716 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:42 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:35:42.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:42 smithi016 ceph-mon[30375]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:42.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:42 smithi016 ceph-mon[30375]: Deploying daemon mon.b on smithi123 2024-09-21T03:35:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:42 smithi016 ceph-mon[30375]: from='client.? 172.21.15.134:0/3515229273' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:35:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:42 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:35:44.012 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:35:44.012 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:35:44.252 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:35:44.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:44 smithi134 ceph-mon[35031]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:44.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:44 smithi016 ceph-mon[30375]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:44.964 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:35:44.964 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":2,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","modified":"2024-09-21T03:35:36.504349Z","created":"2024-09-21T03:33:49.528201Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T03:35:44.964 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 2 2024-09-21T03:35:45.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:45 smithi123 systemd[1]: Started Ceph mon.b for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:35:46.576 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:35:46.577 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:35:46.828 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:35:50.536 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:35:50.536 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":3,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","modified":"2024-09-21T03:35:45.217691Z","created":"2024-09-21T03:33:49.528201Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T03:35:50.537 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 3 2024-09-21T03:35:50.537 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:35:50.537 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: mon.a calling monitor election 2024-09-21T03:35:50.537 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: mon.c calling monitor election 2024-09-21T03:35:50.537 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:50.537 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:35:50.537 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:50.537 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:50.537 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:50.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:50.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:50.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:50.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:50.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:50.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:35:50.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: monmap epoch 3 2024-09-21T03:35:50.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: fsid 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:50.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: last_changed 2024-09-21T03:35:45.217691+0000 2024-09-21T03:35:50.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: created 2024-09-21T03:33:49.528201+0000 2024-09-21T03:35:50.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: min_mon_release 19 (squid) 2024-09-21T03:35:50.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: election_strategy: 1 2024-09-21T03:35:50.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-21T03:35:50.539 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-09-21T03:35:50.539 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-09-21T03:35:50.539 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: fsmap 2024-09-21T03:35:50.539 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:35:50.539 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: mgrmap e13: x(active, since 57s) 2024-09-21T03:35:50.539 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: overall HEALTH_OK 2024-09-21T03:35:50.539 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:50.539 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:50.539 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:50.539 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:50.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:35:50.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: mon.a calling monitor election 2024-09-21T03:35:50.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: mon.c calling monitor election 2024-09-21T03:35:50.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:50.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:35:50.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:50.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:50.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:50.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:50.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:50.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:50.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:50.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:50.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:35:50.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: monmap epoch 3 2024-09-21T03:35:50.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: fsid 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:50.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: last_changed 2024-09-21T03:35:45.217691+0000 2024-09-21T03:35:50.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: created 2024-09-21T03:33:49.528201+0000 2024-09-21T03:35:50.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: min_mon_release 19 (squid) 2024-09-21T03:35:50.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: election_strategy: 1 2024-09-21T03:35:50.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-21T03:35:50.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-09-21T03:35:50.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-09-21T03:35:50.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: fsmap 2024-09-21T03:35:50.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:35:50.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: mgrmap e13: x(active, since 57s) 2024-09-21T03:35:50.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: overall HEALTH_OK 2024-09-21T03:35:50.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:50.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:50.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:50.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:51.115 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T03:35:51.115 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T03:35:51.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:51 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:51.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:51 smithi134 ceph-mon[35031]: from='client.? 172.21.15.134:0/2588197489' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:35:51.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:51 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:51.501 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:51.640 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:51 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:51.640 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:51 smithi016 ceph-mon[30375]: from='client.? 172.21.15.134:0/2588197489' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:35:51.640 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:51 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:52.347 INFO:teuthology.orchestra.run.smithi016.stdout:# minimal ceph.conf for 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:52.347 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-21T03:35:52.348 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:52.348 INFO:teuthology.orchestra.run.smithi016.stdout: mon_host = [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-21T03:35:52.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:35:52.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: mon.a calling monitor election 2024-09-21T03:35:52.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: mon.c calling monitor election 2024-09-21T03:35:52.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:52.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:35:52.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:52.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:52.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:52.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:52.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:52.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:52.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:52.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:52.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:35:52.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: monmap epoch 3 2024-09-21T03:35:52.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: fsid 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:52.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: last_changed 2024-09-21T03:35:45.217691+0000 2024-09-21T03:35:52.658 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: created 2024-09-21T03:33:49.528201+0000 2024-09-21T03:35:52.658 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: min_mon_release 19 (squid) 2024-09-21T03:35:52.658 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: election_strategy: 1 2024-09-21T03:35:52.658 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-21T03:35:52.658 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-09-21T03:35:52.658 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-09-21T03:35:52.658 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: fsmap 2024-09-21T03:35:52.658 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:35:52.658 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: mgrmap e13: x(active, since 57s) 2024-09-21T03:35:52.658 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: overall HEALTH_OK 2024-09-21T03:35:52.658 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:52.658 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:52.658 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:52.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:52.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:52.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: from='client.? 172.21.15.134:0/2588197489' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:35:52.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:52.952 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T03:35:52.953 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:35:52.953 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:35:53.020 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:35:53.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:35:53.117 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T03:35:53.117 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:35:53.150 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T03:35:53.150 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:35:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: mon.b calling monitor election 2024-09-21T03:35:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: Updating smithi123:/var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:35:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: Updating smithi016:/var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:35:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: mon.b calling monitor election 2024-09-21T03:35:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: mon.c calling monitor election 2024-09-21T03:35:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: mon.a calling monitor election 2024-09-21T03:35:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:35:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: monmap epoch 3 2024-09-21T03:35:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: fsid 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: last_changed 2024-09-21T03:35:45.217691+0000 2024-09-21T03:35:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: created 2024-09-21T03:33:49.528201+0000 2024-09-21T03:35:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: min_mon_release 19 (squid) 2024-09-21T03:35:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: election_strategy: 1 2024-09-21T03:35:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-21T03:35:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-09-21T03:35:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-09-21T03:35:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: fsmap 2024-09-21T03:35:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:35:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: mgrmap e13: x(active, since 59s) 2024-09-21T03:35:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: overall HEALTH_OK 2024-09-21T03:35:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/901668441' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:35:53.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:35:53.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:35:53.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:53.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:53 smithi016 ceph-mon[30375]: Reconfiguring daemon mon.a on smithi016 2024-09-21T03:35:53.221 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T03:35:53.229 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:35:53.258 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T03:35:53.285 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:35:53.330 INFO:tasks.cephadm:Adding mgr.x on smithi016 2024-09-21T03:35:53.330 INFO:tasks.cephadm:Adding mgr.y on smithi123 2024-09-21T03:35:53.330 INFO:tasks.cephadm:Adding mgr.z on smithi134 2024-09-21T03:35:53.330 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi016=x;smithi123=y;smithi134=z' 2024-09-21T03:35:53.397 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: mon.b calling monitor election 2024-09-21T03:35:53.397 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: Updating smithi123:/var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:35:53.397 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: Updating smithi016:/var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:35:53.397 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: mon.b calling monitor election 2024-09-21T03:35:53.397 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: mon.c calling monitor election 2024-09-21T03:35:53.397 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: mon.a calling monitor election 2024-09-21T03:35:53.397 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:35:53.397 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: monmap epoch 3 2024-09-21T03:35:53.398 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: fsid 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:53.398 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: last_changed 2024-09-21T03:35:45.217691+0000 2024-09-21T03:35:53.398 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: created 2024-09-21T03:33:49.528201+0000 2024-09-21T03:35:53.398 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: min_mon_release 19 (squid) 2024-09-21T03:35:53.398 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: election_strategy: 1 2024-09-21T03:35:53.398 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-21T03:35:53.398 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-09-21T03:35:53.398 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-09-21T03:35:53.398 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: fsmap 2024-09-21T03:35:53.398 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:35:53.398 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: mgrmap e13: x(active, since 59s) 2024-09-21T03:35:53.398 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: overall HEALTH_OK 2024-09-21T03:35:53.398 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/901668441' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:53.399 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.399 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.399 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.399 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.399 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.399 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.399 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.399 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:35:53.399 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:35:53.399 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:35:53.399 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:53.399 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:53 smithi134 ceph-mon[35031]: Reconfiguring daemon mon.a on smithi016 2024-09-21T03:35:53.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: mon.b calling monitor election 2024-09-21T03:35:53.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: Updating smithi123:/var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:35:53.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: Updating smithi016:/var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:35:53.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: mon.b calling monitor election 2024-09-21T03:35:53.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: mon.c calling monitor election 2024-09-21T03:35:53.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: mon.a calling monitor election 2024-09-21T03:35:53.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:35:53.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: monmap epoch 3 2024-09-21T03:35:53.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: fsid 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:35:53.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: last_changed 2024-09-21T03:35:45.217691+0000 2024-09-21T03:35:53.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: created 2024-09-21T03:33:49.528201+0000 2024-09-21T03:35:53.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: min_mon_release 19 (squid) 2024-09-21T03:35:53.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: election_strategy: 1 2024-09-21T03:35:53.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-21T03:35:53.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-09-21T03:35:53.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-09-21T03:35:53.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: fsmap 2024-09-21T03:35:53.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:35:53.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: mgrmap e13: x(active, since 59s) 2024-09-21T03:35:53.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: overall HEALTH_OK 2024-09-21T03:35:53.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/901668441' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:53.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:53.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:35:53.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:35:53.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:35:53.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:53.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:53 smithi123 ceph-mon[34673]: Reconfiguring daemon mon.a on smithi016 2024-09-21T03:35:53.612 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:35:54.235 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mgr update... 2024-09-21T03:35:54.247 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:54.254 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:54 smithi134 ceph-mon[35031]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:54.255 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:54.297 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:54.298 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:54 smithi016 ceph-mon[30375]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:54.298 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:54.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:35:54.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:54 smithi123 ceph-mon[34673]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:54.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:54.818 DEBUG:teuthology.orchestra.run.smithi123:mgr.y> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T03:35:54.821 DEBUG:teuthology.orchestra.run.smithi134:mgr.z> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T03:35:54.863 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T03:35:54.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T03:35:54.867 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T03:35:54.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T03:35:54.917 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T03:35:54.917 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T03:35:54.965 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T03:35:54.965 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T03:35:55.012 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T03:35:55.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T03:35:55.059 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T03:35:55.059 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T03:35:55.107 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T03:35:55.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T03:35:55.165 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T03:35:55.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T03:35:55.216 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T03:35:55.217 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T03:35:55.272 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T03:35:55.272 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T03:35:55.330 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T03:35:55.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T03:35:55.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:55 smithi016 ceph-mon[30375]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi123=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:55.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:55 smithi016 ceph-mon[30375]: Saving service mgr spec with placement smithi016=x;smithi123=y;smithi134=z;count:3 2024-09-21T03:35:55.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:55 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:55.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:55 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:55.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:55 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:55.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:55 smithi016 ceph-mon[30375]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:35:55.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:55 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:35:55.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:55 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:35:55.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:55 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:55.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:55 smithi016 ceph-mon[30375]: Reconfiguring daemon mgr.x on smithi016 2024-09-21T03:35:55.368 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T03:35:55.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T03:35:55.417 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T03:35:55.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T03:35:55.469 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T03:35:55.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T03:35:55.521 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T03:35:55.522 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T03:35:55.583 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T03:35:55.583 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T03:35:55.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:55 smithi134 ceph-mon[35031]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi123=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:55.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:55 smithi134 ceph-mon[35031]: Saving service mgr spec with placement smithi016=x;smithi123=y;smithi134=z;count:3 2024-09-21T03:35:55.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:55 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:55.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:55 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:55.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:55 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:55.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:55 smithi134 ceph-mon[35031]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:35:55.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:55 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:35:55.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:55 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:35:55.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:55 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:55.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:55 smithi134 ceph-mon[35031]: Reconfiguring daemon mgr.x on smithi016 2024-09-21T03:35:55.646 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T03:35:55.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T03:35:55.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:55 smithi123 ceph-mon[34673]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi123=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:55.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:55 smithi123 ceph-mon[34673]: Saving service mgr spec with placement smithi016=x;smithi123=y;smithi134=z;count:3 2024-09-21T03:35:55.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:55 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:55.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:55 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:55.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:55 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:55.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:55 smithi123 ceph-mon[34673]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:35:55.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:55 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:35:55.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:55 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:35:55.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:55 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:55.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:55 smithi123 ceph-mon[34673]: Reconfiguring daemon mgr.x on smithi016 2024-09-21T03:35:55.701 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T03:35:55.701 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T03:35:55.752 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T03:35:55.752 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T03:35:55.806 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T03:35:55.806 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T03:35:55.863 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T03:35:55.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T03:35:55.930 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T03:35:55.931 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T03:35:55.989 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T03:35:55.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T03:35:56.042 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T03:35:56.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T03:35:56.104 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T03:35:56.104 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T03:35:56.160 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T03:35:56.160 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T03:35:56.216 DEBUG:tasks.cephadm:set 26 configs 2024-09-21T03:35:56.217 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi016') 2024-09-21T03:35:56.525 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:56 smithi016 ceph-mon[30375]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:56.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:56 smithi123 ceph-mon[34673]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:56.676 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:56.751 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:56.802 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:56.828 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:56.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:56 smithi134 ceph-mon[35031]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:56.967 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:56.974 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:57.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:57.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:57.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:57 smithi016 ceph-mon[30375]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:35:57.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:35:57.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:35:57.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:57.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:57 smithi016 ceph-mon[30375]: Reconfiguring daemon mon.b on smithi123 2024-09-21T03:35:58.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:58.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:58.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:57 smithi123 ceph-mon[34673]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:35:58.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:35:58.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:35:58.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:58.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:57 smithi123 ceph-mon[34673]: Reconfiguring daemon mon.b on smithi123 2024-09-21T03:35:58.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:58.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:58.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:57 smithi134 ceph-mon[35031]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:35:58.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:35:58.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:35:58.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:58.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:57 smithi134 ceph-mon[35031]: Reconfiguring daemon mon.b on smithi123 2024-09-21T03:35:58.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.474+0000 7f5f00400640 1 -- 172.21.15.16:0/1504719140 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f5ef80ffb00 con 0x7f5ef81046d0 2024-09-21T03:35:58.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.481+0000 7f5ef6000640 1 -- 172.21.15.16:0/1504719140 <== mon.1 v2:172.21.15.134:3300/0 8 ==== config(23 keys) ==== 1014+0+0 (secure 0 0 0) 0x7f5ee401fe30 con 0x7f5ef81046d0 2024-09-21T03:35:58.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.481+0000 7f5ef6000640 1 -- 172.21.15.16:0/1504719140 <== mon.1 v2:172.21.15.134:3300/0 9 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v11)=0 v11) ==== 123+0+0 (secure 0 0 0) 0x7f5ee4013c20 con 0x7f5ef81046d0 2024-09-21T03:35:58.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.484+0000 7f5f00400640 1 -- 172.21.15.16:0/1504719140 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5ee003fb20 msgr2=0x7f5ee0042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:58.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.484+0000 7f5f00400640 1 --2- 172.21.15.16:0/1504719140 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5ee003fb20 0x7f5ee0042030 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5eec000c00 tx=0x7f5eec004d00 comp rx=0 tx=0).stop 2024-09-21T03:35:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.484+0000 7f5f00400640 1 -- 172.21.15.16:0/1504719140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef81046d0 msgr2=0x7f5ef81a0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.484+0000 7f5f00400640 1 --2- 172.21.15.16:0/1504719140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef81046d0 0x7f5ef81a0000 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400ea60 tx=0x7f5ee400ee60 comp rx=0 tx=0).stop 2024-09-21T03:35:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.484+0000 7f5f00400640 1 -- 172.21.15.16:0/1504719140 shutdown_connections 2024-09-21T03:35:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.484+0000 7f5f00400640 1 --2- 172.21.15.16:0/1504719140 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5ee003fb20 0x7f5ee0042030 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.484+0000 7f5f00400640 1 --2- 172.21.15.16:0/1504719140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ef8107270 0x7f5ef81a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.484+0000 7f5f00400640 1 --2- 172.21.15.16:0/1504719140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef81046d0 0x7f5ef81a0000 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.484+0000 7f5f00400640 1 --2- 172.21.15.16:0/1504719140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef806c660 0x7f5ef819f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.484+0000 7f5f00400640 1 -- 172.21.15.16:0/1504719140 >> 172.21.15.16:0/1504719140 conn(0x7f5ef80fdfe0 msgr2=0x7f5ef8112870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.484+0000 7f5f00400640 1 -- 172.21.15.16:0/1504719140 shutdown_connections 2024-09-21T03:35:58.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.484+0000 7f5f00400640 1 -- 172.21.15.16:0/1504719140 wait complete. 2024-09-21T03:35:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.519+0000 7f2966800640 1 -- 172.21.15.16:0/314851657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29601068d0 msgr2=0x7f2960108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.519+0000 7f2966800640 1 --2- 172.21.15.16:0/314851657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29601068d0 0x7f2960108d30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f294c009860 tx=0x7f294c02a340 comp rx=0 tx=0).stop 2024-09-21T03:35:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.519+0000 7f2966800640 1 -- 172.21.15.16:0/314851657 shutdown_connections 2024-09-21T03:35:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.519+0000 7f2966800640 1 --2- 172.21.15.16:0/314851657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2960109470 0x7f296010b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.519+0000 7f2966800640 1 --2- 172.21.15.16:0/314851657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29601068d0 0x7f2960108d30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.519+0000 7f2966800640 1 --2- 172.21.15.16:0/314851657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f296006b980 0x7f2960106390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.519+0000 7f2966800640 1 -- 172.21.15.16:0/314851657 >> 172.21.15.16:0/314851657 conn(0x7f29600fdfb0 msgr2=0x7f2960100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.519+0000 7f2966800640 1 -- 172.21.15.16:0/314851657 shutdown_connections 2024-09-21T03:35:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.519+0000 7f2966800640 1 -- 172.21.15.16:0/314851657 wait complete. 2024-09-21T03:35:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.521+0000 7f2966800640 1 Processor -- start 2024-09-21T03:35:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.521+0000 7f2966800640 1 -- start start 2024-09-21T03:35:58.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.522+0000 7f2966800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f296006b980 0x7f296019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:58.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.522+0000 7f2966800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29601068d0 0x7f296019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:58.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.522+0000 7f2966800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2960109470 0x7f296019d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:58.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.522+0000 7f2966800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f296010f000 con 0x7f2960109470 2024-09-21T03:35:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.522+0000 7f2966800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f296010ee80 con 0x7f29601068d0 2024-09-21T03:35:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.522+0000 7f2966800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f296010f180 con 0x7f296006b980 2024-09-21T03:35:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.522+0000 7f295ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29601068d0 0x7f296019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.522+0000 7f295f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f296006b980 0x7f296019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.522+0000 7f295fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2960109470 0x7f296019d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.522+0000 7f295ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29601068d0 0x7f296019c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:45826/0 (socket says 172.21.15.16:45826) 2024-09-21T03:35:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.522+0000 7f295f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f296006b980 0x7f296019bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:46938/0 (socket says 172.21.15.16:46938) 2024-09-21T03:35:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.522+0000 7f295ea00640 1 -- 172.21.15.16:0/2475298269 learned_addr learned my addr 172.21.15.16:0/2475298269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:35:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.522+0000 7f295ea00640 1 -- 172.21.15.16:0/2475298269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f296006b980 msgr2=0x7f296019bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.522+0000 7f295ea00640 1 --2- 172.21.15.16:0/2475298269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f296006b980 0x7f296019bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.522+0000 7f295ea00640 1 -- 172.21.15.16:0/2475298269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2960109470 msgr2=0x7f296019d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.522+0000 7f295ea00640 1 --2- 172.21.15.16:0/2475298269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2960109470 0x7f296019d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.522+0000 7f295ea00640 1 -- 172.21.15.16:0/2475298269 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296019d960 con 0x7f29601068d0 2024-09-21T03:35:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.523+0000 7f295ea00640 1 --2- 172.21.15.16:0/2475298269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29601068d0 0x7f296019c5a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f294c02a930 tx=0x7f294c032d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.523+0000 7f2957e00640 1 -- 172.21.15.16:0/2475298269 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c03f020 con 0x7f29601068d0 2024-09-21T03:35:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.523+0000 7f2957e00640 1 -- 172.21.15.16:0/2475298269 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(23 keys) ==== 1014+0+0 (secure 0 0 0) 0x7f294c0044b0 con 0x7f29601068d0 2024-09-21T03:35:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.523+0000 7f2966800640 1 -- 172.21.15.16:0/2475298269 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f296019dbf0 con 0x7f29601068d0 2024-09-21T03:35:58.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.523+0000 7f2957e00640 1 -- 172.21.15.16:0/2475298269 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c037040 con 0x7f29601068d0 2024-09-21T03:35:58.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.523+0000 7f2966800640 1 -- 172.21.15.16:0/2475298269 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29601a4180 con 0x7f29601068d0 2024-09-21T03:35:58.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.524+0000 7f2957e00640 1 -- 172.21.15.16:0/2475298269 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f294c042020 con 0x7f29601068d0 2024-09-21T03:35:58.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.524+0000 7f2966800640 1 -- 172.21.15.16:0/2475298269 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2960106d20 con 0x7f29601068d0 2024-09-21T03:35:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.525+0000 7f2957e00640 1 --2- 172.21.15.16:0/2475298269 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f294403fb20 0x7f2944042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.525+0000 7f2957e00640 1 -- 172.21.15.16:0/2475298269 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f294c030970 con 0x7f29601068d0 2024-09-21T03:35:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.525+0000 7f295f400640 1 --2- 172.21.15.16:0/2475298269 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f294403fb20 0x7f2944042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.526+0000 7f295f400640 1 --2- 172.21.15.16:0/2475298269 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f294403fb20 0x7f2944042030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2950005fd0 tx=0x7f2950009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.531+0000 7f2957e00640 1 -- 172.21.15.16:0/2475298269 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f294c049020 con 0x7f29601068d0 2024-09-21T03:35:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.763+0000 7f2966800640 1 -- 172.21.15.16:0/2475298269 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f296010afb0 con 0x7f29601068d0 2024-09-21T03:35:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.768+0000 7f2957e00640 1 -- 172.21.15.16:0/2475298269 <== mon.2 v2:172.21.15.123:3300/0 7 ==== config(24 keys) ==== 1045+0+0 (secure 0 0 0) 0x7f294c031c50 con 0x7f29601068d0 2024-09-21T03:35:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.768+0000 7f2957e00640 1 -- 172.21.15.16:0/2475298269 <== mon.2 v2:172.21.15.123:3300/0 8 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f294c02ee60 con 0x7f29601068d0 2024-09-21T03:35:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.771+0000 7f2966800640 1 -- 172.21.15.16:0/2475298269 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f294403fb20 msgr2=0x7f2944042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.771+0000 7f2966800640 1 --2- 172.21.15.16:0/2475298269 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f294403fb20 0x7f2944042030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2950005fd0 tx=0x7f2950009260 comp rx=0 tx=0).stop 2024-09-21T03:35:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.771+0000 7f2966800640 1 -- 172.21.15.16:0/2475298269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29601068d0 msgr2=0x7f296019c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.771+0000 7f2966800640 1 --2- 172.21.15.16:0/2475298269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29601068d0 0x7f296019c5a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f294c02a930 tx=0x7f294c032d00 comp rx=0 tx=0).stop 2024-09-21T03:35:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.771+0000 7f2966800640 1 -- 172.21.15.16:0/2475298269 shutdown_connections 2024-09-21T03:35:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.771+0000 7f2966800640 1 --2- 172.21.15.16:0/2475298269 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f294403fb20 0x7f2944042030 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.771+0000 7f2966800640 1 --2- 172.21.15.16:0/2475298269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2960109470 0x7f296019d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.771+0000 7f2966800640 1 --2- 172.21.15.16:0/2475298269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29601068d0 0x7f296019c5a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.771+0000 7f2966800640 1 --2- 172.21.15.16:0/2475298269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f296006b980 0x7f296019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.771+0000 7f2966800640 1 -- 172.21.15.16:0/2475298269 >> 172.21.15.16:0/2475298269 conn(0x7f29600fdfb0 msgr2=0x7f2960104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.771+0000 7f2966800640 1 -- 172.21.15.16:0/2475298269 shutdown_connections 2024-09-21T03:35:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.771+0000 7f2966800640 1 -- 172.21.15.16:0/2475298269 wait complete. 2024-09-21T03:35:59.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:58 smithi123 ceph-mon[34673]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:59.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:59.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:59.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:35:59.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:35:59.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:59.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:58 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1897971007' entity='client.admin' 2024-09-21T03:35:59.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:58 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' 2024-09-21T03:35:59.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:58 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' 2024-09-21T03:35:59.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:58 smithi016 ceph-mon[30375]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:59.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:59.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:59.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:35:59.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:35:59.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:58 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1897971007' entity='client.admin' 2024-09-21T03:35:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:58 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T03:35:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:58 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T03:35:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.263+0000 7f01ab200640 1 -- 172.21.15.16:0/1897971007 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f018c03fb20 msgr2=0x7f018c042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.263+0000 7f01ab200640 1 --2- 172.21.15.16:0/1897971007 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f018c03fb20 0x7f018c042030 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0198009bd0 tx=0x7f0198005f90 comp rx=0 tx=0).stop 2024-09-21T03:35:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.263+0000 7f01ab200640 1 -- 172.21.15.16:0/1897971007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01a410b510 msgr2=0x7f01a419c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.263+0000 7f01ab200640 1 --2- 172.21.15.16:0/1897971007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01a410b510 0x7f01a419c610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f01a000efd0 tx=0x7f01a000c6a0 comp rx=0 tx=0).stop 2024-09-21T03:35:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.263+0000 7f01ab200640 1 -- 172.21.15.16:0/1897971007 shutdown_connections 2024-09-21T03:35:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.263+0000 7f01ab200640 1 --2- 172.21.15.16:0/1897971007 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f018c03fb20 0x7f018c042030 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.263+0000 7f01ab200640 1 --2- 172.21.15.16:0/1897971007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01a410b510 0x7f01a419c610 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.263+0000 7f01ab200640 1 --2- 172.21.15.16:0/1897971007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01a40770e0 0x7f01a419ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.263+0000 7f01ab200640 1 --2- 172.21.15.16:0/1897971007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01a4075cf0 0x7f01a419b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.263+0000 7f01ab200640 1 -- 172.21.15.16:0/1897971007 >> 172.21.15.16:0/1897971007 conn(0x7f01a40fef00 msgr2=0x7f01a4101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.263+0000 7f01ab200640 1 -- 172.21.15.16:0/1897971007 shutdown_connections 2024-09-21T03:35:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:58.263+0000 7f01ab200640 1 -- 172.21.15.16:0/1897971007 wait complete. 2024-09-21T03:35:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:58 smithi134 ceph-mon[35031]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:35:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:35:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:35:59.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:35:59.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:59.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:58 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1897971007' entity='client.admin' 2024-09-21T03:35:59.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:58 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' 2024-09-21T03:35:59.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:58 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' 2024-09-21T03:36:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.894+0000 7fc870e00640 1 -- 172.21.15.16:0/360306813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86c103570 msgr2=0x7fc86c1039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.894+0000 7fc870e00640 1 --2- 172.21.15.16:0/360306813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86c103570 0x7fc86c1039e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009900 tx=0x7fc85c02a480 comp rx=0 tx=0).stop 2024-09-21T03:36:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.894+0000 7fc870e00640 1 -- 172.21.15.16:0/360306813 shutdown_connections 2024-09-21T03:36:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.894+0000 7fc870e00640 1 --2- 172.21.15.16:0/360306813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc86c104fa0 0x7fc86c10b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.894+0000 7fc870e00640 1 --2- 172.21.15.16:0/360306813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc86c104600 0x7fc86c104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.894+0000 7fc870e00640 1 --2- 172.21.15.16:0/360306813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86c103570 0x7fc86c1039e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.894+0000 7fc870e00640 1 -- 172.21.15.16:0/360306813 >> 172.21.15.16:0/360306813 conn(0x7fc86c0feec0 msgr2=0x7fc86c101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.894+0000 7fc870e00640 1 -- 172.21.15.16:0/360306813 shutdown_connections 2024-09-21T03:36:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.895+0000 7fc870e00640 1 -- 172.21.15.16:0/360306813 wait complete. 2024-09-21T03:36:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.896+0000 7fc870e00640 1 Processor -- start 2024-09-21T03:36:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.896+0000 7fc870e00640 1 -- start start 2024-09-21T03:36:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.897+0000 7fc870e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc86c103570 0x7fc86c076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.897+0000 7fc870e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc86c104600 0x7fc86c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.897+0000 7fc870e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86c104fa0 0x7fc86c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.897+0000 7fc870e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c10efb0 con 0x7fc86c104fa0 2024-09-21T03:36:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.897+0000 7fc870e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c10ee30 con 0x7fc86c103570 2024-09-21T03:36:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.897+0000 7fc870e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c10f130 con 0x7fc86c104600 2024-09-21T03:36:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.897+0000 7fc86b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc86c103570 0x7fc86c076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.897+0000 7fc86b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc86c103570 0x7fc86c076e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:45836/0 (socket says 172.21.15.16:45836) 2024-09-21T03:36:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.897+0000 7fc86be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86c104fa0 0x7fc86c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.897+0000 7fc86b400640 1 -- 172.21.15.16:0/4182130084 learned_addr learned my addr 172.21.15.16:0/4182130084 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.897+0000 7fc86aa00640 1 --2- 172.21.15.16:0/4182130084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc86c104600 0x7fc86c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.898+0000 7fc86aa00640 1 -- 172.21.15.16:0/4182130084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc86c103570 msgr2=0x7fc86c076e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.898+0000 7fc86aa00640 1 --2- 172.21.15.16:0/4182130084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc86c103570 0x7fc86c076e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.898+0000 7fc86aa00640 1 -- 172.21.15.16:0/4182130084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86c104fa0 msgr2=0x7fc86c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.898+0000 7fc86aa00640 1 --2- 172.21.15.16:0/4182130084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86c104fa0 0x7fc86c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.898+0000 7fc86aa00640 1 -- 172.21.15.16:0/4182130084 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86c076350 con 0x7fc86c104600 2024-09-21T03:36:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.898+0000 7fc86aa00640 1 --2- 172.21.15.16:0/4182130084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc86c104600 0x7fc86c075710 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc85800e9f0 tx=0x7fc85800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.899+0000 7fc863e00640 1 -- 172.21.15.16:0/4182130084 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85800ccc0 con 0x7fc86c104600 2024-09-21T03:36:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.899+0000 7fc870e00640 1 -- 172.21.15.16:0/4182130084 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86c1a3d00 con 0x7fc86c104600 2024-09-21T03:36:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.899+0000 7fc863e00640 1 -- 172.21.15.16:0/4182130084 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(24 keys) ==== 1045+0+0 (secure 0 0 0) 0x7fc858005020 con 0x7fc86c104600 2024-09-21T03:36:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.899+0000 7fc863e00640 1 -- 172.21.15.16:0/4182130084 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8580044a0 con 0x7fc86c104600 2024-09-21T03:36:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.899+0000 7fc870e00640 1 -- 172.21.15.16:0/4182130084 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc86c1a4160 con 0x7fc86c104600 2024-09-21T03:36:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.900+0000 7fc870e00640 1 -- 172.21.15.16:0/4182130084 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc86c1039e0 con 0x7fc86c104600 2024-09-21T03:36:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.901+0000 7fc863e00640 1 -- 172.21.15.16:0/4182130084 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7fc858004a00 con 0x7fc86c104600 2024-09-21T03:36:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.901+0000 7fc863e00640 1 --2- 172.21.15.16:0/4182130084 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc85003fb20 0x7fc850042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.901+0000 7fc863e00640 1 -- 172.21.15.16:0/4182130084 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc858054530 con 0x7fc86c104600 2024-09-21T03:36:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.901+0000 7fc86b400640 1 --2- 172.21.15.16:0/4182130084 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc85003fb20 0x7fc850042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:00.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.902+0000 7fc86b400640 1 --2- 172.21.15.16:0/4182130084 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc85003fb20 0x7fc850042030 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0096c0 tx=0x7fc85c005a20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:00.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:35:59.907+0000 7fc863e00640 1 -- 172.21.15.16:0/4182130084 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc85800fda0 con 0x7fc86c104600 2024-09-21T03:36:00.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.108+0000 7fc870e00640 1 -- 172.21.15.16:0/4182130084 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fc86c063fc0 con 0x7fc86c104600 2024-09-21T03:36:00.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:59 smithi016 ceph-mon[30375]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:36:00.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:59 smithi016 ceph-mon[30375]: Reconfiguring daemon mon.c on smithi134 2024-09-21T03:36:00.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:59 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:00.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:59 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:00.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:35:59 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.114+0000 7fc863e00640 1 -- 172.21.15.16:0/4182130084 <== mon.1 v2:172.21.15.134:3300/0 7 ==== config(25 keys) ==== 1076+0+0 (secure 0 0 0) 0x7fc858013c20 con 0x7fc86c104600 2024-09-21T03:36:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.114+0000 7fc863e00640 1 -- 172.21.15.16:0/4182130084 <== mon.1 v2:172.21.15.134:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v13)=0 v13) ==== 123+0+0 (secure 0 0 0) 0x7fc85801dd60 con 0x7fc86c104600 2024-09-21T03:36:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.116+0000 7fc870e00640 1 -- 172.21.15.16:0/4182130084 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc85003fb20 msgr2=0x7fc850042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.116+0000 7fc870e00640 1 --2- 172.21.15.16:0/4182130084 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc85003fb20 0x7fc850042030 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0096c0 tx=0x7fc85c005a20 comp rx=0 tx=0).stop 2024-09-21T03:36:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.116+0000 7fc870e00640 1 -- 172.21.15.16:0/4182130084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc86c104600 msgr2=0x7fc86c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.116+0000 7fc870e00640 1 --2- 172.21.15.16:0/4182130084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc86c104600 0x7fc86c075710 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc85800e9f0 tx=0x7fc85800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.116+0000 7fc870e00640 1 -- 172.21.15.16:0/4182130084 shutdown_connections 2024-09-21T03:36:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.116+0000 7fc870e00640 1 --2- 172.21.15.16:0/4182130084 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc85003fb20 0x7fc850042030 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.116+0000 7fc870e00640 1 --2- 172.21.15.16:0/4182130084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86c104fa0 0x7fc86c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.116+0000 7fc870e00640 1 --2- 172.21.15.16:0/4182130084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc86c104600 0x7fc86c075710 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.116+0000 7fc870e00640 1 --2- 172.21.15.16:0/4182130084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc86c103570 0x7fc86c076e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.116+0000 7fc870e00640 1 -- 172.21.15.16:0/4182130084 >> 172.21.15.16:0/4182130084 conn(0x7fc86c0feec0 msgr2=0x7fc86c107920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.117+0000 7fc870e00640 1 -- 172.21.15.16:0/4182130084 shutdown_connections 2024-09-21T03:36:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.117+0000 7fc870e00640 1 -- 172.21.15.16:0/4182130084 wait complete. 2024-09-21T03:36:00.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:59 smithi123 ceph-mon[34673]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:36:00.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:59 smithi123 ceph-mon[34673]: Reconfiguring daemon mon.c on smithi134 2024-09-21T03:36:00.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:59 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:00.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:59 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:00.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:35:59 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:00.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:59 smithi134 ceph-mon[35031]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:36:00.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:59 smithi134 ceph-mon[35031]: Reconfiguring daemon mon.c on smithi134 2024-09-21T03:36:00.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:59 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:00.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:59 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:00.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:35:59 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:00.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.365+0000 7fd984000640 1 -- 172.21.15.16:0/803298060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97c10fee0 msgr2=0x7fd97c112300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:00.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.365+0000 7fd984000640 1 --2- 172.21.15.16:0/803298060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97c10fee0 0x7fd97c112300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd974009ca0 tx=0x7fd97402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.365+0000 7fd984000640 1 -- 172.21.15.16:0/803298060 shutdown_connections 2024-09-21T03:36:00.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.365+0000 7fd984000640 1 --2- 172.21.15.16:0/803298060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97c10fee0 0x7fd97c112300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.365+0000 7fd984000640 1 --2- 172.21.15.16:0/803298060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd97c075cf0 0x7fd97c076100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.365+0000 7fd984000640 1 --2- 172.21.15.16:0/803298060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd97c0770e0 0x7fd97c0757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.365+0000 7fd984000640 1 -- 172.21.15.16:0/803298060 >> 172.21.15.16:0/803298060 conn(0x7fd97c0ff0e0 msgr2=0x7fd97c101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:00.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.366+0000 7fd984000640 1 -- 172.21.15.16:0/803298060 shutdown_connections 2024-09-21T03:36:00.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.366+0000 7fd984000640 1 -- 172.21.15.16:0/803298060 wait complete. 2024-09-21T03:36:00.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.367+0000 7fd984000640 1 Processor -- start 2024-09-21T03:36:00.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.367+0000 7fd984000640 1 -- start start 2024-09-21T03:36:00.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.367+0000 7fd984000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd97c075cf0 0x7fd97c1a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:00.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.367+0000 7fd984000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd97c0770e0 0x7fd97c1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:00.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.367+0000 7fd984000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97c10fee0 0x7fd97c1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:00.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.367+0000 7fd984000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c1158d0 con 0x7fd97c10fee0 2024-09-21T03:36:00.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.367+0000 7fd984000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c115750 con 0x7fd97c0770e0 2024-09-21T03:36:00.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.367+0000 7fd984000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c115a50 con 0x7fd97c075cf0 2024-09-21T03:36:00.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.368+0000 7fd982000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97c10fee0 0x7fd97c1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:00.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.368+0000 7fd981600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd97c075cf0 0x7fd97c1a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:00.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.368+0000 7fd982000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97c10fee0 0x7fd97c1a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59672/0 (socket says 172.21.15.16:59672) 2024-09-21T03:36:00.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.368+0000 7fd982000640 1 -- 172.21.15.16:0/1432196625 learned_addr learned my addr 172.21.15.16:0/1432196625 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.368+0000 7fd980c00640 1 --2- 172.21.15.16:0/1432196625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd97c0770e0 0x7fd97c1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.368+0000 7fd981600640 1 -- 172.21.15.16:0/1432196625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd97c0770e0 msgr2=0x7fd97c1a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.368+0000 7fd981600640 1 --2- 172.21.15.16:0/1432196625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd97c0770e0 0x7fd97c1a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.368+0000 7fd981600640 1 -- 172.21.15.16:0/1432196625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97c10fee0 msgr2=0x7fd97c1a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.368+0000 7fd981600640 1 --2- 172.21.15.16:0/1432196625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97c10fee0 0x7fd97c1a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.368+0000 7fd981600640 1 -- 172.21.15.16:0/1432196625 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c1a1f00 con 0x7fd97c075cf0 2024-09-21T03:36:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.369+0000 7fd981600640 1 --2- 172.21.15.16:0/1432196625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd97c075cf0 0x7fd97c1a04d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd97000e9f0 tx=0x7fd97000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.369+0000 7fd97a000640 1 -- 172.21.15.16:0/1432196625 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97000ccc0 con 0x7fd97c075cf0 2024-09-21T03:36:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.369+0000 7fd984000640 1 -- 172.21.15.16:0/1432196625 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97c1a21f0 con 0x7fd97c075cf0 2024-09-21T03:36:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.369+0000 7fd97a000640 1 -- 172.21.15.16:0/1432196625 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(25 keys) ==== 1076+0+0 (secure 0 0 0) 0x7fd970005020 con 0x7fd97c075cf0 2024-09-21T03:36:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.369+0000 7fd97a000640 1 -- 172.21.15.16:0/1432196625 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970004a00 con 0x7fd97c075cf0 2024-09-21T03:36:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.369+0000 7fd984000640 1 -- 172.21.15.16:0/1432196625 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd97c1a88f0 con 0x7fd97c075cf0 2024-09-21T03:36:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.371+0000 7fd97a000640 1 -- 172.21.15.16:0/1432196625 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7fd970004ba0 con 0x7fd97c075cf0 2024-09-21T03:36:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.370+0000 7fd984000640 1 -- 172.21.15.16:0/1432196625 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd97c1a8ac0 con 0x7fd97c075cf0 2024-09-21T03:36:00.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.371+0000 7fd97a000640 1 --2- 172.21.15.16:0/1432196625 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd96403fb20 0x7fd964042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:00.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.371+0000 7fd97a000640 1 -- 172.21.15.16:0/1432196625 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd970054410 con 0x7fd97c075cf0 2024-09-21T03:36:00.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.371+0000 7fd980c00640 1 --2- 172.21.15.16:0/1432196625 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd96403fb20 0x7fd964042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:00.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.372+0000 7fd980c00640 1 --2- 172.21.15.16:0/1432196625 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd96403fb20 0x7fd964042030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd968009bd0 tx=0x7fd968005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:00.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.376+0000 7fd97a000640 1 -- 172.21.15.16:0/1432196625 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd9700051d0 con 0x7fd97c075cf0 2024-09-21T03:36:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.576+0000 7fd984000640 1 -- 172.21.15.16:0/1432196625 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fd97c1a8c90 con 0x7fd97c075cf0 2024-09-21T03:36:00.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.582+0000 7fd97a000640 1 -- 172.21.15.16:0/1432196625 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v14)=0 v14) ==== 106+0+0 (secure 0 0 0) 0x7fd970020c50 con 0x7fd97c075cf0 2024-09-21T03:36:00.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.584+0000 7fd984000640 1 -- 172.21.15.16:0/1432196625 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd96403fb20 msgr2=0x7fd964042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:00.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.584+0000 7fd984000640 1 --2- 172.21.15.16:0/1432196625 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd96403fb20 0x7fd964042030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd968009bd0 tx=0x7fd968005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:00.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.584+0000 7fd984000640 1 -- 172.21.15.16:0/1432196625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd97c075cf0 msgr2=0x7fd97c1a04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:00.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.584+0000 7fd984000640 1 --2- 172.21.15.16:0/1432196625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd97c075cf0 0x7fd97c1a04d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd97000e9f0 tx=0x7fd97000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.584+0000 7fd984000640 1 -- 172.21.15.16:0/1432196625 shutdown_connections 2024-09-21T03:36:00.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.584+0000 7fd984000640 1 --2- 172.21.15.16:0/1432196625 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd96403fb20 0x7fd964042030 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.584+0000 7fd984000640 1 --2- 172.21.15.16:0/1432196625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97c10fee0 0x7fd97c1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.584+0000 7fd984000640 1 --2- 172.21.15.16:0/1432196625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd97c0770e0 0x7fd97c1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.584+0000 7fd984000640 1 --2- 172.21.15.16:0/1432196625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd97c075cf0 0x7fd97c1a04d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:00.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.584+0000 7fd984000640 1 -- 172.21.15.16:0/1432196625 >> 172.21.15.16:0/1432196625 conn(0x7fd97c0ff0e0 msgr2=0x7fd97c0ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:00.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.584+0000 7fd984000640 1 -- 172.21.15.16:0/1432196625 shutdown_connections 2024-09-21T03:36:00.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:00.584+0000 7fd984000640 1 -- 172.21.15.16:0/1432196625 wait complete. 2024-09-21T03:36:01.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:00 smithi123 ceph-mon[34673]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:01.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:00 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' 2024-09-21T03:36:01.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:00 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' 2024-09-21T03:36:01.161 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:00 smithi016 ceph-mon[30375]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:01.161 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:00 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T03:36:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:00 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T03:36:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:00 smithi134 ceph-mon[35031]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:00 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' 2024-09-21T03:36:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:00 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' 2024-09-21T03:36:02.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.915+0000 7f93f2600640 1 -- 172.21.15.16:0/940050789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93ec0770e0 msgr2=0x7f93ec0757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:02.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.915+0000 7f93f2600640 1 --2- 172.21.15.16:0/940050789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93ec0770e0 0x7f93ec0757b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009900 tx=0x7f93e002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:36:02.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.915+0000 7f93f2600640 1 -- 172.21.15.16:0/940050789 shutdown_connections 2024-09-21T03:36:02.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.915+0000 7f93f2600640 1 --2- 172.21.15.16:0/940050789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93ec10b510 0x7f93ec10d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:02.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.915+0000 7f93f2600640 1 --2- 172.21.15.16:0/940050789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93ec075cf0 0x7f93ec076100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:02.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.915+0000 7f93f2600640 1 --2- 172.21.15.16:0/940050789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93ec0770e0 0x7f93ec0757b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:02.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.915+0000 7f93f2600640 1 -- 172.21.15.16:0/940050789 >> 172.21.15.16:0/940050789 conn(0x7f93ec0fef00 msgr2=0x7f93ec101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:02.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.915+0000 7f93f2600640 1 -- 172.21.15.16:0/940050789 shutdown_connections 2024-09-21T03:36:02.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.915+0000 7f93f2600640 1 -- 172.21.15.16:0/940050789 wait complete. 2024-09-21T03:36:02.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.916+0000 7f93f2600640 1 Processor -- start 2024-09-21T03:36:02.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.916+0000 7f93f2600640 1 -- start start 2024-09-21T03:36:02.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.917+0000 7f93f2600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93ec075cf0 0x7f93ec19bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:02.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.917+0000 7f93f2600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93ec0770e0 0x7f93ec19c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:02.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.917+0000 7f93f2600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93ec10b510 0x7f93ec19d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:02.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.917+0000 7f93f2600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec111320 con 0x7f93ec075cf0 2024-09-21T03:36:02.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.917+0000 7f93f2600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1111a0 con 0x7f93ec0770e0 2024-09-21T03:36:02.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.917+0000 7f93f2600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1114a0 con 0x7f93ec10b510 2024-09-21T03:36:02.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.917+0000 7f93f1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93ec075cf0 0x7f93ec19bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:02.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.917+0000 7f93f1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93ec075cf0 0x7f93ec19bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40512/0 (socket says 172.21.15.16:40512) 2024-09-21T03:36:02.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.917+0000 7f93f1200640 1 -- 172.21.15.16:0/1173459968 learned_addr learned my addr 172.21.15.16:0/1173459968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:02.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.917+0000 7f93f1c00640 1 --2- 172.21.15.16:0/1173459968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93ec10b510 0x7f93ec19d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:02.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.917+0000 7f93ebe00640 1 --2- 172.21.15.16:0/1173459968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93ec0770e0 0x7f93ec19c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:02.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.918+0000 7f93f1200640 1 -- 172.21.15.16:0/1173459968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93ec10b510 msgr2=0x7f93ec19d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:02.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.918+0000 7f93f1200640 1 --2- 172.21.15.16:0/1173459968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93ec10b510 0x7f93ec19d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:02.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.918+0000 7f93f1200640 1 -- 172.21.15.16:0/1173459968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93ec0770e0 msgr2=0x7f93ec19c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:02.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.918+0000 7f93f1200640 1 --2- 172.21.15.16:0/1173459968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93ec0770e0 0x7f93ec19c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:02.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.918+0000 7f93f1200640 1 -- 172.21.15.16:0/1173459968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec19d950 con 0x7f93ec075cf0 2024-09-21T03:36:02.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.918+0000 7f93f1200640 1 --2- 172.21.15.16:0/1173459968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93ec075cf0 0x7f93ec19bf20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f93e0005d90 tx=0x7f93e0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:02.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.918+0000 7f93e9600640 1 -- 172.21.15.16:0/1173459968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e003f070 con 0x7f93ec075cf0 2024-09-21T03:36:02.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.918+0000 7f93e9600640 1 -- 172.21.15.16:0/1173459968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1076+0+0 (secure 0 0 0) 0x7f93e0031bb0 con 0x7f93ec075cf0 2024-09-21T03:36:02.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.918+0000 7f93e9600640 1 -- 172.21.15.16:0/1173459968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0035030 con 0x7f93ec075cf0 2024-09-21T03:36:02.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.918+0000 7f93f2600640 1 -- 172.21.15.16:0/1173459968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ec19dbe0 con 0x7f93ec075cf0 2024-09-21T03:36:02.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.918+0000 7f93f2600640 1 -- 172.21.15.16:0/1173459968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ec1a4170 con 0x7f93ec075cf0 2024-09-21T03:36:02.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.920+0000 7f93f2600640 1 -- 172.21.15.16:0/1173459968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93ec10a320 con 0x7f93ec075cf0 2024-09-21T03:36:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.920+0000 7f93e9600640 1 -- 172.21.15.16:0/1173459968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f93e0042070 con 0x7f93ec075cf0 2024-09-21T03:36:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.920+0000 7f93e9600640 1 --2- 172.21.15.16:0/1173459968 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93d403fb20 0x7f93d4042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.920+0000 7f93e9600640 1 -- 172.21.15.16:0/1173459968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f93e0074e10 con 0x7f93ec075cf0 2024-09-21T03:36:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.920+0000 7f93ebe00640 1 --2- 172.21.15.16:0/1173459968 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93d403fb20 0x7f93d4042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.921+0000 7f93ebe00640 1 --2- 172.21.15.16:0/1173459968 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93d403fb20 0x7f93d4042030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009bd0 tx=0x7f93d8005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:02.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:02.926+0000 7f93e9600640 1 -- 172.21.15.16:0/1173459968 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f93e003b460 con 0x7f93ec075cf0 2024-09-21T03:36:03.096 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:02 smithi016 ceph-mon[30375]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.129+0000 7f93f2600640 1 -- 172.21.15.16:0/1173459968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f93ec02e640 con 0x7f93ec075cf0 2024-09-21T03:36:03.134 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi016') 2024-09-21T03:36:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.135+0000 7f93e9600640 1 -- 172.21.15.16:0/1173459968 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v15)=0 v15) ==== 114+0+0 (secure 0 0 0) 0x7f93e0031710 con 0x7f93ec075cf0 2024-09-21T03:36:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.135+0000 7f93e9600640 1 -- 172.21.15.16:0/1173459968 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e0043970 con 0x7f93ec075cf0 2024-09-21T03:36:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.138+0000 7f93f2600640 1 -- 172.21.15.16:0/1173459968 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93d403fb20 msgr2=0x7f93d4042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.138+0000 7f93f2600640 1 --2- 172.21.15.16:0/1173459968 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93d403fb20 0x7f93d4042030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009bd0 tx=0x7f93d8005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.138+0000 7f93f2600640 1 -- 172.21.15.16:0/1173459968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93ec075cf0 msgr2=0x7f93ec19bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.138+0000 7f93f2600640 1 --2- 172.21.15.16:0/1173459968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93ec075cf0 0x7f93ec19bf20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f93e0005d90 tx=0x7f93e0003c10 comp rx=0 tx=0).stop 2024-09-21T03:36:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.138+0000 7f93f2600640 1 -- 172.21.15.16:0/1173459968 shutdown_connections 2024-09-21T03:36:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.138+0000 7f93f2600640 1 --2- 172.21.15.16:0/1173459968 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93d403fb20 0x7f93d4042030 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.138+0000 7f93f2600640 1 --2- 172.21.15.16:0/1173459968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93ec10b510 0x7f93ec19d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.138+0000 7f93f2600640 1 --2- 172.21.15.16:0/1173459968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93ec0770e0 0x7f93ec19c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.138+0000 7f93f2600640 1 --2- 172.21.15.16:0/1173459968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93ec075cf0 0x7f93ec19bf20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.138+0000 7f93f2600640 1 -- 172.21.15.16:0/1173459968 >> 172.21.15.16:0/1173459968 conn(0x7f93ec0fef00 msgr2=0x7f93ec1074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.138+0000 7f93f2600640 1 -- 172.21.15.16:0/1173459968 shutdown_connections 2024-09-21T03:36:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.138+0000 7f93f2600640 1 -- 172.21.15.16:0/1173459968 wait complete. 2024-09-21T03:36:03.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:02 smithi134 ceph-mon[35031]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:03.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:02 smithi123 ceph-mon[34673]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.701+0000 7f844b200640 1 -- 172.21.15.16:0/650826075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f844c104fa0 msgr2=0x7f844c10b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.701+0000 7f844b200640 1 --2- 172.21.15.16:0/650826075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f844c104fa0 0x7f844c10b940 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8444009be0 tx=0x7f844402a750 comp rx=0 tx=0).stop 2024-09-21T03:36:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.701+0000 7f844b200640 1 -- 172.21.15.16:0/650826075 shutdown_connections 2024-09-21T03:36:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.701+0000 7f844b200640 1 --2- 172.21.15.16:0/650826075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f844c104fa0 0x7f844c10b940 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.701+0000 7f844b200640 1 --2- 172.21.15.16:0/650826075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f844c104600 0x7f844c104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.701+0000 7f844b200640 1 --2- 172.21.15.16:0/650826075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f844c103570 0x7f844c1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.701+0000 7f844b200640 1 -- 172.21.15.16:0/650826075 >> 172.21.15.16:0/650826075 conn(0x7f844c0feec0 msgr2=0x7f844c101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.701+0000 7f844b200640 1 -- 172.21.15.16:0/650826075 shutdown_connections 2024-09-21T03:36:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.701+0000 7f844b200640 1 -- 172.21.15.16:0/650826075 wait complete. 2024-09-21T03:36:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.703+0000 7f844b200640 1 Processor -- start 2024-09-21T03:36:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.703+0000 7f844b200640 1 -- start start 2024-09-21T03:36:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.703+0000 7f844b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f844c103570 0x7f844c199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.703+0000 7f844b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f844c104600 0x7f844c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.703+0000 7f844b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f844c104fa0 0x7f844c19afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.703+0000 7f844b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f844c10f1a0 con 0x7f844c104fa0 2024-09-21T03:36:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.703+0000 7f844b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f844c10f020 con 0x7f844c103570 2024-09-21T03:36:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.703+0000 7f844b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f844c10f320 con 0x7f844c104600 2024-09-21T03:36:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.704+0000 7f844a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f844c104fa0 0x7f844c19afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:03.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.704+0000 7f8449e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f844c103570 0x7f844c199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:03.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.704+0000 7f8449e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f844c103570 0x7f844c199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:59442/0 (socket says 172.21.15.16:59442) 2024-09-21T03:36:03.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.704+0000 7f8449e00640 1 -- 172.21.15.16:0/2035859129 learned_addr learned my addr 172.21.15.16:0/2035859129 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:03.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.704+0000 7f8449400640 1 --2- 172.21.15.16:0/2035859129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f844c104600 0x7f844c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.704+0000 7f8449e00640 1 -- 172.21.15.16:0/2035859129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f844c104600 msgr2=0x7f844c19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.704+0000 7f8449e00640 1 --2- 172.21.15.16:0/2035859129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f844c104600 0x7f844c19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.704+0000 7f8449e00640 1 -- 172.21.15.16:0/2035859129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f844c104fa0 msgr2=0x7f844c19afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.704+0000 7f8449e00640 1 --2- 172.21.15.16:0/2035859129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f844c104fa0 0x7f844c19afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.704+0000 7f8449e00640 1 -- 172.21.15.16:0/2035859129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844c19b7c0 con 0x7f844c103570 2024-09-21T03:36:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.704+0000 7f8449e00640 1 --2- 172.21.15.16:0/2035859129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f844c103570 0x7f844c199d90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f843c00ea60 tx=0x7f843c00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.705+0000 7f8442a00640 1 -- 172.21.15.16:0/2035859129 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c00cd20 con 0x7f844c103570 2024-09-21T03:36:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.705+0000 7f8442a00640 1 -- 172.21.15.16:0/2035859129 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843c0035f0 con 0x7f844c103570 2024-09-21T03:36:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.705+0000 7f844b200640 1 -- 172.21.15.16:0/2035859129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844c19bab0 con 0x7f844c103570 2024-09-21T03:36:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.705+0000 7f8442a00640 1 -- 172.21.15.16:0/2035859129 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c010690 con 0x7f844c103570 2024-09-21T03:36:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.705+0000 7f844b200640 1 -- 172.21.15.16:0/2035859129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844c1a21b0 con 0x7f844c103570 2024-09-21T03:36:03.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.706+0000 7f8442a00640 1 -- 172.21.15.16:0/2035859129 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f843c010830 con 0x7f844c103570 2024-09-21T03:36:03.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.706+0000 7f844b200640 1 -- 172.21.15.16:0/2035859129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f844c10a730 con 0x7f844c103570 2024-09-21T03:36:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.706+0000 7f8442a00640 1 --2- 172.21.15.16:0/2035859129 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f843003fb20 0x7f8430042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.706+0000 7f8442a00640 1 -- 172.21.15.16:0/2035859129 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f843c054830 con 0x7f844c103570 2024-09-21T03:36:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.707+0000 7f8449400640 1 --2- 172.21.15.16:0/2035859129 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f843003fb20 0x7f8430042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.707+0000 7f8449400640 1 --2- 172.21.15.16:0/2035859129 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f843003fb20 0x7f8430042030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8438005fd0 tx=0x7f8438009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.712+0000 7f8442a00640 1 -- 172.21.15.16:0/2035859129 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f843c003810 con 0x7f844c103570 2024-09-21T03:36:03.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.916+0000 7f844b200640 1 -- 172.21.15.16:0/2035859129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f844c02e640 con 0x7f844c103570 2024-09-21T03:36:03.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.924+0000 7f8442a00640 1 -- 172.21.15.16:0/2035859129 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v17)=0 v17) ==== 117+0+0 (secure 0 0 0) 0x7f843c003ac0 con 0x7f844c103570 2024-09-21T03:36:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.926+0000 7f844b200640 1 -- 172.21.15.16:0/2035859129 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f843003fb20 msgr2=0x7f8430042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.926+0000 7f844b200640 1 --2- 172.21.15.16:0/2035859129 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f843003fb20 0x7f8430042030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8438005fd0 tx=0x7f8438009260 comp rx=0 tx=0).stop 2024-09-21T03:36:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.926+0000 7f844b200640 1 -- 172.21.15.16:0/2035859129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f844c103570 msgr2=0x7f844c199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.926+0000 7f844b200640 1 --2- 172.21.15.16:0/2035859129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f844c103570 0x7f844c199d90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f843c00ea60 tx=0x7f843c00ee60 comp rx=0 tx=0).stop 2024-09-21T03:36:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.926+0000 7f844b200640 1 -- 172.21.15.16:0/2035859129 shutdown_connections 2024-09-21T03:36:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.926+0000 7f844b200640 1 --2- 172.21.15.16:0/2035859129 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f843003fb20 0x7f8430042030 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.926+0000 7f844b200640 1 --2- 172.21.15.16:0/2035859129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f844c104fa0 0x7f844c19afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.926+0000 7f844b200640 1 --2- 172.21.15.16:0/2035859129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f844c104600 0x7f844c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.926+0000 7f844b200640 1 --2- 172.21.15.16:0/2035859129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f844c103570 0x7f844c199d90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.926+0000 7f844b200640 1 -- 172.21.15.16:0/2035859129 >> 172.21.15.16:0/2035859129 conn(0x7f844c0feec0 msgr2=0x7f844c107900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.926+0000 7f844b200640 1 -- 172.21.15.16:0/2035859129 shutdown_connections 2024-09-21T03:36:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.926+0000 7f844b200640 1 -- 172.21.15.16:0/2035859129 wait complete. 2024-09-21T03:36:04.300 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:04 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1173459968' entity='client.admin' 2024-09-21T03:36:04.300 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:04 smithi016 ceph-mon[30375]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:04.300 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:04 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3484525341' entity='client.admin' 2024-09-21T03:36:04.300 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:04 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T03:36:04.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:04 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1173459968' entity='client.admin' 2024-09-21T03:36:04.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:04 smithi123 ceph-mon[34673]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:04.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:04 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3484525341' entity='client.admin' 2024-09-21T03:36:04.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:04 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' 2024-09-21T03:36:04.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:04 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1173459968' entity='client.admin' 2024-09-21T03:36:04.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:04 smithi134 ceph-mon[35031]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:04.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:04 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3484525341' entity='client.admin' 2024-09-21T03:36:04.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:04 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' 2024-09-21T03:36:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.419+0000 7f7b6f200640 1 -- 172.21.15.16:0/4250266235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b70104fe0 msgr2=0x7f7b7010b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.419+0000 7f7b6f200640 1 --2- 172.21.15.16:0/4250266235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b70104fe0 0x7f7b7010b980 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009ca0 tx=0x7f7b6802a950 comp rx=0 tx=0).stop 2024-09-21T03:36:05.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.419+0000 7f7b6f200640 1 -- 172.21.15.16:0/4250266235 shutdown_connections 2024-09-21T03:36:05.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.419+0000 7f7b6f200640 1 --2- 172.21.15.16:0/4250266235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b70104fe0 0x7f7b7010b980 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.419+0000 7f7b6f200640 1 --2- 172.21.15.16:0/4250266235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b70104640 0x7f7b70104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.419+0000 7f7b6f200640 1 --2- 172.21.15.16:0/4250266235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b701035b0 0x7f7b70103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.419+0000 7f7b6f200640 1 -- 172.21.15.16:0/4250266235 >> 172.21.15.16:0/4250266235 conn(0x7f7b700fef20 msgr2=0x7f7b701013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:05.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.419+0000 7f7b6f200640 1 -- 172.21.15.16:0/4250266235 shutdown_connections 2024-09-21T03:36:05.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.419+0000 7f7b6f200640 1 -- 172.21.15.16:0/4250266235 wait complete. 2024-09-21T03:36:05.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.421+0000 7f7b6f200640 1 Processor -- start 2024-09-21T03:36:05.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.421+0000 7f7b6f200640 1 -- start start 2024-09-21T03:36:05.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.421+0000 7f7b6f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b701035b0 0x7f7b70076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:05.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.421+0000 7f7b6f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b70104640 0x7f7b70075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:05.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.421+0000 7f7b6f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b70104fe0 0x7f7b70075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:05.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.421+0000 7f7b6f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7010f210 con 0x7f7b701035b0 2024-09-21T03:36:05.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.421+0000 7f7b6f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7010f090 con 0x7f7b70104640 2024-09-21T03:36:05.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.421+0000 7f7b6f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7010f390 con 0x7f7b70104fe0 2024-09-21T03:36:05.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.422+0000 7f7b6de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b701035b0 0x7f7b70076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:05.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.422+0000 7f7b6de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b701035b0 0x7f7b70076e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40532/0 (socket says 172.21.15.16:40532) 2024-09-21T03:36:05.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.422+0000 7f7b6de00640 1 -- 172.21.15.16:0/3484525341 learned_addr learned my addr 172.21.15.16:0/3484525341 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:05.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.422+0000 7f7b6e800640 1 --2- 172.21.15.16:0/3484525341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b70104fe0 0x7f7b70075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:05.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.422+0000 7f7b6d400640 1 --2- 172.21.15.16:0/3484525341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b70104640 0x7f7b70075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:05.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.422+0000 7f7b6de00640 1 -- 172.21.15.16:0/3484525341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b70104fe0 msgr2=0x7f7b70075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.422+0000 7f7b6de00640 1 --2- 172.21.15.16:0/3484525341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b70104fe0 0x7f7b70075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.422+0000 7f7b6de00640 1 -- 172.21.15.16:0/3484525341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b70104640 msgr2=0x7f7b70075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.422+0000 7f7b6de00640 1 --2- 172.21.15.16:0/3484525341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b70104640 0x7f7b70075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.422+0000 7f7b6de00640 1 -- 172.21.15.16:0/3484525341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b70076350 con 0x7f7b701035b0 2024-09-21T03:36:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.422+0000 7f7b6de00640 1 --2- 172.21.15.16:0/3484525341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b701035b0 0x7f7b70076e40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000e9f0 tx=0x7f7b6000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.423+0000 7f7b66a00640 1 -- 172.21.15.16:0/3484525341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6000cd80 con 0x7f7b701035b0 2024-09-21T03:36:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.423+0000 7f7b66a00640 1 -- 172.21.15.16:0/3484525341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b600054d0 con 0x7f7b701035b0 2024-09-21T03:36:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.423+0000 7f7b66a00640 1 -- 172.21.15.16:0/3484525341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60010690 con 0x7f7b701035b0 2024-09-21T03:36:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.423+0000 7f7b6f200640 1 -- 172.21.15.16:0/3484525341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b701a3e60 con 0x7f7b701035b0 2024-09-21T03:36:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.423+0000 7f7b6f200640 1 -- 172.21.15.16:0/3484525341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b701a4200 con 0x7f7b701035b0 2024-09-21T03:36:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.424+0000 7f7b66a00640 1 -- 172.21.15.16:0/3484525341 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f7b60010830 con 0x7f7b701035b0 2024-09-21T03:36:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.424+0000 7f7b6f200640 1 -- 172.21.15.16:0/3484525341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b70103a20 con 0x7f7b701035b0 2024-09-21T03:36:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.425+0000 7f7b66a00640 1 --2- 172.21.15.16:0/3484525341 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7b5403fb20 0x7f7b54042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.425+0000 7f7b66a00640 1 -- 172.21.15.16:0/3484525341 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7b60054a60 con 0x7f7b701035b0 2024-09-21T03:36:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.425+0000 7f7b6d400640 1 --2- 172.21.15.16:0/3484525341 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7b5403fb20 0x7f7b54042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.426+0000 7f7b6d400640 1 --2- 172.21.15.16:0/3484525341 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7b5403fb20 0x7f7b54042030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009bd0 tx=0x7f7b5c005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.430+0000 7f7b66a00640 1 -- 172.21.15.16:0/3484525341 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7b60016330 con 0x7f7b701035b0 2024-09-21T03:36:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.631+0000 7f7b6f200640 1 -- 172.21.15.16:0/3484525341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f7b70063fc0 con 0x7f7b701035b0 2024-09-21T03:36:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.635+0000 7f7b66a00640 1 -- 172.21.15.16:0/3484525341 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v16)=0 v16) ==== 127+0+0 (secure 0 0 0) 0x7f7b600450c0 con 0x7f7b701035b0 2024-09-21T03:36:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.638+0000 7f7b6f200640 1 -- 172.21.15.16:0/3484525341 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7b5403fb20 msgr2=0x7f7b54042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.638+0000 7f7b6f200640 1 --2- 172.21.15.16:0/3484525341 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7b5403fb20 0x7f7b54042030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009bd0 tx=0x7f7b5c005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.638+0000 7f7b6f200640 1 -- 172.21.15.16:0/3484525341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b701035b0 msgr2=0x7f7b70076e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.638+0000 7f7b6f200640 1 --2- 172.21.15.16:0/3484525341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b701035b0 0x7f7b70076e40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000e9f0 tx=0x7f7b6000edf0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.638+0000 7f7b6f200640 1 -- 172.21.15.16:0/3484525341 shutdown_connections 2024-09-21T03:36:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.638+0000 7f7b6f200640 1 --2- 172.21.15.16:0/3484525341 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7b5403fb20 0x7f7b54042030 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.638+0000 7f7b6f200640 1 --2- 172.21.15.16:0/3484525341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b70104fe0 0x7f7b70075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.638+0000 7f7b6f200640 1 --2- 172.21.15.16:0/3484525341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b70104640 0x7f7b70075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.638+0000 7f7b6f200640 1 --2- 172.21.15.16:0/3484525341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b701035b0 0x7f7b70076e40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.638+0000 7f7b6f200640 1 -- 172.21.15.16:0/3484525341 >> 172.21.15.16:0/3484525341 conn(0x7f7b700fef20 msgr2=0x7f7b70107960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.638+0000 7f7b6f200640 1 -- 172.21.15.16:0/3484525341 shutdown_connections 2024-09-21T03:36:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:03.638+0000 7f7b6f200640 1 -- 172.21.15.16:0/3484525341 wait complete. 2024-09-21T03:36:05.847 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi016') 2024-09-21T03:36:05.848 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-09-21T03:36:05.848 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-09-21T03:36:05.848 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi016') 2024-09-21T03:36:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:05.894 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:05.908 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:06.094 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:06.320 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:06.326 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:06.326 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:06.380 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:06 smithi016 ceph-mon[30375]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:06.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:06 smithi134 ceph-mon[35031]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:06.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:06 smithi123 ceph-mon[34673]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.804+0000 7f2eb5800640 1 -- 172.21.15.16:0/2321492393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb006c420 msgr2=0x7f2eb01061d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.804+0000 7f2eb5800640 1 --2- 172.21.15.16:0/2321492393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb006c420 0x7f2eb01061d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009900 tx=0x7f2ea002a410 comp rx=0 tx=0).stop 2024-09-21T03:36:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.804+0000 7f2eb5800640 1 -- 172.21.15.16:0/2321492393 shutdown_connections 2024-09-21T03:36:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.804+0000 7f2eb5800640 1 --2- 172.21.15.16:0/2321492393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eb01092b0 0x7f2eb010b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.804+0000 7f2eb5800640 1 --2- 172.21.15.16:0/2321492393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eb0106710 0x7f2eb0108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.804+0000 7f2eb5800640 1 --2- 172.21.15.16:0/2321492393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb006c420 0x7f2eb01061d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.804+0000 7f2eb5800640 1 -- 172.21.15.16:0/2321492393 >> 172.21.15.16:0/2321492393 conn(0x7f2eb00fde10 msgr2=0x7f2eb01002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.804+0000 7f2eb5800640 1 -- 172.21.15.16:0/2321492393 shutdown_connections 2024-09-21T03:36:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.804+0000 7f2eb5800640 1 -- 172.21.15.16:0/2321492393 wait complete. 2024-09-21T03:36:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.805+0000 7f2eb5800640 1 Processor -- start 2024-09-21T03:36:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.805+0000 7f2eb5800640 1 -- start start 2024-09-21T03:36:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.805+0000 7f2eb5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb006c420 0x7f2eb019c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.806+0000 7f2eb5800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eb0106710 0x7f2eb019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.806+0000 7f2eb5800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eb01092b0 0x7f2eb019d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.806+0000 7f2eb5800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb010f1b0 con 0x7f2eb006c420 2024-09-21T03:36:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.806+0000 7f2eb5800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb010f030 con 0x7f2eb0106710 2024-09-21T03:36:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.806+0000 7f2eb5800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb010f330 con 0x7f2eb01092b0 2024-09-21T03:36:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.806+0000 7f2eafe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb006c420 0x7f2eb019c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.806+0000 7f2eafe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb006c420 0x7f2eb019c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40558/0 (socket says 172.21.15.16:40558) 2024-09-21T03:36:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.806+0000 7f2eafe00640 1 -- 172.21.15.16:0/541174187 learned_addr learned my addr 172.21.15.16:0/541174187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.806+0000 7f2eaf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eb0106710 0x7f2eb019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.806+0000 7f2eb4e00640 1 --2- 172.21.15.16:0/541174187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eb01092b0 0x7f2eb019d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.806+0000 7f2eafe00640 1 -- 172.21.15.16:0/541174187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eb01092b0 msgr2=0x7f2eb019d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.806+0000 7f2eafe00640 1 --2- 172.21.15.16:0/541174187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eb01092b0 0x7f2eb019d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.806+0000 7f2eafe00640 1 -- 172.21.15.16:0/541174187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eb0106710 msgr2=0x7f2eb019c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.806+0000 7f2eafe00640 1 --2- 172.21.15.16:0/541174187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eb0106710 0x7f2eb019c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.806+0000 7f2eafe00640 1 -- 172.21.15.16:0/541174187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb019da70 con 0x7f2eb006c420 2024-09-21T03:36:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.807+0000 7f2eafe00640 1 --2- 172.21.15.16:0/541174187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb006c420 0x7f2eb019c040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0006010 tx=0x7f2ea0004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.807+0000 7f2eacc00640 1 -- 172.21.15.16:0/541174187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea003f030 con 0x7f2eb006c420 2024-09-21T03:36:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.807+0000 7f2eacc00640 1 -- 172.21.15.16:0/541174187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ea0003cb0 con 0x7f2eb006c420 2024-09-21T03:36:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.807+0000 7f2eb5800640 1 -- 172.21.15.16:0/541174187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb019dd00 con 0x7f2eb006c420 2024-09-21T03:36:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.807+0000 7f2eacc00640 1 -- 172.21.15.16:0/541174187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0035070 con 0x7f2eb006c420 2024-09-21T03:36:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.807+0000 7f2eb5800640 1 -- 172.21.15.16:0/541174187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb01a4460 con 0x7f2eb006c420 2024-09-21T03:36:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.808+0000 7f2eb5800640 1 -- 172.21.15.16:0/541174187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb01a4630 con 0x7f2eb006c420 2024-09-21T03:36:06.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.809+0000 7f2eacc00640 1 -- 172.21.15.16:0/541174187 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f2ea0042030 con 0x7f2eb006c420 2024-09-21T03:36:06.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.809+0000 7f2eacc00640 1 --2- 172.21.15.16:0/541174187 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2e9403fb20 0x7f2e94042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:06.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.809+0000 7f2eacc00640 1 -- 172.21.15.16:0/541174187 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2ea0074be0 con 0x7f2eb006c420 2024-09-21T03:36:06.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.809+0000 7f2eaf400640 1 --2- 172.21.15.16:0/541174187 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2e9403fb20 0x7f2e94042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:06.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.810+0000 7f2eaf400640 1 --2- 172.21.15.16:0/541174187 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2e9403fb20 0x7f2e94042030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00bbe0 tx=0x7f2e9c00b390 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:06.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:06.814+0000 7f2eacc00640 1 -- 172.21.15.16:0/541174187 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2ea0003e60 con 0x7f2eb006c420 2024-09-21T03:36:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.022+0000 7f2eb5800640 1 -- 172.21.15.16:0/541174187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f2eb01a4800 con 0x7f2eb006c420 2024-09-21T03:36:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.025+0000 7f2eacc00640 1 -- 172.21.15.16:0/541174187 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v18)=0 v18) ==== 116+0+0 (secure 0 0 0) 0x7f2ea003bc20 con 0x7f2eb006c420 2024-09-21T03:36:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.028+0000 7f2eb5800640 1 -- 172.21.15.16:0/541174187 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2e9403fb20 msgr2=0x7f2e94042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.028+0000 7f2eb5800640 1 --2- 172.21.15.16:0/541174187 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2e9403fb20 0x7f2e94042030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00bbe0 tx=0x7f2e9c00b390 comp rx=0 tx=0).stop 2024-09-21T03:36:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.028+0000 7f2eb5800640 1 -- 172.21.15.16:0/541174187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb006c420 msgr2=0x7f2eb019c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.028+0000 7f2eb5800640 1 --2- 172.21.15.16:0/541174187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb006c420 0x7f2eb019c040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0006010 tx=0x7f2ea0004340 comp rx=0 tx=0).stop 2024-09-21T03:36:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.028+0000 7f2eb5800640 1 -- 172.21.15.16:0/541174187 shutdown_connections 2024-09-21T03:36:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.028+0000 7f2eb5800640 1 --2- 172.21.15.16:0/541174187 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2e9403fb20 0x7f2e94042030 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.028+0000 7f2eb5800640 1 --2- 172.21.15.16:0/541174187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eb01092b0 0x7f2eb019d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.028+0000 7f2eb5800640 1 --2- 172.21.15.16:0/541174187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eb0106710 0x7f2eb019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.028+0000 7f2eb5800640 1 --2- 172.21.15.16:0/541174187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb006c420 0x7f2eb019c040 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.028+0000 7f2eb5800640 1 -- 172.21.15.16:0/541174187 >> 172.21.15.16:0/541174187 conn(0x7f2eb00fde10 msgr2=0x7f2eb0100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.028+0000 7f2eb5800640 1 -- 172.21.15.16:0/541174187 shutdown_connections 2024-09-21T03:36:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.028+0000 7f2eb5800640 1 -- 172.21.15.16:0/541174187 wait complete. 2024-09-21T03:36:07.159 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:07 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:07 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:07 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:07 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:36:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:07 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:36:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:07 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:36:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:07 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:07 smithi134 ceph-mon[35031]: Deploying daemon mgr.z on smithi134 2024-09-21T03:36:07.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:07 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/541174187' entity='client.admin' 2024-09-21T03:36:07.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.333+0000 7f4659c00640 1 -- 172.21.15.16:0/2506652004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4654103750 msgr2=0x7f4654103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.333+0000 7f4659c00640 1 --2- 172.21.15.16:0/2506652004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4654103750 0x7f4654103bc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4644009900 tx=0x7f464402a480 comp rx=0 tx=0).stop 2024-09-21T03:36:07.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.333+0000 7f4659c00640 1 -- 172.21.15.16:0/2506652004 shutdown_connections 2024-09-21T03:36:07.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.333+0000 7f4659c00640 1 --2- 172.21.15.16:0/2506652004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4654105180 0x7f465410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.333+0000 7f4659c00640 1 --2- 172.21.15.16:0/2506652004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46541047e0 0x7f4654104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.333+0000 7f4659c00640 1 --2- 172.21.15.16:0/2506652004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4654103750 0x7f4654103bc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.333+0000 7f4659c00640 1 -- 172.21.15.16:0/2506652004 >> 172.21.15.16:0/2506652004 conn(0x7f46540ff0c0 msgr2=0x7f4654101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:07.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.333+0000 7f4659c00640 1 -- 172.21.15.16:0/2506652004 shutdown_connections 2024-09-21T03:36:07.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.333+0000 7f4659c00640 1 -- 172.21.15.16:0/2506652004 wait complete. 2024-09-21T03:36:07.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.335+0000 7f4659c00640 1 Processor -- start 2024-09-21T03:36:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.335+0000 7f4659c00640 1 -- start start 2024-09-21T03:36:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.335+0000 7f4659c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4654103750 0x7f4654076dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.335+0000 7f4659c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46541047e0 0x7f4654075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.335+0000 7f4659c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4654105180 0x7f4654075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.335+0000 7f4659c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f465410f3c0 con 0x7f4654105180 2024-09-21T03:36:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.335+0000 7f4659c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f465410f240 con 0x7f46541047e0 2024-09-21T03:36:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.335+0000 7f4659c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f465410f540 con 0x7f4654103750 2024-09-21T03:36:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.335+0000 7f4652a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4654103750 0x7f4654076dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.335+0000 7f4652a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4654103750 0x7f4654076dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:47440/0 (socket says 172.21.15.16:47440) 2024-09-21T03:36:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.335+0000 7f4652a00640 1 -- 172.21.15.16:0/3868879009 learned_addr learned my addr 172.21.15.16:0/3868879009 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.336+0000 7f464a000640 1 --2- 172.21.15.16:0/3868879009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46541047e0 0x7f4654075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.336+0000 7f4653400640 1 --2- 172.21.15.16:0/3868879009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4654105180 0x7f4654075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.336+0000 7f4652a00640 1 -- 172.21.15.16:0/3868879009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46541047e0 msgr2=0x7f4654075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.336+0000 7f4652a00640 1 --2- 172.21.15.16:0/3868879009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46541047e0 0x7f4654075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.336+0000 7f4652a00640 1 -- 172.21.15.16:0/3868879009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4654105180 msgr2=0x7f4654075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.336+0000 7f4652a00640 1 --2- 172.21.15.16:0/3868879009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4654105180 0x7f4654075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.336+0000 7f4652a00640 1 -- 172.21.15.16:0/3868879009 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46540762f0 con 0x7f4654103750 2024-09-21T03:36:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.336+0000 7f4652a00640 1 --2- 172.21.15.16:0/3868879009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4654103750 0x7f4654076dd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4644031440 tx=0x7f4644003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.337+0000 7f464be00640 1 -- 172.21.15.16:0/3868879009 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464403f070 con 0x7f4654103750 2024-09-21T03:36:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.337+0000 7f464be00640 1 -- 172.21.15.16:0/3868879009 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4644031c50 con 0x7f4654103750 2024-09-21T03:36:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.337+0000 7f4659c00640 1 -- 172.21.15.16:0/3868879009 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46541a40d0 con 0x7f4654103750 2024-09-21T03:36:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.337+0000 7f464be00640 1 -- 172.21.15.16:0/3868879009 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644035030 con 0x7f4654103750 2024-09-21T03:36:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.337+0000 7f4659c00640 1 -- 172.21.15.16:0/3868879009 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46541a4470 con 0x7f4654103750 2024-09-21T03:36:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.338+0000 7f464be00640 1 -- 172.21.15.16:0/3868879009 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f4644042070 con 0x7f4654103750 2024-09-21T03:36:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.338+0000 7f4659c00640 1 -- 172.21.15.16:0/3868879009 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f465410a930 con 0x7f4654103750 2024-09-21T03:36:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.338+0000 7f464be00640 1 --2- 172.21.15.16:0/3868879009 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f463803fb20 0x7f4638042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.338+0000 7f464be00640 1 -- 172.21.15.16:0/3868879009 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4644074e30 con 0x7f4654103750 2024-09-21T03:36:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.339+0000 7f464a000640 1 --2- 172.21.15.16:0/3868879009 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f463803fb20 0x7f4638042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.339+0000 7f464a000640 1 --2- 172.21.15.16:0/3868879009 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f463803fb20 0x7f4638042030 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f463c009bd0 tx=0x7f463c005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:07.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.344+0000 7f464be00640 1 -- 172.21.15.16:0/3868879009 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f46440746a0 con 0x7f4654103750 2024-09-21T03:36:07.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:07 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:07.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:07 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:07.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:07 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:07.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:07 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:36:07.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:07 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:36:07.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:07 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:36:07.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:07 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:07.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:07 smithi016 ceph-mon[30375]: Deploying daemon mgr.z on smithi134 2024-09-21T03:36:07.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:07 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/541174187' entity='client.admin' 2024-09-21T03:36:07.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.545+0000 7f4659c00640 1 -- 172.21.15.16:0/3868879009 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f465402e640 con 0x7f4654103750 2024-09-21T03:36:07.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.552+0000 7f464be00640 1 -- 172.21.15.16:0/3868879009 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v19)=0 v19) ==== 119+0+0 (secure 0 0 0) 0x7f4644040d50 con 0x7f4654103750 2024-09-21T03:36:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.555+0000 7f4659c00640 1 -- 172.21.15.16:0/3868879009 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f463803fb20 msgr2=0x7f4638042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.555+0000 7f4659c00640 1 --2- 172.21.15.16:0/3868879009 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f463803fb20 0x7f4638042030 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f463c009bd0 tx=0x7f463c005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.555+0000 7f4659c00640 1 -- 172.21.15.16:0/3868879009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4654103750 msgr2=0x7f4654076dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.555+0000 7f4659c00640 1 --2- 172.21.15.16:0/3868879009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4654103750 0x7f4654076dd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4644031440 tx=0x7f4644003c10 comp rx=0 tx=0).stop 2024-09-21T03:36:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.555+0000 7f4659c00640 1 -- 172.21.15.16:0/3868879009 shutdown_connections 2024-09-21T03:36:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.555+0000 7f4659c00640 1 --2- 172.21.15.16:0/3868879009 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f463803fb20 0x7f4638042030 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.555+0000 7f4659c00640 1 --2- 172.21.15.16:0/3868879009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4654105180 0x7f4654075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.555+0000 7f4659c00640 1 --2- 172.21.15.16:0/3868879009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46541047e0 0x7f4654075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.555+0000 7f4659c00640 1 --2- 172.21.15.16:0/3868879009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4654103750 0x7f4654076dd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.555+0000 7f4659c00640 1 -- 172.21.15.16:0/3868879009 >> 172.21.15.16:0/3868879009 conn(0x7f46540ff0c0 msgr2=0x7f4654107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.555+0000 7f4659c00640 1 -- 172.21.15.16:0/3868879009 shutdown_connections 2024-09-21T03:36:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.555+0000 7f4659c00640 1 -- 172.21.15.16:0/3868879009 wait complete. 2024-09-21T03:36:07.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:07 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:07.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:07 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:07.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:07 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:07.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:07 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:36:07.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:07 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:36:07.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:07 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:36:07.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:07 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:07.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:07 smithi123 ceph-mon[34673]: Deploying daemon mgr.z on smithi134 2024-09-21T03:36:07.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:07 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/541174187' entity='client.admin' 2024-09-21T03:36:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.785+0000 7f5186e00640 1 -- 172.21.15.16:0/604117350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5180105980 msgr2=0x7f5180105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.785+0000 7f5186e00640 1 --2- 172.21.15.16:0/604117350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5180105980 0x7f5180105df0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5170009900 tx=0x7f517002a480 comp rx=0 tx=0).stop 2024-09-21T03:36:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.785+0000 7f5186e00640 1 -- 172.21.15.16:0/604117350 shutdown_connections 2024-09-21T03:36:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.785+0000 7f5186e00640 1 --2- 172.21.15.16:0/604117350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51801073b0 0x7f518010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.785+0000 7f5186e00640 1 --2- 172.21.15.16:0/604117350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5180106a10 0x7f5180106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.785+0000 7f5186e00640 1 --2- 172.21.15.16:0/604117350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5180105980 0x7f5180105df0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.785+0000 7f5186e00640 1 -- 172.21.15.16:0/604117350 >> 172.21.15.16:0/604117350 conn(0x7f51801012f0 msgr2=0x7f51801037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.785+0000 7f5186e00640 1 -- 172.21.15.16:0/604117350 shutdown_connections 2024-09-21T03:36:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.785+0000 7f5186e00640 1 -- 172.21.15.16:0/604117350 wait complete. 2024-09-21T03:36:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.786+0000 7f5186e00640 1 Processor -- start 2024-09-21T03:36:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.786+0000 7f5186e00640 1 -- start start 2024-09-21T03:36:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.787+0000 7f5186e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5180105980 0x7f518019c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.787+0000 7f5186e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5180106a10 0x7f518019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.787+0000 7f5186e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51801073b0 0x7f518019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.787+0000 7f5186e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f51801115f0 con 0x7f51801073b0 2024-09-21T03:36:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.787+0000 7f5186e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5180111470 con 0x7f5180106a10 2024-09-21T03:36:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.787+0000 7f5186e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5180111770 con 0x7f5180105980 2024-09-21T03:36:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.787+0000 7f5184e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51801073b0 0x7f518019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.787+0000 7f5184e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51801073b0 0x7f518019d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40582/0 (socket says 172.21.15.16:40582) 2024-09-21T03:36:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.787+0000 7f5184e00640 1 -- 172.21.15.16:0/473815997 learned_addr learned my addr 172.21.15.16:0/473815997 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.787+0000 7f517fe00640 1 --2- 172.21.15.16:0/473815997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5180105980 0x7f518019c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.787+0000 7f517f400640 1 --2- 172.21.15.16:0/473815997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5180106a10 0x7f518019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.788+0000 7f5184e00640 1 -- 172.21.15.16:0/473815997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5180105980 msgr2=0x7f518019c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.788+0000 7f5184e00640 1 --2- 172.21.15.16:0/473815997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5180105980 0x7f518019c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.788+0000 7f5184e00640 1 -- 172.21.15.16:0/473815997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5180106a10 msgr2=0x7f518019c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.788+0000 7f5184e00640 1 --2- 172.21.15.16:0/473815997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5180106a10 0x7f518019c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.788+0000 7f5184e00640 1 -- 172.21.15.16:0/473815997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518019dc10 con 0x7f51801073b0 2024-09-21T03:36:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.788+0000 7f5184e00640 1 --2- 172.21.15.16:0/473815997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51801073b0 0x7f518019d3f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f517800eaa0 tx=0x7f517800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.788+0000 7f517cc00640 1 -- 172.21.15.16:0/473815997 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517800cdb0 con 0x7f51801073b0 2024-09-21T03:36:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.788+0000 7f517cc00640 1 -- 172.21.15.16:0/473815997 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5178005020 con 0x7f51801073b0 2024-09-21T03:36:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.788+0000 7f5186e00640 1 -- 172.21.15.16:0/473815997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518019df00 con 0x7f51801073b0 2024-09-21T03:36:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.788+0000 7f517cc00640 1 -- 172.21.15.16:0/473815997 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51780116d0 con 0x7f51801073b0 2024-09-21T03:36:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.788+0000 7f5186e00640 1 -- 172.21.15.16:0/473815997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51801a4600 con 0x7f51801073b0 2024-09-21T03:36:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.789+0000 7f5186e00640 1 -- 172.21.15.16:0/473815997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f518010cb60 con 0x7f51801073b0 2024-09-21T03:36:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.790+0000 7f517cc00640 1 -- 172.21.15.16:0/473815997 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f5178004b20 con 0x7f51801073b0 2024-09-21T03:36:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.790+0000 7f517cc00640 1 --2- 172.21.15.16:0/473815997 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f516403fb20 0x7f5164042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.790+0000 7f517cc00640 1 -- 172.21.15.16:0/473815997 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5178054730 con 0x7f51801073b0 2024-09-21T03:36:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.790+0000 7f517fe00640 1 --2- 172.21.15.16:0/473815997 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f516403fb20 0x7f5164042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.791+0000 7f517fe00640 1 --2- 172.21.15.16:0/473815997 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f516403fb20 0x7f5164042030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5170004770 tx=0x7f51700314a0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:07.796+0000 7f517cc00640 1 -- 172.21.15.16:0/473815997 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5178016030 con 0x7f51801073b0 2024-09-21T03:36:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:08.002+0000 7f5186e00640 1 -- 172.21.15.16:0/473815997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f518002e640 con 0x7f51801073b0 2024-09-21T03:36:08.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:08.006+0000 7f517cc00640 1 -- 172.21.15.16:0/473815997 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v20)=0 v20) ==== 120+0+0 (secure 0 0 0) 0x7f51780051d0 con 0x7f51801073b0 2024-09-21T03:36:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:08.008+0000 7f5186e00640 1 -- 172.21.15.16:0/473815997 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f516403fb20 msgr2=0x7f5164042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:08.008+0000 7f5186e00640 1 --2- 172.21.15.16:0/473815997 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f516403fb20 0x7f5164042030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5170004770 tx=0x7f51700314a0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:08.008+0000 7f5186e00640 1 -- 172.21.15.16:0/473815997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51801073b0 msgr2=0x7f518019d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:08.008+0000 7f5186e00640 1 --2- 172.21.15.16:0/473815997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51801073b0 0x7f518019d3f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f517800eaa0 tx=0x7f517800ef60 comp rx=0 tx=0).stop 2024-09-21T03:36:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:08.008+0000 7f5186e00640 1 -- 172.21.15.16:0/473815997 shutdown_connections 2024-09-21T03:36:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:08.008+0000 7f5186e00640 1 --2- 172.21.15.16:0/473815997 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f516403fb20 0x7f5164042030 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:08.008+0000 7f5186e00640 1 --2- 172.21.15.16:0/473815997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51801073b0 0x7f518019d3f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:08.008+0000 7f5186e00640 1 --2- 172.21.15.16:0/473815997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5180106a10 0x7f518019c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:08.008+0000 7f5186e00640 1 --2- 172.21.15.16:0/473815997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5180105980 0x7f518019c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:08.008+0000 7f5186e00640 1 -- 172.21.15.16:0/473815997 >> 172.21.15.16:0/473815997 conn(0x7f51801012f0 msgr2=0x7f5180109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:08.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:08.008+0000 7f5186e00640 1 -- 172.21.15.16:0/473815997 shutdown_connections 2024-09-21T03:36:08.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:08.008+0000 7f5186e00640 1 -- 172.21.15.16:0/473815997 wait complete. 2024-09-21T03:36:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:08.296 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 21 03:36:08 smithi134 systemd[1]: Starting Ceph mgr.z for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:36:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:08.552 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 21 03:36:08 smithi134 podman[36278]: 2024-09-21 03:36:08.197477827 +0000 UTC m=+0.016305160 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:36:08.552 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 21 03:36:08 smithi134 podman[36278]: 2024-09-21 03:36:08.311229309 +0000 UTC m=+0.130056642 container create 3cf07047e78696a6e98c49228c594927694375e481224d4a56c70de962fa0aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-z, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:36:08.552 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 21 03:36:08 smithi134 podman[36278]: 2024-09-21 03:36:08.528633474 +0000 UTC m=+0.347460814 container init 3cf07047e78696a6e98c49228c594927694375e481224d4a56c70de962fa0aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:36:08.552 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 21 03:36:08 smithi134 podman[36278]: 2024-09-21 03:36:08.531422798 +0000 UTC m=+0.350250130 container start 3cf07047e78696a6e98c49228c594927694375e481224d4a56c70de962fa0aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-z, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T03:36:08.702 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:08 smithi016 ceph-mon[30375]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:08.702 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:08 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T03:36:08.702 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:08 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/473815997' entity='client.admin' 2024-09-21T03:36:08.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:08 smithi134 ceph-mon[35031]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:08.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:08 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' 2024-09-21T03:36:08.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:08 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/473815997' entity='client.admin' 2024-09-21T03:36:08.852 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 21 03:36:08 smithi134 ceph-mgr[36297]: -- 172.21.15.134:0/601982224 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564932c070e0 con 0x5649379aa480 2024-09-21T03:36:08.852 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 21 03:36:08 smithi134 bash[36278]: 3cf07047e78696a6e98c49228c594927694375e481224d4a56c70de962fa0aad 2024-09-21T03:36:08.852 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 21 03:36:08 smithi134 systemd[1]: Started Ceph mgr.z for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:36:08.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:08 smithi123 ceph-mon[34673]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:08.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:08 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' 2024-09-21T03:36:08.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:08 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/473815997' entity='client.admin' 2024-09-21T03:36:09.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.412+0000 7ff0e8e00640 1 -- 172.21.15.16:0/2912923412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e410b6f0 msgr2=0x7ff0e410db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.412+0000 7ff0e8e00640 1 --2- 172.21.15.16:0/2912923412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e410b6f0 0x7ff0e410db90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009be0 tx=0x7ff0dc02a6c0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.412+0000 7ff0e8e00640 1 -- 172.21.15.16:0/2912923412 shutdown_connections 2024-09-21T03:36:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.412+0000 7ff0e8e00640 1 --2- 172.21.15.16:0/2912923412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e410b6f0 0x7ff0e410db90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.412+0000 7ff0e8e00640 1 --2- 172.21.15.16:0/2912923412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e4075cf0 0x7ff0e4076100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.412+0000 7ff0e8e00640 1 --2- 172.21.15.16:0/2912923412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0e40770e0 0x7ff0e40757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.412+0000 7ff0e8e00640 1 -- 172.21.15.16:0/2912923412 >> 172.21.15.16:0/2912923412 conn(0x7ff0e40ff0e0 msgr2=0x7ff0e4101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.412+0000 7ff0e8e00640 1 -- 172.21.15.16:0/2912923412 shutdown_connections 2024-09-21T03:36:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.412+0000 7ff0e8e00640 1 -- 172.21.15.16:0/2912923412 wait complete. 2024-09-21T03:36:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.413+0000 7ff0e8e00640 1 Processor -- start 2024-09-21T03:36:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.414+0000 7ff0e8e00640 1 -- start start 2024-09-21T03:36:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.414+0000 7ff0e8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e4075cf0 0x7ff0e419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.414+0000 7ff0e8e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e40770e0 0x7ff0e419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.414+0000 7ff0e8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0e410b6f0 0x7ff0e419d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.414+0000 7ff0e8e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e4111590 con 0x7ff0e4075cf0 2024-09-21T03:36:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.414+0000 7ff0e8e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e4111410 con 0x7ff0e410b6f0 2024-09-21T03:36:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.414+0000 7ff0e8e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e4111710 con 0x7ff0e40770e0 2024-09-21T03:36:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.414+0000 7ff0e1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e4075cf0 0x7ff0e419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.414+0000 7ff0e1400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e40770e0 0x7ff0e419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.414+0000 7ff0e1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e4075cf0 0x7ff0e419c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40598/0 (socket says 172.21.15.16:40598) 2024-09-21T03:36:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.414+0000 7ff0e1e00640 1 -- 172.21.15.16:0/2119350654 learned_addr learned my addr 172.21.15.16:0/2119350654 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.414+0000 7ff0e1400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e40770e0 0x7ff0e419c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:47476/0 (socket says 172.21.15.16:47476) 2024-09-21T03:36:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.414+0000 7ff0e2800640 1 --2- 172.21.15.16:0/2119350654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0e410b6f0 0x7ff0e419d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.415+0000 7ff0e1400640 1 -- 172.21.15.16:0/2119350654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0e410b6f0 msgr2=0x7ff0e419d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.415+0000 7ff0e1400640 1 --2- 172.21.15.16:0/2119350654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0e410b6f0 0x7ff0e419d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.415+0000 7ff0e1400640 1 -- 172.21.15.16:0/2119350654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e4075cf0 msgr2=0x7ff0e419c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.415+0000 7ff0e1400640 1 --2- 172.21.15.16:0/2119350654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e4075cf0 0x7ff0e419c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.415+0000 7ff0e1400640 1 -- 172.21.15.16:0/2119350654 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e419dc10 con 0x7ff0e40770e0 2024-09-21T03:36:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.415+0000 7ff0e1400640 1 --2- 172.21.15.16:0/2119350654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e40770e0 0x7ff0e419c850 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009fd0 tx=0x7ff0d000ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.415+0000 7ff0daa00640 1 -- 172.21.15.16:0/2119350654 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d000ccc0 con 0x7ff0e40770e0 2024-09-21T03:36:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.415+0000 7ff0daa00640 1 -- 172.21.15.16:0/2119350654 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0d000ee70 con 0x7ff0e40770e0 2024-09-21T03:36:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.415+0000 7ff0e8e00640 1 -- 172.21.15.16:0/2119350654 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e419df00 con 0x7ff0e40770e0 2024-09-21T03:36:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.415+0000 7ff0daa00640 1 -- 172.21.15.16:0/2119350654 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0004a00 con 0x7ff0e40770e0 2024-09-21T03:36:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.416+0000 7ff0e8e00640 1 -- 172.21.15.16:0/2119350654 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e41a4600 con 0x7ff0e40770e0 2024-09-21T03:36:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.417+0000 7ff0e8e00640 1 -- 172.21.15.16:0/2119350654 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0e41a47d0 con 0x7ff0e40770e0 2024-09-21T03:36:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.417+0000 7ff0daa00640 1 -- 172.21.15.16:0/2119350654 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7ff0d0004ba0 con 0x7ff0e40770e0 2024-09-21T03:36:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.418+0000 7ff0daa00640 1 --2- 172.21.15.16:0/2119350654 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff0c803fb20 0x7ff0c8042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.418+0000 7ff0daa00640 1 -- 172.21.15.16:0/2119350654 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff0d00543f0 con 0x7ff0e40770e0 2024-09-21T03:36:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.418+0000 7ff0e1e00640 1 --2- 172.21.15.16:0/2119350654 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff0c803fb20 0x7ff0c8042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.419+0000 7ff0e1e00640 1 --2- 172.21.15.16:0/2119350654 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff0c803fb20 0x7ff0c8042030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4005fd0 tx=0x7ff0d400b480 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.423+0000 7ff0daa00640 1 -- 172.21.15.16:0/2119350654 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff0d000f3c0 con 0x7ff0e40770e0 2024-09-21T03:36:09.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.642+0000 7ff0e8e00640 1 -- 172.21.15.16:0/2119350654 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7ff0e41a49a0 con 0x7ff0e40770e0 2024-09-21T03:36:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.648+0000 7ff0daa00640 1 -- 172.21.15.16:0/2119350654 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v21)=0 v21) ==== 111+0+0 (secure 0 0 0) 0x7ff0d0005020 con 0x7ff0e40770e0 2024-09-21T03:36:09.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.650+0000 7ff0e8e00640 1 -- 172.21.15.16:0/2119350654 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff0c803fb20 msgr2=0x7ff0c8042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.650+0000 7ff0e8e00640 1 --2- 172.21.15.16:0/2119350654 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff0c803fb20 0x7ff0c8042030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4005fd0 tx=0x7ff0d400b480 comp rx=0 tx=0).stop 2024-09-21T03:36:09.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.650+0000 7ff0e8e00640 1 -- 172.21.15.16:0/2119350654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e40770e0 msgr2=0x7ff0e419c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:09.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.650+0000 7ff0e8e00640 1 --2- 172.21.15.16:0/2119350654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e40770e0 0x7ff0e419c850 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009fd0 tx=0x7ff0d000ec20 comp rx=0 tx=0).stop 2024-09-21T03:36:09.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.650+0000 7ff0e8e00640 1 -- 172.21.15.16:0/2119350654 shutdown_connections 2024-09-21T03:36:09.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.650+0000 7ff0e8e00640 1 --2- 172.21.15.16:0/2119350654 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff0c803fb20 0x7ff0c8042030 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.650+0000 7ff0e8e00640 1 --2- 172.21.15.16:0/2119350654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0e410b6f0 0x7ff0e419d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.650+0000 7ff0e8e00640 1 --2- 172.21.15.16:0/2119350654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e40770e0 0x7ff0e419c850 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.650+0000 7ff0e8e00640 1 --2- 172.21.15.16:0/2119350654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e4075cf0 0x7ff0e419c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:09.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.650+0000 7ff0e8e00640 1 -- 172.21.15.16:0/2119350654 >> 172.21.15.16:0/2119350654 conn(0x7ff0e40ff0e0 msgr2=0x7ff0e40ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:09.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.650+0000 7ff0e8e00640 1 -- 172.21.15.16:0/2119350654 shutdown_connections 2024-09-21T03:36:09.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.650+0000 7ff0e8e00640 1 -- 172.21.15.16:0/2119350654 wait complete. 2024-09-21T03:36:09.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:09 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:09.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:09 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:09.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:09 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:09.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:09 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:36:09.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:09 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:36:09.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:09 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:36:09.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:09 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:09.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:09 smithi016 ceph-mon[30375]: Deploying daemon mgr.y on smithi123 2024-09-21T03:36:09.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:09 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T03:36:09.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:09 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:09.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:09 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:09.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:09 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:09.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:09 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:36:09.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:09 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:36:09.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:09 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:36:09.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:09 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:09.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:09 smithi134 ceph-mon[35031]: Deploying daemon mgr.y on smithi123 2024-09-21T03:36:09.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:09 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' 2024-09-21T03:36:09.979 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:09 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:09.979 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:09 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:09.979 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:09 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:09.979 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:09 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:36:09.979 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:09 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:36:09.980 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:09 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:36:09.980 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:09 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:09.980 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:09 smithi123 ceph-mon[34673]: Deploying daemon mgr.y on smithi123 2024-09-21T03:36:09.980 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:09 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' 2024-09-21T03:36:10.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.854+0000 7fb082a00640 1 -- 172.21.15.16:0/2715254642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb07c0770e0 msgr2=0x7fb07c0757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:10.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.854+0000 7fb082a00640 1 --2- 172.21.15.16:0/2715254642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb07c0770e0 0x7fb07c0757b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb070009900 tx=0x7fb07002a410 comp rx=0 tx=0).stop 2024-09-21T03:36:10.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.854+0000 7fb082a00640 1 -- 172.21.15.16:0/2715254642 shutdown_connections 2024-09-21T03:36:10.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.854+0000 7fb082a00640 1 --2- 172.21.15.16:0/2715254642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb07c10fd00 0x7fb07c112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.854+0000 7fb082a00640 1 --2- 172.21.15.16:0/2715254642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb07c075cf0 0x7fb07c076100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.854+0000 7fb082a00640 1 --2- 172.21.15.16:0/2715254642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb07c0770e0 0x7fb07c0757b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.854+0000 7fb082a00640 1 -- 172.21.15.16:0/2715254642 >> 172.21.15.16:0/2715254642 conn(0x7fb07c0feec0 msgr2=0x7fb07c101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:10.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.854+0000 7fb082a00640 1 -- 172.21.15.16:0/2715254642 shutdown_connections 2024-09-21T03:36:10.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.854+0000 7fb082a00640 1 -- 172.21.15.16:0/2715254642 wait complete. 2024-09-21T03:36:10.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.855+0000 7fb082a00640 1 Processor -- start 2024-09-21T03:36:10.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.855+0000 7fb082a00640 1 -- start start 2024-09-21T03:36:10.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb082a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb07c075cf0 0x7fb07c1a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:10.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb082a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb07c0770e0 0x7fb07c1a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb082a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb07c10fd00 0x7fb07c1a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb082a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c1157b0 con 0x7fb07c10fd00 2024-09-21T03:36:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb082a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c115630 con 0x7fb07c0770e0 2024-09-21T03:36:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb082a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c115930 con 0x7fb07c075cf0 2024-09-21T03:36:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb081600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb07c075cf0 0x7fb07c1a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb081600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb07c075cf0 0x7fb07c1a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:47506/0 (socket says 172.21.15.16:47506) 2024-09-21T03:36:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb081600640 1 -- 172.21.15.16:0/640618123 learned_addr learned my addr 172.21.15.16:0/640618123 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb082000640 1 --2- 172.21.15.16:0/640618123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb07c10fd00 0x7fb07c1a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb080c00640 1 --2- 172.21.15.16:0/640618123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb07c0770e0 0x7fb07c1a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb081600640 1 -- 172.21.15.16:0/640618123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb07c0770e0 msgr2=0x7fb07c1a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb081600640 1 --2- 172.21.15.16:0/640618123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb07c0770e0 0x7fb07c1a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb081600640 1 -- 172.21.15.16:0/640618123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb07c10fd00 msgr2=0x7fb07c1a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb081600640 1 --2- 172.21.15.16:0/640618123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb07c10fd00 0x7fb07c1a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb081600640 1 -- 172.21.15.16:0/640618123 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07c1a1e30 con 0x7fb07c075cf0 2024-09-21T03:36:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb080c00640 1 --2- 172.21.15.16:0/640618123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb07c0770e0 0x7fb07c1a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.856+0000 7fb082000640 1 --2- 172.21.15.16:0/640618123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb07c10fd00 0x7fb07c1a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.857+0000 7fb081600640 1 --2- 172.21.15.16:0/640618123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb07c075cf0 0x7fb07c1a0400 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb070005940 tx=0x7fb0700042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.857+0000 7fb07a000640 1 -- 172.21.15.16:0/640618123 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07003f030 con 0x7fb07c075cf0 2024-09-21T03:36:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.857+0000 7fb07a000640 1 -- 172.21.15.16:0/640618123 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb070003cb0 con 0x7fb07c075cf0 2024-09-21T03:36:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.857+0000 7fb082a00640 1 -- 172.21.15.16:0/640618123 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07c1a20c0 con 0x7fb07c075cf0 2024-09-21T03:36:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.857+0000 7fb07a000640 1 -- 172.21.15.16:0/640618123 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070035070 con 0x7fb07c075cf0 2024-09-21T03:36:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.857+0000 7fb082a00640 1 -- 172.21.15.16:0/640618123 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07c1a8650 con 0x7fb07c075cf0 2024-09-21T03:36:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.858+0000 7fb082a00640 1 -- 172.21.15.16:0/640618123 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb07c105d90 con 0x7fb07c075cf0 2024-09-21T03:36:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.859+0000 7fb07a000640 1 -- 172.21.15.16:0/640618123 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7fb070042030 con 0x7fb07c075cf0 2024-09-21T03:36:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.859+0000 7fb07a000640 1 --2- 172.21.15.16:0/640618123 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb06403fb20 0x7fb064042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.859+0000 7fb07a000640 1 -- 172.21.15.16:0/640618123 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb0700749d0 con 0x7fb07c075cf0 2024-09-21T03:36:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.859+0000 7fb080c00640 1 --2- 172.21.15.16:0/640618123 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb06403fb20 0x7fb064042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:10.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.860+0000 7fb080c00640 1 --2- 172.21.15.16:0/640618123 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb06403fb20 0x7fb064042030 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb068009fc0 tx=0x7fb068005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:10.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:09.864+0000 7fb07a000640 1 -- 172.21.15.16:0/640618123 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb07002e260 con 0x7fb07c075cf0 2024-09-21T03:36:10.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.066+0000 7fb082a00640 1 -- 172.21.15.16:0/640618123 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fb07c02e640 con 0x7fb07c075cf0 2024-09-21T03:36:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.071+0000 7fb07a000640 1 -- 172.21.15.16:0/640618123 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v22)=0 v22) ==== 108+0+0 (secure 0 0 0) 0x7fb07003b460 con 0x7fb07c075cf0 2024-09-21T03:36:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.073+0000 7fb082a00640 1 -- 172.21.15.16:0/640618123 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb06403fb20 msgr2=0x7fb064042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.073+0000 7fb082a00640 1 --2- 172.21.15.16:0/640618123 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb06403fb20 0x7fb064042030 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb068009fc0 tx=0x7fb068005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.074+0000 7fb082a00640 1 -- 172.21.15.16:0/640618123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb07c075cf0 msgr2=0x7fb07c1a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.074+0000 7fb082a00640 1 --2- 172.21.15.16:0/640618123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb07c075cf0 0x7fb07c1a0400 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb070005940 tx=0x7fb0700042f0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.074+0000 7fb082a00640 1 -- 172.21.15.16:0/640618123 shutdown_connections 2024-09-21T03:36:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.074+0000 7fb082a00640 1 --2- 172.21.15.16:0/640618123 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb06403fb20 0x7fb064042030 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.074+0000 7fb082a00640 1 --2- 172.21.15.16:0/640618123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb07c10fd00 0x7fb07c1a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.074+0000 7fb082a00640 1 --2- 172.21.15.16:0/640618123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb07c0770e0 0x7fb07c1a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.074+0000 7fb082a00640 1 --2- 172.21.15.16:0/640618123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb07c075cf0 0x7fb07c1a0400 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.074+0000 7fb082a00640 1 -- 172.21.15.16:0/640618123 >> 172.21.15.16:0/640618123 conn(0x7fb07c0feec0 msgr2=0x7fb07c102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.074+0000 7fb082a00640 1 -- 172.21.15.16:0/640618123 shutdown_connections 2024-09-21T03:36:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.074+0000 7fb082a00640 1 -- 172.21.15.16:0/640618123 wait complete. 2024-09-21T03:36:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.386+0000 7fcc80e00640 1 -- 172.21.15.16:0/2325999363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c1071c0 msgr2=0x7fcc7c10db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.386+0000 7fcc80e00640 1 --2- 172.21.15.16:0/2325999363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c1071c0 0x7fcc7c10db60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009ca0 tx=0x7fcc7402a950 comp rx=0 tx=0).stop 2024-09-21T03:36:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.386+0000 7fcc80e00640 1 -- 172.21.15.16:0/2325999363 shutdown_connections 2024-09-21T03:36:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.386+0000 7fcc80e00640 1 --2- 172.21.15.16:0/2325999363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c1071c0 0x7fcc7c10db60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.386+0000 7fcc80e00640 1 --2- 172.21.15.16:0/2325999363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc7c106820 0x7fcc7c106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.386+0000 7fcc80e00640 1 --2- 172.21.15.16:0/2325999363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc7c105790 0x7fcc7c105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.386+0000 7fcc80e00640 1 -- 172.21.15.16:0/2325999363 >> 172.21.15.16:0/2325999363 conn(0x7fcc7c101100 msgr2=0x7fcc7c1035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.386+0000 7fcc80e00640 1 -- 172.21.15.16:0/2325999363 shutdown_connections 2024-09-21T03:36:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.386+0000 7fcc80e00640 1 -- 172.21.15.16:0/2325999363 wait complete. 2024-09-21T03:36:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.387+0000 7fcc80e00640 1 Processor -- start 2024-09-21T03:36:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.387+0000 7fcc80e00640 1 -- start start 2024-09-21T03:36:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.388+0000 7fcc80e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc7c105790 0x7fcc7c19bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.388+0000 7fcc80e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c106820 0x7fcc7c19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.388+0000 7fcc80e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc7c1071c0 0x7fcc7c19d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.388+0000 7fcc80e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c111430 con 0x7fcc7c106820 2024-09-21T03:36:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.388+0000 7fcc80e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c1112b0 con 0x7fcc7c1071c0 2024-09-21T03:36:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.388+0000 7fcc80e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c1115b0 con 0x7fcc7c105790 2024-09-21T03:36:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.388+0000 7fcc7b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc7c105790 0x7fcc7c19bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.388+0000 7fcc7b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc7c105790 0x7fcc7c19bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:47522/0 (socket says 172.21.15.16:47522) 2024-09-21T03:36:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.388+0000 7fcc7b400640 1 -- 172.21.15.16:0/1703810319 learned_addr learned my addr 172.21.15.16:0/1703810319 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.388+0000 7fcc72a00640 1 --2- 172.21.15.16:0/1703810319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c106820 0x7fcc7c19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.388+0000 7fcc7be00640 1 --2- 172.21.15.16:0/1703810319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc7c1071c0 0x7fcc7c19d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.389+0000 7fcc72a00640 1 -- 172.21.15.16:0/1703810319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc7c105790 msgr2=0x7fcc7c19bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.389+0000 7fcc72a00640 1 --2- 172.21.15.16:0/1703810319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc7c105790 0x7fcc7c19bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.389+0000 7fcc72a00640 1 -- 172.21.15.16:0/1703810319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc7c1071c0 msgr2=0x7fcc7c19d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.389+0000 7fcc72a00640 1 --2- 172.21.15.16:0/1703810319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc7c1071c0 0x7fcc7c19d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.389+0000 7fcc72a00640 1 -- 172.21.15.16:0/1703810319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7c19d9d0 con 0x7fcc7c106820 2024-09-21T03:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.389+0000 7fcc7be00640 1 --2- 172.21.15.16:0/1703810319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc7c1071c0 0x7fcc7c19d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.389+0000 7fcc72a00640 1 --2- 172.21.15.16:0/1703810319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c106820 0x7fcc7c19c610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400e9f0 tx=0x7fcc6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.389+0000 7fcc78c00640 1 -- 172.21.15.16:0/1703810319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6400ccc0 con 0x7fcc7c106820 2024-09-21T03:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.390+0000 7fcc78c00640 1 -- 172.21.15.16:0/1703810319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc64005020 con 0x7fcc7c106820 2024-09-21T03:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.390+0000 7fcc80e00640 1 -- 172.21.15.16:0/1703810319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7c19dcc0 con 0x7fcc7c106820 2024-09-21T03:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.390+0000 7fcc78c00640 1 -- 172.21.15.16:0/1703810319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64004a00 con 0x7fcc7c106820 2024-09-21T03:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.390+0000 7fcc80e00640 1 -- 172.21.15.16:0/1703810319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc7c1a43c0 con 0x7fcc7c106820 2024-09-21T03:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.391+0000 7fcc80e00640 1 -- 172.21.15.16:0/1703810319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc7c10c950 con 0x7fcc7c106820 2024-09-21T03:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.391+0000 7fcc78c00640 1 -- 172.21.15.16:0/1703810319 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7fcc64004ba0 con 0x7fcc7c106820 2024-09-21T03:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.392+0000 7fcc78c00640 1 --2- 172.21.15.16:0/1703810319 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcc6003fb20 0x7fcc60042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.392+0000 7fcc78c00640 1 -- 172.21.15.16:0/1703810319 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcc64054530 con 0x7fcc7c106820 2024-09-21T03:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.392+0000 7fcc7b400640 1 --2- 172.21.15.16:0/1703810319 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcc6003fb20 0x7fcc60042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.393+0000 7fcc7b400640 1 --2- 172.21.15.16:0/1703810319 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcc6003fb20 0x7fcc60042030 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0095f0 tx=0x7fcc6c005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.398+0000 7fcc78c00640 1 -- 172.21.15.16:0/1703810319 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcc6401fe10 con 0x7fcc7c106820 2024-09-21T03:36:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.603+0000 7fcc80e00640 1 -- 172.21.15.16:0/1703810319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fcc7c063fc0 con 0x7fcc7c106820 2024-09-21T03:36:10.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.606+0000 7fcc78c00640 1 -- 172.21.15.16:0/1703810319 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v23)=0 v23) ==== 122+0+0 (secure 0 0 0) 0x7fcc640054f0 con 0x7fcc7c106820 2024-09-21T03:36:10.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.609+0000 7fcc80e00640 1 -- 172.21.15.16:0/1703810319 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcc6003fb20 msgr2=0x7fcc60042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:10.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.609+0000 7fcc80e00640 1 --2- 172.21.15.16:0/1703810319 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcc6003fb20 0x7fcc60042030 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0095f0 tx=0x7fcc6c005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:10.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.609+0000 7fcc80e00640 1 -- 172.21.15.16:0/1703810319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c106820 msgr2=0x7fcc7c19c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:10.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.609+0000 7fcc80e00640 1 --2- 172.21.15.16:0/1703810319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c106820 0x7fcc7c19c610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400e9f0 tx=0x7fcc6400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.609+0000 7fcc80e00640 1 -- 172.21.15.16:0/1703810319 shutdown_connections 2024-09-21T03:36:10.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.609+0000 7fcc80e00640 1 --2- 172.21.15.16:0/1703810319 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcc6003fb20 0x7fcc60042030 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.609+0000 7fcc80e00640 1 --2- 172.21.15.16:0/1703810319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc7c1071c0 0x7fcc7c19d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.609+0000 7fcc80e00640 1 --2- 172.21.15.16:0/1703810319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c106820 0x7fcc7c19c610 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.609+0000 7fcc80e00640 1 --2- 172.21.15.16:0/1703810319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc7c105790 0x7fcc7c19bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:10.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.609+0000 7fcc80e00640 1 -- 172.21.15.16:0/1703810319 >> 172.21.15.16:0/1703810319 conn(0x7fcc7c101100 msgr2=0x7fcc7c109b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:10.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.609+0000 7fcc80e00640 1 -- 172.21.15.16:0/1703810319 shutdown_connections 2024-09-21T03:36:10.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:10.609+0000 7fcc80e00640 1 -- 172.21.15.16:0/1703810319 wait complete. 2024-09-21T03:36:10.634 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 21 03:36:10 smithi123 systemd[1]: Starting Ceph mgr.y for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:36:10.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:10 smithi123 ceph-mon[34673]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:10.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:10 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' 2024-09-21T03:36:10.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:10 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1703810319' entity='client.admin' 2024-09-21T03:36:10.907 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 21 03:36:10 smithi123 podman[35476]: 2024-09-21 03:36:10.535285117 +0000 UTC m=+0.015611804 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:36:10.907 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 21 03:36:10 smithi123 podman[35476]: 2024-09-21 03:36:10.655288236 +0000 UTC m=+0.135614903 container create 40fbbada280ecf136d60f6a810bed83bf2d928e061a058ec8b05011a4ee973aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-y, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:36:10.907 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 21 03:36:10 smithi123 podman[35476]: 2024-09-21 03:36:10.841006956 +0000 UTC m=+0.321333638 container init 40fbbada280ecf136d60f6a810bed83bf2d928e061a058ec8b05011a4ee973aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-y, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T03:36:10.907 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 21 03:36:10 smithi123 podman[35476]: 2024-09-21 03:36:10.843648869 +0000 UTC m=+0.323975537 container start 40fbbada280ecf136d60f6a810bed83bf2d928e061a058ec8b05011a4ee973aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-y, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-21T03:36:10.907 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 21 03:36:10 smithi123 ceph-mgr[35495]: -- 172.21.15.123:0/3434376091 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5558f66210e0 con 0x5558fb3c4000 2024-09-21T03:36:10.969 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:10 smithi016 ceph-mon[30375]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:10.969 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:10 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T03:36:10.969 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:10 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1703810319' entity='client.admin' 2024-09-21T03:36:10.970 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:10 smithi134 ceph-mon[35031]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:10.970 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:10 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' 2024-09-21T03:36:10.970 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:10 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1703810319' entity='client.admin' 2024-09-21T03:36:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.275+0000 7feee3400640 1 -- 172.21.15.16:0/3935141585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc103750 msgr2=0x7feedc103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.275+0000 7feee3400640 1 --2- 172.21.15.16:0/3935141585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc103750 0x7feedc103bc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feed0009900 tx=0x7feed002a480 comp rx=0 tx=0).stop 2024-09-21T03:36:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.275+0000 7feee3400640 1 -- 172.21.15.16:0/3935141585 shutdown_connections 2024-09-21T03:36:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.275+0000 7feee3400640 1 --2- 172.21.15.16:0/3935141585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feedc105180 0x7feedc10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.275+0000 7feee3400640 1 --2- 172.21.15.16:0/3935141585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feedc1047e0 0x7feedc104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.275+0000 7feee3400640 1 --2- 172.21.15.16:0/3935141585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc103750 0x7feedc103bc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.275+0000 7feee3400640 1 -- 172.21.15.16:0/3935141585 >> 172.21.15.16:0/3935141585 conn(0x7feedc0ff0c0 msgr2=0x7feedc101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.275+0000 7feee3400640 1 -- 172.21.15.16:0/3935141585 shutdown_connections 2024-09-21T03:36:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.275+0000 7feee3400640 1 -- 172.21.15.16:0/3935141585 wait complete. 2024-09-21T03:36:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.277+0000 7feee3400640 1 Processor -- start 2024-09-21T03:36:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.277+0000 7feee3400640 1 -- start start 2024-09-21T03:36:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.277+0000 7feee3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feedc103750 0x7feedc19c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.277+0000 7feee3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feedc1047e0 0x7feedc19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.277+0000 7feee3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc105180 0x7feedc19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.277+0000 7feee3400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feedc10f3c0 con 0x7feedc1047e0 2024-09-21T03:36:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.277+0000 7feee3400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feedc10f240 con 0x7feedc105180 2024-09-21T03:36:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.277+0000 7feee3400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feedc10f540 con 0x7feedc103750 2024-09-21T03:36:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.278+0000 7feee0a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feedc103750 0x7feedc19c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.278+0000 7feee0a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feedc103750 0x7feedc19c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:47540/0 (socket says 172.21.15.16:47540) 2024-09-21T03:36:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.278+0000 7feee0a00640 1 -- 172.21.15.16:0/1467519042 learned_addr learned my addr 172.21.15.16:0/1467519042 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.278+0000 7feee1400640 1 --2- 172.21.15.16:0/1467519042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc105180 0x7feedc19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.278+0000 7feedbe00640 1 --2- 172.21.15.16:0/1467519042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feedc1047e0 0x7feedc19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.278+0000 7feee0a00640 1 -- 172.21.15.16:0/1467519042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc105180 msgr2=0x7feedc19d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.278+0000 7feee0a00640 1 --2- 172.21.15.16:0/1467519042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc105180 0x7feedc19d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.278+0000 7feee0a00640 1 -- 172.21.15.16:0/1467519042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feedc1047e0 msgr2=0x7feedc19c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.278+0000 7feee0a00640 1 --2- 172.21.15.16:0/1467519042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feedc1047e0 0x7feedc19c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.278+0000 7feee0a00640 1 -- 172.21.15.16:0/1467519042 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feedc19dbf0 con 0x7feedc103750 2024-09-21T03:36:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.278+0000 7feee0a00640 1 --2- 172.21.15.16:0/1467519042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feedc103750 0x7feedc19c1c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feed0031440 tx=0x7feed0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.279+0000 7feed9600640 1 -- 172.21.15.16:0/1467519042 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed003f070 con 0x7feedc103750 2024-09-21T03:36:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.279+0000 7feed9600640 1 -- 172.21.15.16:0/1467519042 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed0031c50 con 0x7feedc103750 2024-09-21T03:36:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.279+0000 7feee3400640 1 -- 172.21.15.16:0/1467519042 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feedc19de80 con 0x7feedc103750 2024-09-21T03:36:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.279+0000 7feed9600640 1 -- 172.21.15.16:0/1467519042 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0035030 con 0x7feedc103750 2024-09-21T03:36:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.279+0000 7feee3400640 1 -- 172.21.15.16:0/1467519042 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feedc1a4410 con 0x7feedc103750 2024-09-21T03:36:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.280+0000 7feee3400640 1 -- 172.21.15.16:0/1467519042 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feedc1a4730 con 0x7feedc103750 2024-09-21T03:36:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.281+0000 7feed9600640 1 -- 172.21.15.16:0/1467519042 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7feed0042070 con 0x7feedc103750 2024-09-21T03:36:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.282+0000 7feed9600640 1 --2- 172.21.15.16:0/1467519042 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7feec403fb20 0x7feec4042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.282+0000 7feed9600640 1 -- 172.21.15.16:0/1467519042 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7feed0074e90 con 0x7feedc103750 2024-09-21T03:36:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.282+0000 7feedbe00640 1 --2- 172.21.15.16:0/1467519042 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7feec403fb20 0x7feec4042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.283+0000 7feedbe00640 1 --2- 172.21.15.16:0/1467519042 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7feec403fb20 0x7feec4042030 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7feec80095f0 tx=0x7feec8005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.287+0000 7feed9600640 1 -- 172.21.15.16:0/1467519042 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7feed0045d70 con 0x7feedc103750 2024-09-21T03:36:11.406 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 21 03:36:10 smithi123 bash[35476]: 40fbbada280ecf136d60f6a810bed83bf2d928e061a058ec8b05011a4ee973aa 2024-09-21T03:36:11.406 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 21 03:36:10 smithi123 systemd[1]: Started Ceph mgr.y for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:36:11.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.493+0000 7feee3400640 1 -- 172.21.15.16:0/1467519042 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7feedc1a4900 con 0x7feedc103750 2024-09-21T03:36:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.499+0000 7feed9600640 1 -- 172.21.15.16:0/1467519042 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v24)=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7feed003b460 con 0x7feedc103750 2024-09-21T03:36:11.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.501+0000 7feee3400640 1 -- 172.21.15.16:0/1467519042 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7feec403fb20 msgr2=0x7feec4042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:11.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.501+0000 7feee3400640 1 --2- 172.21.15.16:0/1467519042 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7feec403fb20 0x7feec4042030 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7feec80095f0 tx=0x7feec8005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:11.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.501+0000 7feee3400640 1 -- 172.21.15.16:0/1467519042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feedc103750 msgr2=0x7feedc19c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:11.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.501+0000 7feee3400640 1 --2- 172.21.15.16:0/1467519042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feedc103750 0x7feedc19c1c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feed0031440 tx=0x7feed0003c10 comp rx=0 tx=0).stop 2024-09-21T03:36:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.502+0000 7feee3400640 1 -- 172.21.15.16:0/1467519042 shutdown_connections 2024-09-21T03:36:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.502+0000 7feee3400640 1 --2- 172.21.15.16:0/1467519042 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7feec403fb20 0x7feec4042030 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.502+0000 7feee3400640 1 --2- 172.21.15.16:0/1467519042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc105180 0x7feedc19d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.502+0000 7feee3400640 1 --2- 172.21.15.16:0/1467519042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feedc1047e0 0x7feedc19c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.502+0000 7feee3400640 1 --2- 172.21.15.16:0/1467519042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feedc103750 0x7feedc19c1c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.502+0000 7feee3400640 1 -- 172.21.15.16:0/1467519042 >> 172.21.15.16:0/1467519042 conn(0x7feedc0ff0c0 msgr2=0x7feedc0755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.502+0000 7feee3400640 1 -- 172.21.15.16:0/1467519042 shutdown_connections 2024-09-21T03:36:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.503+0000 7feee3400640 1 -- 172.21.15.16:0/1467519042 wait complete. 2024-09-21T03:36:12.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:11 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:12.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:11 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:12.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:11 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:12.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:11 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:12.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:11 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:12.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:11 smithi016 ceph-mon[30375]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:12.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:11 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T03:36:12.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:11 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T03:36:12.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:11 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:12.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:11 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:12.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:11 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:12.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:11 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:12.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:11 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:12.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:11 smithi134 ceph-mon[35031]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:12.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:11 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' 2024-09-21T03:36:12.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:11 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' 2024-09-21T03:36:12.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:11 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:12.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:11 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:12.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:11 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:12.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:11 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:12.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:11 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:12.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:11 smithi123 ceph-mon[34673]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:12.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:11 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' 2024-09-21T03:36:12.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:11 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' 2024-09-21T03:36:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.109+0000 7f87ab800640 1 -- 172.21.15.16:0/2724877360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87a4105180 msgr2=0x7f87a410bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.109+0000 7f87ab800640 1 --2- 172.21.15.16:0/2724877360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87a4105180 0x7f87a410bb20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f879c009be0 tx=0x7f879c02a750 comp rx=0 tx=0).stop 2024-09-21T03:36:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.109+0000 7f87ab800640 1 -- 172.21.15.16:0/2724877360 shutdown_connections 2024-09-21T03:36:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.109+0000 7f87ab800640 1 --2- 172.21.15.16:0/2724877360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87a4105180 0x7f87a410bb20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.109+0000 7f87ab800640 1 --2- 172.21.15.16:0/2724877360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a41047e0 0x7f87a4104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.109+0000 7f87ab800640 1 --2- 172.21.15.16:0/2724877360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87a4103750 0x7f87a4103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.109+0000 7f87ab800640 1 -- 172.21.15.16:0/2724877360 >> 172.21.15.16:0/2724877360 conn(0x7f87a40ff0c0 msgr2=0x7f87a4101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:13.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.109+0000 7f87ab800640 1 -- 172.21.15.16:0/2724877360 shutdown_connections 2024-09-21T03:36:13.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.109+0000 7f87ab800640 1 -- 172.21.15.16:0/2724877360 wait complete. 2024-09-21T03:36:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.110+0000 7f87ab800640 1 Processor -- start 2024-09-21T03:36:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.111+0000 7f87ab800640 1 -- start start 2024-09-21T03:36:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.111+0000 7f87ab800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a4103750 0x7f87a419c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.111+0000 7f87ab800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87a41047e0 0x7f87a419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.111+0000 7f87ab800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87a4105180 0x7f87a419d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.111+0000 7f87ab800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87a410f380 con 0x7f87a4103750 2024-09-21T03:36:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.111+0000 7f87ab800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f87a410f200 con 0x7f87a41047e0 2024-09-21T03:36:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.111+0000 7f87ab800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87a410f500 con 0x7f87a4105180 2024-09-21T03:36:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.111+0000 7f879be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87a41047e0 0x7f87a419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.111+0000 7f87a8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a4103750 0x7f87a419c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.111+0000 7f87a9800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87a4105180 0x7f87a419d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:13.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.111+0000 7f879be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87a41047e0 0x7f87a419c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34874/0 (socket says 172.21.15.16:34874) 2024-09-21T03:36:13.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.111+0000 7f879be00640 1 -- 172.21.15.16:0/2075064140 learned_addr learned my addr 172.21.15.16:0/2075064140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:13.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.112+0000 7f87a9800640 1 -- 172.21.15.16:0/2075064140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87a41047e0 msgr2=0x7f87a419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.112+0000 7f87a9800640 1 --2- 172.21.15.16:0/2075064140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87a41047e0 0x7f87a419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.112+0000 7f87a9800640 1 -- 172.21.15.16:0/2075064140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a4103750 msgr2=0x7f87a419c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.112+0000 7f87a9800640 1 --2- 172.21.15.16:0/2075064140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a4103750 0x7f87a419c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.112+0000 7f87a9800640 1 -- 172.21.15.16:0/2075064140 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a419db90 con 0x7f87a4105180 2024-09-21T03:36:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.112+0000 7f87a9800640 1 --2- 172.21.15.16:0/2075064140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87a4105180 0x7f87a419d370 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f879c034040 tx=0x7f879c030d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.112+0000 7f87a2000640 1 -- 172.21.15.16:0/2075064140 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c041020 con 0x7f87a4105180 2024-09-21T03:36:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.112+0000 7f87a2000640 1 -- 172.21.15.16:0/2075064140 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f879c0318f0 con 0x7f87a4105180 2024-09-21T03:36:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.112+0000 7f87a2000640 1 -- 172.21.15.16:0/2075064140 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c036040 con 0x7f87a4105180 2024-09-21T03:36:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.112+0000 7f87ab800640 1 -- 172.21.15.16:0/2075064140 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a419de20 con 0x7f87a4105180 2024-09-21T03:36:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.113+0000 7f87ab800640 1 -- 172.21.15.16:0/2075064140 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a41a43b0 con 0x7f87a4105180 2024-09-21T03:36:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.114+0000 7f87ab800640 1 -- 172.21.15.16:0/2075064140 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a41a4610 con 0x7f87a4105180 2024-09-21T03:36:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.114+0000 7f87a2000640 1 -- 172.21.15.16:0/2075064140 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f879c044020 con 0x7f87a4105180 2024-09-21T03:36:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.115+0000 7f87a2000640 1 --2- 172.21.15.16:0/2075064140 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f878803fb20 0x7f8788042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.115+0000 7f87a2000640 1 -- 172.21.15.16:0/2075064140 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f879c075140 con 0x7f87a4105180 2024-09-21T03:36:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.115+0000 7f87a8e00640 1 --2- 172.21.15.16:0/2075064140 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f878803fb20 0x7f8788042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.116+0000 7f87a8e00640 1 --2- 172.21.15.16:0/2075064140 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f878803fb20 0x7f8788042030 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8794009530 tx=0x7f8794009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.120+0000 7f87a2000640 1 -- 172.21.15.16:0/2075064140 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f879c04b020 con 0x7f87a4105180 2024-09-21T03:36:13.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.326+0000 7f87ab800640 1 -- 172.21.15.16:0/2075064140 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f87a41a48c0 con 0x7f87a4105180 2024-09-21T03:36:13.328 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:13.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:13.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:13.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:13.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:13.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:13 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T03:36:13.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:13.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:13.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:13.330 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.334+0000 7f87a2000640 1 -- 172.21.15.16:0/2075064140 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v27)=0 v27) ==== 116+0+0 (secure 0 0 0) 0x7f879c03bd80 con 0x7f87a4105180 2024-09-21T03:36:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.337+0000 7f87ab800640 1 -- 172.21.15.16:0/2075064140 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f878803fb20 msgr2=0x7f8788042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.337+0000 7f87ab800640 1 --2- 172.21.15.16:0/2075064140 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f878803fb20 0x7f8788042030 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8794009530 tx=0x7f8794009260 comp rx=0 tx=0).stop 2024-09-21T03:36:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.337+0000 7f87ab800640 1 -- 172.21.15.16:0/2075064140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87a4105180 msgr2=0x7f87a419d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.337+0000 7f87ab800640 1 --2- 172.21.15.16:0/2075064140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87a4105180 0x7f87a419d370 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f879c034040 tx=0x7f879c030d20 comp rx=0 tx=0).stop 2024-09-21T03:36:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.337+0000 7f87ab800640 1 -- 172.21.15.16:0/2075064140 shutdown_connections 2024-09-21T03:36:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.337+0000 7f87ab800640 1 --2- 172.21.15.16:0/2075064140 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f878803fb20 0x7f8788042030 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.337+0000 7f87ab800640 1 --2- 172.21.15.16:0/2075064140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87a4105180 0x7f87a419d370 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.337+0000 7f87ab800640 1 --2- 172.21.15.16:0/2075064140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87a41047e0 0x7f87a419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.337+0000 7f87ab800640 1 --2- 172.21.15.16:0/2075064140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87a4103750 0x7f87a419c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.337+0000 7f87ab800640 1 -- 172.21.15.16:0/2075064140 >> 172.21.15.16:0/2075064140 conn(0x7f87a40ff0c0 msgr2=0x7f87a40755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.337+0000 7f87ab800640 1 -- 172.21.15.16:0/2075064140 shutdown_connections 2024-09-21T03:36:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.337+0000 7f87ab800640 1 -- 172.21.15.16:0/2075064140 wait complete. 2024-09-21T03:36:13.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:13.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:13.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:13.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:13.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:13.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:13 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' 2024-09-21T03:36:13.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:13.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:13.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:13.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:13.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:13.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:13.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:13.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:13.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:13 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' 2024-09-21T03:36:13.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:13.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:13.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:13.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:13.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.855+0000 7f25cd800640 1 -- 172.21.15.16:0/2754590250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c81047e0 msgr2=0x7f25c8104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.855+0000 7f25cd800640 1 --2- 172.21.15.16:0/2754590250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c81047e0 0x7f25c8104bf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009900 tx=0x7f25b002a480 comp rx=0 tx=0).stop 2024-09-21T03:36:13.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.855+0000 7f25cd800640 1 -- 172.21.15.16:0/2754590250 shutdown_connections 2024-09-21T03:36:13.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.855+0000 7f25cd800640 1 --2- 172.21.15.16:0/2754590250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8105180 0x7f25c810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.855+0000 7f25cd800640 1 --2- 172.21.15.16:0/2754590250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c81047e0 0x7f25c8104bf0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.855+0000 7f25cd800640 1 --2- 172.21.15.16:0/2754590250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25c8103750 0x7f25c8103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.855+0000 7f25cd800640 1 -- 172.21.15.16:0/2754590250 >> 172.21.15.16:0/2754590250 conn(0x7f25c80ff0e0 msgr2=0x7f25c8101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:13.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.855+0000 7f25cd800640 1 -- 172.21.15.16:0/2754590250 shutdown_connections 2024-09-21T03:36:13.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.855+0000 7f25cd800640 1 -- 172.21.15.16:0/2754590250 wait complete. 2024-09-21T03:36:13.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.856+0000 7f25cd800640 1 Processor -- start 2024-09-21T03:36:13.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.856+0000 7f25cd800640 1 -- start start 2024-09-21T03:36:13.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.857+0000 7f25cd800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c8103750 0x7f25c819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:13.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.857+0000 7f25cd800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25c81047e0 0x7f25c819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:13.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.857+0000 7f25cd800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8105180 0x7f25c819d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:13.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.857+0000 7f25cd800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f25c810f3d0 con 0x7f25c8105180 2024-09-21T03:36:13.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.857+0000 7f25cd800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25c810f250 con 0x7f25c81047e0 2024-09-21T03:36:13.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.857+0000 7f25cd800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f25c810f550 con 0x7f25c8103750 2024-09-21T03:36:13.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.857+0000 7f25c6a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c8103750 0x7f25c819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:13.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.857+0000 7f25bfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25c81047e0 0x7f25c819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:13.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.857+0000 7f25c6a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c8103750 0x7f25c819c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:48510/0 (socket says 172.21.15.16:48510) 2024-09-21T03:36:13.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.857+0000 7f25c6a00640 1 -- 172.21.15.16:0/166021684 learned_addr learned my addr 172.21.15.16:0/166021684 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:13.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.857+0000 7f25c7400640 1 --2- 172.21.15.16:0/166021684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8105180 0x7f25c819d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:13.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.857+0000 7f25c6a00640 1 -- 172.21.15.16:0/166021684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25c81047e0 msgr2=0x7f25c819c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.857+0000 7f25c6a00640 1 --2- 172.21.15.16:0/166021684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25c81047e0 0x7f25c819c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.857+0000 7f25c6a00640 1 -- 172.21.15.16:0/166021684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8105180 msgr2=0x7f25c819d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:13.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.857+0000 7f25c6a00640 1 --2- 172.21.15.16:0/166021684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8105180 0x7f25c819d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:13.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.857+0000 7f25c6a00640 1 -- 172.21.15.16:0/166021684 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c819dc30 con 0x7f25c8103750 2024-09-21T03:36:13.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.858+0000 7f25c6a00640 1 --2- 172.21.15.16:0/166021684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c8103750 0x7f25c819c270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f25b800e9f0 tx=0x7f25b800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:13.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.858+0000 7f25bf400640 1 -- 172.21.15.16:0/166021684 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b800cd80 con 0x7f25c8103750 2024-09-21T03:36:13.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.858+0000 7f25bf400640 1 -- 172.21.15.16:0/166021684 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25b80108e0 con 0x7f25c8103750 2024-09-21T03:36:13.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.858+0000 7f25bf400640 1 -- 172.21.15.16:0/166021684 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8018980 con 0x7f25c8103750 2024-09-21T03:36:13.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.858+0000 7f25cd800640 1 -- 172.21.15.16:0/166021684 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c819df20 con 0x7f25c8103750 2024-09-21T03:36:13.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.858+0000 7f25cd800640 1 -- 172.21.15.16:0/166021684 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c81a4620 con 0x7f25c8103750 2024-09-21T03:36:13.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.859+0000 7f25bf400640 1 -- 172.21.15.16:0/166021684 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f25b8018b20 con 0x7f25c8103750 2024-09-21T03:36:13.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.859+0000 7f25cd800640 1 -- 172.21.15.16:0/166021684 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c81a47f0 con 0x7f25c8103750 2024-09-21T03:36:13.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.860+0000 7f25bf400640 1 --2- 172.21.15.16:0/166021684 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f25ac03fb20 0x7f25ac042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:13.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.860+0000 7f25bf400640 1 -- 172.21.15.16:0/166021684 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f25b8054a00 con 0x7f25c8103750 2024-09-21T03:36:13.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.860+0000 7f25bfe00640 1 --2- 172.21.15.16:0/166021684 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f25ac03fb20 0x7f25ac042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:13.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.861+0000 7f25bfe00640 1 --2- 172.21.15.16:0/166021684 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f25ac03fb20 0x7f25ac042030 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f25b0004770 tx=0x7f25b00314a0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:13.869+0000 7f25bf400640 1 -- 172.21.15.16:0/166021684 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f25b80071c0 con 0x7f25c8103750 2024-09-21T03:36:14.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:14.077+0000 7f25cd800640 1 -- 172.21.15.16:0/166021684 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f25c81a49c0 con 0x7f25c8103750 2024-09-21T03:36:14.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:14.078+0000 7f25bf400640 1 -- 172.21.15.16:0/166021684 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v27)=0 v27) ==== 108+0+0 (secure 0 0 0) 0x7f25b8018e10 con 0x7f25c8103750 2024-09-21T03:36:14.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:14.080+0000 7f25cd800640 1 -- 172.21.15.16:0/166021684 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f25ac03fb20 msgr2=0x7f25ac042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:14.080+0000 7f25cd800640 1 --2- 172.21.15.16:0/166021684 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f25ac03fb20 0x7f25ac042030 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f25b0004770 tx=0x7f25b00314a0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:14.080+0000 7f25cd800640 1 -- 172.21.15.16:0/166021684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c8103750 msgr2=0x7f25c819c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:14.080+0000 7f25cd800640 1 --2- 172.21.15.16:0/166021684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c8103750 0x7f25c819c270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f25b800e9f0 tx=0x7f25b800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:14.081+0000 7f25cd800640 1 -- 172.21.15.16:0/166021684 shutdown_connections 2024-09-21T03:36:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:14.081+0000 7f25cd800640 1 --2- 172.21.15.16:0/166021684 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f25ac03fb20 0x7f25ac042030 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:14.081+0000 7f25cd800640 1 --2- 172.21.15.16:0/166021684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8105180 0x7f25c819d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:14.081+0000 7f25cd800640 1 --2- 172.21.15.16:0/166021684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25c81047e0 0x7f25c819c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:14.081+0000 7f25cd800640 1 --2- 172.21.15.16:0/166021684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c8103750 0x7f25c819c270 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:14.081+0000 7f25cd800640 1 -- 172.21.15.16:0/166021684 >> 172.21.15.16:0/166021684 conn(0x7f25c80ff0e0 msgr2=0x7f25c80755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:14.081+0000 7f25cd800640 1 -- 172.21.15.16:0/166021684 shutdown_connections 2024-09-21T03:36:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:14.081+0000 7f25cd800640 1 -- 172.21.15.16:0/166021684 wait complete. 2024-09-21T03:36:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.299+0000 7f804be00640 1 -- 172.21.15.16:0/2300154431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f804c103570 msgr2=0x7f804c1039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.299+0000 7f804be00640 1 --2- 172.21.15.16:0/2300154431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f804c103570 0x7f804c1039e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f803c009900 tx=0x7f803c02a480 comp rx=0 tx=0).stop 2024-09-21T03:36:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.299+0000 7f804be00640 1 -- 172.21.15.16:0/2300154431 shutdown_connections 2024-09-21T03:36:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.299+0000 7f804be00640 1 --2- 172.21.15.16:0/2300154431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f804c104fa0 0x7f804c10b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.299+0000 7f804be00640 1 --2- 172.21.15.16:0/2300154431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c104600 0x7f804c104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.299+0000 7f804be00640 1 --2- 172.21.15.16:0/2300154431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f804c103570 0x7f804c1039e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.299+0000 7f804be00640 1 -- 172.21.15.16:0/2300154431 >> 172.21.15.16:0/2300154431 conn(0x7f804c0feec0 msgr2=0x7f804c101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.299+0000 7f804be00640 1 -- 172.21.15.16:0/2300154431 shutdown_connections 2024-09-21T03:36:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.299+0000 7f804be00640 1 -- 172.21.15.16:0/2300154431 wait complete. 2024-09-21T03:36:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.300+0000 7f804be00640 1 Processor -- start 2024-09-21T03:36:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.300+0000 7f804be00640 1 -- start start 2024-09-21T03:36:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.301+0000 7f804be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f804c103570 0x7f804c19c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.301+0000 7f804be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c104600 0x7f804c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.301+0000 7f804be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f804c104fa0 0x7f804c19d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.301+0000 7f804be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f804c10f1f0 con 0x7f804c104600 2024-09-21T03:36:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.301+0000 7f804be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f804c10f070 con 0x7f804c103570 2024-09-21T03:36:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.301+0000 7f804be00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f804c10f370 con 0x7f804c104fa0 2024-09-21T03:36:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.301+0000 7f804aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f804c103570 0x7f804c19c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.301+0000 7f804aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f804c103570 0x7f804c19c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34854/0 (socket says 172.21.15.16:34854) 2024-09-21T03:36:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.301+0000 7f804aa00640 1 -- 172.21.15.16:0/1384807131 learned_addr learned my addr 172.21.15.16:0/1384807131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.301+0000 7f804a000640 1 --2- 172.21.15.16:0/1384807131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c104600 0x7f804c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.301+0000 7f804b400640 1 --2- 172.21.15.16:0/1384807131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f804c104fa0 0x7f804c19d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.301+0000 7f804aa00640 1 -- 172.21.15.16:0/1384807131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f804c104fa0 msgr2=0x7f804c19d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.301+0000 7f804aa00640 1 --2- 172.21.15.16:0/1384807131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f804c104fa0 0x7f804c19d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.301+0000 7f804aa00640 1 -- 172.21.15.16:0/1384807131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c104600 msgr2=0x7f804c19c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.301+0000 7f804aa00640 1 --2- 172.21.15.16:0/1384807131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c104600 0x7f804c19c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.301+0000 7f804aa00640 1 -- 172.21.15.16:0/1384807131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c19da50 con 0x7f804c103570 2024-09-21T03:36:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.302+0000 7f804aa00640 1 --2- 172.21.15.16:0/1384807131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f804c103570 0x7f804c19c020 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f803c031440 tx=0x7f803c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.302+0000 7f8043400640 1 -- 172.21.15.16:0/1384807131 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c03f070 con 0x7f804c103570 2024-09-21T03:36:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.302+0000 7f804be00640 1 -- 172.21.15.16:0/1384807131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804c19dce0 con 0x7f804c103570 2024-09-21T03:36:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.302+0000 7f8043400640 1 -- 172.21.15.16:0/1384807131 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f803c031c50 con 0x7f804c103570 2024-09-21T03:36:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.302+0000 7f8043400640 1 -- 172.21.15.16:0/1384807131 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c035030 con 0x7f804c103570 2024-09-21T03:36:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.302+0000 7f804be00640 1 -- 172.21.15.16:0/1384807131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f804c1a4440 con 0x7f804c103570 2024-09-21T03:36:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.303+0000 7f804be00640 1 -- 172.21.15.16:0/1384807131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f804c1039e0 con 0x7f804c103570 2024-09-21T03:36:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.304+0000 7f8043400640 1 -- 172.21.15.16:0/1384807131 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f803c042070 con 0x7f804c103570 2024-09-21T03:36:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.304+0000 7f8043400640 1 --2- 172.21.15.16:0/1384807131 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f803003fb20 0x7f8030042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.304+0000 7f8043400640 1 -- 172.21.15.16:0/1384807131 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f803c074da0 con 0x7f804c103570 2024-09-21T03:36:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.304+0000 7f804a000640 1 --2- 172.21.15.16:0/1384807131 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f803003fb20 0x7f8030042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.305+0000 7f804a000640 1 --2- 172.21.15.16:0/1384807131 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f803003fb20 0x7f8030042030 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8038009bd0 tx=0x7f8038005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.314+0000 7f8043400640 1 -- 172.21.15.16:0/1384807131 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f803c03b460 con 0x7f804c103570 2024-09-21T03:36:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.517+0000 7f804be00640 1 -- 172.21.15.16:0/1384807131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f804c1a47d0 con 0x7f804c103570 2024-09-21T03:36:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.523+0000 7f8043400640 1 -- 172.21.15.16:0/1384807131 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v26)=0 v26) ==== 125+0+0 (secure 0 0 0) 0x7f803c0317a0 con 0x7f804c103570 2024-09-21T03:36:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.525+0000 7f804be00640 1 -- 172.21.15.16:0/1384807131 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f803003fb20 msgr2=0x7f8030042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.525+0000 7f804be00640 1 --2- 172.21.15.16:0/1384807131 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f803003fb20 0x7f8030042030 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8038009bd0 tx=0x7f8038005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.525+0000 7f804be00640 1 -- 172.21.15.16:0/1384807131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f804c103570 msgr2=0x7f804c19c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.525+0000 7f804be00640 1 --2- 172.21.15.16:0/1384807131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f804c103570 0x7f804c19c020 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f803c031440 tx=0x7f803c003c10 comp rx=0 tx=0).stop 2024-09-21T03:36:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.525+0000 7f804be00640 1 -- 172.21.15.16:0/1384807131 shutdown_connections 2024-09-21T03:36:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.525+0000 7f804be00640 1 --2- 172.21.15.16:0/1384807131 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f803003fb20 0x7f8030042030 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.525+0000 7f804be00640 1 --2- 172.21.15.16:0/1384807131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f804c104fa0 0x7f804c19d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.525+0000 7f804be00640 1 --2- 172.21.15.16:0/1384807131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c104600 0x7f804c19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.525+0000 7f804be00640 1 --2- 172.21.15.16:0/1384807131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f804c103570 0x7f804c19c020 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.525+0000 7f804be00640 1 -- 172.21.15.16:0/1384807131 >> 172.21.15.16:0/1384807131 conn(0x7f804c0feec0 msgr2=0x7f804c0755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.525+0000 7f804be00640 1 -- 172.21.15.16:0/1384807131 shutdown_connections 2024-09-21T03:36:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:12.526+0000 7f804be00640 1 -- 172.21.15.16:0/1384807131 wait complete. 2024-09-21T03:36:14.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:14 smithi134 ceph-mon[35031]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:14.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:14 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' 2024-09-21T03:36:14.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:14 smithi134 ceph-mon[35031]: Standby manager daemon z started 2024-09-21T03:36:14.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:14 smithi134 ceph-mon[35031]: from='mgr.? 172.21.15.134:0/3886928278' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:36:14.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:14 smithi134 ceph-mon[35031]: from='mgr.? 172.21.15.134:0/3886928278' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:36:14.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:14 smithi134 ceph-mon[35031]: from='mgr.? 172.21.15.134:0/3886928278' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:36:14.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:14 smithi134 ceph-mon[35031]: from='mgr.? 172.21.15.134:0/3886928278' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:36:14.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:14 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:14.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:14 smithi123 ceph-mon[34673]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:14.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:14 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' 2024-09-21T03:36:14.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:14 smithi123 ceph-mon[34673]: Standby manager daemon z started 2024-09-21T03:36:14.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:14 smithi123 ceph-mon[34673]: from='mgr.? 172.21.15.134:0/3886928278' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:36:14.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:14 smithi123 ceph-mon[34673]: from='mgr.? 172.21.15.134:0/3886928278' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:36:14.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:14 smithi123 ceph-mon[34673]: from='mgr.? 172.21.15.134:0/3886928278' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:36:14.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:14 smithi123 ceph-mon[34673]: from='mgr.? 172.21.15.134:0/3886928278' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:36:14.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:14 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:14.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:14 smithi016 ceph-mon[30375]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:14.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:14 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T03:36:14.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:14 smithi016 ceph-mon[30375]: Standby manager daemon z started 2024-09-21T03:36:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:14 smithi016 ceph-mon[30375]: from='mgr.? 172.21.15.134:0/3886928278' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:36:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:14 smithi016 ceph-mon[30375]: from='mgr.? 172.21.15.134:0/3886928278' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:36:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:14 smithi016 ceph-mon[30375]: from='mgr.? 172.21.15.134:0/3886928278' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:36:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:14 smithi016 ceph-mon[30375]: from='mgr.? 172.21.15.134:0/3886928278' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:36:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:14 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:15.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:15 smithi134 ceph-mon[35031]: mgrmap e14: x(active, since 81s), standbys: z 2024-09-21T03:36:15.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:15 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:36:15.652 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:15 smithi016 ceph-mon[30375]: mgrmap e14: x(active, since 81s), standbys: z 2024-09-21T03:36:15.652 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:15 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:36:15.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:15 smithi123 ceph-mon[34673]: mgrmap e14: x(active, since 81s), standbys: z 2024-09-21T03:36:15.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:15 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:36:15.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.831+0000 7f8667000640 1 -- 172.21.15.16:0/1997475788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8660104fe0 msgr2=0x7f866010b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:15.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.831+0000 7f8667000640 1 --2- 172.21.15.16:0/1997475788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8660104fe0 0x7f866010b980 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8658009ca0 tx=0x7f865802a950 comp rx=0 tx=0).stop 2024-09-21T03:36:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.831+0000 7f8667000640 1 -- 172.21.15.16:0/1997475788 shutdown_connections 2024-09-21T03:36:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.831+0000 7f8667000640 1 --2- 172.21.15.16:0/1997475788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8660104fe0 0x7f866010b980 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.831+0000 7f8667000640 1 --2- 172.21.15.16:0/1997475788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8660104640 0x7f8660104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.831+0000 7f8667000640 1 --2- 172.21.15.16:0/1997475788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86601035b0 0x7f8660103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.831+0000 7f8667000640 1 -- 172.21.15.16:0/1997475788 >> 172.21.15.16:0/1997475788 conn(0x7f86600fef20 msgr2=0x7f86601013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.831+0000 7f8667000640 1 -- 172.21.15.16:0/1997475788 shutdown_connections 2024-09-21T03:36:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.832+0000 7f8667000640 1 -- 172.21.15.16:0/1997475788 wait complete. 2024-09-21T03:36:15.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.832+0000 7f8667000640 1 Processor -- start 2024-09-21T03:36:15.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.833+0000 7f8667000640 1 -- start start 2024-09-21T03:36:15.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.833+0000 7f8667000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86601035b0 0x7f8660076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:15.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.833+0000 7f8667000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8660104640 0x7f8660075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:15.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.833+0000 7f8667000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8660104fe0 0x7f8660075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.833+0000 7f8667000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f866010f230 con 0x7f8660104fe0 2024-09-21T03:36:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.833+0000 7f8667000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f866010f0b0 con 0x7f86601035b0 2024-09-21T03:36:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.833+0000 7f8667000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f866010f3b0 con 0x7f8660104640 2024-09-21T03:36:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.833+0000 7f8665c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86601035b0 0x7f8660076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.833+0000 7f8665c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86601035b0 0x7f8660076e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34928/0 (socket says 172.21.15.16:34928) 2024-09-21T03:36:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.833+0000 7f8665c00640 1 -- 172.21.15.16:0/2365886860 learned_addr learned my addr 172.21.15.16:0/2365886860 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.833+0000 7f8665200640 1 --2- 172.21.15.16:0/2365886860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8660104640 0x7f8660075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.833+0000 7f8666600640 1 --2- 172.21.15.16:0/2365886860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8660104fe0 0x7f8660075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.834+0000 7f8665c00640 1 -- 172.21.15.16:0/2365886860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8660104640 msgr2=0x7f8660075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.834+0000 7f8665c00640 1 --2- 172.21.15.16:0/2365886860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8660104640 0x7f8660075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.834+0000 7f8665c00640 1 -- 172.21.15.16:0/2365886860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8660104fe0 msgr2=0x7f8660075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.834+0000 7f8665c00640 1 --2- 172.21.15.16:0/2365886860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8660104fe0 0x7f8660075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.834+0000 7f8665c00640 1 -- 172.21.15.16:0/2365886860 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86600762c0 con 0x7f86601035b0 2024-09-21T03:36:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.834+0000 7f8665c00640 1 --2- 172.21.15.16:0/2365886860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86601035b0 0x7f8660076e40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f865400e9f0 tx=0x7f865400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.834+0000 7f865e000640 1 -- 172.21.15.16:0/2365886860 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865400ccc0 con 0x7f86601035b0 2024-09-21T03:36:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.834+0000 7f865e000640 1 -- 172.21.15.16:0/2365886860 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86540108e0 con 0x7f86601035b0 2024-09-21T03:36:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.834+0000 7f8667000640 1 -- 172.21.15.16:0/2365886860 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86601a3ee0 con 0x7f86601035b0 2024-09-21T03:36:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.834+0000 7f865e000640 1 -- 172.21.15.16:0/2365886860 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654018880 con 0x7f86601035b0 2024-09-21T03:36:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.835+0000 7f8667000640 1 -- 172.21.15.16:0/2365886860 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86601a4280 con 0x7f86601035b0 2024-09-21T03:36:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.836+0000 7f8667000640 1 -- 172.21.15.16:0/2365886860 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8660103a20 con 0x7f86601035b0 2024-09-21T03:36:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.837+0000 7f865e000640 1 -- 172.21.15.16:0/2365886860 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 14) ==== 103177+0+0 (secure 0 0 0) 0x7f8654010430 con 0x7f86601035b0 2024-09-21T03:36:15.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.838+0000 7f865e000640 1 --2- 172.21.15.16:0/2365886860 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f864807afd0 0x7f864807d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:15.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.838+0000 7f865e000640 1 -- 172.21.15.16:0/2365886860 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f865409d110 con 0x7f86601035b0 2024-09-21T03:36:15.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.838+0000 7f8665200640 1 --2- 172.21.15.16:0/2365886860 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f864807afd0 0x7f864807d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.839+0000 7f8665200640 1 --2- 172.21.15.16:0/2365886860 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f864807afd0 0x7f864807d4e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f864c00ab60 tx=0x7f864c00a2d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:15.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:15.846+0000 7f865e000640 1 -- 172.21.15.16:0/2365886860 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8654069b50 con 0x7f86601035b0 2024-09-21T03:36:16.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.049+0000 7f8667000640 1 -- 172.21.15.16:0/2365886860 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f866002e640 con 0x7f86601035b0 2024-09-21T03:36:16.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.055+0000 7f865e000640 1 -- 172.21.15.16:0/2365886860 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v28)=0 v28) ==== 121+0+0 (secure 0 0 0) 0x7f865406ea00 con 0x7f86601035b0 2024-09-21T03:36:16.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.059+0000 7f8667000640 1 -- 172.21.15.16:0/2365886860 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f864807afd0 msgr2=0x7f864807d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.059+0000 7f8667000640 1 --2- 172.21.15.16:0/2365886860 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f864807afd0 0x7f864807d4e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f864c00ab60 tx=0x7f864c00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.059+0000 7f8667000640 1 -- 172.21.15.16:0/2365886860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86601035b0 msgr2=0x7f8660076e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.059+0000 7f8667000640 1 --2- 172.21.15.16:0/2365886860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86601035b0 0x7f8660076e40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f865400e9f0 tx=0x7f865400edf0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.059+0000 7f8667000640 1 -- 172.21.15.16:0/2365886860 shutdown_connections 2024-09-21T03:36:16.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.059+0000 7f8667000640 1 --2- 172.21.15.16:0/2365886860 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f864807afd0 0x7f864807d4e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.059+0000 7f8667000640 1 --2- 172.21.15.16:0/2365886860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8660104fe0 0x7f8660075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.059+0000 7f8667000640 1 --2- 172.21.15.16:0/2365886860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8660104640 0x7f8660075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.059+0000 7f8667000640 1 --2- 172.21.15.16:0/2365886860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86601035b0 0x7f8660076e40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.059+0000 7f8667000640 1 -- 172.21.15.16:0/2365886860 >> 172.21.15.16:0/2365886860 conn(0x7f86600fef20 msgr2=0x7f8660107940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.059+0000 7f8667000640 1 -- 172.21.15.16:0/2365886860 shutdown_connections 2024-09-21T03:36:16.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.059+0000 7f8667000640 1 -- 172.21.15.16:0/2365886860 wait complete. 2024-09-21T03:36:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.208+0000 7fd537200640 1 -- 172.21.15.16:0/24189953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd538104fa0 msgr2=0x7fd53810b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.208+0000 7fd537200640 1 --2- 172.21.15.16:0/24189953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd538104fa0 0x7fd53810b940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd530009ca0 tx=0x7fd53002a950 comp rx=0 tx=0).stop 2024-09-21T03:36:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.208+0000 7fd537200640 1 -- 172.21.15.16:0/24189953 shutdown_connections 2024-09-21T03:36:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.208+0000 7fd537200640 1 --2- 172.21.15.16:0/24189953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd538104fa0 0x7fd53810b940 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.208+0000 7fd537200640 1 --2- 172.21.15.16:0/24189953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd538104600 0x7fd538104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.208+0000 7fd537200640 1 --2- 172.21.15.16:0/24189953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd538103570 0x7fd5381039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.208+0000 7fd537200640 1 -- 172.21.15.16:0/24189953 >> 172.21.15.16:0/24189953 conn(0x7fd5380feec0 msgr2=0x7fd538101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.208+0000 7fd537200640 1 -- 172.21.15.16:0/24189953 shutdown_connections 2024-09-21T03:36:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.208+0000 7fd537200640 1 -- 172.21.15.16:0/24189953 wait complete. 2024-09-21T03:36:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.209+0000 7fd537200640 1 Processor -- start 2024-09-21T03:36:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.210+0000 7fd537200640 1 -- start start 2024-09-21T03:36:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.210+0000 7fd537200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd538103570 0x7fd538076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.210+0000 7fd537200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd538104600 0x7fd538075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.210+0000 7fd537200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd538104fa0 0x7fd538075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.210+0000 7fd537200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd53810f220 con 0x7fd538104fa0 2024-09-21T03:36:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.210+0000 7fd537200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd53810f0a0 con 0x7fd538104600 2024-09-21T03:36:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.210+0000 7fd537200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd53810f3a0 con 0x7fd538103570 2024-09-21T03:36:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.210+0000 7fd535e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd538103570 0x7fd538076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.210+0000 7fd52fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd538104600 0x7fd538075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.210+0000 7fd536800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd538104fa0 0x7fd538075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.210+0000 7fd535e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd538103570 0x7fd538076e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:48546/0 (socket says 172.21.15.16:48546) 2024-09-21T03:36:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.210+0000 7fd535e00640 1 -- 172.21.15.16:0/2092076202 learned_addr learned my addr 172.21.15.16:0/2092076202 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.211+0000 7fd52fe00640 1 -- 172.21.15.16:0/2092076202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd538103570 msgr2=0x7fd538076e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.211+0000 7fd52fe00640 1 --2- 172.21.15.16:0/2092076202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd538103570 0x7fd538076e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.211+0000 7fd52fe00640 1 -- 172.21.15.16:0/2092076202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd538104fa0 msgr2=0x7fd538075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.211+0000 7fd52fe00640 1 --2- 172.21.15.16:0/2092076202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd538104fa0 0x7fd538075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.211+0000 7fd52fe00640 1 -- 172.21.15.16:0/2092076202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd538076350 con 0x7fd538104600 2024-09-21T03:36:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.211+0000 7fd52fe00640 1 --2- 172.21.15.16:0/2092076202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd538104600 0x7fd538075710 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd52000e9f0 tx=0x7fd52000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:16.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.211+0000 7fd52ea00640 1 -- 172.21.15.16:0/2092076202 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52000ccc0 con 0x7fd538104600 2024-09-21T03:36:16.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.211+0000 7fd52ea00640 1 -- 172.21.15.16:0/2092076202 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5200108e0 con 0x7fd538104600 2024-09-21T03:36:16.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.211+0000 7fd537200640 1 -- 172.21.15.16:0/2092076202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5381a3e40 con 0x7fd538104600 2024-09-21T03:36:16.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.211+0000 7fd52ea00640 1 -- 172.21.15.16:0/2092076202 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd520018880 con 0x7fd538104600 2024-09-21T03:36:16.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.211+0000 7fd537200640 1 -- 172.21.15.16:0/2092076202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5381a4230 con 0x7fd538104600 2024-09-21T03:36:16.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.213+0000 7fd537200640 1 -- 172.21.15.16:0/2092076202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5381039e0 con 0x7fd538104600 2024-09-21T03:36:16.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.215+0000 7fd52ea00640 1 -- 172.21.15.16:0/2092076202 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 14) ==== 103177+0+0 (secure 0 0 0) 0x7fd520010430 con 0x7fd538104600 2024-09-21T03:36:16.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.216+0000 7fd52ea00640 1 --2- 172.21.15.16:0/2092076202 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd51c07afd0 0x7fd51c07d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.216+0000 7fd52ea00640 1 -- 172.21.15.16:0/2092076202 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd52009d0d0 con 0x7fd538104600 2024-09-21T03:36:16.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.216+0000 7fd535e00640 1 --2- 172.21.15.16:0/2092076202 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd51c07afd0 0x7fd51c07d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.217+0000 7fd535e00640 1 --2- 172.21.15.16:0/2092076202 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd51c07afd0 0x7fd51c07d4e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd528009cd0 tx=0x7fd528005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:16.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.220+0000 7fd52ea00640 1 -- 172.21.15.16:0/2092076202 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd520069b10 con 0x7fd538104600 2024-09-21T03:36:16.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.362+0000 7fd52ea00640 1 -- 172.21.15.16:0/2092076202 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fd520061350 con 0x7fd538104600 2024-09-21T03:36:16.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.430+0000 7fd537200640 1 -- 172.21.15.16:0/2092076202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7fd538063fc0 con 0x7fd538104600 2024-09-21T03:36:16.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.435+0000 7fd52ea00640 1 -- 172.21.15.16:0/2092076202 <== mon.2 v2:172.21.15.123:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v29)=0 v29) ==== 119+0+0 (secure 0 0 0) 0x7fd5200b6bd0 con 0x7fd538104600 2024-09-21T03:36:16.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.438+0000 7fd537200640 1 -- 172.21.15.16:0/2092076202 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd51c07afd0 msgr2=0x7fd51c07d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.438+0000 7fd537200640 1 --2- 172.21.15.16:0/2092076202 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd51c07afd0 0x7fd51c07d4e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd528009cd0 tx=0x7fd528005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:16.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.438+0000 7fd537200640 1 -- 172.21.15.16:0/2092076202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd538104600 msgr2=0x7fd538075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.438+0000 7fd537200640 1 --2- 172.21.15.16:0/2092076202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd538104600 0x7fd538075710 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd52000e9f0 tx=0x7fd52000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.438+0000 7fd537200640 1 -- 172.21.15.16:0/2092076202 shutdown_connections 2024-09-21T03:36:16.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.438+0000 7fd537200640 1 --2- 172.21.15.16:0/2092076202 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd51c07afd0 0x7fd51c07d4e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.438+0000 7fd537200640 1 --2- 172.21.15.16:0/2092076202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd538104fa0 0x7fd538075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.438+0000 7fd537200640 1 --2- 172.21.15.16:0/2092076202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd538104600 0x7fd538075710 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.438+0000 7fd537200640 1 --2- 172.21.15.16:0/2092076202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd538103570 0x7fd538076e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.438+0000 7fd537200640 1 -- 172.21.15.16:0/2092076202 >> 172.21.15.16:0/2092076202 conn(0x7fd5380feec0 msgr2=0x7fd538107920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.438+0000 7fd537200640 1 -- 172.21.15.16:0/2092076202 shutdown_connections 2024-09-21T03:36:16.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.438+0000 7fd537200640 1 -- 172.21.15.16:0/2092076202 wait complete. 2024-09-21T03:36:16.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.290+0000 7f4d53a00640 1 -- 172.21.15.16:0/1987754990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c06c410 msgr2=0x7f4d4c108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.290+0000 7f4d53a00640 1 --2- 172.21.15.16:0/1987754990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c06c410 0x7f4d4c108840 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009860 tx=0x7f4d4002a340 comp rx=0 tx=0).stop 2024-09-21T03:36:16.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.290+0000 7f4d53a00640 1 -- 172.21.15.16:0/1987754990 shutdown_connections 2024-09-21T03:36:16.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.290+0000 7f4d53a00640 1 --2- 172.21.15.16:0/1987754990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d4c10b6e0 0x7f4d4c10dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.290+0000 7f4d53a00640 1 --2- 172.21.15.16:0/1987754990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d4c108d80 0x7f4d4c10b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.290+0000 7f4d53a00640 1 --2- 172.21.15.16:0/1987754990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c06c410 0x7f4d4c108840 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.290+0000 7f4d53a00640 1 -- 172.21.15.16:0/1987754990 >> 172.21.15.16:0/1987754990 conn(0x7f4d4c0fdfe0 msgr2=0x7f4d4c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.290+0000 7f4d53a00640 1 -- 172.21.15.16:0/1987754990 shutdown_connections 2024-09-21T03:36:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.290+0000 7f4d53a00640 1 -- 172.21.15.16:0/1987754990 wait complete. 2024-09-21T03:36:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.291+0000 7f4d53a00640 1 Processor -- start 2024-09-21T03:36:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.292+0000 7f4d53a00640 1 -- start start 2024-09-21T03:36:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.292+0000 7f4d53a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c06c410 0x7f4d4c19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.292+0000 7f4d53a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d4c108d80 0x7f4d4c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.292+0000 7f4d53a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d4c10b6e0 0x7f4d4c19d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.292+0000 7f4d53a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c111590 con 0x7f4d4c108d80 2024-09-21T03:36:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.292+0000 7f4d53a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c111410 con 0x7f4d4c06c410 2024-09-21T03:36:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.292+0000 7f4d53a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c111710 con 0x7f4d4c10b6e0 2024-09-21T03:36:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.292+0000 7f4d51000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c06c410 0x7f4d4c19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.292+0000 7f4d51000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c06c410 0x7f4d4c19c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34970/0 (socket says 172.21.15.16:34970) 2024-09-21T03:36:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.292+0000 7f4d51000640 1 -- 172.21.15.16:0/2951711549 learned_addr learned my addr 172.21.15.16:0/2951711549 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.292+0000 7f4d4be00640 1 --2- 172.21.15.16:0/2951711549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d4c108d80 0x7f4d4c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.293+0000 7f4d51a00640 1 --2- 172.21.15.16:0/2951711549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d4c10b6e0 0x7f4d4c19d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.293+0000 7f4d51000640 1 -- 172.21.15.16:0/2951711549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d4c10b6e0 msgr2=0x7f4d4c19d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.293+0000 7f4d51000640 1 --2- 172.21.15.16:0/2951711549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d4c10b6e0 0x7f4d4c19d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.293+0000 7f4d51000640 1 -- 172.21.15.16:0/2951711549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d4c108d80 msgr2=0x7f4d4c19c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.293+0000 7f4d51000640 1 --2- 172.21.15.16:0/2951711549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d4c108d80 0x7f4d4c19c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.293+0000 7f4d51000640 1 -- 172.21.15.16:0/2951711549 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4c19dbe0 con 0x7f4d4c06c410 2024-09-21T03:36:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.293+0000 7f4d51000640 1 --2- 172.21.15.16:0/2951711549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c06c410 0x7f4d4c19c1b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4d40032d50 tx=0x7f4d40032d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.293+0000 7f4d49600640 1 -- 172.21.15.16:0/2951711549 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40037040 con 0x7f4d4c06c410 2024-09-21T03:36:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.293+0000 7f4d49600640 1 -- 172.21.15.16:0/2951711549 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d40030c10 con 0x7f4d4c06c410 2024-09-21T03:36:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.293+0000 7f4d53a00640 1 -- 172.21.15.16:0/2951711549 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4c19de70 con 0x7f4d4c06c410 2024-09-21T03:36:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.293+0000 7f4d49600640 1 -- 172.21.15.16:0/2951711549 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40003850 con 0x7f4d4c06c410 2024-09-21T03:36:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.294+0000 7f4d53a00640 1 -- 172.21.15.16:0/2951711549 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d4c1a4400 con 0x7f4d4c06c410 2024-09-21T03:36:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.295+0000 7f4d53a00640 1 -- 172.21.15.16:0/2951711549 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d4c0ffb00 con 0x7f4d4c06c410 2024-09-21T03:36:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.296+0000 7f4d49600640 1 -- 172.21.15.16:0/2951711549 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 14) ==== 103177+0+0 (secure 0 0 0) 0x7f4d40043020 con 0x7f4d4c06c410 2024-09-21T03:36:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.297+0000 7f4d49600640 1 --2- 172.21.15.16:0/2951711549 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4d3407afd0 0x7f4d3407d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.297+0000 7f4d49600640 1 -- 172.21.15.16:0/2951711549 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4d400bc080 con 0x7f4d4c06c410 2024-09-21T03:36:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.297+0000 7f4d4be00640 1 --2- 172.21.15.16:0/2951711549 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4d3407afd0 0x7f4d3407d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.298+0000 7f4d4be00640 1 --2- 172.21.15.16:0/2951711549 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4d3407afd0 0x7f4d3407d4e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4d380095f0 tx=0x7f4d38005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.301+0000 7f4d49600640 1 -- 172.21.15.16:0/2951711549 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4d4004a060 con 0x7f4d4c06c410 2024-09-21T03:36:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.362+0000 7f4d49600640 1 -- 172.21.15.16:0/2951711549 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f4d40004050 con 0x7f4d4c06c410 2024-09-21T03:36:16.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.510+0000 7f4d53a00640 1 -- 172.21.15.16:0/2951711549 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f4d4c02e640 con 0x7f4d4c06c410 2024-09-21T03:36:16.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.518+0000 7f4d49600640 1 -- 172.21.15.16:0/2951711549 <== mon.2 v2:172.21.15.123:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v30)=0 v30) ==== 120+0+0 (secure 0 0 0) 0x7f4d40084760 con 0x7f4d4c06c410 2024-09-21T03:36:16.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.520+0000 7f4d53a00640 1 -- 172.21.15.16:0/2951711549 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4d3407afd0 msgr2=0x7f4d3407d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.520+0000 7f4d53a00640 1 --2- 172.21.15.16:0/2951711549 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4d3407afd0 0x7f4d3407d4e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4d380095f0 tx=0x7f4d38005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:16.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.520+0000 7f4d53a00640 1 -- 172.21.15.16:0/2951711549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c06c410 msgr2=0x7f4d4c19c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.520+0000 7f4d53a00640 1 --2- 172.21.15.16:0/2951711549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c06c410 0x7f4d4c19c1b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4d40032d50 tx=0x7f4d40032d80 comp rx=0 tx=0).stop 2024-09-21T03:36:16.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.520+0000 7f4d53a00640 1 -- 172.21.15.16:0/2951711549 shutdown_connections 2024-09-21T03:36:16.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.520+0000 7f4d53a00640 1 --2- 172.21.15.16:0/2951711549 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4d3407afd0 0x7f4d3407d4e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.520+0000 7f4d53a00640 1 --2- 172.21.15.16:0/2951711549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d4c10b6e0 0x7f4d4c19d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.520+0000 7f4d53a00640 1 --2- 172.21.15.16:0/2951711549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d4c108d80 0x7f4d4c19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.520+0000 7f4d53a00640 1 --2- 172.21.15.16:0/2951711549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c06c410 0x7f4d4c19c1b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.521+0000 7f4d53a00640 1 -- 172.21.15.16:0/2951711549 >> 172.21.15.16:0/2951711549 conn(0x7f4d4c0fdfe0 msgr2=0x7f4d4c109c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.521+0000 7f4d53a00640 1 -- 172.21.15.16:0/2951711549 shutdown_connections 2024-09-21T03:36:16.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:16.521+0000 7f4d53a00640 1 -- 172.21.15.16:0/2951711549 wait complete. 2024-09-21T03:36:16.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:16 smithi016 ceph-mon[30375]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:16.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:16 smithi016 ceph-mon[30375]: Standby manager daemon y started 2024-09-21T03:36:16.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:16 smithi016 ceph-mon[30375]: from='mgr.? 172.21.15.123:0/1576534712' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:36:16.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:16 smithi016 ceph-mon[30375]: from='mgr.? 172.21.15.123:0/1576534712' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:36:16.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:16 smithi016 ceph-mon[30375]: from='mgr.? 172.21.15.123:0/1576534712' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:36:16.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:16 smithi016 ceph-mon[30375]: from='mgr.? 172.21.15.123:0/1576534712' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:36:16.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:16 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T03:36:16.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:16 smithi134 ceph-mon[35031]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:16.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:16 smithi134 ceph-mon[35031]: Standby manager daemon y started 2024-09-21T03:36:16.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:16 smithi134 ceph-mon[35031]: from='mgr.? 172.21.15.123:0/1576534712' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:36:16.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:16 smithi134 ceph-mon[35031]: from='mgr.? 172.21.15.123:0/1576534712' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:36:16.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:16 smithi134 ceph-mon[35031]: from='mgr.? 172.21.15.123:0/1576534712' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:36:16.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:16 smithi134 ceph-mon[35031]: from='mgr.? 172.21.15.123:0/1576534712' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:36:16.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:16 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' 2024-09-21T03:36:16.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:16 smithi123 ceph-mon[34673]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:16.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:16 smithi123 ceph-mon[34673]: Standby manager daemon y started 2024-09-21T03:36:16.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:16 smithi123 ceph-mon[34673]: from='mgr.? 172.21.15.123:0/1576534712' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:36:16.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:16 smithi123 ceph-mon[34673]: from='mgr.? 172.21.15.123:0/1576534712' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:36:16.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:16 smithi123 ceph-mon[34673]: from='mgr.? 172.21.15.123:0/1576534712' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:36:16.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:16 smithi123 ceph-mon[34673]: from='mgr.? 172.21.15.123:0/1576534712' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:36:16.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:16 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' 2024-09-21T03:36:17.035 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:17.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.639+0000 7fd9f1400640 1 -- 172.21.15.16:0/1097153784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9ec1073b0 msgr2=0x7fd9ec10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:17.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.639+0000 7fd9f1400640 1 --2- 172.21.15.16:0/1097153784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9ec1073b0 0x7fd9ec10dd50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009ca0 tx=0x7fd9e402a950 comp rx=0 tx=0).stop 2024-09-21T03:36:17.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.640+0000 7fd9f1400640 1 -- 172.21.15.16:0/1097153784 shutdown_connections 2024-09-21T03:36:17.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.640+0000 7fd9f1400640 1 --2- 172.21.15.16:0/1097153784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9ec1073b0 0x7fd9ec10dd50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.640+0000 7fd9f1400640 1 --2- 172.21.15.16:0/1097153784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec106a10 0x7fd9ec106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.640+0000 7fd9f1400640 1 --2- 172.21.15.16:0/1097153784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9ec105980 0x7fd9ec105df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.640+0000 7fd9f1400640 1 -- 172.21.15.16:0/1097153784 >> 172.21.15.16:0/1097153784 conn(0x7fd9ec1012f0 msgr2=0x7fd9ec1037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:17.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.640+0000 7fd9f1400640 1 -- 172.21.15.16:0/1097153784 shutdown_connections 2024-09-21T03:36:17.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.640+0000 7fd9f1400640 1 -- 172.21.15.16:0/1097153784 wait complete. 2024-09-21T03:36:17.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.641+0000 7fd9f1400640 1 Processor -- start 2024-09-21T03:36:17.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.641+0000 7fd9f1400640 1 -- start start 2024-09-21T03:36:17.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.641+0000 7fd9f1400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9ec105980 0x7fd9ec19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:17.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.641+0000 7fd9f1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec106a10 0x7fd9ec19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:17.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.641+0000 7fd9f1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9ec1073b0 0x7fd9ec19d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:17.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.641+0000 7fd9f1400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec1115b0 con 0x7fd9ec1073b0 2024-09-21T03:36:17.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.641+0000 7fd9f1400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec111430 con 0x7fd9ec106a10 2024-09-21T03:36:17.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.641+0000 7fd9f1400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec111730 con 0x7fd9ec105980 2024-09-21T03:36:17.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.642+0000 7fd9ea800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9ec105980 0x7fd9ec19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:17.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.642+0000 7fd9ea800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9ec105980 0x7fd9ec19c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:47562/0 (socket says 172.21.15.16:47562) 2024-09-21T03:36:17.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.642+0000 7fd9ea800640 1 -- 172.21.15.16:0/1982531050 learned_addr learned my addr 172.21.15.16:0/1982531050 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:17.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.642+0000 7fd9e9e00640 1 --2- 172.21.15.16:0/1982531050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec106a10 0x7fd9ec19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:17.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.642+0000 7fd9eb200640 1 --2- 172.21.15.16:0/1982531050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9ec1073b0 0x7fd9ec19d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:17.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.642+0000 7fd9ea800640 1 -- 172.21.15.16:0/1982531050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec106a10 msgr2=0x7fd9ec19c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:17.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.642+0000 7fd9ea800640 1 --2- 172.21.15.16:0/1982531050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec106a10 0x7fd9ec19c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.642+0000 7fd9ea800640 1 -- 172.21.15.16:0/1982531050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9ec1073b0 msgr2=0x7fd9ec19d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:17.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.642+0000 7fd9ea800640 1 --2- 172.21.15.16:0/1982531050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9ec1073b0 0x7fd9ec19d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.642+0000 7fd9ea800640 1 -- 172.21.15.16:0/1982531050 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ec19dbe0 con 0x7fd9ec105980 2024-09-21T03:36:17.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.642+0000 7fd9ea800640 1 --2- 172.21.15.16:0/1982531050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9ec105980 0x7fd9ec19c1b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00ea60 tx=0x7fd9dc00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:17.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.642+0000 7fd9e3400640 1 -- 172.21.15.16:0/1982531050 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc00cde0 con 0x7fd9ec105980 2024-09-21T03:36:17.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.642+0000 7fd9e3400640 1 -- 172.21.15.16:0/1982531050 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9dc0035f0 con 0x7fd9ec105980 2024-09-21T03:36:17.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.642+0000 7fd9f1400640 1 -- 172.21.15.16:0/1982531050 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ec19ded0 con 0x7fd9ec105980 2024-09-21T03:36:17.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.643+0000 7fd9e3400640 1 -- 172.21.15.16:0/1982531050 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc010690 con 0x7fd9ec105980 2024-09-21T03:36:17.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.643+0000 7fd9f1400640 1 -- 172.21.15.16:0/1982531050 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ec1a45d0 con 0x7fd9ec105980 2024-09-21T03:36:17.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.644+0000 7fd9f1400640 1 -- 172.21.15.16:0/1982531050 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9ec10cb60 con 0x7fd9ec105980 2024-09-21T03:36:17.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.644+0000 7fd9e3400640 1 -- 172.21.15.16:0/1982531050 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7fd9dc010830 con 0x7fd9ec105980 2024-09-21T03:36:17.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.644+0000 7fd9e3400640 1 --2- 172.21.15.16:0/1982531050 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd9d003fb20 0x7fd9d0042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:17.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.644+0000 7fd9e3400640 1 -- 172.21.15.16:0/1982531050 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd9dc054870 con 0x7fd9ec105980 2024-09-21T03:36:17.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.644+0000 7fd9e9e00640 1 --2- 172.21.15.16:0/1982531050 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd9d003fb20 0x7fd9d0042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:17.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.645+0000 7fd9e9e00640 1 --2- 172.21.15.16:0/1982531050 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd9d003fb20 0x7fd9d0042030 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009bd0 tx=0x7fd9d8005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:17.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.649+0000 7fd9e3400640 1 -- 172.21.15.16:0/1982531050 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd9dc00f3c0 con 0x7fd9ec105980 2024-09-21T03:36:17.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.856+0000 7fd9f1400640 1 -- 172.21.15.16:0/1982531050 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fd9ec02e640 con 0x7fd9ec105980 2024-09-21T03:36:17.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.861+0000 7fd9e3400640 1 -- 172.21.15.16:0/1982531050 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v25)=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7fd9dc020e00 con 0x7fd9ec105980 2024-09-21T03:36:17.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.864+0000 7fd9f1400640 1 -- 172.21.15.16:0/1982531050 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd9d003fb20 msgr2=0x7fd9d0042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:17.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.864+0000 7fd9f1400640 1 --2- 172.21.15.16:0/1982531050 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd9d003fb20 0x7fd9d0042030 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009bd0 tx=0x7fd9d8005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:17.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.864+0000 7fd9f1400640 1 -- 172.21.15.16:0/1982531050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9ec105980 msgr2=0x7fd9ec19c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:17.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.864+0000 7fd9f1400640 1 --2- 172.21.15.16:0/1982531050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9ec105980 0x7fd9ec19c1b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00ea60 tx=0x7fd9dc00ef20 comp rx=0 tx=0).stop 2024-09-21T03:36:17.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.864+0000 7fd9f1400640 1 -- 172.21.15.16:0/1982531050 shutdown_connections 2024-09-21T03:36:17.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.864+0000 7fd9f1400640 1 --2- 172.21.15.16:0/1982531050 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd9d003fb20 0x7fd9d0042030 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.864+0000 7fd9f1400640 1 --2- 172.21.15.16:0/1982531050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9ec1073b0 0x7fd9ec19d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.864+0000 7fd9f1400640 1 --2- 172.21.15.16:0/1982531050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec106a10 0x7fd9ec19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.864+0000 7fd9f1400640 1 --2- 172.21.15.16:0/1982531050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9ec105980 0x7fd9ec19c1b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.864+0000 7fd9f1400640 1 -- 172.21.15.16:0/1982531050 >> 172.21.15.16:0/1982531050 conn(0x7fd9ec1012f0 msgr2=0x7fd9ec109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:17.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.864+0000 7fd9f1400640 1 -- 172.21.15.16:0/1982531050 shutdown_connections 2024-09-21T03:36:17.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:11.864+0000 7fd9f1400640 1 -- 172.21.15.16:0/1982531050 wait complete. 2024-09-21T03:36:17.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:17 smithi123 ceph-mon[34673]: mgrmap e15: x(active, since 83s), standbys: z, y 2024-09-21T03:36:17.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:17 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:36:17.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:17 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' 2024-09-21T03:36:17.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:17 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' 2024-09-21T03:36:17.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:17 smithi016 ceph-mon[30375]: mgrmap e15: x(active, since 83s), standbys: z, y 2024-09-21T03:36:17.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:17 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:36:17.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:17 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T03:36:17.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:17 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T03:36:17.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:17 smithi134 ceph-mon[35031]: mgrmap e15: x(active, since 83s), standbys: z, y 2024-09-21T03:36:17.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:17 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:36:17.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:17 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' 2024-09-21T03:36:17.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:17 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' 2024-09-21T03:36:18.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:18 smithi016 ceph-mon[30375]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:18.824 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:18.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:18 smithi134 ceph-mon[35031]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:18.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:18 smithi123 ceph-mon[34673]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:20.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.381+0000 7fc29ec00640 1 -- 172.21.15.16:0/3177894872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc29810fee0 msgr2=0x7fc298112300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.381+0000 7fc29ec00640 1 --2- 172.21.15.16:0/3177894872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc29810fee0 0x7fc298112300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc290009ca0 tx=0x7fc29002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.382+0000 7fc29ec00640 1 -- 172.21.15.16:0/3177894872 shutdown_connections 2024-09-21T03:36:20.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.382+0000 7fc29ec00640 1 --2- 172.21.15.16:0/3177894872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc29810fee0 0x7fc298112300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.382+0000 7fc29ec00640 1 --2- 172.21.15.16:0/3177894872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc298075cf0 0x7fc298076100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.382+0000 7fc29ec00640 1 --2- 172.21.15.16:0/3177894872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2980770e0 0x7fc2980757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.382+0000 7fc29ec00640 1 -- 172.21.15.16:0/3177894872 >> 172.21.15.16:0/3177894872 conn(0x7fc2980ff0c0 msgr2=0x7fc298101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:20.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.382+0000 7fc29ec00640 1 -- 172.21.15.16:0/3177894872 shutdown_connections 2024-09-21T03:36:20.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.382+0000 7fc29ec00640 1 -- 172.21.15.16:0/3177894872 wait complete. 2024-09-21T03:36:20.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.383+0000 7fc29ec00640 1 Processor -- start 2024-09-21T03:36:20.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.383+0000 7fc29ec00640 1 -- start start 2024-09-21T03:36:20.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.383+0000 7fc29ec00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc298075cf0 0x7fc2981a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.383+0000 7fc29ec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2980770e0 0x7fc2981a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.383+0000 7fc29ec00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc29810fee0 0x7fc2981a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.383+0000 7fc29ec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2981159f0 con 0x7fc2980770e0 2024-09-21T03:36:20.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.383+0000 7fc29ec00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc298115870 con 0x7fc298075cf0 2024-09-21T03:36:20.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.383+0000 7fc29ec00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc298115b70 con 0x7fc29810fee0 2024-09-21T03:36:20.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.383+0000 7fc28fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2980770e0 0x7fc2981a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.383+0000 7fc28fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2980770e0 0x7fc2981a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59058/0 (socket says 172.21.15.16:59058) 2024-09-21T03:36:20.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.384+0000 7fc28fe00640 1 -- 172.21.15.16:0/318457304 learned_addr learned my addr 172.21.15.16:0/318457304 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:20.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.384+0000 7fc29cc00640 1 --2- 172.21.15.16:0/318457304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc29810fee0 0x7fc2981a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.384+0000 7fc297e00640 1 --2- 172.21.15.16:0/318457304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc298075cf0 0x7fc2981a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.384+0000 7fc28fe00640 1 -- 172.21.15.16:0/318457304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc29810fee0 msgr2=0x7fc2981a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.384+0000 7fc28fe00640 1 --2- 172.21.15.16:0/318457304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc29810fee0 0x7fc2981a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.384+0000 7fc28fe00640 1 -- 172.21.15.16:0/318457304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc298075cf0 msgr2=0x7fc2981a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.384+0000 7fc28fe00640 1 --2- 172.21.15.16:0/318457304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc298075cf0 0x7fc2981a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.384+0000 7fc28fe00640 1 -- 172.21.15.16:0/318457304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2981a1fe0 con 0x7fc2980770e0 2024-09-21T03:36:20.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.384+0000 7fc28fe00640 1 --2- 172.21.15.16:0/318457304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2980770e0 0x7fc2981a0c20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc28000e9f0 tx=0x7fc28000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:20.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.385+0000 7fc295600640 1 -- 172.21.15.16:0/318457304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28000cd80 con 0x7fc2980770e0 2024-09-21T03:36:20.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.385+0000 7fc295600640 1 -- 172.21.15.16:0/318457304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2800054d0 con 0x7fc2980770e0 2024-09-21T03:36:20.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.385+0000 7fc29ec00640 1 -- 172.21.15.16:0/318457304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2981a22d0 con 0x7fc2980770e0 2024-09-21T03:36:20.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.385+0000 7fc295600640 1 -- 172.21.15.16:0/318457304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc280010690 con 0x7fc2980770e0 2024-09-21T03:36:20.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.385+0000 7fc29ec00640 1 -- 172.21.15.16:0/318457304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2981a89d0 con 0x7fc2980770e0 2024-09-21T03:36:20.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.386+0000 7fc29ec00640 1 -- 172.21.15.16:0/318457304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2981a8ba0 con 0x7fc2980770e0 2024-09-21T03:36:20.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.388+0000 7fc295600640 1 -- 172.21.15.16:0/318457304 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fc280004e30 con 0x7fc2980770e0 2024-09-21T03:36:20.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.390+0000 7fc295600640 1 --2- 172.21.15.16:0/318457304 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc27c0b6500 0x7fc27c0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.390+0000 7fc295600640 1 -- 172.21.15.16:0/318457304 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc2800e5c10 con 0x7fc2980770e0 2024-09-21T03:36:20.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.390+0000 7fc297e00640 1 --2- 172.21.15.16:0/318457304 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc27c0b6500 0x7fc27c0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.391+0000 7fc297e00640 1 --2- 172.21.15.16:0/318457304 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc27c0b6500 0x7fc27c0b8a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc288005d00 tx=0x7fc28800b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:20.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.392+0000 7fc295600640 1 -- 172.21.15.16:0/318457304 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc2800ae0d0 con 0x7fc2980770e0 2024-09-21T03:36:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.591+0000 7fc29ec00640 1 -- 172.21.15.16:0/318457304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fc2981a8d70 con 0x7fc2980770e0 2024-09-21T03:36:20.593 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:20 smithi016 ceph-mon[30375]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:20.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.596+0000 7fc295600640 1 -- 172.21.15.16:0/318457304 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v31)=0 v31) ==== 115+0+0 (secure 0 0 0) 0x7fc2800b24b0 con 0x7fc2980770e0 2024-09-21T03:36:20.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.598+0000 7fc29ec00640 1 -- 172.21.15.16:0/318457304 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc27c0b6500 msgr2=0x7fc27c0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.598+0000 7fc29ec00640 1 --2- 172.21.15.16:0/318457304 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc27c0b6500 0x7fc27c0b8a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc288005d00 tx=0x7fc28800b410 comp rx=0 tx=0).stop 2024-09-21T03:36:20.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.598+0000 7fc29ec00640 1 -- 172.21.15.16:0/318457304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2980770e0 msgr2=0x7fc2981a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.598+0000 7fc29ec00640 1 --2- 172.21.15.16:0/318457304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2980770e0 0x7fc2981a0c20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc28000e9f0 tx=0x7fc28000edf0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.598+0000 7fc29ec00640 1 -- 172.21.15.16:0/318457304 shutdown_connections 2024-09-21T03:36:20.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.598+0000 7fc29ec00640 1 --2- 172.21.15.16:0/318457304 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc27c0b6500 0x7fc27c0b8a10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.598+0000 7fc29ec00640 1 --2- 172.21.15.16:0/318457304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc29810fee0 0x7fc2981a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.598+0000 7fc29ec00640 1 --2- 172.21.15.16:0/318457304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2980770e0 0x7fc2981a0c20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.598+0000 7fc29ec00640 1 --2- 172.21.15.16:0/318457304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc298075cf0 0x7fc2981a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.598+0000 7fc29ec00640 1 -- 172.21.15.16:0/318457304 >> 172.21.15.16:0/318457304 conn(0x7fc2980ff0c0 msgr2=0x7fc2980ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:20.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.598+0000 7fc29ec00640 1 -- 172.21.15.16:0/318457304 shutdown_connections 2024-09-21T03:36:20.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:20.599+0000 7fc29ec00640 1 -- 172.21.15.16:0/318457304 wait complete. 2024-09-21T03:36:20.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:20 smithi134 ceph-mon[35031]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:20.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:20 smithi123 ceph-mon[34673]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:21.633 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:21 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/318457304' entity='client.admin' 2024-09-21T03:36:21.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.378+0000 7fa8b5a00640 1 -- 172.21.15.16:0/2209018127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b01047e0 msgr2=0x7fa8b0104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:21.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.378+0000 7fa8b5a00640 1 --2- 172.21.15.16:0/2209018127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b01047e0 0x7fa8b0104bf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa898009900 tx=0x7fa89802a480 comp rx=0 tx=0).stop 2024-09-21T03:36:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.379+0000 7fa8b5a00640 1 -- 172.21.15.16:0/2209018127 shutdown_connections 2024-09-21T03:36:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.379+0000 7fa8b5a00640 1 --2- 172.21.15.16:0/2209018127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8b0105180 0x7fa8b010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.379+0000 7fa8b5a00640 1 --2- 172.21.15.16:0/2209018127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b01047e0 0x7fa8b0104bf0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.379+0000 7fa8b5a00640 1 --2- 172.21.15.16:0/2209018127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8b0103750 0x7fa8b0103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.379+0000 7fa8b5a00640 1 -- 172.21.15.16:0/2209018127 >> 172.21.15.16:0/2209018127 conn(0x7fa8b00ff0e0 msgr2=0x7fa8b0101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.379+0000 7fa8b5a00640 1 -- 172.21.15.16:0/2209018127 shutdown_connections 2024-09-21T03:36:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.379+0000 7fa8b5a00640 1 -- 172.21.15.16:0/2209018127 wait complete. 2024-09-21T03:36:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.380+0000 7fa8b5a00640 1 Processor -- start 2024-09-21T03:36:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.380+0000 7fa8b5a00640 1 -- start start 2024-09-21T03:36:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.381+0000 7fa8b5a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8b0103750 0x7fa8b0076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.381+0000 7fa8b5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b01047e0 0x7fa8b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.381+0000 7fa8b5a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8b0105180 0x7fa8b0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.381+0000 7fa8b5a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b010f380 con 0x7fa8b01047e0 2024-09-21T03:36:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.381+0000 7fa8b5a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b010f200 con 0x7fa8b0105180 2024-09-21T03:36:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.381+0000 7fa8b5a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b010f500 con 0x7fa8b0103750 2024-09-21T03:36:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.381+0000 7fa8a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b01047e0 0x7fa8b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.381+0000 7fa8a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b01047e0 0x7fa8b0075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59068/0 (socket says 172.21.15.16:59068) 2024-09-21T03:36:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.381+0000 7fa8a7e00640 1 -- 172.21.15.16:0/2164726478 learned_addr learned my addr 172.21.15.16:0/2164726478 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.381+0000 7fa8aea00640 1 --2- 172.21.15.16:0/2164726478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8b0103750 0x7fa8b0076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.381+0000 7fa8af400640 1 --2- 172.21.15.16:0/2164726478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8b0105180 0x7fa8b0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.382+0000 7fa8a7e00640 1 -- 172.21.15.16:0/2164726478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8b0103750 msgr2=0x7fa8b0076e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.382+0000 7fa8a7e00640 1 --2- 172.21.15.16:0/2164726478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8b0103750 0x7fa8b0076e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.382+0000 7fa8a7e00640 1 -- 172.21.15.16:0/2164726478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8b0105180 msgr2=0x7fa8b0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.382+0000 7fa8a7e00640 1 --2- 172.21.15.16:0/2164726478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8b0105180 0x7fa8b0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.382+0000 7fa8a7e00640 1 -- 172.21.15.16:0/2164726478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b0076350 con 0x7fa8b01047e0 2024-09-21T03:36:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.382+0000 7fa8a7e00640 1 --2- 172.21.15.16:0/2164726478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b01047e0 0x7fa8b0075710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa898006010 tx=0x7fa8980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.382+0000 7fa8a7400640 1 -- 172.21.15.16:0/2164726478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89803f030 con 0x7fa8b01047e0 2024-09-21T03:36:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.382+0000 7fa8a7400640 1 -- 172.21.15.16:0/2164726478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa898031910 con 0x7fa8b01047e0 2024-09-21T03:36:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.383+0000 7fa8b5a00640 1 -- 172.21.15.16:0/2164726478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b01a4040 con 0x7fa8b01047e0 2024-09-21T03:36:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.383+0000 7fa8a7400640 1 -- 172.21.15.16:0/2164726478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898035070 con 0x7fa8b01047e0 2024-09-21T03:36:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.383+0000 7fa8b5a00640 1 -- 172.21.15.16:0/2164726478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b01a4430 con 0x7fa8b01047e0 2024-09-21T03:36:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.384+0000 7fa8b5a00640 1 -- 172.21.15.16:0/2164726478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b0103bc0 con 0x7fa8b01047e0 2024-09-21T03:36:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.386+0000 7fa8a7400640 1 -- 172.21.15.16:0/2164726478 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fa898042030 con 0x7fa8b01047e0 2024-09-21T03:36:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.389+0000 7fa8a7400640 1 --2- 172.21.15.16:0/2164726478 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa8940b6500 0x7fa8940b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.389+0000 7fa8aea00640 1 --2- 172.21.15.16:0/2164726478 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa8940b6500 0x7fa8940b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.389+0000 7fa8a7400640 1 -- 172.21.15.16:0/2164726478 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa8981061d0 con 0x7fa8b01047e0 2024-09-21T03:36:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.390+0000 7fa8aea00640 1 --2- 172.21.15.16:0/2164726478 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa8940b6500 0x7fa8940b8a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009bd0 tx=0x7fa8a0005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.391+0000 7fa8a7400640 1 -- 172.21.15.16:0/2164726478 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa8980d2a70 con 0x7fa8b01047e0 2024-09-21T03:36:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.594+0000 7fa8b5a00640 1 -- 172.21.15.16:0/2164726478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fa8b010a930 con 0x7fa8b01047e0 2024-09-21T03:36:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.609+0000 7fa8a7400640 1 -- 172.21.15.16:0/2164726478 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v32)=0 v32) ==== 121+0+0 (secure 0 0 0) 0x7fa8980d79b0 con 0x7fa8b01047e0 2024-09-21T03:36:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.611+0000 7fa8b5a00640 1 -- 172.21.15.16:0/2164726478 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa8940b6500 msgr2=0x7fa8940b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.611+0000 7fa8b5a00640 1 --2- 172.21.15.16:0/2164726478 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa8940b6500 0x7fa8940b8a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009bd0 tx=0x7fa8a0005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.611+0000 7fa8b5a00640 1 -- 172.21.15.16:0/2164726478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b01047e0 msgr2=0x7fa8b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.611+0000 7fa8b5a00640 1 --2- 172.21.15.16:0/2164726478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b01047e0 0x7fa8b0075710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa898006010 tx=0x7fa8980042f0 comp rx=0 tx=0).stop 2024-09-21T03:36:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.612+0000 7fa8b5a00640 1 -- 172.21.15.16:0/2164726478 shutdown_connections 2024-09-21T03:36:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.612+0000 7fa8b5a00640 1 --2- 172.21.15.16:0/2164726478 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa8940b6500 0x7fa8940b8a10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.612+0000 7fa8b5a00640 1 --2- 172.21.15.16:0/2164726478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8b0105180 0x7fa8b0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.612+0000 7fa8b5a00640 1 --2- 172.21.15.16:0/2164726478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b01047e0 0x7fa8b0075710 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.612+0000 7fa8b5a00640 1 --2- 172.21.15.16:0/2164726478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8b0103750 0x7fa8b0076e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.612+0000 7fa8b5a00640 1 -- 172.21.15.16:0/2164726478 >> 172.21.15.16:0/2164726478 conn(0x7fa8b00ff0e0 msgr2=0x7fa8b0107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.612+0000 7fa8b5a00640 1 -- 172.21.15.16:0/2164726478 shutdown_connections 2024-09-21T03:36:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:21.612+0000 7fa8b5a00640 1 -- 172.21.15.16:0/2164726478 wait complete. 2024-09-21T03:36:21.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:21 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/318457304' entity='client.admin' 2024-09-21T03:36:22.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:21 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/318457304' entity='client.admin' 2024-09-21T03:36:22.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:22 smithi016 ceph-mon[30375]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:22.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:22 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2164726478' entity='client.admin' 2024-09-21T03:36:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.663+0000 7fbaf9c00640 1 -- 172.21.15.16:0/1365525640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf4103750 msgr2=0x7fbaf4103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.663+0000 7fbaf9c00640 1 --2- 172.21.15.16:0/1365525640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf4103750 0x7fbaf4103bc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009900 tx=0x7fbae402a480 comp rx=0 tx=0).stop 2024-09-21T03:36:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.663+0000 7fbaf9c00640 1 -- 172.21.15.16:0/1365525640 shutdown_connections 2024-09-21T03:36:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.663+0000 7fbaf9c00640 1 --2- 172.21.15.16:0/1365525640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaf4105180 0x7fbaf410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.663+0000 7fbaf9c00640 1 --2- 172.21.15.16:0/1365525640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaf41047e0 0x7fbaf4104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.663+0000 7fbaf9c00640 1 --2- 172.21.15.16:0/1365525640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf4103750 0x7fbaf4103bc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.663+0000 7fbaf9c00640 1 -- 172.21.15.16:0/1365525640 >> 172.21.15.16:0/1365525640 conn(0x7fbaf40ff0c0 msgr2=0x7fbaf4101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.663+0000 7fbaf9c00640 1 -- 172.21.15.16:0/1365525640 shutdown_connections 2024-09-21T03:36:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.663+0000 7fbaf9c00640 1 -- 172.21.15.16:0/1365525640 wait complete. 2024-09-21T03:36:22.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.664+0000 7fbaf9c00640 1 Processor -- start 2024-09-21T03:36:22.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.664+0000 7fbaf9c00640 1 -- start start 2024-09-21T03:36:22.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.665+0000 7fbaf9c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaf4103750 0x7fbaf4076dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.665+0000 7fbaf9c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaf41047e0 0x7fbaf4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.665+0000 7fbaf9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf4105180 0x7fbaf4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.665+0000 7fbaf9c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf410f3f0 con 0x7fbaf4105180 2024-09-21T03:36:22.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.665+0000 7fbaf9c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf410f270 con 0x7fbaf41047e0 2024-09-21T03:36:22.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.665+0000 7fbaf9c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf410f570 con 0x7fbaf4103750 2024-09-21T03:36:22.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.665+0000 7fbaf3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf4105180 0x7fbaf4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.665+0000 7fbaf3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf4105180 0x7fbaf4075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41184/0 (socket says 172.21.15.16:41184) 2024-09-21T03:36:22.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.665+0000 7fbaf3400640 1 -- 172.21.15.16:0/1659467659 learned_addr learned my addr 172.21.15.16:0/1659467659 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:22.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.665+0000 7fbaf2a00640 1 --2- 172.21.15.16:0/1659467659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaf4103750 0x7fbaf4076dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.665+0000 7fbaebe00640 1 --2- 172.21.15.16:0/1659467659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaf41047e0 0x7fbaf4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.665+0000 7fbaf3400640 1 -- 172.21.15.16:0/1659467659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaf4103750 msgr2=0x7fbaf4076dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.665+0000 7fbaf3400640 1 --2- 172.21.15.16:0/1659467659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaf4103750 0x7fbaf4076dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.665+0000 7fbaf3400640 1 -- 172.21.15.16:0/1659467659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaf41047e0 msgr2=0x7fbaf4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.665+0000 7fbaf3400640 1 --2- 172.21.15.16:0/1659467659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaf41047e0 0x7fbaf4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.666+0000 7fbaf3400640 1 -- 172.21.15.16:0/1659467659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf40762f0 con 0x7fbaf4105180 2024-09-21T03:36:22.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.666+0000 7fbaf3400640 1 --2- 172.21.15.16:0/1659467659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf4105180 0x7fbaf4075c50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009fd0 tx=0x7fbaec00def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:22.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.666+0000 7fbaeb400640 1 -- 172.21.15.16:0/1659467659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec017070 con 0x7fbaf4105180 2024-09-21T03:36:22.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.666+0000 7fbaeb400640 1 -- 172.21.15.16:0/1659467659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaec003e20 con 0x7fbaf4105180 2024-09-21T03:36:22.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.666+0000 7fbaf9c00640 1 -- 172.21.15.16:0/1659467659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf41a4130 con 0x7fbaf4105180 2024-09-21T03:36:22.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.666+0000 7fbaeb400640 1 -- 172.21.15.16:0/1659467659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec010460 con 0x7fbaf4105180 2024-09-21T03:36:22.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.666+0000 7fbaf9c00640 1 -- 172.21.15.16:0/1659467659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf41a4530 con 0x7fbaf4105180 2024-09-21T03:36:22.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.668+0000 7fbaf9c00640 1 -- 172.21.15.16:0/1659467659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf410a930 con 0x7fbaf4105180 2024-09-21T03:36:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.670+0000 7fbaeb400640 1 -- 172.21.15.16:0/1659467659 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fbaec004020 con 0x7fbaf4105180 2024-09-21T03:36:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.671+0000 7fbaeb400640 1 --2- 172.21.15.16:0/1659467659 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbad80b6500 0x7fbad80b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.672+0000 7fbaeb400640 1 -- 172.21.15.16:0/1659467659 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbaec0e5490 con 0x7fbaf4105180 2024-09-21T03:36:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.672+0000 7fbaf2a00640 1 --2- 172.21.15.16:0/1659467659 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbad80b6500 0x7fbad80b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.672+0000 7fbaf2a00640 1 --2- 172.21.15.16:0/1659467659 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbad80b6500 0x7fbad80b8a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbae4000c00 tx=0x7fbae4031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.673+0000 7fbaeb400640 1 -- 172.21.15.16:0/1659467659 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbaec0ad950 con 0x7fbaf4105180 2024-09-21T03:36:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.874+0000 7fbaf9c00640 1 -- 172.21.15.16:0/1659467659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fbaf402e640 con 0x7fbaf4105180 2024-09-21T03:36:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.880+0000 7fbaeb400640 1 -- 172.21.15.16:0/1659467659 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v33)=0 v33) ==== 118+0+0 (secure 0 0 0) 0x7fbaec0b1d30 con 0x7fbaf4105180 2024-09-21T03:36:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.883+0000 7fbaf9c00640 1 -- 172.21.15.16:0/1659467659 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbad80b6500 msgr2=0x7fbad80b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.883+0000 7fbaf9c00640 1 --2- 172.21.15.16:0/1659467659 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbad80b6500 0x7fbad80b8a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbae4000c00 tx=0x7fbae4031400 comp rx=0 tx=0).stop 2024-09-21T03:36:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.883+0000 7fbaf9c00640 1 -- 172.21.15.16:0/1659467659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf4105180 msgr2=0x7fbaf4075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.883+0000 7fbaf9c00640 1 --2- 172.21.15.16:0/1659467659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf4105180 0x7fbaf4075c50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009fd0 tx=0x7fbaec00def0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.883+0000 7fbaf9c00640 1 -- 172.21.15.16:0/1659467659 shutdown_connections 2024-09-21T03:36:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.883+0000 7fbaf9c00640 1 --2- 172.21.15.16:0/1659467659 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbad80b6500 0x7fbad80b8a10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.883+0000 7fbaf9c00640 1 --2- 172.21.15.16:0/1659467659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf4105180 0x7fbaf4075c50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.883+0000 7fbaf9c00640 1 --2- 172.21.15.16:0/1659467659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaf41047e0 0x7fbaf4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.883+0000 7fbaf9c00640 1 --2- 172.21.15.16:0/1659467659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaf4103750 0x7fbaf4076dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.883+0000 7fbaf9c00640 1 -- 172.21.15.16:0/1659467659 >> 172.21.15.16:0/1659467659 conn(0x7fbaf40ff0c0 msgr2=0x7fbaf4107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.883+0000 7fbaf9c00640 1 -- 172.21.15.16:0/1659467659 shutdown_connections 2024-09-21T03:36:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:22.883+0000 7fbaf9c00640 1 -- 172.21.15.16:0/1659467659 wait complete. 2024-09-21T03:36:22.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:22 smithi123 ceph-mon[34673]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:22.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:22 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2164726478' entity='client.admin' 2024-09-21T03:36:23.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:22 smithi134 ceph-mon[35031]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:23.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:22 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2164726478' entity='client.admin' 2024-09-21T03:36:23.200 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi016') 2024-09-21T03:36:23.200 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi016') 2024-09-21T03:36:23.200 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi016') 2024-09-21T03:36:23.200 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi016') 2024-09-21T03:36:23.200 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi016') 2024-09-21T03:36:23.201 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi016') 2024-09-21T03:36:24.067 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:23 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1659467659' entity='client.admin' 2024-09-21T03:36:24.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:23 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1659467659' entity='client.admin' 2024-09-21T03:36:24.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:23 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1659467659' entity='client.admin' 2024-09-21T03:36:24.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:24 smithi016 ceph-mon[30375]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.935+0000 7fe571a00640 1 -- 172.21.15.16:0/4082720846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c075cf0 msgr2=0x7fe56c076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.935+0000 7fe571a00640 1 --2- 172.21.15.16:0/4082720846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c075cf0 0x7fe56c076100 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe558009900 tx=0x7fe55802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.935+0000 7fe571a00640 1 -- 172.21.15.16:0/4082720846 shutdown_connections 2024-09-21T03:36:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.935+0000 7fe571a00640 1 --2- 172.21.15.16:0/4082720846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c10fee0 0x7fe56c112300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.935+0000 7fe571a00640 1 --2- 172.21.15.16:0/4082720846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c075cf0 0x7fe56c076100 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.935+0000 7fe571a00640 1 --2- 172.21.15.16:0/4082720846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe56c0770e0 0x7fe56c0757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.935+0000 7fe571a00640 1 -- 172.21.15.16:0/4082720846 >> 172.21.15.16:0/4082720846 conn(0x7fe56c0ff0e0 msgr2=0x7fe56c101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.936+0000 7fe571a00640 1 -- 172.21.15.16:0/4082720846 shutdown_connections 2024-09-21T03:36:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.936+0000 7fe571a00640 1 -- 172.21.15.16:0/4082720846 wait complete. 2024-09-21T03:36:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.937+0000 7fe571a00640 1 Processor -- start 2024-09-21T03:36:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.938+0000 7fe571a00640 1 -- start start 2024-09-21T03:36:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.938+0000 7fe571a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe56c075cf0 0x7fe56c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.938+0000 7fe571a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c0770e0 0x7fe56c1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.938+0000 7fe571a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c10fee0 0x7fe56c1a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.938+0000 7fe571a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c1159f0 con 0x7fe56c10fee0 2024-09-21T03:36:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.938+0000 7fe571a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c115870 con 0x7fe56c0770e0 2024-09-21T03:36:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.938+0000 7fe571a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c115b70 con 0x7fe56c075cf0 2024-09-21T03:36:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.938+0000 7fe56aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe56c075cf0 0x7fe56c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:24.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.938+0000 7fe56aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe56c075cf0 0x7fe56c1a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:48164/0 (socket says 172.21.15.16:48164) 2024-09-21T03:36:24.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.939+0000 7fe56aa00640 1 -- 172.21.15.16:0/1555247548 learned_addr learned my addr 172.21.15.16:0/1555247548 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:24.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.939+0000 7fe56a000640 1 --2- 172.21.15.16:0/1555247548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c0770e0 0x7fe56c1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:24.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.939+0000 7fe56b400640 1 --2- 172.21.15.16:0/1555247548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c10fee0 0x7fe56c1a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:24.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.939+0000 7fe56aa00640 1 -- 172.21.15.16:0/1555247548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c0770e0 msgr2=0x7fe56c1a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:24.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.939+0000 7fe56aa00640 1 --2- 172.21.15.16:0/1555247548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c0770e0 0x7fe56c1a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.939+0000 7fe56aa00640 1 -- 172.21.15.16:0/1555247548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c10fee0 msgr2=0x7fe56c1a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:24.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.939+0000 7fe56aa00640 1 --2- 172.21.15.16:0/1555247548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c10fee0 0x7fe56c1a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.939+0000 7fe56aa00640 1 -- 172.21.15.16:0/1555247548 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c1a2050 con 0x7fe56c075cf0 2024-09-21T03:36:24.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.939+0000 7fe56aa00640 1 --2- 172.21.15.16:0/1555247548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe56c075cf0 0x7fe56c1a0620 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00e9f0 tx=0x7fe55c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:24.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.940+0000 7fe563400640 1 -- 172.21.15.16:0/1555247548 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c00cd80 con 0x7fe56c075cf0 2024-09-21T03:36:24.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.940+0000 7fe563400640 1 -- 172.21.15.16:0/1555247548 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe55c0054d0 con 0x7fe56c075cf0 2024-09-21T03:36:24.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.940+0000 7fe563400640 1 -- 172.21.15.16:0/1555247548 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c010690 con 0x7fe56c075cf0 2024-09-21T03:36:24.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.940+0000 7fe571a00640 1 -- 172.21.15.16:0/1555247548 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56c1a2340 con 0x7fe56c075cf0 2024-09-21T03:36:24.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.940+0000 7fe571a00640 1 -- 172.21.15.16:0/1555247548 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe56c1a8a40 con 0x7fe56c075cf0 2024-09-21T03:36:24.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.941+0000 7fe571a00640 1 -- 172.21.15.16:0/1555247548 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56c105f70 con 0x7fe56c075cf0 2024-09-21T03:36:24.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.943+0000 7fe563400640 1 -- 172.21.15.16:0/1555247548 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fe55c004e90 con 0x7fe56c075cf0 2024-09-21T03:36:24.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.945+0000 7fe563400640 1 --2- 172.21.15.16:0/1555247548 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5500b6500 0x7fe5500b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:24.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.945+0000 7fe563400640 1 -- 172.21.15.16:0/1555247548 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe55c0e5de0 con 0x7fe56c075cf0 2024-09-21T03:36:24.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.945+0000 7fe56a000640 1 --2- 172.21.15.16:0/1555247548 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5500b6500 0x7fe5500b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:24.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.946+0000 7fe56a000640 1 --2- 172.21.15.16:0/1555247548 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5500b6500 0x7fe5500b8a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe558031490 tx=0x7fe558005a20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:24.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:24.947+0000 7fe563400640 1 -- 172.21.15.16:0/1555247548 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe55c0ae2a0 con 0x7fe56c075cf0 2024-09-21T03:36:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:25.153+0000 7fe571a00640 1 -- 172.21.15.16:0/1555247548 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fe56c02e640 con 0x7fe56c075cf0 2024-09-21T03:36:25.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:24 smithi123 ceph-mon[34673]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:25.155+0000 7fe563400640 1 -- 172.21.15.16:0/1555247548 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v33)=0 v33) ==== 106+0+0 (secure 0 0 0) 0x7fe55c0b2680 con 0x7fe56c075cf0 2024-09-21T03:36:25.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:25.157+0000 7fe571a00640 1 -- 172.21.15.16:0/1555247548 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5500b6500 msgr2=0x7fe5500b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:25.157+0000 7fe571a00640 1 --2- 172.21.15.16:0/1555247548 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5500b6500 0x7fe5500b8a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe558031490 tx=0x7fe558005a20 comp rx=0 tx=0).stop 2024-09-21T03:36:25.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:25.158+0000 7fe571a00640 1 -- 172.21.15.16:0/1555247548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe56c075cf0 msgr2=0x7fe56c1a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:25.158+0000 7fe571a00640 1 --2- 172.21.15.16:0/1555247548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe56c075cf0 0x7fe56c1a0620 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00e9f0 tx=0x7fe55c00edf0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:25.158+0000 7fe571a00640 1 -- 172.21.15.16:0/1555247548 shutdown_connections 2024-09-21T03:36:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:25.158+0000 7fe571a00640 1 --2- 172.21.15.16:0/1555247548 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5500b6500 0x7fe5500b8a10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:25.158+0000 7fe571a00640 1 --2- 172.21.15.16:0/1555247548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe56c10fee0 0x7fe56c1a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:25.158+0000 7fe571a00640 1 --2- 172.21.15.16:0/1555247548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c0770e0 0x7fe56c1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:25.158+0000 7fe571a00640 1 --2- 172.21.15.16:0/1555247548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe56c075cf0 0x7fe56c1a0620 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:25.158+0000 7fe571a00640 1 -- 172.21.15.16:0/1555247548 >> 172.21.15.16:0/1555247548 conn(0x7fe56c0ff0e0 msgr2=0x7fe56c103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:25.158+0000 7fe571a00640 1 -- 172.21.15.16:0/1555247548 shutdown_connections 2024-09-21T03:36:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:25.158+0000 7fe571a00640 1 -- 172.21.15.16:0/1555247548 wait complete. 2024-09-21T03:36:25.229 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi016') 2024-09-21T03:36:25.229 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi016') 2024-09-21T03:36:25.229 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi016') 2024-09-21T03:36:25.229 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi016') 2024-09-21T03:36:25.229 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi016') 2024-09-21T03:36:25.229 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-09-21T03:36:25.229 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-09-21T03:36:25.230 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi016') 2024-09-21T03:36:25.230 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi016') 2024-09-21T03:36:25.230 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi016') 2024-09-21T03:36:25.230 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi016') 2024-09-21T03:36:25.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:24 smithi134 ceph-mon[35031]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:25.833 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi016') 2024-09-21T03:36:25.833 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi016') 2024-09-21T03:36:25.834 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2b324840-77ca-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi016') 2024-09-21T03:36:25.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T03:36:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:26.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.776+0000 7f21e7200640 1 -- 172.21.15.16:0/3172201908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e010fd20 msgr2=0x7f21e0112140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.776+0000 7f21e7200640 1 --2- 172.21.15.16:0/3172201908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e010fd20 0x7f21e0112140 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009ca0 tx=0x7f21dc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.776+0000 7f21e7200640 1 -- 172.21.15.16:0/3172201908 shutdown_connections 2024-09-21T03:36:26.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.776+0000 7f21e7200640 1 --2- 172.21.15.16:0/3172201908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e010fd20 0x7f21e0112140 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.776+0000 7f21e7200640 1 --2- 172.21.15.16:0/3172201908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21e0075cf0 0x7f21e0076100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.776+0000 7f21e7200640 1 --2- 172.21.15.16:0/3172201908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e00770e0 0x7f21e00757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.776+0000 7f21e7200640 1 -- 172.21.15.16:0/3172201908 >> 172.21.15.16:0/3172201908 conn(0x7f21e00fef00 msgr2=0x7f21e01013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:26.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.776+0000 7f21e7200640 1 -- 172.21.15.16:0/3172201908 shutdown_connections 2024-09-21T03:36:26.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.776+0000 7f21e7200640 1 -- 172.21.15.16:0/3172201908 wait complete. 2024-09-21T03:36:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.777+0000 7f21e7200640 1 Processor -- start 2024-09-21T03:36:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.777+0000 7f21e7200640 1 -- start start 2024-09-21T03:36:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.777+0000 7f21e7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21e0075cf0 0x7f21e01a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.777+0000 7f21e7200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e00770e0 0x7f21e01a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.778+0000 7f21e7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e010fd20 0x7f21e01a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.778+0000 7f21e7200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21e0115830 con 0x7f21e00770e0 2024-09-21T03:36:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.778+0000 7f21e7200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01156b0 con 0x7f21e0075cf0 2024-09-21T03:36:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.778+0000 7f21e7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01159b0 con 0x7f21e010fd20 2024-09-21T03:36:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.778+0000 7f21e5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e00770e0 0x7f21e01a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.778+0000 7f21e5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21e0075cf0 0x7f21e01a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.778+0000 7f21e5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21e0075cf0 0x7f21e01a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37068/0 (socket says 172.21.15.16:37068) 2024-09-21T03:36:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.778+0000 7f21e5e00640 1 -- 172.21.15.16:0/2686133779 learned_addr learned my addr 172.21.15.16:0/2686133779 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.778+0000 7f21e6800640 1 --2- 172.21.15.16:0/2686133779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e010fd20 0x7f21e01a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.778+0000 7f21e5e00640 1 -- 172.21.15.16:0/2686133779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e010fd20 msgr2=0x7f21e01a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.778+0000 7f21e5e00640 1 --2- 172.21.15.16:0/2686133779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e010fd20 0x7f21e01a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.778+0000 7f21e5e00640 1 -- 172.21.15.16:0/2686133779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e00770e0 msgr2=0x7f21e01a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.778+0000 7f21e5e00640 1 --2- 172.21.15.16:0/2686133779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e00770e0 0x7f21e01a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.778+0000 7f21e5e00640 1 -- 172.21.15.16:0/2686133779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e01a1e10 con 0x7f21e0075cf0 2024-09-21T03:36:26.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.778+0000 7f21e5400640 1 --2- 172.21.15.16:0/2686133779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e00770e0 0x7f21e01a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:26.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.778+0000 7f21e6800640 1 --2- 172.21.15.16:0/2686133779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e010fd20 0x7f21e01a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:26.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.779+0000 7f21e5e00640 1 --2- 172.21.15.16:0/2686133779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21e0075cf0 0x7f21e01a0450 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f21d400e9f0 tx=0x7f21d400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:26.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.779+0000 7f21daa00640 1 -- 172.21.15.16:0/2686133779 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d400cd80 con 0x7f21e0075cf0 2024-09-21T03:36:26.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.779+0000 7f21e7200640 1 -- 172.21.15.16:0/2686133779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e01a2100 con 0x7f21e0075cf0 2024-09-21T03:36:26.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.779+0000 7f21daa00640 1 -- 172.21.15.16:0/2686133779 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21d40054d0 con 0x7f21e0075cf0 2024-09-21T03:36:26.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.779+0000 7f21daa00640 1 -- 172.21.15.16:0/2686133779 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4010690 con 0x7f21e0075cf0 2024-09-21T03:36:26.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.779+0000 7f21e7200640 1 -- 172.21.15.16:0/2686133779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e01a8800 con 0x7f21e0075cf0 2024-09-21T03:36:26.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.780+0000 7f21e7200640 1 -- 172.21.15.16:0/2686133779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e0105db0 con 0x7f21e0075cf0 2024-09-21T03:36:26.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.782+0000 7f21daa00640 1 -- 172.21.15.16:0/2686133779 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f21d4004e90 con 0x7f21e0075cf0 2024-09-21T03:36:26.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.784+0000 7f21daa00640 1 --2- 172.21.15.16:0/2686133779 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f21c80b6480 0x7f21c80b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.784+0000 7f21daa00640 1 -- 172.21.15.16:0/2686133779 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f21d40e5b20 con 0x7f21e0075cf0 2024-09-21T03:36:26.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.784+0000 7f21e5400640 1 --2- 172.21.15.16:0/2686133779 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f21c80b6480 0x7f21c80b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.785+0000 7f21e5400640 1 --2- 172.21.15.16:0/2686133779 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f21c80b6480 0x7f21c80b8990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009fc0 tx=0x7f21d0005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:26.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.788+0000 7f21daa00640 1 -- 172.21.15.16:0/2686133779 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f21d40adfe0 con 0x7f21e0075cf0 2024-09-21T03:36:26.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.987+0000 7f21e7200640 1 -- 172.21.15.16:0/2686133779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f21e0063fc0 con 0x7f21e0075cf0 2024-09-21T03:36:26.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.988+0000 7f21daa00640 1 -- 172.21.15.16:0/2686133779 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+20909 (secure 0 0 0) 0x7f21d40b23c0 con 0x7f21e0075cf0 2024-09-21T03:36:26.990 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T03:36:26.990 INFO:teuthology.orchestra.run.smithi016.stdout:global dev auth_debug true 2024-09-21T03:36:26.990 INFO:teuthology.orchestra.run.smithi016.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T03:36:26.990 INFO:teuthology.orchestra.run.smithi016.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T03:36:26.990 INFO:teuthology.orchestra.run.smithi016.stdout:global basic log_to_file true 2024-09-21T03:36:26.990 INFO:teuthology.orchestra.run.smithi016.stdout:global basic log_to_journald false 2024-09-21T03:36:26.990 INFO:teuthology.orchestra.run.smithi016.stdout:global basic log_to_stderr false 2024-09-21T03:36:26.990 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_allow_pool_delete true 2024-09-21T03:36:26.990 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T03:36:26.990 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T03:36:26.991 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T03:36:26.991 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T03:36:26.991 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T03:36:26.991 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T03:36:26.991 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T03:36:26.991 INFO:teuthology.orchestra.run.smithi016.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T03:36:26.991 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T03:36:26.991 INFO:teuthology.orchestra.run.smithi016.stdout:global dev ms_die_on_bug true 2024-09-21T03:36:26.991 INFO:teuthology.orchestra.run.smithi016.stdout:global dev ms_die_on_old_message true 2024-09-21T03:36:26.991 INFO:teuthology.orchestra.run.smithi016.stdout:global dev ms_die_on_skipped_message false 2024-09-21T03:36:26.991 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T03:36:26.991 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T03:36:26.991 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T03:36:26.991 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T03:36:26.992 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T03:36:26.992 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced debug_mon 20/20 2024-09-21T03:36:26.992 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced debug_ms 1/1 2024-09-21T03:36:26.992 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced debug_paxos 20/20 2024-09-21T03:36:26.992 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T03:36:26.992 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T03:36:26.992 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T03:36:26.992 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T03:36:26.992 INFO:teuthology.orchestra.run.smithi016.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T03:36:26.992 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T03:36:26.992 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T03:36:26.992 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T03:36:26.992 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T03:36:26.992 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T03:36:26.992 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T03:36:26.993 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T03:36:26.993 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced client_mount_timeout 30 2024-09-21T03:36:26.993 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_client 20/20 2024-09-21T03:36:26.993 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_mgr 20/20 2024-09-21T03:36:26.993 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_ms 1/1 2024-09-21T03:36:26.993 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T03:36:26.994 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T03:36:26.994 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T03:36:26.994 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T03:36:26.994 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T03:36:26.994 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T03:36:26.994 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T03:36:26.994 INFO:teuthology.orchestra.run.smithi016.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T03:36:26.994 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev bdev_debug_aio true 2024-09-21T03:36:26.994 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T03:36:26.994 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T03:36:26.994 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced debug_bluefs 20/20 2024-09-21T03:36:26.994 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced debug_bluestore 20/20 2024-09-21T03:36:26.994 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced debug_ms 1/1 2024-09-21T03:36:26.994 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced debug_osd 20/20 2024-09-21T03:36:26.995 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced debug_rocksdb 10/10 2024-09-21T03:36:26.995 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T03:36:26.995 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev osd_debug_op_order true 2024-09-21T03:36:26.995 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T03:36:26.995 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev osd_debug_shutdown true 2024-09-21T03:36:26.995 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T03:36:26.995 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T03:36:26.995 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T03:36:26.995 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T03:36:26.995 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T03:36:26.995 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_max_omap_entries_per_request 10000 2024-09-21T03:36:26.995 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T03:36:26.995 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T03:36:26.995 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T03:36:26.996 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T03:36:26.996 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T03:36:26.996 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T03:36:26.996 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T03:36:26.996 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T03:36:26.996 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T03:36:26.996 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T03:36:26.996 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T03:36:26.996 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-21T03:36:26.996 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T03:36:26.996 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-21T03:36:26.996 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T03:36:26.996 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T03:36:26.996 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T03:36:26.996 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_bal_split_size 100 2024-09-21T03:36:26.997 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-21T03:36:26.997 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-21T03:36:26.997 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T03:36:26.997 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-21T03:36:26.997 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T03:36:26.997 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T03:36:26.997 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T03:36:26.997 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-21T03:36:26.997 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-21T03:36:26.997 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-21T03:36:26.997 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T03:36:26.997 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T03:36:26.997 INFO:teuthology.orchestra.run.smithi016.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T03:36:26.997 INFO:teuthology.orchestra.run.smithi016.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T03:36:26.997 INFO:teuthology.orchestra.run.smithi016.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T03:36:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.990+0000 7f21e7200640 1 -- 172.21.15.16:0/2686133779 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f21c80b6480 msgr2=0x7f21c80b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.990+0000 7f21e7200640 1 --2- 172.21.15.16:0/2686133779 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f21c80b6480 0x7f21c80b8990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009fc0 tx=0x7f21d0005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.990+0000 7f21e7200640 1 -- 172.21.15.16:0/2686133779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21e0075cf0 msgr2=0x7f21e01a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.990+0000 7f21e7200640 1 --2- 172.21.15.16:0/2686133779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21e0075cf0 0x7f21e01a0450 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f21d400e9f0 tx=0x7f21d400edf0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.990+0000 7f21e7200640 1 -- 172.21.15.16:0/2686133779 shutdown_connections 2024-09-21T03:36:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.990+0000 7f21e7200640 1 --2- 172.21.15.16:0/2686133779 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f21c80b6480 0x7f21c80b8990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.990+0000 7f21e7200640 1 --2- 172.21.15.16:0/2686133779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e010fd20 0x7f21e01a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.990+0000 7f21e7200640 1 --2- 172.21.15.16:0/2686133779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e00770e0 0x7f21e01a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.990+0000 7f21e7200640 1 --2- 172.21.15.16:0/2686133779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21e0075cf0 0x7f21e01a0450 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.990+0000 7f21e7200640 1 -- 172.21.15.16:0/2686133779 >> 172.21.15.16:0/2686133779 conn(0x7f21e00fef00 msgr2=0x7f21e0102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.991+0000 7f21e7200640 1 -- 172.21.15.16:0/2686133779 shutdown_connections 2024-09-21T03:36:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:26.991+0000 7f21e7200640 1 -- 172.21.15.16:0/2686133779 wait complete. 2024-09-21T03:36:27.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:26 smithi123 ceph-mon[34673]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:27.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:26 smithi016 ceph-mon[30375]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:27.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:26 smithi134 ceph-mon[35031]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:27.665 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T03:36:27.665 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:36:27.665 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:36:27.701 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:36:27.701 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-09-21T03:36:27.777 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:36:27.777 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:36:27.777 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 638 Links: 1 2024-09-21T03:36:27.777 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:36:27.777 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:36:27.777 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-21 03:35:11.075442371 +0000 2024-09-21T03:36:27.777 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-21 03:30:15.192432348 +0000 2024-09-21T03:36:27.777 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-21 03:30:15.192432348 +0000 2024-09-21T03:36:27.777 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-21 03:30:15.192432348 +0000 2024-09-21T03:36:27.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:36:27.872 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-21T03:36:27.872 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-21T03:36:27.872 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000180677 s, 2.8 MB/s 2024-09-21T03:36:27.874 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:36:27.912 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-09-21T03:36:27.988 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:36:27.988 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:36:27.988 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 658 Links: 1 2024-09-21T03:36:27.988 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:36:27.988 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:36:27.988 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-21 03:35:11.113441783 +0000 2024-09-21T03:36:27.988 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-21 03:30:15.657425106 +0000 2024-09-21T03:36:27.988 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-21 03:30:15.657425106 +0000 2024-09-21T03:36:27.988 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-21 03:30:15.657425106 +0000 2024-09-21T03:36:27.988 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:36:28.080 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-21T03:36:28.080 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-21T03:36:28.081 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000181964 s, 2.8 MB/s 2024-09-21T03:36:28.083 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:36:28.119 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-09-21T03:36:28.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:27 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2686133779' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:36:28.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:27 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2686133779' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:36:28.195 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:36:28.195 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:36:28.195 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 676 Links: 1 2024-09-21T03:36:28.196 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:36:28.196 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:36:28.196 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-21 03:35:11.153441165 +0000 2024-09-21T03:36:28.196 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-21 03:30:16.105418129 +0000 2024-09-21T03:36:28.196 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-21 03:30:16.105418129 +0000 2024-09-21T03:36:28.196 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-21 03:30:16.105418129 +0000 2024-09-21T03:36:28.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:36:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-21T03:36:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-21T03:36:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000180372 s, 2.8 MB/s 2024-09-21T03:36:28.290 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:36:28.327 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-09-21T03:36:28.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:27 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2686133779' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:36:28.402 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:36:28.402 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:36:28.402 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 696 Links: 1 2024-09-21T03:36:28.402 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:36:28.403 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:36:28.403 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-21 03:35:11.192440562 +0000 2024-09-21T03:36:28.403 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-21 03:30:16.564410980 +0000 2024-09-21T03:36:28.403 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-21 03:30:16.564410980 +0000 2024-09-21T03:36:28.403 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-21 03:30:16.564410980 +0000 2024-09-21T03:36:28.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:36:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-21T03:36:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-21T03:36:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000184674 s, 2.8 MB/s 2024-09-21T03:36:28.497 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:36:28.534 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T03:36:28.534 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:36:28.554 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:36:28.555 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-09-21T03:36:28.612 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:36:28.612 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:36:28.613 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T03:36:28.613 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:36:28.613 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:36:28.613 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-21 03:36:10.361893583 +0000 2024-09-21T03:36:28.613 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-21 03:29:36.471723251 +0000 2024-09-21T03:36:28.613 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-21 03:29:36.471723251 +0000 2024-09-21T03:36:28.613 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-21 03:29:36.471723251 +0000 2024-09-21T03:36:28.613 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:36:28.681 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-21T03:36:28.681 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-21T03:36:28.681 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000229903 s, 2.2 MB/s 2024-09-21T03:36:28.683 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:36:28.748 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-09-21T03:36:28.810 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:36:28.810 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:36:28.810 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T03:36:28.810 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:36:28.810 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:36:28.810 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-21 03:36:10.362893559 +0000 2024-09-21T03:36:28.810 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-21 03:29:36.813714725 +0000 2024-09-21T03:36:28.810 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-21 03:29:36.813714725 +0000 2024-09-21T03:36:28.810 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-21 03:29:36.813714725 +0000 2024-09-21T03:36:28.811 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:36:28.881 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-21T03:36:28.882 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-21T03:36:28.882 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000179813 s, 2.8 MB/s 2024-09-21T03:36:28.883 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:36:28.943 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-09-21T03:36:29.001 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:36:29.001 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:36:29.001 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-21T03:36:29.001 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:36:29.001 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:36:29.001 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-21 03:36:10.362893559 +0000 2024-09-21T03:36:29.001 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-21 03:29:37.142706524 +0000 2024-09-21T03:36:29.001 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-21 03:29:37.142706524 +0000 2024-09-21T03:36:29.001 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-21 03:29:37.142706524 +0000 2024-09-21T03:36:29.002 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:36:29.067 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:28 smithi123 ceph-mon[34673]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:29.069 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-21T03:36:29.070 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-21T03:36:29.070 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.00022063 s, 2.3 MB/s 2024-09-21T03:36:29.071 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:36:29.132 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-09-21T03:36:29.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:28 smithi016 ceph-mon[30375]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:29.190 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:36:29.190 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:36:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-21T03:36:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:36:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:36:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-21 03:36:10.362893559 +0000 2024-09-21T03:36:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-21 03:29:37.464698498 +0000 2024-09-21T03:36:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-21 03:29:37.464698498 +0000 2024-09-21T03:36:29.190 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-21 03:29:37.464698498 +0000 2024-09-21T03:36:29.191 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:36:29.256 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-21T03:36:29.256 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-21T03:36:29.256 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000200051 s, 2.6 MB/s 2024-09-21T03:36:29.257 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:36:29.314 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T03:36:29.314 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:36:29.337 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:36:29.337 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-09-21T03:36:29.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:28 smithi134 ceph-mon[35031]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:36:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:36:29.369 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T03:36:29.369 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:36:29.369 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:36:29.369 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-21 03:36:08.021728760 +0000 2024-09-21T03:36:29.369 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-21 03:29:45.517152833 +0000 2024-09-21T03:36:29.369 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-21 03:29:45.517152833 +0000 2024-09-21T03:36:29.370 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-21 03:29:45.517152833 +0000 2024-09-21T03:36:29.370 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:36:29.439 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-21T03:36:29.439 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-21T03:36:29.439 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000261464 s, 2.0 MB/s 2024-09-21T03:36:29.440 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:36:29.500 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-09-21T03:36:29.556 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:36:29.557 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:36:29.557 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-21T03:36:29.557 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:36:29.557 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:36:29.557 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-21 03:36:08.021728760 +0000 2024-09-21T03:36:29.557 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-21 03:29:45.844144819 +0000 2024-09-21T03:36:29.557 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-21 03:29:45.844144819 +0000 2024-09-21T03:36:29.557 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-21 03:29:45.844144819 +0000 2024-09-21T03:36:29.557 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:36:29.623 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-21T03:36:29.623 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-21T03:36:29.623 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000239778 s, 2.1 MB/s 2024-09-21T03:36:29.624 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:36:29.682 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-09-21T03:36:29.737 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:36:29.738 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:36:29.738 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-21T03:36:29.738 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:36:29.738 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:36:29.738 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-21 03:36:08.021728760 +0000 2024-09-21T03:36:29.738 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-21 03:29:46.170136829 +0000 2024-09-21T03:36:29.738 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-21 03:29:46.170136829 +0000 2024-09-21T03:36:29.738 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-21 03:29:46.170136829 +0000 2024-09-21T03:36:29.738 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:36:29.802 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-21T03:36:29.802 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-21T03:36:29.802 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000220072 s, 2.3 MB/s 2024-09-21T03:36:29.803 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:36:29.860 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-09-21T03:36:29.916 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:36:29.916 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:36:29.916 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-21T03:36:29.917 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:36:29.917 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:36:29.917 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-21 03:36:08.022728736 +0000 2024-09-21T03:36:29.917 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-21 03:29:46.495128865 +0000 2024-09-21T03:36:29.917 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-21 03:29:46.495128865 +0000 2024-09-21T03:36:29.917 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-21 03:29:46.495128865 +0000 2024-09-21T03:36:29.917 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:36:29.980 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-21T03:36:29.980 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-21T03:36:29.980 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000194874 s, 2.6 MB/s 2024-09-21T03:36:29.981 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:36:30.038 INFO:tasks.cephadm:Deploying osd.0 on smithi016 with /dev/vg_nvme/lv_4... 2024-09-21T03:36:30.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:36:30.414 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:31.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:30 smithi123 ceph-mon[34673]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:31.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:30 smithi016 ceph-mon[30375]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:31.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:30 smithi134 ceph-mon[35031]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:33.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:32 smithi123 ceph-mon[34673]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:32 smithi016 ceph-mon[30375]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:33.213 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:36:33.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi016:vg_nvme/lv_4 2024-09-21T03:36:33.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:32 smithi134 ceph-mon[35031]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:34.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.239+0000 7f1bf2800640 1 -- 172.21.15.16:0/616294103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bec106710 msgr2=0x7f1bec108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.239+0000 7f1bf2800640 1 --2- 172.21.15.16:0/616294103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bec106710 0x7f1bec108b70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009900 tx=0x7f1bd802a410 comp rx=0 tx=0).stop 2024-09-21T03:36:34.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.239+0000 7f1bf2800640 1 -- 172.21.15.16:0/616294103 shutdown_connections 2024-09-21T03:36:34.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.239+0000 7f1bf2800640 1 --2- 172.21.15.16:0/616294103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bec1092b0 0x7f1bec10b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.239+0000 7f1bf2800640 1 --2- 172.21.15.16:0/616294103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bec106710 0x7f1bec108b70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.239+0000 7f1bf2800640 1 --2- 172.21.15.16:0/616294103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bec06c420 0x7f1bec1061d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.239+0000 7f1bf2800640 1 -- 172.21.15.16:0/616294103 >> 172.21.15.16:0/616294103 conn(0x7f1bec0fde10 msgr2=0x7f1bec1002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:34.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.239+0000 7f1bf2800640 1 -- 172.21.15.16:0/616294103 shutdown_connections 2024-09-21T03:36:34.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.239+0000 7f1bf2800640 1 -- 172.21.15.16:0/616294103 wait complete. 2024-09-21T03:36:34.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.240+0000 7f1bf2800640 1 Processor -- start 2024-09-21T03:36:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.240+0000 7f1bf2800640 1 -- start start 2024-09-21T03:36:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bec06c420 0x7f1bec199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf2800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bec106710 0x7f1bec19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bec1092b0 0x7f1bec19b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf2800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec10f220 con 0x7f1bec06c420 2024-09-21T03:36:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf2800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec10f0a0 con 0x7f1bec1092b0 2024-09-21T03:36:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf2800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec10f3a0 con 0x7f1bec106710 2024-09-21T03:36:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bec06c420 0x7f1bec199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bec06c420 0x7f1bec199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38178/0 (socket says 172.21.15.16:38178) 2024-09-21T03:36:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf1400640 1 -- 172.21.15.16:0/3668247331 learned_addr learned my addr 172.21.15.16:0/3668247331 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:36:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf0a00640 1 --2- 172.21.15.16:0/3668247331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bec106710 0x7f1bec19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf1e00640 1 --2- 172.21.15.16:0/3668247331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bec1092b0 0x7f1bec19b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf1400640 1 -- 172.21.15.16:0/3668247331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bec106710 msgr2=0x7f1bec19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf1400640 1 --2- 172.21.15.16:0/3668247331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bec106710 0x7f1bec19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf1400640 1 -- 172.21.15.16:0/3668247331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bec1092b0 msgr2=0x7f1bec19b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf1400640 1 --2- 172.21.15.16:0/3668247331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bec1092b0 0x7f1bec19b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf1400640 1 -- 172.21.15.16:0/3668247331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bec19b880 con 0x7f1bec06c420 2024-09-21T03:36:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf0a00640 1 --2- 172.21.15.16:0/3668247331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bec106710 0x7f1bec19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.241+0000 7f1bf1e00640 1 --2- 172.21.15.16:0/3668247331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bec1092b0 0x7f1bec19b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:34.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.242+0000 7f1bf1400640 1 --2- 172.21.15.16:0/3668247331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bec06c420 0x7f1bec199e50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1be000e9f0 tx=0x7f1be000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:34.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.242+0000 7f1bea000640 1 -- 172.21.15.16:0/3668247331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be000cd80 con 0x7f1bec06c420 2024-09-21T03:36:34.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.242+0000 7f1bea000640 1 -- 172.21.15.16:0/3668247331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1be00108e0 con 0x7f1bec06c420 2024-09-21T03:36:34.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.242+0000 7f1bea000640 1 -- 172.21.15.16:0/3668247331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be0018980 con 0x7f1bec06c420 2024-09-21T03:36:34.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.242+0000 7f1bf2800640 1 -- 172.21.15.16:0/3668247331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bec19bb70 con 0x7f1bec06c420 2024-09-21T03:36:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.242+0000 7f1bf2800640 1 -- 172.21.15.16:0/3668247331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bec1a2270 con 0x7f1bec06c420 2024-09-21T03:36:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.243+0000 7f1bf2800640 1 -- 172.21.15.16:0/3668247331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bec0ff930 con 0x7f1bec06c420 2024-09-21T03:36:34.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.246+0000 7f1bea000640 1 -- 172.21.15.16:0/3668247331 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f1be0010430 con 0x7f1bec06c420 2024-09-21T03:36:34.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.247+0000 7f1bea000640 1 --2- 172.21.15.16:0/3668247331 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1bd40b6500 0x7f1bd40b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:34.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.247+0000 7f1bea000640 1 -- 172.21.15.16:0/3668247331 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1be00e5f50 con 0x7f1bec06c420 2024-09-21T03:36:34.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.247+0000 7f1bf0a00640 1 --2- 172.21.15.16:0/3668247331 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1bd40b6500 0x7f1bd40b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:34.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.248+0000 7f1bf0a00640 1 --2- 172.21.15.16:0/3668247331 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1bd40b6500 0x7f1bd40b8a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8000c00 tx=0x7f1bd8031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:34.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.249+0000 7f1bea000640 1 -- 172.21.15.16:0/3668247331 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1be00ae410 con 0x7f1bec06c420 2024-09-21T03:36:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:36:34.457+0000 7f1bf2800640 1 -- 172.21.15.16:0/3668247331 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1bec063800 con 0x7f1bd40b6500 2024-09-21T03:36:35.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:34 smithi123 ceph-mon[34673]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:35.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:34 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:36:35.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:34 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:36:35.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:34 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:35.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:34 smithi016 ceph-mon[30375]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:35.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:34 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:36:35.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:34 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:36:35.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:34 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:35.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:34 smithi134 ceph-mon[35031]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:35.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:34 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:36:35.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:34 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:36:35.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:34 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:36.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:35 smithi016 ceph-mon[30375]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:36.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:35 smithi134 ceph-mon[35031]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:36.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:35 smithi123 ceph-mon[34673]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:37.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:36 smithi016 ceph-mon[30375]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:37.253 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:36 smithi134 ceph-mon[35031]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:37.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:36 smithi123 ceph-mon[34673]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:38.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:37 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1063588186' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6a7e4b9e-078f-4ae9-ab9e-37be995e4deb"} : dispatch 2024-09-21T03:36:38.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:37 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1063588186' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a7e4b9e-078f-4ae9-ab9e-37be995e4deb"}]': finished 2024-09-21T03:36:38.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:37 smithi016 ceph-mon[30375]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T03:36:38.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:37 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:36:38.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:37 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1063588186' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6a7e4b9e-078f-4ae9-ab9e-37be995e4deb"} : dispatch 2024-09-21T03:36:38.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:37 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1063588186' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a7e4b9e-078f-4ae9-ab9e-37be995e4deb"}]': finished 2024-09-21T03:36:38.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:37 smithi134 ceph-mon[35031]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T03:36:38.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:37 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:36:38.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:37 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1063588186' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6a7e4b9e-078f-4ae9-ab9e-37be995e4deb"} : dispatch 2024-09-21T03:36:38.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:37 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1063588186' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a7e4b9e-078f-4ae9-ab9e-37be995e4deb"}]': finished 2024-09-21T03:36:38.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:37 smithi123 ceph-mon[34673]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T03:36:38.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:37 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:36:39.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:38 smithi134 ceph-mon[35031]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:39.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:38 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1169061040' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:36:39.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:38 smithi123 ceph-mon[34673]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:39.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:38 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1169061040' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:36:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:38 smithi016 ceph-mon[30375]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:38 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1169061040' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:36:41.208 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:40 smithi123 ceph-mon[34673]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:41.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:40 smithi134 ceph-mon[35031]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:41.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:40 smithi016 ceph-mon[30375]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:42 smithi134 ceph-mon[35031]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:43.389 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:42 smithi016 ceph-mon[30375]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:43.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:42 smithi123 ceph-mon[34673]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:45.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:44 smithi016 ceph-mon[30375]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:45.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:44 smithi134 ceph-mon[35031]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:45.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:44 smithi123 ceph-mon[34673]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:47.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:46 smithi016 ceph-mon[30375]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:47.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:46 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:36:47.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:46 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:47.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:46 smithi134 ceph-mon[35031]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:47.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:46 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:36:47.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:46 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:47.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:46 smithi123 ceph-mon[34673]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:47.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:46 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:36:47.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:46 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:48.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:47 smithi016 ceph-mon[30375]: Deploying daemon osd.0 on smithi016 2024-09-21T03:36:48.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:47 smithi134 ceph-mon[35031]: Deploying daemon osd.0 on smithi016 2024-09-21T03:36:48.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:47 smithi123 ceph-mon[34673]: Deploying daemon osd.0 on smithi016 2024-09-21T03:36:49.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:48 smithi016 ceph-mon[30375]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:49.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:48 smithi134 ceph-mon[35031]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:49.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:48 smithi123 ceph-mon[34673]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:51.200 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:50 smithi016 ceph-mon[30375]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:51.355 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:50 smithi134 ceph-mon[35031]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:51.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:50 smithi123 ceph-mon[34673]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:53.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:52 smithi134 ceph-mon[35031]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:53.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:52 smithi123 ceph-mon[34673]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:53.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:52 smithi016 ceph-mon[30375]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:55.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:54 smithi016 ceph-mon[30375]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:55.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:54 smithi134 ceph-mon[35031]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:55.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:54 smithi123 ceph-mon[34673]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:56.211 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:55 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:56.211 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:55 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:56.211 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:55 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:56.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:55 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:56.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:55 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:56.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:55 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:56.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:55 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:56.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:55 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:56.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:55 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:57.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:56 smithi016 ceph-mon[30375]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:57.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:56 smithi134 ceph-mon[35031]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:57.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:56 smithi123 ceph-mon[34673]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:59.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:58 smithi123 ceph-mon[34673]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:59.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:59.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:59.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:59.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:59.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:36:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:59.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:58 smithi016 ceph-mon[30375]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:59.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:59.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:59.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:59.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:59.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:36:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:58 smithi134 ceph-mon[35031]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:36:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:36:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:36:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:00.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:00.809+0000 7f1bea000640 1 -- 172.21.15.16:0/3668247331 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1bec063800 con 0x7f1bd40b6500 2024-09-21T03:37:00.812 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 0 on host 'smithi016' 2024-09-21T03:37:00.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:00.812+0000 7f1bf2800640 1 -- 172.21.15.16:0/3668247331 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1bd40b6500 msgr2=0x7f1bd40b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:00.812+0000 7f1bf2800640 1 --2- 172.21.15.16:0/3668247331 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1bd40b6500 0x7f1bd40b8a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8000c00 tx=0x7f1bd8031400 comp rx=0 tx=0).stop 2024-09-21T03:37:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:00.812+0000 7f1bf2800640 1 -- 172.21.15.16:0/3668247331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bec06c420 msgr2=0x7f1bec199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:00.812+0000 7f1bf2800640 1 --2- 172.21.15.16:0/3668247331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bec06c420 0x7f1bec199e50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1be000e9f0 tx=0x7f1be000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:00.812+0000 7f1bf2800640 1 -- 172.21.15.16:0/3668247331 shutdown_connections 2024-09-21T03:37:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:00.813+0000 7f1bf2800640 1 --2- 172.21.15.16:0/3668247331 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1bd40b6500 0x7f1bd40b8a10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:00.813+0000 7f1bf2800640 1 --2- 172.21.15.16:0/3668247331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bec1092b0 0x7f1bec19b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:00.813+0000 7f1bf2800640 1 --2- 172.21.15.16:0/3668247331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bec106710 0x7f1bec19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:00.813+0000 7f1bf2800640 1 --2- 172.21.15.16:0/3668247331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bec06c420 0x7f1bec199e50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:00.813+0000 7f1bf2800640 1 -- 172.21.15.16:0/3668247331 >> 172.21.15.16:0/3668247331 conn(0x7f1bec0fde10 msgr2=0x7f1bec104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:00.813+0000 7f1bf2800640 1 -- 172.21.15.16:0/3668247331 shutdown_connections 2024-09-21T03:37:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:00.813+0000 7f1bf2800640 1 -- 172.21.15.16:0/3668247331 wait complete. 2024-09-21T03:37:01.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:00 smithi123 ceph-mon[34673]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:01.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:00 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:01.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:00 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:01.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:00 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:00 smithi016 ceph-mon[30375]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:00 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:00 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:00 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:00 smithi134 ceph-mon[35031]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:00 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:00 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:00 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:01.569 DEBUG:teuthology.orchestra.run.smithi016:osd.0> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T03:37:01.573 INFO:tasks.cephadm:Deploying osd.1 on smithi123 with /dev/vg_nvme/lv_4... 2024-09-21T03:37:01.573 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:37:01.823 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:37:02.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:01 smithi016 ceph-mon[30375]: from='osd.0 [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:37:02.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:01 smithi123 ceph-mon[34673]: from='osd.0 [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:37:02.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:01 smithi134 ceph-mon[35031]: from='osd.0 [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:37:03.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:02 smithi016 ceph-mon[30375]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:03.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:02 smithi016 ceph-mon[30375]: from='osd.0 [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:37:03.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:02 smithi016 ceph-mon[30375]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:37:03.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:02 smithi016 ceph-mon[30375]: from='osd.0 [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:37:03.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:02 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:37:03.136 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:37:02 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-0[45832]: 2024-09-21T03:37:02.912+0000 7f5f35600640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T03:37:03.136 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:37:02 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-0[45832]: 2024-09-21T03:37:02.916+0000 7f5f30600640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:37:03.151 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:02 smithi123 ceph-mon[34673]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:03.151 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:02 smithi123 ceph-mon[34673]: from='osd.0 [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:37:03.151 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:02 smithi123 ceph-mon[34673]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:37:03.151 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:02 smithi123 ceph-mon[34673]: from='osd.0 [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:37:03.152 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:02 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:37:03.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:02 smithi134 ceph-mon[35031]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:03.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:02 smithi134 ceph-mon[35031]: from='osd.0 [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:37:03.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:02 smithi134 ceph-mon[35031]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:37:03.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:02 smithi134 ceph-mon[35031]: from='osd.0 [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:37:03.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:02 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:37:04.025 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:37:04.050 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi123:vg_nvme/lv_4 2024-09-21T03:37:04.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:03 smithi016 ceph-mon[30375]: from='osd.0 [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T03:37:04.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:03 smithi016 ceph-mon[30375]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:37:04.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:03 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:37:04.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:03 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:37:04.294 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:37:04.319 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:03 smithi123 ceph-mon[34673]: from='osd.0 [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T03:37:04.319 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:03 smithi123 ceph-mon[34673]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:37:04.319 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:03 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:37:04.319 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:03 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:37:04.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:03 smithi134 ceph-mon[35031]: from='osd.0 [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T03:37:04.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:03 smithi134 ceph-mon[35031]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:37:04.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:03 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:37:04.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:03 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:37:04.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.744+0000 7f42d451f640 1 -- 172.21.15.123:0/4098873960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc1026f0 msgr2=0x7f42cc102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:04.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.744+0000 7f42d451f640 1 --2- 172.21.15.123:0/4098873960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc1026f0 0x7f42cc102b60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009900 tx=0x7f42c802a480 comp rx=0 tx=0).stop 2024-09-21T03:37:04.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.744+0000 7f42d451f640 1 -- 172.21.15.123:0/4098873960 shutdown_connections 2024-09-21T03:37:04.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.744+0000 7f42d451f640 1 --2- 172.21.15.123:0/4098873960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc104120 0x7f42cc10aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:04.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.744+0000 7f42d451f640 1 --2- 172.21.15.123:0/4098873960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42cc103780 0x7f42cc103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:04.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.744+0000 7f42d451f640 1 --2- 172.21.15.123:0/4098873960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc1026f0 0x7f42cc102b60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:04.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.744+0000 7f42d451f640 1 -- 172.21.15.123:0/4098873960 >> 172.21.15.123:0/4098873960 conn(0x7f42cc0fe080 msgr2=0x7f42cc100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:04.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.744+0000 7f42d451f640 1 -- 172.21.15.123:0/4098873960 shutdown_connections 2024-09-21T03:37:04.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.744+0000 7f42d451f640 1 -- 172.21.15.123:0/4098873960 wait complete. 2024-09-21T03:37:04.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d451f640 1 Processor -- start 2024-09-21T03:37:04.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d451f640 1 -- start start 2024-09-21T03:37:04.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d451f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc1026f0 0x7f42cc1062e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:04.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d451f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc103780 0x7f42cc104bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:04.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d451f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42cc104120 0x7f42cc1050f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:04.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d451f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc10e2a0 con 0x7f42cc1026f0 2024-09-21T03:37:04.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d451f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc10e120 con 0x7f42cc103780 2024-09-21T03:37:04.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d451f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc10e420 con 0x7f42cc104120 2024-09-21T03:37:04.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d1aba640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc103780 0x7f42cc104bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:04.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d1aba640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc103780 0x7f42cc104bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34564/0 (socket says 172.21.15.123:34564) 2024-09-21T03:37:04.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d1aba640 1 -- 172.21.15.123:0/2443383615 learned_addr learned my addr 172.21.15.123:0/2443383615 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-21T03:37:04.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d22bb640 1 --2- 172.21.15.123:0/2443383615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc1026f0 0x7f42cc1062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:04.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d2abc640 1 --2- 172.21.15.123:0/2443383615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42cc104120 0x7f42cc1050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:04.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d1aba640 1 -- 172.21.15.123:0/2443383615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42cc104120 msgr2=0x7f42cc1050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d1aba640 1 --2- 172.21.15.123:0/2443383615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42cc104120 0x7f42cc1050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d1aba640 1 -- 172.21.15.123:0/2443383615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc1026f0 msgr2=0x7f42cc1062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d1aba640 1 --2- 172.21.15.123:0/2443383615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc1026f0 0x7f42cc1062e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.745+0000 7f42d1aba640 1 -- 172.21.15.123:0/2443383615 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42cc1057f0 con 0x7f42cc103780 2024-09-21T03:37:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.746+0000 7f42d1aba640 1 --2- 172.21.15.123:0/2443383615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc103780 0x7f42cc104bb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f42c000d950 tx=0x7f42c000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.746+0000 7f42bf7fe640 1 -- 172.21.15.123:0/2443383615 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0017070 con 0x7f42cc103780 2024-09-21T03:37:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.746+0000 7f42bf7fe640 1 -- 172.21.15.123:0/2443383615 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42c0009c30 con 0x7f42cc103780 2024-09-21T03:37:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.746+0000 7f42d451f640 1 -- 172.21.15.123:0/2443383615 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42cc1a40b0 con 0x7f42cc103780 2024-09-21T03:37:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.746+0000 7f42bf7fe640 1 -- 172.21.15.123:0/2443383615 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c000f620 con 0x7f42cc103780 2024-09-21T03:37:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.746+0000 7f42d451f640 1 -- 172.21.15.123:0/2443383615 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42cc1a4510 con 0x7f42cc103780 2024-09-21T03:37:04.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.747+0000 7f42d451f640 1 -- 172.21.15.123:0/2443383615 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42cc1a4770 con 0x7f42cc103780 2024-09-21T03:37:04.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.749+0000 7f42bf7fe640 1 -- 172.21.15.123:0/2443383615 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f42c0010460 con 0x7f42cc103780 2024-09-21T03:37:04.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.749+0000 7f42bf7fe640 1 --2- 172.21.15.123:0/2443383615 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42b80b6480 0x7f42b80b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:04.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.750+0000 7f42bf7fe640 1 -- 172.21.15.123:0/2443383615 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f42c00e5410 con 0x7f42cc103780 2024-09-21T03:37:04.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.750+0000 7f42d22bb640 1 --2- 172.21.15.123:0/2443383615 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42b80b6480 0x7f42b80b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:04.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.750+0000 7f42d22bb640 1 --2- 172.21.15.123:0/2443383615 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42b80b6480 0x7f42b80b8990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42c8000c00 tx=0x7f42c8031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:04.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.757+0000 7f42bf7fe640 1 -- 172.21.15.123:0/2443383615 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f42c0009de0 con 0x7f42cc103780 2024-09-21T03:37:04.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:04.950+0000 7f42d451f640 1 -- 172.21.15.123:0/2443383615 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f42cc100030 con 0x7f42b80b6480 2024-09-21T03:37:05.127 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:04 smithi016 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T03:37:05.127 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:04 smithi016 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T03:37:05.127 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:04 smithi016 ceph-mon[30375]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:05.127 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:04 smithi016 ceph-mon[30375]: osd.0 [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551] boot 2024-09-21T03:37:05.127 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:04 smithi016 ceph-mon[30375]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T03:37:05.128 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:04 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:37:05.314 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:04 smithi123 ceph-mon[34673]: purged_snaps scrub starts 2024-09-21T03:37:05.314 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:04 smithi123 ceph-mon[34673]: purged_snaps scrub ok 2024-09-21T03:37:05.314 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:04 smithi123 ceph-mon[34673]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:05.314 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:04 smithi123 ceph-mon[34673]: osd.0 [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551] boot 2024-09-21T03:37:05.314 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:04 smithi123 ceph-mon[34673]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T03:37:05.314 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:04 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:37:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:04 smithi134 ceph-mon[35031]: purged_snaps scrub starts 2024-09-21T03:37:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:04 smithi134 ceph-mon[35031]: purged_snaps scrub ok 2024-09-21T03:37:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:04 smithi134 ceph-mon[35031]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:37:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:04 smithi134 ceph-mon[35031]: osd.0 [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551] boot 2024-09-21T03:37:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:04 smithi134 ceph-mon[35031]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T03:37:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:04 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:37:06.231 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:05 smithi123 ceph-mon[34673]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:37:06.232 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:05 smithi123 ceph-mon[34673]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:06.232 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:37:06.232 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:37:06.232 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:06.232 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:06.232 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:06.232 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:06.232 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:06.232 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:06.232 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:06.232 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:06.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:05 smithi134 ceph-mon[35031]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:37:06.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:05 smithi134 ceph-mon[35031]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:06.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:37:06.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:37:06.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:06.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:06.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:06.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:06.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:06.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:06.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:06.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:06.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:05 smithi016 ceph-mon[30375]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:37:06.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:05 smithi016 ceph-mon[30375]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:06.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:37:06.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:37:06.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:06.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:06.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:06.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:06.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:06.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:06.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:06.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:07.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:06 smithi123 ceph-mon[34673]: Detected new or changed devices on smithi016 2024-09-21T03:37:07.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:06 smithi123 ceph-mon[34673]: Adjusting osd_memory_target on smithi016 to 17094M 2024-09-21T03:37:07.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:06 smithi123 ceph-mon[34673]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:07.253 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:06 smithi134 ceph-mon[35031]: Detected new or changed devices on smithi016 2024-09-21T03:37:07.253 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:06 smithi134 ceph-mon[35031]: Adjusting osd_memory_target on smithi016 to 17094M 2024-09-21T03:37:07.253 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:06 smithi134 ceph-mon[35031]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:07.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:06 smithi016 ceph-mon[30375]: Detected new or changed devices on smithi016 2024-09-21T03:37:07.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:06 smithi016 ceph-mon[30375]: Adjusting osd_memory_target on smithi016 to 17094M 2024-09-21T03:37:07.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:06 smithi016 ceph-mon[30375]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:08.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:07 smithi123 ceph-mon[34673]: from='client.? 172.21.15.123:0/2732159784' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3a497a64-e1a8-43ef-8509-0a79515da0aa"} : dispatch 2024-09-21T03:37:08.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:07 smithi123 ceph-mon[34673]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3a497a64-e1a8-43ef-8509-0a79515da0aa"} : dispatch 2024-09-21T03:37:08.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:07 smithi123 ceph-mon[34673]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a497a64-e1a8-43ef-8509-0a79515da0aa"}]': finished 2024-09-21T03:37:08.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:07 smithi123 ceph-mon[34673]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T03:37:08.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:07 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:08.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:07 smithi123 ceph-mon[34673]: from='client.? 172.21.15.123:0/412520426' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:37:08.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:07 smithi134 ceph-mon[35031]: from='client.? 172.21.15.123:0/2732159784' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3a497a64-e1a8-43ef-8509-0a79515da0aa"} : dispatch 2024-09-21T03:37:08.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:07 smithi134 ceph-mon[35031]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3a497a64-e1a8-43ef-8509-0a79515da0aa"} : dispatch 2024-09-21T03:37:08.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:07 smithi134 ceph-mon[35031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a497a64-e1a8-43ef-8509-0a79515da0aa"}]': finished 2024-09-21T03:37:08.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:07 smithi134 ceph-mon[35031]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T03:37:08.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:07 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:08.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:07 smithi134 ceph-mon[35031]: from='client.? 172.21.15.123:0/412520426' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:37:08.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:07 smithi016 ceph-mon[30375]: from='client.? 172.21.15.123:0/2732159784' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3a497a64-e1a8-43ef-8509-0a79515da0aa"} : dispatch 2024-09-21T03:37:08.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:07 smithi016 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3a497a64-e1a8-43ef-8509-0a79515da0aa"} : dispatch 2024-09-21T03:37:08.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:07 smithi016 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a497a64-e1a8-43ef-8509-0a79515da0aa"}]': finished 2024-09-21T03:37:08.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:07 smithi016 ceph-mon[30375]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T03:37:08.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:07 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:08.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:07 smithi016 ceph-mon[30375]: from='client.? 172.21.15.123:0/412520426' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:37:09.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:08 smithi134 ceph-mon[35031]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:09.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:08 smithi123 ceph-mon[34673]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:09.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:08 smithi016 ceph-mon[30375]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:11.208 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:10 smithi123 ceph-mon[34673]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:11.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:10 smithi134 ceph-mon[35031]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:11.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:10 smithi016 ceph-mon[30375]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:12.939 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:12 smithi123 ceph-mon[34673]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:13.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:12 smithi134 ceph-mon[35031]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:13.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:12 smithi016 ceph-mon[30375]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:15.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:14 smithi123 ceph-mon[34673]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:15.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:14 smithi134 ceph-mon[35031]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:15.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:14 smithi016 ceph-mon[30375]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:16.110 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:15 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:37:16.110 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:15 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:16.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:15 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:37:16.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:15 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:16.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:15 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:37:16.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:15 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:17.212 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:16 smithi123 ceph-mon[34673]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:17.212 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:16 smithi123 ceph-mon[34673]: Deploying daemon osd.1 on smithi123 2024-09-21T03:37:17.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:16 smithi134 ceph-mon[35031]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:17.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:16 smithi134 ceph-mon[35031]: Deploying daemon osd.1 on smithi123 2024-09-21T03:37:17.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:16 smithi016 ceph-mon[30375]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:17.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:16 smithi016 ceph-mon[30375]: Deploying daemon osd.1 on smithi123 2024-09-21T03:37:19.328 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:18 smithi123 ceph-mon[34673]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:19.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:18 smithi134 ceph-mon[35031]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:19.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:18 smithi016 ceph-mon[30375]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:21.248 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:20 smithi123 ceph-mon[34673]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:21.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:20 smithi134 ceph-mon[35031]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:21.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:20 smithi016 ceph-mon[30375]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:22.329 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:21 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:22.329 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:21 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:22.329 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:21 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:22.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:21 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:22.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:21 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:22.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:21 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:22.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:21 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:22.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:21 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:22.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:21 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:23.204 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:22 smithi123 ceph-mon[34673]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:23.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:22 smithi134 ceph-mon[35031]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:23.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:22 smithi016 ceph-mon[30375]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:25.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:24 smithi123 ceph-mon[34673]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:25.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:24 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:25.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:24 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:25.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:24 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:25.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:24 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:25.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:24 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:25.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:24 smithi016 ceph-mon[30375]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:25.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:24 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:25.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:24 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:25.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:24 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:24 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:24 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:25.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:24 smithi134 ceph-mon[35031]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:25.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:24 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:25.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:24 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:25.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:24 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:25.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:24 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:25.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:24 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:25.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:25.864+0000 7f42bf7fe640 1 -- 172.21.15.123:0/2443383615 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f42cc100030 con 0x7f42b80b6480 2024-09-21T03:37:25.867 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 1 on host 'smithi123' 2024-09-21T03:37:25.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:25.867+0000 7f42d451f640 1 -- 172.21.15.123:0/2443383615 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42b80b6480 msgr2=0x7f42b80b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:25.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:25.867+0000 7f42d451f640 1 --2- 172.21.15.123:0/2443383615 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42b80b6480 0x7f42b80b8990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42c8000c00 tx=0x7f42c8031400 comp rx=0 tx=0).stop 2024-09-21T03:37:25.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:25.867+0000 7f42d451f640 1 -- 172.21.15.123:0/2443383615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc103780 msgr2=0x7f42cc104bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:25.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:25.867+0000 7f42d451f640 1 --2- 172.21.15.123:0/2443383615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc103780 0x7f42cc104bb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f42c000d950 tx=0x7f42c000de10 comp rx=0 tx=0).stop 2024-09-21T03:37:25.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:25.867+0000 7f42d451f640 1 -- 172.21.15.123:0/2443383615 shutdown_connections 2024-09-21T03:37:25.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:25.867+0000 7f42d451f640 1 --2- 172.21.15.123:0/2443383615 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42b80b6480 0x7f42b80b8990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:25.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:25.867+0000 7f42d451f640 1 --2- 172.21.15.123:0/2443383615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42cc104120 0x7f42cc1050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:25.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:25.867+0000 7f42d451f640 1 --2- 172.21.15.123:0/2443383615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc103780 0x7f42cc104bb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:25.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:25.867+0000 7f42d451f640 1 --2- 172.21.15.123:0/2443383615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc1026f0 0x7f42cc1062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:25.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:25.867+0000 7f42d451f640 1 -- 172.21.15.123:0/2443383615 >> 172.21.15.123:0/2443383615 conn(0x7f42cc0fe080 msgr2=0x7f42cc0ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:25.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:25.867+0000 7f42d451f640 1 -- 172.21.15.123:0/2443383615 shutdown_connections 2024-09-21T03:37:25.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:37:25.867+0000 7f42d451f640 1 -- 172.21.15.123:0/2443383615 wait complete. 2024-09-21T03:37:26.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:25 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:26.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:25 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:26.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:25 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:26.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:25 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:26.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:25 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:26.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:25 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:26.480 DEBUG:teuthology.orchestra.run.smithi123:osd.1> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T03:37:26.483 INFO:tasks.cephadm:Deploying osd.2 on smithi134 with /dev/vg_nvme/lv_4... 2024-09-21T03:37:26.483 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:37:26.733 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:37:27.031 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:26 smithi123 ceph-mon[34673]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:27.031 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:26 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:27.150 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:26 smithi134 ceph-mon[35031]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:27.151 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:26 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:27.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:26 smithi016 ceph-mon[30375]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:27.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:26 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:27.903 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:37:27 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-1[38603]: 2024-09-21T03:37:27.871+0000 7ff990844640 -1 osd.1 0 log_to_monitors true 2024-09-21T03:37:29.062 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:37:29.087 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi134:vg_nvme/lv_4 2024-09-21T03:37:29.110 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:28 smithi134 ceph-mon[35031]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:29.110 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:28 smithi134 ceph-mon[35031]: from='osd.1 [v2:172.21.15.123:6800/2152486571,v1:172.21.15.123:6801/2152486571]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:37:29.110 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:28 smithi134 ceph-mon[35031]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:37:29.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:28 smithi123 ceph-mon[34673]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:29.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:28 smithi123 ceph-mon[34673]: from='osd.1 [v2:172.21.15.123:6800/2152486571,v1:172.21.15.123:6801/2152486571]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:37:29.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:28 smithi123 ceph-mon[34673]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:37:29.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:28 smithi016 ceph-mon[30375]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:29.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:28 smithi016 ceph-mon[30375]: from='osd.1 [v2:172.21.15.123:6800/2152486571,v1:172.21.15.123:6801/2152486571]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:37:29.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:28 smithi016 ceph-mon[30375]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:37:29.337 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:37:29.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.833+0000 7f029967a640 1 -- 172.21.15.134:0/3333070507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0294102640 msgr2=0x7f0294102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:29.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.833+0000 7f029967a640 1 --2- 172.21.15.134:0/3333070507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0294102640 0x7f0294102ab0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f028c009900 tx=0x7f028c02a480 comp rx=0 tx=0).stop 2024-09-21T03:37:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.833+0000 7f029967a640 1 -- 172.21.15.134:0/3333070507 shutdown_connections 2024-09-21T03:37:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.833+0000 7f029967a640 1 --2- 172.21.15.134:0/3333070507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0294104070 0x7f029410aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.833+0000 7f029967a640 1 --2- 172.21.15.134:0/3333070507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02941036d0 0x7f0294103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.833+0000 7f029967a640 1 --2- 172.21.15.134:0/3333070507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0294102640 0x7f0294102ab0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.833+0000 7f029967a640 1 -- 172.21.15.134:0/3333070507 >> 172.21.15.134:0/3333070507 conn(0x7f02940fdfb0 msgr2=0x7f0294100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.833+0000 7f029967a640 1 -- 172.21.15.134:0/3333070507 shutdown_connections 2024-09-21T03:37:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.833+0000 7f029967a640 1 -- 172.21.15.134:0/3333070507 wait complete. 2024-09-21T03:37:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.834+0000 7f029967a640 1 Processor -- start 2024-09-21T03:37:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.834+0000 7f029967a640 1 -- start start 2024-09-21T03:37:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.834+0000 7f029967a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0294102640 0x7f0294106230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.834+0000 7f029967a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02941036d0 0x7f0294104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.834+0000 7f029967a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0294104070 0x7f0294105040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.834+0000 7f029967a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f029410e300 con 0x7f0294104070 2024-09-21T03:37:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.834+0000 7f029967a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f029410e180 con 0x7f0294102640 2024-09-21T03:37:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.834+0000 7f029967a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f029410e480 con 0x7f02941036d0 2024-09-21T03:37:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.834+0000 7f02927fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02941036d0 0x7f0294104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.834+0000 7f02927fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02941036d0 0x7f0294104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:33378/0 (socket says 172.21.15.134:33378) 2024-09-21T03:37:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.834+0000 7f02927fc640 1 -- 172.21.15.134:0/538615963 learned_addr learned my addr 172.21.15.134:0/538615963 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-21T03:37:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.834+0000 7f0292ffd640 1 --2- 172.21.15.134:0/538615963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0294102640 0x7f0294106230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:29.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.835+0000 7f02927fc640 1 -- 172.21.15.134:0/538615963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0294102640 msgr2=0x7f0294106230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:29.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.835+0000 7f02927fc640 1 --2- 172.21.15.134:0/538615963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0294102640 0x7f0294106230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:29.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.835+0000 7f02927fc640 1 -- 172.21.15.134:0/538615963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0294104070 msgr2=0x7f0294105040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:29.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.835+0000 7f02927fc640 1 --2- 172.21.15.134:0/538615963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0294104070 0x7f0294105040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:29.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.835+0000 7f02927fc640 1 -- 172.21.15.134:0/538615963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0294105740 con 0x7f02941036d0 2024-09-21T03:37:29.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.835+0000 7f02927fc640 1 --2- 172.21.15.134:0/538615963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02941036d0 0x7f0294104b00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f028800ba50 tx=0x7f028800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:29.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.835+0000 7f027bfff640 1 -- 172.21.15.134:0/538615963 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288019070 con 0x7f02941036d0 2024-09-21T03:37:29.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.835+0000 7f029967a640 1 -- 172.21.15.134:0/538615963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02941a40b0 con 0x7f02941036d0 2024-09-21T03:37:29.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.835+0000 7f027bfff640 1 -- 172.21.15.134:0/538615963 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f028800cbe0 con 0x7f02941036d0 2024-09-21T03:37:29.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.835+0000 7f027bfff640 1 -- 172.21.15.134:0/538615963 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02880075d0 con 0x7f02941036d0 2024-09-21T03:37:29.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.835+0000 7f029967a640 1 -- 172.21.15.134:0/538615963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02941a45e0 con 0x7f02941036d0 2024-09-21T03:37:29.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.836+0000 7f029967a640 1 -- 172.21.15.134:0/538615963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02941a48a0 con 0x7f02941036d0 2024-09-21T03:37:29.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.838+0000 7f027bfff640 1 -- 172.21.15.134:0/538615963 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f028801c070 con 0x7f02941036d0 2024-09-21T03:37:29.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.839+0000 7f027bfff640 1 --2- 172.21.15.134:0/538615963 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f027c0b6480 0x7f027c0b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:29.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.839+0000 7f027bfff640 1 -- 172.21.15.134:0/538615963 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1903+0+0 (secure 0 0 0) 0x7f02880e53b0 con 0x7f02941036d0 2024-09-21T03:37:29.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.839+0000 7f0292ffd640 1 --2- 172.21.15.134:0/538615963 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f027c0b6480 0x7f027c0b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:29.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.840+0000 7f0292ffd640 1 --2- 172.21.15.134:0/538615963 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f027c0b6480 0x7f027c0b8990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f028c031440 tx=0x7f028c0314a0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:29.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:29.842+0000 7f027bfff640 1 -- 172.21.15.134:0/538615963 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f02880ad590 con 0x7f02941036d0 2024-09-21T03:37:30.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:30.038+0000 7f029967a640 1 -- 172.21.15.134:0/538615963 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f02940fff50 con 0x7f027c0b6480 2024-09-21T03:37:30.040 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:29 smithi134 ceph-mon[35031]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:37:30.040 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:29 smithi134 ceph-mon[35031]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:37:30.040 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:29 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:30.040 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:29 smithi134 ceph-mon[35031]: from='osd.1 [v2:172.21.15.123:6800/2152486571,v1:172.21.15.123:6801/2152486571]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:37:30.040 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:29 smithi134 ceph-mon[35031]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:37:30.040 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:29 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:30.040 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:29 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:30.040 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:29 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:30.041 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:29 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:30.041 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:29 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:30.041 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:29 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:30.041 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:29 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:30.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:29 smithi123 ceph-mon[34673]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:37:30.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:29 smithi123 ceph-mon[34673]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:37:30.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:29 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:30.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:29 smithi123 ceph-mon[34673]: from='osd.1 [v2:172.21.15.123:6800/2152486571,v1:172.21.15.123:6801/2152486571]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:37:30.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:29 smithi123 ceph-mon[34673]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:37:30.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:29 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:30.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:29 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:30.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:29 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:30.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:29 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:30.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:29 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:30.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:29 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:30.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:29 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:30.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:29 smithi016 ceph-mon[30375]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:37:30.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:29 smithi016 ceph-mon[30375]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:37:30.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:29 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:30.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:29 smithi016 ceph-mon[30375]: from='osd.1 [v2:172.21.15.123:6800/2152486571,v1:172.21.15.123:6801/2152486571]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:37:30.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:29 smithi016 ceph-mon[30375]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:37:30.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:29 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:30.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:29 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:30.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:29 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:30.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:29 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:30.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:29 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:30.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:29 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:30.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:29 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:30.887 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:37:30 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-1[38603]: 2024-09-21T03:37:30.552+0000 7ff98c7c6640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T03:37:30.887 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:37:30 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-1[38603]: 2024-09-21T03:37:30.556+0000 7ff9845bc640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:37:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:30 smithi134 ceph-mon[35031]: purged_snaps scrub starts 2024-09-21T03:37:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:30 smithi134 ceph-mon[35031]: purged_snaps scrub ok 2024-09-21T03:37:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:30 smithi134 ceph-mon[35031]: Detected new or changed devices on smithi123 2024-09-21T03:37:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:30 smithi134 ceph-mon[35031]: Adjusting osd_memory_target on smithi123 to 17088M 2024-09-21T03:37:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:30 smithi134 ceph-mon[35031]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:31.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:30 smithi134 ceph-mon[35031]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T03:37:31.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:30 smithi134 ceph-mon[35031]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:37:31.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:30 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:31.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:30 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:31.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:30 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:37:31.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:30 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:37:31.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:30 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:31.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:30 smithi134 ceph-mon[35031]: from='osd.1 ' entity='osd.1' 2024-09-21T03:37:31.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:30 smithi123 ceph-mon[34673]: purged_snaps scrub starts 2024-09-21T03:37:31.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:30 smithi123 ceph-mon[34673]: purged_snaps scrub ok 2024-09-21T03:37:31.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:30 smithi123 ceph-mon[34673]: Detected new or changed devices on smithi123 2024-09-21T03:37:31.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:30 smithi123 ceph-mon[34673]: Adjusting osd_memory_target on smithi123 to 17088M 2024-09-21T03:37:31.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:30 smithi123 ceph-mon[34673]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:31.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:30 smithi123 ceph-mon[34673]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T03:37:31.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:30 smithi123 ceph-mon[34673]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:37:31.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:30 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:31.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:30 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:31.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:30 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:37:31.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:30 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:37:31.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:30 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:31.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:30 smithi123 ceph-mon[34673]: from='osd.1 ' entity='osd.1' 2024-09-21T03:37:31.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:30 smithi016 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T03:37:31.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:30 smithi016 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T03:37:31.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:30 smithi016 ceph-mon[30375]: Detected new or changed devices on smithi123 2024-09-21T03:37:31.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:30 smithi016 ceph-mon[30375]: Adjusting osd_memory_target on smithi123 to 17088M 2024-09-21T03:37:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:30 smithi016 ceph-mon[30375]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:30 smithi016 ceph-mon[30375]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T03:37:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:30 smithi016 ceph-mon[30375]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:37:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:30 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:30 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:30 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:37:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:30 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:37:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:30 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:30 smithi016 ceph-mon[30375]: from='osd.1 ' entity='osd.1' 2024-09-21T03:37:32.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:31 smithi123 ceph-mon[34673]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:32.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:31 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:32.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:31 smithi123 ceph-mon[34673]: osd.1 [v2:172.21.15.123:6800/2152486571,v1:172.21.15.123:6801/2152486571] boot 2024-09-21T03:37:32.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:31 smithi123 ceph-mon[34673]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T03:37:32.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:31 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:32.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:31 smithi016 ceph-mon[30375]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:32.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:31 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:32.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:31 smithi016 ceph-mon[30375]: osd.1 [v2:172.21.15.123:6800/2152486571,v1:172.21.15.123:6801/2152486571] boot 2024-09-21T03:37:32.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:31 smithi016 ceph-mon[30375]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T03:37:32.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:31 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:32.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:31 smithi134 ceph-mon[35031]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:32.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:31 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:32.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:31 smithi134 ceph-mon[35031]: osd.1 [v2:172.21.15.123:6800/2152486571,v1:172.21.15.123:6801/2152486571] boot 2024-09-21T03:37:32.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:31 smithi134 ceph-mon[35031]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T03:37:32.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:31 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:37:33.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:32 smithi123 ceph-mon[34673]: pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:33.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:32 smithi123 ceph-mon[34673]: from='client.? 172.21.15.134:0/306279088' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1c39518e-6cce-4a0f-9ee8-55c60ad325f9"} : dispatch 2024-09-21T03:37:33.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:32 smithi123 ceph-mon[34673]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1c39518e-6cce-4a0f-9ee8-55c60ad325f9"} : dispatch 2024-09-21T03:37:33.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:32 smithi123 ceph-mon[34673]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c39518e-6cce-4a0f-9ee8-55c60ad325f9"}]': finished 2024-09-21T03:37:33.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:32 smithi123 ceph-mon[34673]: osdmap e14: 3 total, 2 up, 3 in 2024-09-21T03:37:33.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:32 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:33.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:32 smithi123 ceph-mon[34673]: from='client.? 172.21.15.134:0/937145951' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:37:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:32 smithi016 ceph-mon[30375]: pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:32 smithi016 ceph-mon[30375]: from='client.? 172.21.15.134:0/306279088' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1c39518e-6cce-4a0f-9ee8-55c60ad325f9"} : dispatch 2024-09-21T03:37:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:32 smithi016 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1c39518e-6cce-4a0f-9ee8-55c60ad325f9"} : dispatch 2024-09-21T03:37:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:32 smithi016 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c39518e-6cce-4a0f-9ee8-55c60ad325f9"}]': finished 2024-09-21T03:37:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:32 smithi016 ceph-mon[30375]: osdmap e14: 3 total, 2 up, 3 in 2024-09-21T03:37:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:32 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:32 smithi016 ceph-mon[30375]: from='client.? 172.21.15.134:0/937145951' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:37:33.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:32 smithi134 ceph-mon[35031]: pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:37:33.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:32 smithi134 ceph-mon[35031]: from='client.? 172.21.15.134:0/306279088' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1c39518e-6cce-4a0f-9ee8-55c60ad325f9"} : dispatch 2024-09-21T03:37:33.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:32 smithi134 ceph-mon[35031]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1c39518e-6cce-4a0f-9ee8-55c60ad325f9"} : dispatch 2024-09-21T03:37:33.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:32 smithi134 ceph-mon[35031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c39518e-6cce-4a0f-9ee8-55c60ad325f9"}]': finished 2024-09-21T03:37:33.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:32 smithi134 ceph-mon[35031]: osdmap e14: 3 total, 2 up, 3 in 2024-09-21T03:37:33.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:32 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:33.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:32 smithi134 ceph-mon[35031]: from='client.? 172.21.15.134:0/937145951' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:37:35.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:34 smithi123 ceph-mon[34673]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:35.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:34 smithi016 ceph-mon[30375]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:35.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:34 smithi134 ceph-mon[35031]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:37.154 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:36 smithi134 ceph-mon[35031]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:37.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:36 smithi123 ceph-mon[34673]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:37.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:36 smithi016 ceph-mon[30375]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:39.057 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:38 smithi134 ceph-mon[35031]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:39.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:38 smithi123 ceph-mon[34673]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:39.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:38 smithi016 ceph-mon[30375]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:41.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:40 smithi016 ceph-mon[30375]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:41.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:37:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:41.208 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:40 smithi134 ceph-mon[35031]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:41.208 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:37:41.208 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:41.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:40 smithi123 ceph-mon[34673]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:41.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:37:41.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:42.205 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:41 smithi134 ceph-mon[35031]: Deploying daemon osd.2 on smithi134 2024-09-21T03:37:42.307 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:41 smithi123 ceph-mon[34673]: Deploying daemon osd.2 on smithi134 2024-09-21T03:37:42.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:41 smithi016 ceph-mon[30375]: Deploying daemon osd.2 on smithi134 2024-09-21T03:37:43.338 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:42 smithi134 ceph-mon[35031]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:43.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:42 smithi123 ceph-mon[34673]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:43.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:42 smithi016 ceph-mon[30375]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:45.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:44 smithi134 ceph-mon[35031]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:45.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:44 smithi123 ceph-mon[34673]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:45.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:44 smithi016 ceph-mon[30375]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:47.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:46 smithi134 ceph-mon[35031]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:47.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:46 smithi016 ceph-mon[30375]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:47.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:46 smithi123 ceph-mon[34673]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:48.293 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:47 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:48.293 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:47 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:48.293 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:47 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:48.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:47 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:48.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:47 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:48.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:47 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:48.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:47 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:48.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:47 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:48.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:47 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:49.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:48 smithi134 ceph-mon[35031]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:49.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:48 smithi123 ceph-mon[34673]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:49.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:48 smithi016 ceph-mon[30375]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:50.489 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 2 on host 'smithi134' 2024-09-21T03:37:50.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:50.487+0000 7f027bfff640 1 -- 172.21.15.134:0/538615963 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f02940fff50 con 0x7f027c0b6480 2024-09-21T03:37:50.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:50.489+0000 7f029967a640 1 -- 172.21.15.134:0/538615963 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f027c0b6480 msgr2=0x7f027c0b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:50.489+0000 7f029967a640 1 --2- 172.21.15.134:0/538615963 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f027c0b6480 0x7f027c0b8990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f028c031440 tx=0x7f028c0314a0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:50.490+0000 7f029967a640 1 -- 172.21.15.134:0/538615963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02941036d0 msgr2=0x7f0294104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:50.490+0000 7f029967a640 1 --2- 172.21.15.134:0/538615963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02941036d0 0x7f0294104b00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f028800ba50 tx=0x7f028800bf10 comp rx=0 tx=0).stop 2024-09-21T03:37:50.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:50.490+0000 7f029967a640 1 -- 172.21.15.134:0/538615963 shutdown_connections 2024-09-21T03:37:50.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:50.490+0000 7f029967a640 1 --2- 172.21.15.134:0/538615963 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f027c0b6480 0x7f027c0b8990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:50.490+0000 7f029967a640 1 --2- 172.21.15.134:0/538615963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0294104070 0x7f0294105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:50.490+0000 7f029967a640 1 --2- 172.21.15.134:0/538615963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02941036d0 0x7f0294104b00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:50.490+0000 7f029967a640 1 --2- 172.21.15.134:0/538615963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0294102640 0x7f0294106230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:50.490+0000 7f029967a640 1 -- 172.21.15.134:0/538615963 >> 172.21.15.134:0/538615963 conn(0x7f02940fdfb0 msgr2=0x7f02940ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:50.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:50.490+0000 7f029967a640 1 -- 172.21.15.134:0/538615963 shutdown_connections 2024-09-21T03:37:50.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:37:50.490+0000 7f029967a640 1 -- 172.21.15.134:0/538615963 wait complete. 2024-09-21T03:37:50.797 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:50 smithi134 ceph-mon[35031]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:50.797 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:50.797 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:50.797 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:50.797 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:50.797 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:50.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:50 smithi123 ceph-mon[34673]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:50.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:50 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:50.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:50 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:50.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:50 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:50.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:50 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:50.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:50 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:50.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:50 smithi016 ceph-mon[30375]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:50.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:50.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:50.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:50.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:50.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:51.065 DEBUG:teuthology.orchestra.run.smithi134:osd.2> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T03:37:51.107 INFO:tasks.cephadm:Deploying osd.3 on smithi016 with /dev/vg_nvme/lv_3... 2024-09-21T03:37:51.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:37:51.491 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:51.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:51 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:51.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:51 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:51.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:51 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:51.815 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:51 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:51.815 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:51 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:51.815 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:51 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:51.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:51 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:51.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:51 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:51.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:51 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:52.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:52 smithi134 ceph-mon[35031]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:52.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:52 smithi123 ceph-mon[34673]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:52.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:52 smithi016 ceph-mon[30375]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:53.486 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:37:53 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-2[39432]: 2024-09-21T03:37:53.289+0000 7f71749d8640 -1 osd.2 0 log_to_monitors true 2024-09-21T03:37:53.828 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:53 smithi134 ceph-mon[35031]: from='osd.2 [v2:172.21.15.134:6800/2812351279,v1:172.21.15.134:6801/2812351279]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:37:53.828 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:53 smithi134 ceph-mon[35031]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:37:53.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:53 smithi123 ceph-mon[34673]: from='osd.2 [v2:172.21.15.134:6800/2812351279,v1:172.21.15.134:6801/2812351279]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:37:53.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:53 smithi123 ceph-mon[34673]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:37:53.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:53 smithi016 ceph-mon[30375]: from='osd.2 [v2:172.21.15.134:6800/2812351279,v1:172.21.15.134:6801/2812351279]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:37:53.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:53 smithi016 ceph-mon[30375]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:37:54.852 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:37:54 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-2[39432]: 2024-09-21T03:37:54.505+0000 7f717095a640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T03:37:54.852 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:37:54 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-2[39432]: 2024-09-21T03:37:54.509+0000 7f716c758640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:37:54.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:54 smithi134 ceph-mon[35031]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:54.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:54 smithi134 ceph-mon[35031]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:37:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:54 smithi134 ceph-mon[35031]: from='osd.2 [v2:172.21.15.134:6800/2812351279,v1:172.21.15.134:6801/2812351279]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:37:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:54 smithi134 ceph-mon[35031]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T03:37:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:54 smithi134 ceph-mon[35031]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:37:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:54 smithi134 ceph-mon[35031]: Detected new or changed devices on smithi134 2024-09-21T03:37:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:54 smithi134 ceph-mon[35031]: Adjusting osd_memory_target on smithi134 to 17088M 2024-09-21T03:37:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:54.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:54 smithi016 ceph-mon[30375]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:54.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:54 smithi016 ceph-mon[30375]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:37:54.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:54 smithi016 ceph-mon[30375]: from='osd.2 [v2:172.21.15.134:6800/2812351279,v1:172.21.15.134:6801/2812351279]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:37:54.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:54 smithi016 ceph-mon[30375]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T03:37:54.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:54.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:54 smithi016 ceph-mon[30375]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:37:54.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:54 smithi016 ceph-mon[30375]: Detected new or changed devices on smithi134 2024-09-21T03:37:54.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:54.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:54.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:54.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:54 smithi016 ceph-mon[30375]: Adjusting osd_memory_target on smithi134 to 17088M 2024-09-21T03:37:54.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:54.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:54.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:54.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:54 smithi123 ceph-mon[34673]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:54 smithi123 ceph-mon[34673]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:37:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:54 smithi123 ceph-mon[34673]: from='osd.2 [v2:172.21.15.134:6800/2812351279,v1:172.21.15.134:6801/2812351279]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:37:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:54 smithi123 ceph-mon[34673]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T03:37:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:54 smithi123 ceph-mon[34673]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:37:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:54 smithi123 ceph-mon[34673]: Detected new or changed devices on smithi134 2024-09-21T03:37:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:37:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:54 smithi123 ceph-mon[34673]: Adjusting osd_memory_target on smithi134 to 17088M 2024-09-21T03:37:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:54.908 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:54.908 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:54.908 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:54.988 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:37:55.019 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi016:vg_nvme/lv_3 2024-09-21T03:37:55.392 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:55.816 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:55 smithi016 ceph-mon[30375]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-21T03:37:55.816 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:55 smithi016 ceph-mon[30375]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:37:55.816 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:55 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:55.816 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:55 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:55.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:55 smithi134 ceph-mon[35031]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-21T03:37:55.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:55 smithi134 ceph-mon[35031]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:37:55.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:55 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:55.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:55 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:55.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:55 smithi123 ceph-mon[34673]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-21T03:37:55.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:55 smithi123 ceph-mon[34673]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:37:55.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:55 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:55.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:55 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.275+0000 7fba21a00640 1 -- 172.21.15.16:0/1473315224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba1c10fd00 msgr2=0x7fba1c112120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.275+0000 7fba21a00640 1 --2- 172.21.15.16:0/1473315224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba1c10fd00 0x7fba1c112120 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fba14009c10 tx=0x7fba1402a6f0 comp rx=0 tx=0).stop 2024-09-21T03:37:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.276+0000 7fba21a00640 1 -- 172.21.15.16:0/1473315224 shutdown_connections 2024-09-21T03:37:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.276+0000 7fba21a00640 1 --2- 172.21.15.16:0/1473315224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba1c10fd00 0x7fba1c112120 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.276+0000 7fba21a00640 1 --2- 172.21.15.16:0/1473315224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba1c075cf0 0x7fba1c076100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.276+0000 7fba21a00640 1 --2- 172.21.15.16:0/1473315224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba1c0770e0 0x7fba1c0757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.276+0000 7fba21a00640 1 -- 172.21.15.16:0/1473315224 >> 172.21.15.16:0/1473315224 conn(0x7fba1c0feec0 msgr2=0x7fba1c101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.276+0000 7fba21a00640 1 -- 172.21.15.16:0/1473315224 shutdown_connections 2024-09-21T03:37:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.276+0000 7fba21a00640 1 -- 172.21.15.16:0/1473315224 wait complete. 2024-09-21T03:37:56.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.277+0000 7fba21a00640 1 Processor -- start 2024-09-21T03:37:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.277+0000 7fba21a00640 1 -- start start 2024-09-21T03:37:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.277+0000 7fba21a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba1c075cf0 0x7fba1c1a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.277+0000 7fba21a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba1c0770e0 0x7fba1c1a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.277+0000 7fba21a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba1c10fd00 0x7fba1c1a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.277+0000 7fba21a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1157a0 con 0x7fba1c10fd00 2024-09-21T03:37:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.277+0000 7fba21a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c115620 con 0x7fba1c0770e0 2024-09-21T03:37:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.277+0000 7fba21a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c115920 con 0x7fba1c075cf0 2024-09-21T03:37:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.278+0000 7fba1be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba1c075cf0 0x7fba1c1a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.278+0000 7fba1be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba1c075cf0 0x7fba1c1a03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:58968/0 (socket says 172.21.15.16:58968) 2024-09-21T03:37:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.278+0000 7fba1be00640 1 -- 172.21.15.16:0/3288157931 learned_addr learned my addr 172.21.15.16:0/3288157931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:37:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.278+0000 7fba13400640 1 --2- 172.21.15.16:0/3288157931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba1c0770e0 0x7fba1c1a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.278+0000 7fba21000640 1 --2- 172.21.15.16:0/3288157931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba1c10fd00 0x7fba1c1a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.278+0000 7fba13400640 1 -- 172.21.15.16:0/3288157931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba1c075cf0 msgr2=0x7fba1c1a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.278+0000 7fba13400640 1 --2- 172.21.15.16:0/3288157931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba1c075cf0 0x7fba1c1a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.278+0000 7fba13400640 1 -- 172.21.15.16:0/3288157931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba1c10fd00 msgr2=0x7fba1c1a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.278+0000 7fba13400640 1 --2- 172.21.15.16:0/3288157931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba1c10fd00 0x7fba1c1a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.278+0000 7fba13400640 1 -- 172.21.15.16:0/3288157931 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1c1a1df0 con 0x7fba1c0770e0 2024-09-21T03:37:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.278+0000 7fba13400640 1 --2- 172.21.15.16:0/3288157931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba1c0770e0 0x7fba1c1a0a30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fba0400e9f0 tx=0x7fba0400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:56.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.279+0000 7fba19600640 1 -- 172.21.15.16:0/3288157931 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0400ccc0 con 0x7fba1c0770e0 2024-09-21T03:37:56.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.279+0000 7fba19600640 1 -- 172.21.15.16:0/3288157931 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba040108e0 con 0x7fba1c0770e0 2024-09-21T03:37:56.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.279+0000 7fba21a00640 1 -- 172.21.15.16:0/3288157931 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1c1a20e0 con 0x7fba1c0770e0 2024-09-21T03:37:56.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.279+0000 7fba19600640 1 -- 172.21.15.16:0/3288157931 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04018880 con 0x7fba1c0770e0 2024-09-21T03:37:56.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.279+0000 7fba21a00640 1 -- 172.21.15.16:0/3288157931 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1c1a87e0 con 0x7fba1c0770e0 2024-09-21T03:37:56.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.280+0000 7fba21a00640 1 -- 172.21.15.16:0/3288157931 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba1c1a89b0 con 0x7fba1c0770e0 2024-09-21T03:37:56.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.282+0000 7fba19600640 1 -- 172.21.15.16:0/3288157931 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fba04010430 con 0x7fba1c0770e0 2024-09-21T03:37:56.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.284+0000 7fba19600640 1 --2- 172.21.15.16:0/3288157931 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fba000b6480 0x7fba000b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:56.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.284+0000 7fba19600640 1 -- 172.21.15.16:0/3288157931 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2851+0+0 (secure 0 0 0) 0x7fba040e64d0 con 0x7fba1c0770e0 2024-09-21T03:37:56.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.284+0000 7fba1be00640 1 --2- 172.21.15.16:0/3288157931 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fba000b6480 0x7fba000b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:56.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.285+0000 7fba1be00640 1 --2- 172.21.15.16:0/3288157931 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fba000b6480 0x7fba000b8990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0058c0 tx=0x7fba0c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.286+0000 7fba19600640 1 -- 172.21.15.16:0/3288157931 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fba040b26d0 con 0x7fba1c0770e0 2024-09-21T03:37:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:56.497+0000 7fba21a00640 1 -- 172.21.15.16:0/3288157931 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fba1c0ffc90 con 0x7fba000b6480 2024-09-21T03:37:56.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:56 smithi134 ceph-mon[35031]: purged_snaps scrub starts 2024-09-21T03:37:56.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:56 smithi134 ceph-mon[35031]: purged_snaps scrub ok 2024-09-21T03:37:56.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:56 smithi134 ceph-mon[35031]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:56.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:56 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:56.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:56 smithi134 ceph-mon[35031]: osd.2 [v2:172.21.15.134:6800/2812351279,v1:172.21.15.134:6801/2812351279] boot 2024-09-21T03:37:56.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:56 smithi134 ceph-mon[35031]: osdmap e17: 3 total, 3 up, 3 in 2024-09-21T03:37:56.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:56 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:56.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:56 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:37:56.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:56 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:37:56.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:56 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:37:56.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:56 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:56.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:56 smithi123 ceph-mon[34673]: purged_snaps scrub starts 2024-09-21T03:37:56.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:56 smithi123 ceph-mon[34673]: purged_snaps scrub ok 2024-09-21T03:37:56.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:56 smithi123 ceph-mon[34673]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:56.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:56 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:56.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:56 smithi123 ceph-mon[34673]: osd.2 [v2:172.21.15.134:6800/2812351279,v1:172.21.15.134:6801/2812351279] boot 2024-09-21T03:37:56.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:56 smithi123 ceph-mon[34673]: osdmap e17: 3 total, 3 up, 3 in 2024-09-21T03:37:56.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:56 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:56.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:56 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:37:56.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:56 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:37:56.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:56 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:37:56.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:56 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:56.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:56 smithi016 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T03:37:56.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:56 smithi016 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T03:37:56.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:56 smithi016 ceph-mon[30375]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:37:56.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:56 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:56.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:56 smithi016 ceph-mon[30375]: osd.2 [v2:172.21.15.134:6800/2812351279,v1:172.21.15.134:6801/2812351279] boot 2024-09-21T03:37:56.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:56 smithi016 ceph-mon[30375]: osdmap e17: 3 total, 3 up, 3 in 2024-09-21T03:37:56.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:56 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:37:56.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:56 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:37:56.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:56 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:37:56.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:56 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:37:56.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:56 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:57.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:57 smithi016 ceph-mon[30375]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:57.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:37:57.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:57 smithi016 ceph-mon[30375]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T03:37:57.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:37:57.795 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:57 smithi134 ceph-mon[35031]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:57.795 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:37:57.795 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:57 smithi134 ceph-mon[35031]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T03:37:57.795 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:37:57.876 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:37:57 smithi016 sudo[49900]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:37:57.876 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:37:57 smithi016 sudo[49900]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:37:57.877 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:37:57 smithi016 sudo[49900]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:37:57.877 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:37:57 smithi016 sudo[49900]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:37:57.877 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:37:57 smithi016 sudo[49903]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:37:57.877 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:37:57 smithi016 sudo[49903]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:37:57.877 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:37:57 smithi016 sudo[49903]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:37:57.877 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:37:57 smithi016 sudo[49903]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:37:57.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:57 smithi123 ceph-mon[34673]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:57.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:37:57.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:57 smithi123 ceph-mon[34673]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T03:37:57.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:37:57.906 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:37:57 smithi123 sudo[42137]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:37:57.906 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:37:57 smithi123 sudo[42137]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:37:57.907 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:37:57 smithi123 sudo[42137]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:37:57.907 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:37:57 smithi123 sudo[42137]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:37:57.907 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:37:57 smithi123 sudo[42140]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:37:57.907 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:37:57 smithi123 sudo[42140]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:37:57.907 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:37:57 smithi123 sudo[42140]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:37:57.907 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:37:57 smithi123 sudo[42140]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:37:58.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:57 smithi134 sudo[43194]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:37:58.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:57 smithi134 sudo[43194]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:37:58.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:57 smithi134 sudo[43194]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:37:58.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:57 smithi134 sudo[43194]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:37:58.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:57 smithi134 sudo[43197]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:37:58.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:57 smithi134 sudo[43197]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:37:58.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:57 smithi134 sudo[43197]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:37:58.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:57 smithi134 sudo[43197]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:37:58.103 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:37:57 smithi134 sudo[43188]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:37:58.103 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:37:57 smithi134 sudo[43188]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:37:58.103 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:37:57 smithi134 sudo[43188]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:37:58.103 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:37:57 smithi134 sudo[43188]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:37:58.103 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:37:57 smithi134 sudo[43191]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:37:58.103 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:37:57 smithi134 sudo[43191]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:37:58.103 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:37:57 smithi134 sudo[43191]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:37:58.103 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:37:57 smithi134 sudo[43191]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:37:58.130 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:57 smithi016 sudo[49907]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:37:58.130 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:57 smithi016 sudo[49907]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:37:58.130 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:57 smithi016 sudo[49907]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:37:58.130 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:57 smithi016 sudo[49907]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:37:58.130 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:57 smithi016 sudo[49910]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:37:58.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:57 smithi016 sudo[49910]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:37:58.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:57 smithi016 sudo[49910]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:37:58.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:57 smithi016 sudo[49910]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:37:58.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:57 smithi123 sudo[42143]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:37:58.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:57 smithi123 sudo[42143]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:37:58.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:57 smithi123 sudo[42143]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:37:58.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:57 smithi123 sudo[42143]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:37:58.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 sudo[42146]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:37:58.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 sudo[42146]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:37:58.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 sudo[42146]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:37:58.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 sudo[42146]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:37:58.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: pgmap v100: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:37:58.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:37:58.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:37:58.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:58.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:58.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:58.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:37:58.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:37:58.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:58.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:58.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:58.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:37:58.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:58.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:58.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:58.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:37:58.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:37:58.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:58.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:58.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:58.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:58 smithi016 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:37:58.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: pgmap v100: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:37:58.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:37:58.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:37:58.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:58.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:58.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:58.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:37:58.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:37:58.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:58.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:58.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:58.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:37:58.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:58.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:58.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:58.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:37:58.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:37:58.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:58.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:58.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:58.854 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:58 smithi134 ceph-mon[35031]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:37:58.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: pgmap v100: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:37:58.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:37:58.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:37:58.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:58.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:58.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:37:58.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:37:58.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:37:58.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:58.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:58.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:58.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:37:58.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:58.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:58.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:58.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:37:58.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:37:58.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:37:58.908 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:37:58.908 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:37:58.908 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:58 smithi123 ceph-mon[34673]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:37:59.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:37:59.547+0000 7fba19600640 1 -- 172.21.15.16:0/3288157931 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fba040a9f10 con 0x7fba1c0770e0 2024-09-21T03:37:59.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:37:59 smithi134 ceph-mon[35031]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T03:37:59.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:37:59 smithi123 ceph-mon[34673]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T03:37:59.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:37:59 smithi016 ceph-mon[30375]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T03:38:00.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:00 smithi134 ceph-mon[35031]: pgmap v103: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:00.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:00 smithi134 ceph-mon[35031]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-09-21T03:38:00.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:00 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2294003042' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf"} : dispatch 2024-09-21T03:38:00.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:00 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2294003042' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf"}]': finished 2024-09-21T03:38:00.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:00 smithi134 ceph-mon[35031]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T03:38:00.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:00 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:00.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:00 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1677790239' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:38:00.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:00 smithi123 ceph-mon[34673]: pgmap v103: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:00.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:00 smithi123 ceph-mon[34673]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-09-21T03:38:00.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:00 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2294003042' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf"} : dispatch 2024-09-21T03:38:00.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:00 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2294003042' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf"}]': finished 2024-09-21T03:38:00.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:00 smithi123 ceph-mon[34673]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T03:38:00.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:00 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:00.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:00 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1677790239' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:38:00.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:00 smithi016 ceph-mon[30375]: pgmap v103: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:00.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:00 smithi016 ceph-mon[30375]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-09-21T03:38:00.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:00 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2294003042' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf"} : dispatch 2024-09-21T03:38:00.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:00 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2294003042' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf"}]': finished 2024-09-21T03:38:00.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:00 smithi016 ceph-mon[30375]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T03:38:00.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:00 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:00.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:00 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1677790239' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:38:02.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:02 smithi134 ceph-mon[35031]: pgmap v105: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:02.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:02 smithi123 ceph-mon[34673]: pgmap v105: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:02.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:02 smithi016 ceph-mon[30375]: pgmap v105: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:04.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:04 smithi134 ceph-mon[35031]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:04.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:04 smithi016 ceph-mon[30375]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:04.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:04 smithi123 ceph-mon[34673]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:06.816 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:06 smithi016 ceph-mon[30375]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:06.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:06 smithi134 ceph-mon[35031]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:06.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:06 smithi123 ceph-mon[34673]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:08.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:08 smithi134 ceph-mon[35031]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:08.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:08 smithi123 ceph-mon[34673]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:08.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:08 smithi016 ceph-mon[30375]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:09.845 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:09 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:38:09.845 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:09 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:09.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:09 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:38:09.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:09 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:09.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:09 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:38:09.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:09 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:10.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:10 smithi134 ceph-mon[35031]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:10.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:10 smithi134 ceph-mon[35031]: Deploying daemon osd.3 on smithi016 2024-09-21T03:38:10.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:10 smithi016 ceph-mon[30375]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:10.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:10 smithi016 ceph-mon[30375]: Deploying daemon osd.3 on smithi016 2024-09-21T03:38:10.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:10 smithi123 ceph-mon[34673]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:10.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:10 smithi123 ceph-mon[34673]: Deploying daemon osd.3 on smithi016 2024-09-21T03:38:12.723 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:12 smithi016 ceph-mon[30375]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:12.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:12 smithi134 ceph-mon[35031]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:12.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:12 smithi123 ceph-mon[34673]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:14.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:14 smithi134 ceph-mon[35031]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:14.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:14 smithi123 ceph-mon[34673]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:14.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:14 smithi016 ceph-mon[30375]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:16.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:16 smithi134 ceph-mon[35031]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:16.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:16 smithi123 ceph-mon[34673]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:16.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:16 smithi016 ceph-mon[30375]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:18.676 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:18 smithi016 ceph-mon[30375]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:18.676 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:18 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:18.676 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:18 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:18.676 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:18 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:18.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:18 smithi134 ceph-mon[35031]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:18.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:18 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:18.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:18 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:18.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:18 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:18.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:18 smithi123 ceph-mon[34673]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:18.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:18 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:18.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:18 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:18.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:18 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:20.778 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:20 smithi016 ceph-mon[30375]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:20.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:20 smithi134 ceph-mon[35031]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:20.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:20 smithi123 ceph-mon[34673]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:22.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:22.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:22.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:22.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:22.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:22.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:22 smithi134 ceph-mon[35031]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:22.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:22.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:22.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:22.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:22.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:22.403 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:22 smithi016 ceph-mon[30375]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:22.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:22.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:22.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:22.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:22.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:22.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:22 smithi123 ceph-mon[34673]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:23.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:38:23.253+0000 7fba19600640 1 -- 172.21.15.16:0/3288157931 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fba1c0ffc90 con 0x7fba000b6480 2024-09-21T03:38:23.255 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 3 on host 'smithi016' 2024-09-21T03:38:23.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:38:23.255+0000 7fba21a00640 1 -- 172.21.15.16:0/3288157931 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fba000b6480 msgr2=0x7fba000b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:23.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:38:23.255+0000 7fba21a00640 1 --2- 172.21.15.16:0/3288157931 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fba000b6480 0x7fba000b8990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0058c0 tx=0x7fba0c009260 comp rx=0 tx=0).stop 2024-09-21T03:38:23.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:38:23.255+0000 7fba21a00640 1 -- 172.21.15.16:0/3288157931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba1c0770e0 msgr2=0x7fba1c1a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:23.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:38:23.255+0000 7fba21a00640 1 --2- 172.21.15.16:0/3288157931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba1c0770e0 0x7fba1c1a0a30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fba0400e9f0 tx=0x7fba0400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:38:23.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:38:23.255+0000 7fba21a00640 1 -- 172.21.15.16:0/3288157931 shutdown_connections 2024-09-21T03:38:23.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:38:23.255+0000 7fba21a00640 1 --2- 172.21.15.16:0/3288157931 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fba000b6480 0x7fba000b8990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:23.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:38:23.255+0000 7fba21a00640 1 --2- 172.21.15.16:0/3288157931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba1c10fd00 0x7fba1c1a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:23.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:38:23.255+0000 7fba21a00640 1 --2- 172.21.15.16:0/3288157931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba1c0770e0 0x7fba1c1a0a30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:23.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:38:23.255+0000 7fba21a00640 1 --2- 172.21.15.16:0/3288157931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba1c075cf0 0x7fba1c1a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:23.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:38:23.255+0000 7fba21a00640 1 -- 172.21.15.16:0/3288157931 >> 172.21.15.16:0/3288157931 conn(0x7fba1c0feec0 msgr2=0x7fba1c0ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:23.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:38:23.256+0000 7fba21a00640 1 -- 172.21.15.16:0/3288157931 shutdown_connections 2024-09-21T03:38:23.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:38:23.256+0000 7fba21a00640 1 -- 172.21.15.16:0/3288157931 wait complete. 2024-09-21T03:38:23.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:23 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:23.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:23 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:23.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:23 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:23.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:23 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:23.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:23 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:23.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:23 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:23.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:23 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:23.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:23 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:23.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:23 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:24.101 DEBUG:teuthology.orchestra.run.smithi016:osd.3> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T03:38:24.106 INFO:tasks.cephadm:Deploying osd.4 on smithi123 with /dev/vg_nvme/lv_3... 2024-09-21T03:38:24.106 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:38:24.374 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:38:24.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:24 smithi123 ceph-mon[34673]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:24.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:24 smithi123 ceph-mon[34673]: from='osd.3 [v2:172.21.15.16:6810/2366216026,v1:172.21.15.16:6811/2366216026]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:38:24.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:24 smithi123 ceph-mon[34673]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:38:24.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:24 smithi016 ceph-mon[30375]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:24.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:24 smithi016 ceph-mon[30375]: from='osd.3 [v2:172.21.15.16:6810/2366216026,v1:172.21.15.16:6811/2366216026]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:38:24.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:24 smithi016 ceph-mon[30375]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:38:24.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:24 smithi134 ceph-mon[35031]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:24.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:24 smithi134 ceph-mon[35031]: from='osd.3 [v2:172.21.15.16:6810/2366216026,v1:172.21.15.16:6811/2366216026]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:38:24.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:24 smithi134 ceph-mon[35031]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:38:25.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:25 smithi134 ceph-mon[35031]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:38:25.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:25 smithi134 ceph-mon[35031]: from='osd.3 [v2:172.21.15.16:6810/2366216026,v1:172.21.15.16:6811/2366216026]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:38:25.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:25 smithi134 ceph-mon[35031]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:38:25.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:25 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:25.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:25 smithi134 ceph-mon[35031]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:38:25.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:25 smithi123 ceph-mon[34673]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:38:25.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:25 smithi123 ceph-mon[34673]: from='osd.3 [v2:172.21.15.16:6810/2366216026,v1:172.21.15.16:6811/2366216026]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:38:25.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:25 smithi123 ceph-mon[34673]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:38:25.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:25 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:25.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:25 smithi123 ceph-mon[34673]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:38:25.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:25 smithi016 ceph-mon[30375]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:38:25.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:25 smithi016 ceph-mon[30375]: from='osd.3 [v2:172.21.15.16:6810/2366216026,v1:172.21.15.16:6811/2366216026]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:38:25.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:25 smithi016 ceph-mon[30375]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:38:25.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:25 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:25.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:25 smithi016 ceph-mon[30375]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:38:25.663 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:38:25 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-3[51972]: 2024-09-21T03:38:25.334+0000 7f489ec00640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T03:38:25.663 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:38:25 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-3[51972]: 2024-09-21T03:38:25.341+0000 7f4899c00640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:38:26.581 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:26 smithi123 ceph-mon[34673]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:26.582 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:26 smithi123 ceph-mon[34673]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T03:38:26.582 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:26 smithi123 ceph-mon[34673]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:38:26.582 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:26 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:26.582 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:26 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:26.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:26 smithi134 ceph-mon[35031]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:26.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:26 smithi134 ceph-mon[35031]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T03:38:26.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:26 smithi134 ceph-mon[35031]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:38:26.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:26 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:26.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:26 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:26.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:26 smithi016 ceph-mon[30375]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:26.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:26 smithi016 ceph-mon[30375]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T03:38:26.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:26 smithi016 ceph-mon[30375]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:38:26.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:26 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:26.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:26 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:26.914 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:38:26.939 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi123:vg_nvme/lv_3 2024-09-21T03:38:27.187 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:38:27.560 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:27 smithi123 ceph-mon[34673]: purged_snaps scrub starts 2024-09-21T03:38:27.561 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:27 smithi123 ceph-mon[34673]: purged_snaps scrub ok 2024-09-21T03:38:27.561 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:27 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:27.561 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:27 smithi123 ceph-mon[34673]: osd.3 [v2:172.21.15.16:6810/2366216026,v1:172.21.15.16:6811/2366216026] boot 2024-09-21T03:38:27.561 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:27 smithi123 ceph-mon[34673]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T03:38:27.561 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:27 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:27.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:27 smithi134 ceph-mon[35031]: purged_snaps scrub starts 2024-09-21T03:38:27.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:27 smithi134 ceph-mon[35031]: purged_snaps scrub ok 2024-09-21T03:38:27.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:27 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:27.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:27 smithi134 ceph-mon[35031]: osd.3 [v2:172.21.15.16:6810/2366216026,v1:172.21.15.16:6811/2366216026] boot 2024-09-21T03:38:27.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:27 smithi134 ceph-mon[35031]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T03:38:27.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:27 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:27.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:27 smithi016 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T03:38:27.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:27 smithi016 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T03:38:27.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:27 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:27.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:27 smithi016 ceph-mon[30375]: osd.3 [v2:172.21.15.16:6810/2366216026,v1:172.21.15.16:6811/2366216026] boot 2024-09-21T03:38:27.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:27 smithi016 ceph-mon[30375]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T03:38:27.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:27 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:38:27.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.647+0000 7f54ddbd8640 1 -- 172.21.15.123:0/1478499731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54d81068d0 msgr2=0x7f54d8108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:27.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.647+0000 7f54ddbd8640 1 --2- 172.21.15.123:0/1478499731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54d81068d0 0x7f54d8108d30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009900 tx=0x7f54c002a410 comp rx=0 tx=0).stop 2024-09-21T03:38:27.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.647+0000 7f54ddbd8640 1 -- 172.21.15.123:0/1478499731 shutdown_connections 2024-09-21T03:38:27.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.647+0000 7f54ddbd8640 1 --2- 172.21.15.123:0/1478499731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54d8109470 0x7f54d810b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:27.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.647+0000 7f54ddbd8640 1 --2- 172.21.15.123:0/1478499731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54d81068d0 0x7f54d8108d30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:27.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.647+0000 7f54ddbd8640 1 --2- 172.21.15.123:0/1478499731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d806b980 0x7f54d8106390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:27.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.647+0000 7f54ddbd8640 1 -- 172.21.15.123:0/1478499731 >> 172.21.15.123:0/1478499731 conn(0x7f54d80fdfb0 msgr2=0x7f54d8100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:27.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.647+0000 7f54ddbd8640 1 -- 172.21.15.123:0/1478499731 shutdown_connections 2024-09-21T03:38:27.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.647+0000 7f54ddbd8640 1 -- 172.21.15.123:0/1478499731 wait complete. 2024-09-21T03:38:27.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.648+0000 7f54ddbd8640 1 Processor -- start 2024-09-21T03:38:27.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.648+0000 7f54ddbd8640 1 -- start start 2024-09-21T03:38:27.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.648+0000 7f54ddbd8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54d806b980 0x7f54d8199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:27.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.648+0000 7f54ddbd8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54d81068d0 0x7f54d819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:27.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.648+0000 7f54ddbd8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d8109470 0x7f54d819b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:27.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.648+0000 7f54ddbd8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54d810f380 con 0x7f54d8109470 2024-09-21T03:38:27.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.648+0000 7f54ddbd8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54d810f200 con 0x7f54d81068d0 2024-09-21T03:38:27.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.648+0000 7f54ddbd8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54d810f500 con 0x7f54d806b980 2024-09-21T03:38:27.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.648+0000 7f54ceffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54d81068d0 0x7f54d819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:27.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.648+0000 7f54ceffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54d81068d0 0x7f54d819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44768/0 (socket says 172.21.15.123:44768) 2024-09-21T03:38:27.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.648+0000 7f54ceffd640 1 -- 172.21.15.123:0/731666989 learned_addr learned my addr 172.21.15.123:0/731666989 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-21T03:38:27.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.648+0000 7f54d77fe640 1 --2- 172.21.15.123:0/731666989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54d806b980 0x7f54d8199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:27.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.649+0000 7f54d7fff640 1 --2- 172.21.15.123:0/731666989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d8109470 0x7f54d819b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:27.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.649+0000 7f54ceffd640 1 -- 172.21.15.123:0/731666989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54d806b980 msgr2=0x7f54d8199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:27.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.649+0000 7f54ceffd640 1 --2- 172.21.15.123:0/731666989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54d806b980 0x7f54d8199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:27.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.649+0000 7f54ceffd640 1 -- 172.21.15.123:0/731666989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d8109470 msgr2=0x7f54d819b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:27.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.649+0000 7f54ceffd640 1 --2- 172.21.15.123:0/731666989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d8109470 0x7f54d819b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:27.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.649+0000 7f54ceffd640 1 -- 172.21.15.123:0/731666989 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d819b9b0 con 0x7f54d81068d0 2024-09-21T03:38:27.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.649+0000 7f54ceffd640 1 --2- 172.21.15.123:0/731666989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54d81068d0 0x7f54d819a5f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f54c0006010 tx=0x7f54c0004340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:27.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.649+0000 7f54d57fa640 1 -- 172.21.15.123:0/731666989 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c003f030 con 0x7f54d81068d0 2024-09-21T03:38:27.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.649+0000 7f54ddbd8640 1 -- 172.21.15.123:0/731666989 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d819bc40 con 0x7f54d81068d0 2024-09-21T03:38:27.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.649+0000 7f54d57fa640 1 -- 172.21.15.123:0/731666989 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54c0003cb0 con 0x7f54d81068d0 2024-09-21T03:38:27.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.649+0000 7f54d57fa640 1 -- 172.21.15.123:0/731666989 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c0035070 con 0x7f54d81068d0 2024-09-21T03:38:27.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.649+0000 7f54ddbd8640 1 -- 172.21.15.123:0/731666989 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d81a23a0 con 0x7f54d81068d0 2024-09-21T03:38:27.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.650+0000 7f54ddbd8640 1 -- 172.21.15.123:0/731666989 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d8106d20 con 0x7f54d81068d0 2024-09-21T03:38:27.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.652+0000 7f54d57fa640 1 -- 172.21.15.123:0/731666989 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f54c0042030 con 0x7f54d81068d0 2024-09-21T03:38:27.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.653+0000 7f54d57fa640 1 --2- 172.21.15.123:0/731666989 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f54c40b6510 0x7f54c40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:27.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.653+0000 7f54d57fa640 1 -- 172.21.15.123:0/731666989 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3669+0+0 (secure 0 0 0) 0x7f54c0106e00 con 0x7f54d81068d0 2024-09-21T03:38:27.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.653+0000 7f54d77fe640 1 --2- 172.21.15.123:0/731666989 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f54c40b6510 0x7f54c40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:27.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.654+0000 7f54d77fe640 1 --2- 172.21.15.123:0/731666989 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f54c40b6510 0x7f54c40b8a20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f54c80058c0 tx=0x7f54c800a2d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:27.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.657+0000 7f54d57fa640 1 -- 172.21.15.123:0/731666989 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f54c00ce8b0 con 0x7f54d81068d0 2024-09-21T03:38:27.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:27.843+0000 7f54ddbd8640 1 -- 172.21.15.123:0/731666989 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f54d8063800 con 0x7f54c40b6510 2024-09-21T03:38:28.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:28 smithi123 ceph-mon[34673]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:28.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:28 smithi123 ceph-mon[34673]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T03:38:28.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:28 smithi123 ceph-mon[34673]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:38:28.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:28 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:38:28.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:28 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:38:28.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:28 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:28.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:28 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:28.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:28 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:28.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:28 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:28.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:28 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:28.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:28 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:28.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:28 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:28.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:28 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:28.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:28 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:28.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:28 smithi134 ceph-mon[35031]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:28.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:28 smithi134 ceph-mon[35031]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T03:38:28.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:28 smithi134 ceph-mon[35031]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:38:28.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:28 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:38:28.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:28 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:38:28.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:28 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:28.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:28 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:28.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:28 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:28.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:28 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:28.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:28 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:28.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:28 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:28.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:28 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:28.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:28 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:28.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:28 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:28.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:28 smithi016 ceph-mon[30375]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:38:28.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:28 smithi016 ceph-mon[30375]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T03:38:28.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:28 smithi016 ceph-mon[30375]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:38:28.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:28 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:38:28.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:28 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:38:28.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:28 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:28.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:28 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:28.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:28 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:28.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:28 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:28.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:28 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:28.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:28 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:28.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:28 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:28.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:28 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:28.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:28 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:29.498 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:29 smithi123 ceph-mon[34673]: Detected new or changed devices on smithi016 2024-09-21T03:38:29.498 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:29 smithi123 ceph-mon[34673]: Adjusting osd_memory_target on smithi016 to 8547M 2024-09-21T03:38:29.498 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:29 smithi123 ceph-mon[34673]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:38:29.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:29 smithi134 ceph-mon[35031]: Detected new or changed devices on smithi016 2024-09-21T03:38:29.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:29 smithi134 ceph-mon[35031]: Adjusting osd_memory_target on smithi016 to 8547M 2024-09-21T03:38:29.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:29 smithi134 ceph-mon[35031]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:38:29.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:29 smithi016 ceph-mon[30375]: Detected new or changed devices on smithi016 2024-09-21T03:38:29.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:29 smithi016 ceph-mon[30375]: Adjusting osd_memory_target on smithi016 to 8547M 2024-09-21T03:38:29.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:29 smithi016 ceph-mon[30375]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:38:30.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:30 smithi134 ceph-mon[35031]: pgmap v124: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:30.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:30 smithi134 ceph-mon[35031]: from='client.? 172.21.15.123:0/4283325337' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ad5e6d0e-31e8-4779-9e74-f215032777ba"} : dispatch 2024-09-21T03:38:30.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:30 smithi134 ceph-mon[35031]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ad5e6d0e-31e8-4779-9e74-f215032777ba"} : dispatch 2024-09-21T03:38:30.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:30 smithi134 ceph-mon[35031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad5e6d0e-31e8-4779-9e74-f215032777ba"}]': finished 2024-09-21T03:38:30.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:30 smithi134 ceph-mon[35031]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T03:38:30.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:30 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:30.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:30 smithi123 ceph-mon[34673]: pgmap v124: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:30.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:30 smithi123 ceph-mon[34673]: from='client.? 172.21.15.123:0/4283325337' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ad5e6d0e-31e8-4779-9e74-f215032777ba"} : dispatch 2024-09-21T03:38:30.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:30 smithi123 ceph-mon[34673]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ad5e6d0e-31e8-4779-9e74-f215032777ba"} : dispatch 2024-09-21T03:38:30.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:30 smithi123 ceph-mon[34673]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad5e6d0e-31e8-4779-9e74-f215032777ba"}]': finished 2024-09-21T03:38:30.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:30 smithi123 ceph-mon[34673]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T03:38:30.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:30 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:30.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:30 smithi016 ceph-mon[30375]: pgmap v124: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:30.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:30 smithi016 ceph-mon[30375]: from='client.? 172.21.15.123:0/4283325337' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ad5e6d0e-31e8-4779-9e74-f215032777ba"} : dispatch 2024-09-21T03:38:30.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:30 smithi016 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ad5e6d0e-31e8-4779-9e74-f215032777ba"} : dispatch 2024-09-21T03:38:30.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:30 smithi016 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad5e6d0e-31e8-4779-9e74-f215032777ba"}]': finished 2024-09-21T03:38:30.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:30 smithi016 ceph-mon[30375]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T03:38:30.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:30 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:31.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:31 smithi134 ceph-mon[35031]: from='client.? 172.21.15.123:0/616710831' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:38:31.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:31 smithi123 ceph-mon[34673]: from='client.? 172.21.15.123:0/616710831' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:38:31.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:31 smithi016 ceph-mon[30375]: from='client.? 172.21.15.123:0/616710831' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:38:32.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:32 smithi134 ceph-mon[35031]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:32.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:32 smithi123 ceph-mon[34673]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:32.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:32 smithi016 ceph-mon[30375]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:34.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:34 smithi134 ceph-mon[35031]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:38:34.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:34 smithi123 ceph-mon[34673]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:38:34.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:34 smithi016 ceph-mon[30375]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:38:36.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:36 smithi134 ceph-mon[35031]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T03:38:36.606 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:36 smithi123 ceph-mon[34673]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T03:38:36.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:36 smithi016 ceph-mon[30375]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T03:38:38.618 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:38 smithi123 ceph-mon[34673]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T03:38:38.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:38 smithi016 ceph-mon[30375]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T03:38:38.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:38 smithi134 ceph-mon[35031]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T03:38:39.626 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:39 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:38:39.626 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:39 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:39.626 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:39 smithi134 ceph-mon[35031]: Deploying daemon osd.4 on smithi123 2024-09-21T03:38:39.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:39 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:38:39.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:39 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:39.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:39 smithi123 ceph-mon[34673]: Deploying daemon osd.4 on smithi123 2024-09-21T03:38:39.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:39 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:38:39.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:39 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:39.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:39 smithi016 ceph-mon[30375]: Deploying daemon osd.4 on smithi123 2024-09-21T03:38:40.583 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:40 smithi123 ceph-mon[34673]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:38:40.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:40 smithi016 ceph-mon[30375]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:38:40.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:40 smithi134 ceph-mon[35031]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:38:42.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:42 smithi123 ceph-mon[34673]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T03:38:42.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:42 smithi016 ceph-mon[30375]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T03:38:42.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:42 smithi134 ceph-mon[35031]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T03:38:44.481 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:44 smithi123 ceph-mon[34673]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:38:44.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:44 smithi016 ceph-mon[30375]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:38:44.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:44 smithi134 ceph-mon[35031]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:38:45.626 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:45 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:45.626 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:45 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:45.626 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:45 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:45.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:45 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:45.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:45 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:45.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:45 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:45.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:45 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:45.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:45 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:45.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:45 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:46.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:46 smithi123 ceph-mon[34673]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:46.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:46 smithi016 ceph-mon[30375]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:46.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:46 smithi134 ceph-mon[35031]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:49.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:48 smithi134 ceph-mon[35031]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:49.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:48 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:49.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:48 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:49.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:48 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:49.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:48 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:49.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:48 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:49.119 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:48 smithi123 ceph-mon[34673]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:49.119 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:48 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:49.119 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:48 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:49.119 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:48 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:49.119 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:48 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:49.119 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:48 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:49.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:48 smithi016 ceph-mon[30375]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:49.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:48 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:49.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:48 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:48 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:48 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:48 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:49.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:49.161+0000 7f54d57fa640 1 -- 172.21.15.123:0/731666989 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f54d8063800 con 0x7f54c40b6510 2024-09-21T03:38:49.163 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 4 on host 'smithi123' 2024-09-21T03:38:49.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:49.163+0000 7f54ddbd8640 1 -- 172.21.15.123:0/731666989 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f54c40b6510 msgr2=0x7f54c40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:49.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:49.163+0000 7f54ddbd8640 1 --2- 172.21.15.123:0/731666989 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f54c40b6510 0x7f54c40b8a20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f54c80058c0 tx=0x7f54c800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:49.163+0000 7f54ddbd8640 1 -- 172.21.15.123:0/731666989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54d81068d0 msgr2=0x7f54d819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:49.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:49.164+0000 7f54ddbd8640 1 --2- 172.21.15.123:0/731666989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54d81068d0 0x7f54d819a5f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f54c0006010 tx=0x7f54c0004340 comp rx=0 tx=0).stop 2024-09-21T03:38:49.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:49.164+0000 7f54ddbd8640 1 -- 172.21.15.123:0/731666989 shutdown_connections 2024-09-21T03:38:49.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:49.164+0000 7f54ddbd8640 1 --2- 172.21.15.123:0/731666989 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f54c40b6510 0x7f54c40b8a20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:49.164+0000 7f54ddbd8640 1 --2- 172.21.15.123:0/731666989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54d8109470 0x7f54d819b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:49.164+0000 7f54ddbd8640 1 --2- 172.21.15.123:0/731666989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54d81068d0 0x7f54d819a5f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:49.164+0000 7f54ddbd8640 1 --2- 172.21.15.123:0/731666989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54d806b980 0x7f54d8199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:49.164+0000 7f54ddbd8640 1 -- 172.21.15.123:0/731666989 >> 172.21.15.123:0/731666989 conn(0x7f54d80fdfb0 msgr2=0x7f54d8104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:49.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:49.164+0000 7f54ddbd8640 1 -- 172.21.15.123:0/731666989 shutdown_connections 2024-09-21T03:38:49.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:38:49.164+0000 7f54ddbd8640 1 -- 172.21.15.123:0/731666989 wait complete. 2024-09-21T03:38:49.831 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:49 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:49.831 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:49 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:49.831 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:49 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:49.857 DEBUG:teuthology.orchestra.run.smithi123:osd.4> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T03:38:49.859 INFO:tasks.cephadm:Deploying osd.5 on smithi134 with /dev/vg_nvme/lv_3... 2024-09-21T03:38:49.859 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:38:50.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:49 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:50.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:49 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:50.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:49 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:50.104 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:38:50.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:49 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:50.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:49 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:50.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:49 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:50.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:50 smithi134 ceph-mon[35031]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:51.034 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:50 smithi123 ceph-mon[34673]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:51.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:50 smithi016 ceph-mon[30375]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:51.657 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:38:51 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-4[44742]: 2024-09-21T03:38:51.454+0000 7f21f8a1c640 -1 osd.4 0 log_to_monitors true 2024-09-21T03:38:51.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:51 smithi134 ceph-mon[35031]: from='osd.4 [v2:172.21.15.123:6808/2354279116,v1:172.21.15.123:6809/2354279116]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:38:51.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:51 smithi134 ceph-mon[35031]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:38:52.019 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:51 smithi123 ceph-mon[34673]: from='osd.4 [v2:172.21.15.123:6808/2354279116,v1:172.21.15.123:6809/2354279116]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:38:52.020 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:51 smithi123 ceph-mon[34673]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:38:52.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:51 smithi016 ceph-mon[30375]: from='osd.4 [v2:172.21.15.123:6808/2354279116,v1:172.21.15.123:6809/2354279116]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:38:52.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:51 smithi016 ceph-mon[30375]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:38:52.600 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:38:52.626 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi134:vg_nvme/lv_3 2024-09-21T03:38:52.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:52 smithi134 ceph-mon[35031]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:52.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:52 smithi134 ceph-mon[35031]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:38:52.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:52 smithi134 ceph-mon[35031]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:38:52.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:52.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:52 smithi134 ceph-mon[35031]: from='osd.4 [v2:172.21.15.123:6808/2354279116,v1:172.21.15.123:6809/2354279116]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:38:52.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:52 smithi134 ceph-mon[35031]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:38:52.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:52.877 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:38:53.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:52 smithi123 ceph-mon[34673]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:53.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:52 smithi123 ceph-mon[34673]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:38:53.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:52 smithi123 ceph-mon[34673]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:38:53.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:53.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:52 smithi123 ceph-mon[34673]: from='osd.4 [v2:172.21.15.123:6808/2354279116,v1:172.21.15.123:6809/2354279116]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:38:53.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:52 smithi123 ceph-mon[34673]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:38:53.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:53.157 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:38:52 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-4[44742]: 2024-09-21T03:38:52.745+0000 7f21f499e640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T03:38:53.157 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:38:52 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-4[44742]: 2024-09-21T03:38:52.754+0000 7f21f079c640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:38:53.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:52 smithi016 ceph-mon[30375]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:53.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:52 smithi016 ceph-mon[30375]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:38:53.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:52 smithi016 ceph-mon[30375]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:38:53.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:53.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:52 smithi016 ceph-mon[30375]: from='osd.4 [v2:172.21.15.123:6808/2354279116,v1:172.21.15.123:6809/2354279116]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:38:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:52 smithi016 ceph-mon[30375]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:38:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.335+0000 7fb81c609640 1 -- 172.21.15.134:0/2664474442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8141047e0 msgr2=0x7fb814104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.335+0000 7fb81c609640 1 --2- 172.21.15.134:0/2664474442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8141047e0 0x7fb814104bf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb804009900 tx=0x7fb80402a480 comp rx=0 tx=0).stop 2024-09-21T03:38:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.335+0000 7fb81c609640 1 -- 172.21.15.134:0/2664474442 shutdown_connections 2024-09-21T03:38:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.335+0000 7fb81c609640 1 --2- 172.21.15.134:0/2664474442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb814105180 0x7fb81410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.335+0000 7fb81c609640 1 --2- 172.21.15.134:0/2664474442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8141047e0 0x7fb814104bf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.335+0000 7fb81c609640 1 --2- 172.21.15.134:0/2664474442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb814103750 0x7fb814103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.335+0000 7fb81c609640 1 -- 172.21.15.134:0/2664474442 >> 172.21.15.134:0/2664474442 conn(0x7fb8140ff0c0 msgr2=0x7fb814101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.335+0000 7fb81c609640 1 -- 172.21.15.134:0/2664474442 shutdown_connections 2024-09-21T03:38:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.335+0000 7fb81c609640 1 -- 172.21.15.134:0/2664474442 wait complete. 2024-09-21T03:38:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.335+0000 7fb81c609640 1 Processor -- start 2024-09-21T03:38:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.335+0000 7fb81c609640 1 -- start start 2024-09-21T03:38:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb81c609640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb814103750 0x7fb81419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb81c609640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8141047e0 0x7fb81419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb81c609640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb814105180 0x7fb81419b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb81c609640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb81410f420 con 0x7fb814105180 2024-09-21T03:38:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb81c609640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb81410f2a0 con 0x7fb814103750 2024-09-21T03:38:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb81c609640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb81410f5a0 con 0x7fb8141047e0 2024-09-21T03:38:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb819ba4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8141047e0 0x7fb81419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb819ba4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8141047e0 0x7fb81419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:44550/0 (socket says 172.21.15.134:44550) 2024-09-21T03:38:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb819ba4640 1 -- 172.21.15.134:0/2492321227 learned_addr learned my addr 172.21.15.134:0/2492321227 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-21T03:38:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb81a3a5640 1 --2- 172.21.15.134:0/2492321227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb814103750 0x7fb81419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb819ba4640 1 -- 172.21.15.134:0/2492321227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb814103750 msgr2=0x7fb81419a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb819ba4640 1 --2- 172.21.15.134:0/2492321227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb814103750 0x7fb81419a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb81aba6640 1 --2- 172.21.15.134:0/2492321227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb814105180 0x7fb81419b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb819ba4640 1 -- 172.21.15.134:0/2492321227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb814105180 msgr2=0x7fb81419b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb819ba4640 1 --2- 172.21.15.134:0/2492321227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb814105180 0x7fb81419b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb819ba4640 1 -- 172.21.15.134:0/2492321227 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81419ba80 con 0x7fb8141047e0 2024-09-21T03:38:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb819ba4640 1 --2- 172.21.15.134:0/2492321227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8141047e0 0x7fb81419a6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb804006010 tx=0x7fb804004340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb80b7fe640 1 -- 172.21.15.134:0/2492321227 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80403f030 con 0x7fb8141047e0 2024-09-21T03:38:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb80b7fe640 1 -- 172.21.15.134:0/2492321227 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb804003cb0 con 0x7fb8141047e0 2024-09-21T03:38:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb81c609640 1 -- 172.21.15.134:0/2492321227 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81419bd10 con 0x7fb8141047e0 2024-09-21T03:38:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb80b7fe640 1 -- 172.21.15.134:0/2492321227 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804035070 con 0x7fb8141047e0 2024-09-21T03:38:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.336+0000 7fb81c609640 1 -- 172.21.15.134:0/2492321227 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8141a2320 con 0x7fb8141047e0 2024-09-21T03:38:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.338+0000 7fb81c609640 1 -- 172.21.15.134:0/2492321227 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb81410a930 con 0x7fb8141047e0 2024-09-21T03:38:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.339+0000 7fb80b7fe640 1 -- 172.21.15.134:0/2492321227 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fb804042030 con 0x7fb8141047e0 2024-09-21T03:38:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.340+0000 7fb80b7fe640 1 --2- 172.21.15.134:0/2492321227 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8000b6510 0x7fb8000b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.340+0000 7fb80b7fe640 1 -- 172.21.15.134:0/2492321227 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3821+0+0 (secure 0 0 0) 0x7fb804106eb0 con 0x7fb8141047e0 2024-09-21T03:38:53.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.341+0000 7fb81a3a5640 1 --2- 172.21.15.134:0/2492321227 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8000b6510 0x7fb8000b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:53.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.341+0000 7fb81a3a5640 1 --2- 172.21.15.134:0/2492321227 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8000b6510 0x7fb8000b8a20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb810005fd0 tx=0x7fb81000b390 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:53.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.344+0000 7fb80b7fe640 1 -- 172.21.15.134:0/2492321227 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb804138710 con 0x7fb8141047e0 2024-09-21T03:38:53.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:38:53.531+0000 7fb81c609640 1 -- 172.21.15.134:0/2492321227 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb814063800 con 0x7fb8000b6510 2024-09-21T03:38:54.038 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:53 smithi123 ceph-mon[34673]: Detected new or changed devices on smithi123 2024-09-21T03:38:54.038 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:54.038 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:54.038 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:53 smithi123 ceph-mon[34673]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T03:38:54.039 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:53 smithi123 ceph-mon[34673]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:38:54.039 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:54.039 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:54.039 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:53 smithi123 ceph-mon[34673]: Adjusting osd_memory_target on smithi123 to 8544M 2024-09-21T03:38:54.039 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:54.039 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:54.039 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:54.039 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:54.039 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:54.039 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:38:54.039 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:38:54.039 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:54.058 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:53 smithi134 ceph-mon[35031]: Detected new or changed devices on smithi123 2024-09-21T03:38:54.058 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:54.058 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:54.058 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:53 smithi134 ceph-mon[35031]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T03:38:54.058 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:53 smithi134 ceph-mon[35031]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:38:54.058 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:54.058 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:54.058 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:53 smithi134 ceph-mon[35031]: Adjusting osd_memory_target on smithi123 to 8544M 2024-09-21T03:38:54.058 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:54.059 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:54.059 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:54.059 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:54.059 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:54.059 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:38:54.059 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:38:54.059 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:54.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:53 smithi016 ceph-mon[30375]: Detected new or changed devices on smithi123 2024-09-21T03:38:54.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:54.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:54.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:53 smithi016 ceph-mon[30375]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T03:38:54.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:53 smithi016 ceph-mon[30375]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:38:54.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:38:54.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:54.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:53 smithi016 ceph-mon[30375]: Adjusting osd_memory_target on smithi123 to 8544M 2024-09-21T03:38:54.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:54.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:54.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:54.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:54.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:38:54.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:38:54.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:38:54.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:54.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:54 smithi123 ceph-mon[34673]: purged_snaps scrub starts 2024-09-21T03:38:54.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:54 smithi123 ceph-mon[34673]: purged_snaps scrub ok 2024-09-21T03:38:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:54 smithi123 ceph-mon[34673]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:54 smithi123 ceph-mon[34673]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:38:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:54 smithi123 ceph-mon[34673]: osd.4 [v2:172.21.15.123:6808/2354279116,v1:172.21.15.123:6809/2354279116] boot 2024-09-21T03:38:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:54 smithi123 ceph-mon[34673]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T03:38:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:54.912 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:54 smithi134 ceph-mon[35031]: purged_snaps scrub starts 2024-09-21T03:38:54.912 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:54 smithi134 ceph-mon[35031]: purged_snaps scrub ok 2024-09-21T03:38:54.912 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:54 smithi134 ceph-mon[35031]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:54.912 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:54 smithi134 ceph-mon[35031]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:38:54.912 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:54.912 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:54 smithi134 ceph-mon[35031]: osd.4 [v2:172.21.15.123:6808/2354279116,v1:172.21.15.123:6809/2354279116] boot 2024-09-21T03:38:54.912 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:54 smithi134 ceph-mon[35031]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T03:38:54.912 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:55.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:54 smithi016 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T03:38:55.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:54 smithi016 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T03:38:55.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:54 smithi016 ceph-mon[30375]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:38:55.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:54 smithi016 ceph-mon[30375]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:38:55.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:55.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:54 smithi016 ceph-mon[30375]: osd.4 [v2:172.21.15.123:6808/2354279116,v1:172.21.15.123:6809/2354279116] boot 2024-09-21T03:38:55.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:54 smithi016 ceph-mon[30375]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T03:38:55.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:38:56.092 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:55 smithi016 ceph-mon[30375]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:38:56.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:55 smithi134 ceph-mon[35031]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:38:56.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:55 smithi123 ceph-mon[34673]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:38:56.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:56 smithi134 ceph-mon[35031]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:38:56.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:56 smithi134 ceph-mon[35031]: from='client.? 172.21.15.134:0/2892350039' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca09bf88-1772-4dc0-8711-cbf9a5b31977"} : dispatch 2024-09-21T03:38:56.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:56 smithi134 ceph-mon[35031]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca09bf88-1772-4dc0-8711-cbf9a5b31977"} : dispatch 2024-09-21T03:38:56.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:56 smithi134 ceph-mon[35031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca09bf88-1772-4dc0-8711-cbf9a5b31977"}]': finished 2024-09-21T03:38:56.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:56 smithi134 ceph-mon[35031]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T03:38:56.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:56 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:38:56.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:56 smithi134 ceph-mon[35031]: from='client.? 172.21.15.134:0/3483101087' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:38:57.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:56 smithi123 ceph-mon[34673]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:38:57.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:56 smithi123 ceph-mon[34673]: from='client.? 172.21.15.134:0/2892350039' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca09bf88-1772-4dc0-8711-cbf9a5b31977"} : dispatch 2024-09-21T03:38:57.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:56 smithi123 ceph-mon[34673]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca09bf88-1772-4dc0-8711-cbf9a5b31977"} : dispatch 2024-09-21T03:38:57.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:56 smithi123 ceph-mon[34673]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca09bf88-1772-4dc0-8711-cbf9a5b31977"}]': finished 2024-09-21T03:38:57.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:56 smithi123 ceph-mon[34673]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T03:38:57.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:56 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:38:57.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:56 smithi123 ceph-mon[34673]: from='client.? 172.21.15.134:0/3483101087' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:38:57.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:56 smithi016 ceph-mon[30375]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:38:57.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:56 smithi016 ceph-mon[30375]: from='client.? 172.21.15.134:0/2892350039' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca09bf88-1772-4dc0-8711-cbf9a5b31977"} : dispatch 2024-09-21T03:38:57.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:56 smithi016 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca09bf88-1772-4dc0-8711-cbf9a5b31977"} : dispatch 2024-09-21T03:38:57.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:56 smithi016 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca09bf88-1772-4dc0-8711-cbf9a5b31977"}]': finished 2024-09-21T03:38:57.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:56 smithi016 ceph-mon[30375]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T03:38:57.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:56 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:38:57.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:56 smithi016 ceph-mon[30375]: from='client.? 172.21.15.134:0/3483101087' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:38:59.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:38:58 smithi134 ceph-mon[35031]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:38:59.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:38:58 smithi123 ceph-mon[34673]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:38:59.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:38:58 smithi016 ceph-mon[30375]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:01.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:00 smithi134 ceph-mon[35031]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:01.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:00 smithi123 ceph-mon[34673]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:00 smithi016 ceph-mon[30375]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:02.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:02 smithi134 ceph-mon[35031]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:03.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:02 smithi123 ceph-mon[34673]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:03.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:02 smithi016 ceph-mon[30375]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:05.090 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:04 smithi134 ceph-mon[35031]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:05.090 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:04 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:39:05.090 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:04 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:05.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:04 smithi123 ceph-mon[34673]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:05.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:04 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:39:05.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:04 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:05.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:04 smithi016 ceph-mon[30375]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:05.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:04 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:39:05.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:04 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:06.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:05 smithi134 ceph-mon[35031]: Deploying daemon osd.5 on smithi134 2024-09-21T03:39:06.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:05 smithi123 ceph-mon[34673]: Deploying daemon osd.5 on smithi134 2024-09-21T03:39:06.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:05 smithi016 ceph-mon[30375]: Deploying daemon osd.5 on smithi134 2024-09-21T03:39:07.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:06 smithi016 ceph-mon[30375]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:07.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:06 smithi134 ceph-mon[35031]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:07.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:06 smithi123 ceph-mon[34673]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:09.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:08 smithi134 ceph-mon[35031]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:09.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:08 smithi123 ceph-mon[34673]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:09.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:08 smithi016 ceph-mon[30375]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:11.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:10 smithi134 ceph-mon[35031]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:11.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:10 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:11.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:10 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:11.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:10 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:11.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:10 smithi123 ceph-mon[34673]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:11.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:10 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:11.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:10 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:11.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:10 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:11.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:10 smithi016 ceph-mon[30375]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:11.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:10 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:11.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:10 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:11.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:10 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:12.948 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:12 smithi134 ceph-mon[35031]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:13.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:12 smithi123 ceph-mon[34673]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:13.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:12 smithi016 ceph-mon[30375]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:14.515 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:14 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:14.515 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:14 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:14.515 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:14 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:14.516 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:14 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:14.516 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:14 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:14.516 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:14 smithi123 ceph-mon[34673]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:14.587 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:14 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:14.587 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:14 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:14.587 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:14 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:14.587 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:14 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:14.587 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:14 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:14.587 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:14 smithi134 ceph-mon[35031]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:14.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:14 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:14.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:14 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:14 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:14 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:14 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:14 smithi016 ceph-mon[30375]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:14.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:39:14.924+0000 7fb80b7fe640 1 -- 172.21.15.134:0/2492321227 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb814063800 con 0x7fb8000b6510 2024-09-21T03:39:14.926 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 5 on host 'smithi134' 2024-09-21T03:39:14.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:39:14.926+0000 7fb81c609640 1 -- 172.21.15.134:0/2492321227 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8000b6510 msgr2=0x7fb8000b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:14.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:39:14.926+0000 7fb81c609640 1 --2- 172.21.15.134:0/2492321227 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8000b6510 0x7fb8000b8a20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb810005fd0 tx=0x7fb81000b390 comp rx=0 tx=0).stop 2024-09-21T03:39:14.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:39:14.927+0000 7fb81c609640 1 -- 172.21.15.134:0/2492321227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8141047e0 msgr2=0x7fb81419a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:14.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:39:14.927+0000 7fb81c609640 1 --2- 172.21.15.134:0/2492321227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8141047e0 0x7fb81419a6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb804006010 tx=0x7fb804004340 comp rx=0 tx=0).stop 2024-09-21T03:39:14.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:39:14.927+0000 7fb81c609640 1 -- 172.21.15.134:0/2492321227 shutdown_connections 2024-09-21T03:39:14.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:39:14.927+0000 7fb81c609640 1 --2- 172.21.15.134:0/2492321227 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8000b6510 0x7fb8000b8a20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:39:14.927+0000 7fb81c609640 1 --2- 172.21.15.134:0/2492321227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb814105180 0x7fb81419b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:39:14.927+0000 7fb81c609640 1 --2- 172.21.15.134:0/2492321227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8141047e0 0x7fb81419a6c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:39:14.927+0000 7fb81c609640 1 --2- 172.21.15.134:0/2492321227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb814103750 0x7fb81419a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:14.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:39:14.927+0000 7fb81c609640 1 -- 172.21.15.134:0/2492321227 >> 172.21.15.134:0/2492321227 conn(0x7fb8140ff0c0 msgr2=0x7fb814107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:14.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:39:14.927+0000 7fb81c609640 1 -- 172.21.15.134:0/2492321227 shutdown_connections 2024-09-21T03:39:14.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:39:14.927+0000 7fb81c609640 1 -- 172.21.15.134:0/2492321227 wait complete. 2024-09-21T03:39:15.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:15 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:15.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:15 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:15.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:15 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:15.635 DEBUG:teuthology.orchestra.run.smithi134:osd.5> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T03:39:15.637 INFO:tasks.cephadm:Deploying osd.6 on smithi016 with /dev/vg_nvme/lv_2... 2024-09-21T03:39:15.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:39:15.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:15 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:15.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:15 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:15.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:15 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:15.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:15 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:15.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:15 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:15.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:15 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:16.022 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:16.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:16 smithi016 ceph-mon[30375]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:16.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:16 smithi134 ceph-mon[35031]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:16.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:16 smithi123 ceph-mon[34673]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:17.044 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:39:16 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:39:16.841+0000 7fa6f0aba640 -1 osd.5 0 log_to_monitors true 2024-09-21T03:39:17.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:17 smithi134 ceph-mon[35031]: from='osd.5 [v2:172.21.15.134:6808/2678416182,v1:172.21.15.134:6809/2678416182]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:39:17.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:17 smithi134 ceph-mon[35031]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:39:17.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:17 smithi016 ceph-mon[30375]: from='osd.5 [v2:172.21.15.134:6808/2678416182,v1:172.21.15.134:6809/2678416182]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:39:17.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:17 smithi016 ceph-mon[30375]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:39:17.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:17 smithi123 ceph-mon[34673]: from='osd.5 [v2:172.21.15.134:6808/2678416182,v1:172.21.15.134:6809/2678416182]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:39:17.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:17 smithi123 ceph-mon[34673]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:39:18.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:18 smithi016 ceph-mon[30375]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:39:18.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:18 smithi016 ceph-mon[30375]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:39:18.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:18 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:39:18.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:18 smithi016 ceph-mon[30375]: from='osd.5 [v2:172.21.15.134:6808/2678416182,v1:172.21.15.134:6809/2678416182]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:39:18.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:18 smithi016 ceph-mon[30375]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:39:18.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:18 smithi016 ceph-mon[30375]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:18.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:18 smithi134 ceph-mon[35031]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:39:18.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:18 smithi134 ceph-mon[35031]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:39:18.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:18 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:39:18.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:18 smithi134 ceph-mon[35031]: from='osd.5 [v2:172.21.15.134:6808/2678416182,v1:172.21.15.134:6809/2678416182]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:39:18.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:18 smithi134 ceph-mon[35031]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:39:18.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:18 smithi134 ceph-mon[35031]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:18.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:18 smithi123 ceph-mon[34673]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:39:18.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:18 smithi123 ceph-mon[34673]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:39:18.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:18 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:39:18.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:18 smithi123 ceph-mon[34673]: from='osd.5 [v2:172.21.15.134:6808/2678416182,v1:172.21.15.134:6809/2678416182]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:39:18.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:18 smithi123 ceph-mon[34673]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:39:18.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:18 smithi123 ceph-mon[34673]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:39:19.271 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:39:19.003+0000 7fa6eca3c640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T03:39:19.271 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:39:19.009+0000 7fa6e4832640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:39:19.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:39:19.364 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi016:vg_nvme/lv_2 2024-09-21T03:39:19.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-21T03:39:19.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:39:19.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:39:19.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:39:19.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: Detected new or changed devices on smithi134 2024-09-21T03:39:19.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:19.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:19.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:19.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:19.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: Adjusting osd_memory_target on smithi134 to 8544M 2024-09-21T03:39:19.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:19.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:19.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:19.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:19.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: from='osd.5 ' entity='osd.5' 2024-09-21T03:39:19.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: osd.5 [v2:172.21.15.134:6808/2678416182,v1:172.21.15.134:6809/2678416182] boot 2024-09-21T03:39:19.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T03:39:19.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:39:19.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-21T03:39:19.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:39:19.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:39:19.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:39:19.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: Detected new or changed devices on smithi134 2024-09-21T03:39:19.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:19.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:19.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:19.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:19.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: Adjusting osd_memory_target on smithi134 to 8544M 2024-09-21T03:39:19.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:19.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:19.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:19.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:19.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: from='osd.5 ' entity='osd.5' 2024-09-21T03:39:19.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: osd.5 [v2:172.21.15.134:6808/2678416182,v1:172.21.15.134:6809/2678416182] boot 2024-09-21T03:39:19.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T03:39:19.658 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:39:19.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-21T03:39:19.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:39:19.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:39:19.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:39:19.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: Detected new or changed devices on smithi134 2024-09-21T03:39:19.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:19.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:19.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:19.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:19.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: Adjusting osd_memory_target on smithi134 to 8544M 2024-09-21T03:39:19.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:19.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:19.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:19.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:19.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: from='osd.5 ' entity='osd.5' 2024-09-21T03:39:19.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: osd.5 [v2:172.21.15.134:6808/2678416182,v1:172.21.15.134:6809/2678416182] boot 2024-09-21T03:39:19.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T03:39:19.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:39:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:20.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.428+0000 7efee7000640 1 -- 172.21.15.16:0/1917592805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee0104640 msgr2=0x7efee0104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:20.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.428+0000 7efee7000640 1 --2- 172.21.15.16:0/1917592805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee0104640 0x7efee0104a50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efecc009900 tx=0x7efecc02a480 comp rx=0 tx=0).stop 2024-09-21T03:39:20.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.429+0000 7efee7000640 1 -- 172.21.15.16:0/1917592805 shutdown_connections 2024-09-21T03:39:20.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.429+0000 7efee7000640 1 --2- 172.21.15.16:0/1917592805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee0104fe0 0x7efee010b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.429+0000 7efee7000640 1 --2- 172.21.15.16:0/1917592805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee0104640 0x7efee0104a50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.429+0000 7efee7000640 1 --2- 172.21.15.16:0/1917592805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee01035b0 0x7efee0103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.429+0000 7efee7000640 1 -- 172.21.15.16:0/1917592805 >> 172.21.15.16:0/1917592805 conn(0x7efee00fef20 msgr2=0x7efee01013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:20.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.429+0000 7efee7000640 1 -- 172.21.15.16:0/1917592805 shutdown_connections 2024-09-21T03:39:20.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.429+0000 7efee7000640 1 -- 172.21.15.16:0/1917592805 wait complete. 2024-09-21T03:39:20.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.430+0000 7efee7000640 1 Processor -- start 2024-09-21T03:39:20.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.430+0000 7efee7000640 1 -- start start 2024-09-21T03:39:20.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.430+0000 7efee7000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee01035b0 0x7efee0076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.430+0000 7efee7000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee0104640 0x7efee0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.430+0000 7efee7000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee0104fe0 0x7efee0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.430+0000 7efee7000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efee010f210 con 0x7efee01035b0 2024-09-21T03:39:20.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.430+0000 7efee7000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efee010f090 con 0x7efee0104640 2024-09-21T03:39:20.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.430+0000 7efee7000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efee010f390 con 0x7efee0104fe0 2024-09-21T03:39:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.431+0000 7efee5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee01035b0 0x7efee0076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.431+0000 7efee5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee01035b0 0x7efee0076e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54886/0 (socket says 172.21.15.16:54886) 2024-09-21T03:39:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.431+0000 7efee5c00640 1 -- 172.21.15.16:0/1513652351 learned_addr learned my addr 172.21.15.16:0/1513652351 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:39:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.431+0000 7efee5200640 1 --2- 172.21.15.16:0/1513652351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee0104640 0x7efee0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.431+0000 7efee6600640 1 --2- 172.21.15.16:0/1513652351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee0104fe0 0x7efee0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.431+0000 7efee5c00640 1 -- 172.21.15.16:0/1513652351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee0104fe0 msgr2=0x7efee0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.431+0000 7efee5c00640 1 --2- 172.21.15.16:0/1513652351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee0104fe0 0x7efee0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.431+0000 7efee5c00640 1 -- 172.21.15.16:0/1513652351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee0104640 msgr2=0x7efee0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.431+0000 7efee5c00640 1 --2- 172.21.15.16:0/1513652351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee0104640 0x7efee0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.431+0000 7efee5c00640 1 -- 172.21.15.16:0/1513652351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee0076350 con 0x7efee01035b0 2024-09-21T03:39:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.431+0000 7efee5200640 1 --2- 172.21.15.16:0/1513652351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee0104640 0x7efee0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.432+0000 7efee5c00640 1 --2- 172.21.15.16:0/1513652351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee01035b0 0x7efee0076e40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efed400e9f0 tx=0x7efed400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.432+0000 7efede000640 1 -- 172.21.15.16:0/1513652351 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed400ccc0 con 0x7efee01035b0 2024-09-21T03:39:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.432+0000 7efede000640 1 -- 172.21.15.16:0/1513652351 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed400ce60 con 0x7efee01035b0 2024-09-21T03:39:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.432+0000 7efee7000640 1 -- 172.21.15.16:0/1513652351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee01a3f00 con 0x7efee01035b0 2024-09-21T03:39:20.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.432+0000 7efede000640 1 -- 172.21.15.16:0/1513652351 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4010430 con 0x7efee01035b0 2024-09-21T03:39:20.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.432+0000 7efee7000640 1 -- 172.21.15.16:0/1513652351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee01a42a0 con 0x7efee01035b0 2024-09-21T03:39:20.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.433+0000 7efee7000640 1 -- 172.21.15.16:0/1513652351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee0103a20 con 0x7efee01035b0 2024-09-21T03:39:20.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.436+0000 7efede000640 1 -- 172.21.15.16:0/1513652351 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7efed40046c0 con 0x7efee01035b0 2024-09-21T03:39:20.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.437+0000 7efede000640 1 --2- 172.21.15.16:0/1513652351 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efec80b6590 0x7efec80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.437+0000 7efede000640 1 -- 172.21.15.16:0/1513652351 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4533+0+0 (secure 0 0 0) 0x7efed40e6a20 con 0x7efee01035b0 2024-09-21T03:39:20.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.438+0000 7efee5200640 1 --2- 172.21.15.16:0/1513652351 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efec80b6590 0x7efec80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.438+0000 7efee5200640 1 --2- 172.21.15.16:0/1513652351 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efec80b6590 0x7efec80b8aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efecc000c00 tx=0x7efecc031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:20.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.440+0000 7efede000640 1 -- 172.21.15.16:0/1513652351 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efed4056710 con 0x7efee01035b0 2024-09-21T03:39:20.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:20 smithi134 ceph-mon[35031]: purged_snaps scrub starts 2024-09-21T03:39:20.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:20 smithi134 ceph-mon[35031]: purged_snaps scrub ok 2024-09-21T03:39:20.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:20 smithi134 ceph-mon[35031]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 574 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:20.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:20 smithi134 ceph-mon[35031]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:39:20.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:20.650+0000 7efee7000640 1 -- 172.21.15.16:0/1513652351 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7efee0063800 con 0x7efec80b6590 2024-09-21T03:39:20.653 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:20 smithi016 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T03:39:20.653 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:20 smithi016 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T03:39:20.653 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:20 smithi016 ceph-mon[30375]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 574 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:20.653 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:20 smithi016 ceph-mon[30375]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:39:20.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:20 smithi123 ceph-mon[34673]: purged_snaps scrub starts 2024-09-21T03:39:20.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:20 smithi123 ceph-mon[34673]: purged_snaps scrub ok 2024-09-21T03:39:20.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:20 smithi123 ceph-mon[34673]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 574 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:20.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:20 smithi123 ceph-mon[34673]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:39:21.481 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:21 smithi016 ceph-mon[30375]: from='client.14499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:21.482 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:21 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:39:21.482 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:21 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:39:21.482 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:21 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:21.482 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:21 smithi016 ceph-mon[30375]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:39:21.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:21 smithi134 ceph-mon[35031]: from='client.14499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:21.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:21 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:39:21.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:21 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:39:21.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:21 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:21.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:21 smithi134 ceph-mon[35031]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:39:21.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:21 smithi123 ceph-mon[34673]: from='client.14499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:21.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:21 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:39:21.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:21 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:39:21.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:21 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:21.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:21 smithi123 ceph-mon[34673]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:39:22.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:22 smithi016 ceph-mon[30375]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 574 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:22.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:22 smithi134 ceph-mon[35031]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 574 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:22.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:22 smithi123 ceph-mon[34673]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 574 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:24.595 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:24 smithi123 ceph-mon[34673]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 573 MiB used, 536 GiB / 536 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T03:39:24.595 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:24 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2862245433' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0d41ed4e-8699-4d88-9254-12d3b84941e3"} : dispatch 2024-09-21T03:39:24.595 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:24 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2862245433' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d41ed4e-8699-4d88-9254-12d3b84941e3"}]': finished 2024-09-21T03:39:24.595 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:24 smithi123 ceph-mon[34673]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T03:39:24.595 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:24 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:24.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:24 smithi134 ceph-mon[35031]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 573 MiB used, 536 GiB / 536 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T03:39:24.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:24 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2862245433' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0d41ed4e-8699-4d88-9254-12d3b84941e3"} : dispatch 2024-09-21T03:39:24.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:24 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2862245433' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d41ed4e-8699-4d88-9254-12d3b84941e3"}]': finished 2024-09-21T03:39:24.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:24 smithi134 ceph-mon[35031]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T03:39:24.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:24 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:24.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:24 smithi016 ceph-mon[30375]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 573 MiB used, 536 GiB / 536 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T03:39:24.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:24 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2862245433' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0d41ed4e-8699-4d88-9254-12d3b84941e3"} : dispatch 2024-09-21T03:39:24.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:24 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2862245433' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d41ed4e-8699-4d88-9254-12d3b84941e3"}]': finished 2024-09-21T03:39:24.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:24 smithi016 ceph-mon[30375]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T03:39:24.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:24 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:25.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:25 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/409468998' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:39:25.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:25 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/409468998' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:39:25.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:25 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/409468998' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:39:26.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:26 smithi134 ceph-mon[35031]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:39:26.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:26 smithi123 ceph-mon[34673]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:39:26.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:26 smithi016 ceph-mon[30375]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:39:28.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:28 smithi134 ceph-mon[35031]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-21T03:39:28.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:28 smithi123 ceph-mon[34673]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-21T03:39:28.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:28 smithi016 ceph-mon[30375]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-21T03:39:30.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:30 smithi016 ceph-mon[30375]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:39:30.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:30 smithi134 ceph-mon[35031]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:39:30.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:30 smithi123 ceph-mon[34673]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:39:32.565 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:32 smithi016 ceph-mon[30375]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:39:32.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:32 smithi134 ceph-mon[35031]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:39:32.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:32 smithi123 ceph-mon[34673]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:39:34.573 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:34 smithi016 ceph-mon[30375]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:34.573 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:34 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:39:34.573 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:34 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:34.573 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:34 smithi016 ceph-mon[30375]: Deploying daemon osd.6 on smithi016 2024-09-21T03:39:34.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:34 smithi134 ceph-mon[35031]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:34.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:34 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:39:34.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:34 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:34.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:34 smithi134 ceph-mon[35031]: Deploying daemon osd.6 on smithi016 2024-09-21T03:39:34.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:34 smithi123 ceph-mon[34673]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:34.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:34 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:39:34.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:34 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:34.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:34 smithi123 ceph-mon[34673]: Deploying daemon osd.6 on smithi016 2024-09-21T03:39:36.470 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:36 smithi016 ceph-mon[30375]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:36.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:36 smithi134 ceph-mon[35031]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:36.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:36 smithi123 ceph-mon[34673]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:38.588 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:38 smithi016 ceph-mon[30375]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:38.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:38 smithi134 ceph-mon[35031]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:38.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:38 smithi123 ceph-mon[34673]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:40.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:40 smithi134 ceph-mon[35031]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:40.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:40 smithi123 ceph-mon[34673]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:40.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:40 smithi016 ceph-mon[30375]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:42.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:42 smithi016 ceph-mon[30375]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:42.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:42 smithi134 ceph-mon[35031]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:42.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:42 smithi123 ceph-mon[34673]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:43.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:43 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:43.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:43 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:43.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:43 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:43.619 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:43 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:43.619 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:43 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:43.620 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:43 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:43.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:43 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:43.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:43 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:43.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:43 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:45.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:44 smithi016 ceph-mon[30375]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:45.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:44 smithi134 ceph-mon[35031]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:45.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:44 smithi123 ceph-mon[34673]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:47.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:46 smithi134 ceph-mon[35031]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:47.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:46 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:47.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:46 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:47.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:46 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:47.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:46 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:47.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:46 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:47.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:46 smithi123 ceph-mon[34673]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:47.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:46 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:47.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:46 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:47.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:46 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:47.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:46 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:47.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:46 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:47.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:46 smithi016 ceph-mon[30375]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:47.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:46 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:47.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:46 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:47.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:46 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:47.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:46 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:47.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:46 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:48.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:48.517+0000 7efede000640 1 -- 172.21.15.16:0/1513652351 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efee0063800 con 0x7efec80b6590 2024-09-21T03:39:48.520 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 6 on host 'smithi016' 2024-09-21T03:39:48.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:48.520+0000 7efee7000640 1 -- 172.21.15.16:0/1513652351 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efec80b6590 msgr2=0x7efec80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:48.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:48.520+0000 7efee7000640 1 --2- 172.21.15.16:0/1513652351 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efec80b6590 0x7efec80b8aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efecc000c00 tx=0x7efecc031400 comp rx=0 tx=0).stop 2024-09-21T03:39:48.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:48.521+0000 7efee7000640 1 -- 172.21.15.16:0/1513652351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee01035b0 msgr2=0x7efee0076e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:48.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:48.521+0000 7efee7000640 1 --2- 172.21.15.16:0/1513652351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee01035b0 0x7efee0076e40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efed400e9f0 tx=0x7efed400edf0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:48.521+0000 7efee7000640 1 -- 172.21.15.16:0/1513652351 shutdown_connections 2024-09-21T03:39:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:48.521+0000 7efee7000640 1 --2- 172.21.15.16:0/1513652351 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efec80b6590 0x7efec80b8aa0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:48.521+0000 7efee7000640 1 --2- 172.21.15.16:0/1513652351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee0104fe0 0x7efee0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:48.521+0000 7efee7000640 1 --2- 172.21.15.16:0/1513652351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee0104640 0x7efee0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:48.521+0000 7efee7000640 1 --2- 172.21.15.16:0/1513652351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee01035b0 0x7efee0076e40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:48.521+0000 7efee7000640 1 -- 172.21.15.16:0/1513652351 >> 172.21.15.16:0/1513652351 conn(0x7efee00fef20 msgr2=0x7efee0107960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:48.521+0000 7efee7000640 1 -- 172.21.15.16:0/1513652351 shutdown_connections 2024-09-21T03:39:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:39:48.521+0000 7efee7000640 1 -- 172.21.15.16:0/1513652351 wait complete. 2024-09-21T03:39:49.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:48 smithi134 ceph-mon[35031]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:49.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:48 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:49.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:48 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:49.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:48 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:49.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:48 smithi123 ceph-mon[34673]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:49.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:48 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:49.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:48 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:49.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:48 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:49.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:48 smithi016 ceph-mon[30375]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:49.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:48 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:48 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:48 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:49.250 DEBUG:teuthology.orchestra.run.smithi016:osd.6> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T03:39:49.254 INFO:tasks.cephadm:Deploying osd.7 on smithi123 with /dev/vg_nvme/lv_2... 2024-09-21T03:39:49.254 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:39:49.504 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:39:49 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:39:49.434+0000 7f3fca097640 -1 osd.6 0 log_to_monitors true 2024-09-21T03:39:49.508 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:39:49.900 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:49 smithi123 ceph-mon[34673]: from='osd.6 [v2:172.21.15.16:6818/3377807744,v1:172.21.15.16:6819/3377807744]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:39:49.900 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:49 smithi123 ceph-mon[34673]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:39:50.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:49 smithi134 ceph-mon[35031]: from='osd.6 [v2:172.21.15.16:6818/3377807744,v1:172.21.15.16:6819/3377807744]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:39:50.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:49 smithi134 ceph-mon[35031]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:39:50.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:49 smithi016 ceph-mon[30375]: from='osd.6 [v2:172.21.15.16:6818/3377807744,v1:172.21.15.16:6819/3377807744]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:39:50.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:49 smithi016 ceph-mon[30375]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:39:50.992 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:50 smithi016 ceph-mon[30375]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:50.992 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:50 smithi016 ceph-mon[30375]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:39:50.992 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:50 smithi016 ceph-mon[30375]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:39:50.992 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:50.992 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:50 smithi016 ceph-mon[30375]: from='osd.6 [v2:172.21.15.16:6818/3377807744,v1:172.21.15.16:6819/3377807744]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:39:50.992 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:50 smithi016 ceph-mon[30375]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:39:51.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:50 smithi134 ceph-mon[35031]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:51.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:50 smithi134 ceph-mon[35031]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:39:51.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:50 smithi134 ceph-mon[35031]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:39:51.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:51.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:50 smithi134 ceph-mon[35031]: from='osd.6 [v2:172.21.15.16:6818/3377807744,v1:172.21.15.16:6819/3377807744]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:39:51.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:50 smithi134 ceph-mon[35031]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:39:51.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:50 smithi123 ceph-mon[34673]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:51.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:50 smithi123 ceph-mon[34673]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:39:51.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:50 smithi123 ceph-mon[34673]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:39:51.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:50 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:51.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:50 smithi123 ceph-mon[34673]: from='osd.6 [v2:172.21.15.16:6818/3377807744,v1:172.21.15.16:6819/3377807744]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:39:51.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:50 smithi123 ceph-mon[34673]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:39:51.662 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:39:51 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:39:51.461+0000 7f3fc5000640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T03:39:51.662 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:39:51 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:39:51.486+0000 7f3fbb000640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:39:52.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:51 smithi134 ceph-mon[35031]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T03:39:52.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:51 smithi134 ceph-mon[35031]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:39:52.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:51 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:52.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:51 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:52.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:51 smithi134 ceph-mon[35031]: from='osd.6 ' entity='osd.6' 2024-09-21T03:39:52.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:51 smithi123 ceph-mon[34673]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T03:39:52.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:51 smithi123 ceph-mon[34673]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:39:52.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:51 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:52.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:51 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:52.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:51 smithi123 ceph-mon[34673]: from='osd.6 ' entity='osd.6' 2024-09-21T03:39:52.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:51 smithi016 ceph-mon[30375]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T03:39:52.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:51 smithi016 ceph-mon[30375]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:39:52.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:51 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:52.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:51 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:52.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:51 smithi016 ceph-mon[30375]: from='osd.6 ' entity='osd.6' 2024-09-21T03:39:52.308 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:39:52.334 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi123:vg_nvme/lv_2 2024-09-21T03:39:52.584 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:39:52.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:52 smithi123 ceph-mon[34673]: purged_snaps scrub starts 2024-09-21T03:39:52.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:52 smithi123 ceph-mon[34673]: purged_snaps scrub ok 2024-09-21T03:39:52.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:52 smithi123 ceph-mon[34673]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:52.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:52.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:52 smithi123 ceph-mon[34673]: osd.6 [v2:172.21.15.16:6818/3377807744,v1:172.21.15.16:6819/3377807744] boot 2024-09-21T03:39:52.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:52 smithi123 ceph-mon[34673]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T03:39:52.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:53.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.053+0000 7f64a6b95640 1 -- 172.21.15.123:0/2048837098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a0108370 msgr2=0x7f64a010a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:53.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.053+0000 7f64a6b95640 1 --2- 172.21.15.123:0/2048837098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a0108370 0x7f64a010a880 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6498009ca0 tx=0x7f649802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.053+0000 7f64a6b95640 1 -- 172.21.15.123:0/2048837098 shutdown_connections 2024-09-21T03:39:53.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.053+0000 7f64a6b95640 1 --2- 172.21.15.123:0/2048837098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a0108370 0x7f64a010a880 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.053+0000 7f64a6b95640 1 --2- 172.21.15.123:0/2048837098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64a0105750 0x7f64a0107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.053+0000 7f64a6b95640 1 --2- 172.21.15.123:0/2048837098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a0102c90 0x7f64a0105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.053+0000 7f64a6b95640 1 -- 172.21.15.123:0/2048837098 >> 172.21.15.123:0/2048837098 conn(0x7f64a00fd770 msgr2=0x7f64a00ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:53.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.053+0000 7f64a6b95640 1 -- 172.21.15.123:0/2048837098 shutdown_connections 2024-09-21T03:39:53.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.053+0000 7f64a6b95640 1 -- 172.21.15.123:0/2048837098 wait complete. 2024-09-21T03:39:53.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f64a6b95640 1 Processor -- start 2024-09-21T03:39:53.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f64a6b95640 1 -- start start 2024-09-21T03:39:53.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f64a6b95640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64a0102c90 0x7f64a01021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f64a6b95640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a0105750 0x7f64a019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f64a6b95640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a0108370 0x7f64a01a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f64a6b95640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64a010e2d0 con 0x7f64a0108370 2024-09-21T03:39:53.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f64a6b95640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64a010e150 con 0x7f64a0105750 2024-09-21T03:39:53.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f64a6b95640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64a010e450 con 0x7f64a0102c90 2024-09-21T03:39:53.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f649ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a0105750 0x7f64a019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f649ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a0105750 0x7f64a019ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48592/0 (socket says 172.21.15.123:48592) 2024-09-21T03:39:53.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f649ffff640 1 -- 172.21.15.123:0/2775860785 learned_addr learned my addr 172.21.15.123:0/2775860785 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-21T03:39:53.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f64a4931640 1 --2- 172.21.15.123:0/2775860785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64a0102c90 0x7f64a01021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f64a5132640 1 --2- 172.21.15.123:0/2775860785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a0108370 0x7f64a01a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f649ffff640 1 -- 172.21.15.123:0/2775860785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64a0102c90 msgr2=0x7f64a01021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:53.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f649ffff640 1 --2- 172.21.15.123:0/2775860785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64a0102c90 0x7f64a01021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f649ffff640 1 -- 172.21.15.123:0/2775860785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a0108370 msgr2=0x7f64a01a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:53.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f649ffff640 1 --2- 172.21.15.123:0/2775860785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a0108370 0x7f64a01a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.054+0000 7f649ffff640 1 -- 172.21.15.123:0/2775860785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a01a1a40 con 0x7f64a0105750 2024-09-21T03:39:53.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.055+0000 7f649ffff640 1 --2- 172.21.15.123:0/2775860785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a0105750 0x7f64a019ecf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f648c00c890 tx=0x7f648c00cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:53.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.055+0000 7f649dffb640 1 -- 172.21.15.123:0/2775860785 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c004220 con 0x7f64a0105750 2024-09-21T03:39:53.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.055+0000 7f64a6b95640 1 -- 172.21.15.123:0/2775860785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a01a1d30 con 0x7f64a0105750 2024-09-21T03:39:53.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.055+0000 7f649dffb640 1 -- 172.21.15.123:0/2775860785 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f648c0043c0 con 0x7f64a0105750 2024-09-21T03:39:53.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.055+0000 7f649dffb640 1 -- 172.21.15.123:0/2775860785 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c004a00 con 0x7f64a0105750 2024-09-21T03:39:53.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.055+0000 7f64a6b95640 1 -- 172.21.15.123:0/2775860785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a01a2280 con 0x7f64a0105750 2024-09-21T03:39:53.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.056+0000 7f64a6b95640 1 -- 172.21.15.123:0/2775860785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a00ff290 con 0x7f64a0105750 2024-09-21T03:39:53.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.058+0000 7f649dffb640 1 -- 172.21.15.123:0/2775860785 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f648c0079b0 con 0x7f64a0105750 2024-09-21T03:39:53.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.059+0000 7f649dffb640 1 --2- 172.21.15.123:0/2775860785 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64900b6510 0x7f64900b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.059+0000 7f649dffb640 1 -- 172.21.15.123:0/2775860785 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4965+0+0 (secure 0 0 0) 0x7f648c0e64a0 con 0x7f64a0105750 2024-09-21T03:39:53.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.059+0000 7f64a4931640 1 --2- 172.21.15.123:0/2775860785 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64900b6510 0x7f64900b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.060+0000 7f64a4931640 1 --2- 172.21.15.123:0/2775860785 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64900b6510 0x7f64900b8a20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6494009890 tx=0x7f6494008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:53.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.063+0000 7f649dffb640 1 -- 172.21.15.123:0/2775860785 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f648c014030 con 0x7f64a0105750 2024-09-21T03:39:53.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:52 smithi134 ceph-mon[35031]: purged_snaps scrub starts 2024-09-21T03:39:53.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:52 smithi134 ceph-mon[35031]: purged_snaps scrub ok 2024-09-21T03:39:53.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:52 smithi134 ceph-mon[35031]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:53.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:53.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:52 smithi134 ceph-mon[35031]: osd.6 [v2:172.21.15.16:6818/3377807744,v1:172.21.15.16:6819/3377807744] boot 2024-09-21T03:39:53.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:52 smithi134 ceph-mon[35031]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T03:39:53.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:53.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:52 smithi016 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T03:39:53.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:52 smithi016 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T03:39:53.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:52 smithi016 ceph-mon[30375]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:39:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:52 smithi016 ceph-mon[30375]: osd.6 [v2:172.21.15.16:6818/3377807744,v1:172.21.15.16:6819/3377807744] boot 2024-09-21T03:39:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:52 smithi016 ceph-mon[30375]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T03:39:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:39:53.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:39:53.258+0000 7f64a6b95640 1 -- 172.21.15.123:0/2775860785 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f64a0063800 con 0x7f64900b6510 2024-09-21T03:39:54.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:54 smithi134 ceph-mon[35031]: Detected new or changed devices on smithi016 2024-09-21T03:39:54.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:54.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:54.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:54.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:54.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:54.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:54 smithi134 ceph-mon[35031]: Adjusting osd_memory_target on smithi016 to 5698M 2024-09-21T03:39:54.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:54.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:54.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:54.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:54.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:54 smithi134 ceph-mon[35031]: from='client.24461 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:54.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:39:54.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:39:54.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:54.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:54 smithi134 ceph-mon[35031]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:39:54.616 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:54 smithi123 ceph-mon[34673]: Detected new or changed devices on smithi016 2024-09-21T03:39:54.616 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:54.617 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:54.617 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:54.617 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:54.617 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:54.617 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:54 smithi123 ceph-mon[34673]: Adjusting osd_memory_target on smithi016 to 5698M 2024-09-21T03:39:54.617 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:54.617 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:54.617 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:54.617 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:54.617 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:54 smithi123 ceph-mon[34673]: from='client.24461 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:54.617 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:39:54.617 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:39:54.617 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:54.618 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:54 smithi123 ceph-mon[34673]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:39:54.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:54 smithi016 ceph-mon[30375]: Detected new or changed devices on smithi016 2024-09-21T03:39:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:39:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:54 smithi016 ceph-mon[30375]: Adjusting osd_memory_target on smithi016 to 5698M 2024-09-21T03:39:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:39:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:54 smithi016 ceph-mon[30375]: from='client.24461 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:39:54.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:39:54.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:54.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:54 smithi016 ceph-mon[30375]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:39:55.564 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:55 smithi016 ceph-mon[30375]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T03:39:55.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:55 smithi134 ceph-mon[35031]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T03:39:55.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:55 smithi123 ceph-mon[34673]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T03:39:56.578 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:56 smithi016 ceph-mon[30375]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:39:56.578 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:56 smithi016 ceph-mon[30375]: from='client.? 172.21.15.123:0/519880378' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1541a272-c524-4f12-9b94-763f69bb1c8d"} : dispatch 2024-09-21T03:39:56.578 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:56 smithi016 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1541a272-c524-4f12-9b94-763f69bb1c8d"} : dispatch 2024-09-21T03:39:56.578 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:56 smithi016 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1541a272-c524-4f12-9b94-763f69bb1c8d"}]': finished 2024-09-21T03:39:56.579 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:56 smithi016 ceph-mon[30375]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T03:39:56.579 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:56 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:39:56.579 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:56 smithi016 ceph-mon[30375]: from='client.? 172.21.15.123:0/3451000873' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:39:56.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:56 smithi134 ceph-mon[35031]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:39:56.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:56 smithi134 ceph-mon[35031]: from='client.? 172.21.15.123:0/519880378' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1541a272-c524-4f12-9b94-763f69bb1c8d"} : dispatch 2024-09-21T03:39:56.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:56 smithi134 ceph-mon[35031]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1541a272-c524-4f12-9b94-763f69bb1c8d"} : dispatch 2024-09-21T03:39:56.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:56 smithi134 ceph-mon[35031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1541a272-c524-4f12-9b94-763f69bb1c8d"}]': finished 2024-09-21T03:39:56.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:56 smithi134 ceph-mon[35031]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T03:39:56.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:56 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:39:56.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:56 smithi134 ceph-mon[35031]: from='client.? 172.21.15.123:0/3451000873' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:39:56.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:56 smithi123 ceph-mon[34673]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:39:56.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:56 smithi123 ceph-mon[34673]: from='client.? 172.21.15.123:0/519880378' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1541a272-c524-4f12-9b94-763f69bb1c8d"} : dispatch 2024-09-21T03:39:56.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:56 smithi123 ceph-mon[34673]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1541a272-c524-4f12-9b94-763f69bb1c8d"} : dispatch 2024-09-21T03:39:56.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:56 smithi123 ceph-mon[34673]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1541a272-c524-4f12-9b94-763f69bb1c8d"}]': finished 2024-09-21T03:39:56.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:56 smithi123 ceph-mon[34673]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T03:39:56.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:56 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:39:56.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:56 smithi123 ceph-mon[34673]: from='client.? 172.21.15.123:0/3451000873' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:39:58.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:39:58 smithi134 ceph-mon[35031]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:39:58.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:39:58 smithi123 ceph-mon[34673]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:39:58.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:39:58 smithi016 ceph-mon[30375]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:00.509 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:00 smithi123 ceph-mon[34673]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:00.509 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:00 smithi123 ceph-mon[34673]: overall HEALTH_OK 2024-09-21T03:40:00.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:00 smithi134 ceph-mon[35031]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:00.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:00 smithi134 ceph-mon[35031]: overall HEALTH_OK 2024-09-21T03:40:00.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:00 smithi016 ceph-mon[30375]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:00.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:00 smithi016 ceph-mon[30375]: overall HEALTH_OK 2024-09-21T03:40:02.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:02 smithi123 ceph-mon[34673]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:02.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:02 smithi134 ceph-mon[35031]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:02.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:02 smithi016 ceph-mon[30375]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:04.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:04 smithi134 ceph-mon[35031]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:04.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:04 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:40:04.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:04 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:04.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:04 smithi134 ceph-mon[35031]: Deploying daemon osd.7 on smithi123 2024-09-21T03:40:04.642 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:04 smithi123 ceph-mon[34673]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:04.643 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:04 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:40:04.643 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:04 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:04.643 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:04 smithi123 ceph-mon[34673]: Deploying daemon osd.7 on smithi123 2024-09-21T03:40:04.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:04 smithi016 ceph-mon[30375]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:04.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:04 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:40:04.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:04 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:04.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:04 smithi016 ceph-mon[30375]: Deploying daemon osd.7 on smithi123 2024-09-21T03:40:06.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:06 smithi134 ceph-mon[35031]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:06.627 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:06 smithi016 ceph-mon[30375]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:06.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:06 smithi123 ceph-mon[34673]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:08.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:08 smithi123 ceph-mon[34673]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:08.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:08 smithi016 ceph-mon[30375]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:08.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:08 smithi134 ceph-mon[35031]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:10.626 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:10 smithi123 ceph-mon[34673]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:10.626 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:10 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:10.626 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:10 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:10.626 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:10 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:10.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:10 smithi016 ceph-mon[30375]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:10.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:10 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:10.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:10 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:10.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:10 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:10.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:10 smithi134 ceph-mon[35031]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:10.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:10 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:10.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:10 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:10.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:10 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:12.425 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:12 smithi123 ceph-mon[34673]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:12.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:12 smithi016 ceph-mon[30375]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:12.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:12 smithi134 ceph-mon[35031]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:13.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:40:13.859+0000 7f649dffb640 1 -- 172.21.15.123:0/2775860785 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f64a0063800 con 0x7f64900b6510 2024-09-21T03:40:13.862 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 7 on host 'smithi123' 2024-09-21T03:40:13.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:40:13.861+0000 7f64a6b95640 1 -- 172.21.15.123:0/2775860785 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64900b6510 msgr2=0x7f64900b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:40:13.861+0000 7f64a6b95640 1 --2- 172.21.15.123:0/2775860785 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64900b6510 0x7f64900b8a20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6494009890 tx=0x7f6494008040 comp rx=0 tx=0).stop 2024-09-21T03:40:13.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:40:13.862+0000 7f64a6b95640 1 -- 172.21.15.123:0/2775860785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a0105750 msgr2=0x7f64a019ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:13.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:40:13.862+0000 7f64a6b95640 1 --2- 172.21.15.123:0/2775860785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a0105750 0x7f64a019ecf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f648c00c890 tx=0x7f648c00cd50 comp rx=0 tx=0).stop 2024-09-21T03:40:13.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:40:13.862+0000 7f64a6b95640 1 -- 172.21.15.123:0/2775860785 shutdown_connections 2024-09-21T03:40:13.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:40:13.862+0000 7f64a6b95640 1 --2- 172.21.15.123:0/2775860785 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64900b6510 0x7f64900b8a20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:40:13.862+0000 7f64a6b95640 1 --2- 172.21.15.123:0/2775860785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64a0108370 0x7f64a01a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:40:13.862+0000 7f64a6b95640 1 --2- 172.21.15.123:0/2775860785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a0105750 0x7f64a019ecf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:40:13.862+0000 7f64a6b95640 1 --2- 172.21.15.123:0/2775860785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64a0102c90 0x7f64a01021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:13.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:40:13.862+0000 7f64a6b95640 1 -- 172.21.15.123:0/2775860785 >> 172.21.15.123:0/2775860785 conn(0x7f64a00fd770 msgr2=0x7f64a0103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:13.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:40:13.862+0000 7f64a6b95640 1 -- 172.21.15.123:0/2775860785 shutdown_connections 2024-09-21T03:40:13.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:40:13.862+0000 7f64a6b95640 1 -- 172.21.15.123:0/2775860785 wait complete. 2024-09-21T03:40:14.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:14.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:14.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:14.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:14.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:14.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:14.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:14.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:14.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:14.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:14.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:14.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:14.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:14.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:14.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:14.503 DEBUG:teuthology.orchestra.run.smithi123:osd.7> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T03:40:14.505 INFO:tasks.cephadm:Deploying osd.8 on smithi134 with /dev/vg_nvme/lv_2... 2024-09-21T03:40:14.505 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:40:14.763 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:40:14.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:14 smithi134 ceph-mon[35031]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:14.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:14 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:14.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:14 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:14.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:14 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:15.050 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:14 smithi123 ceph-mon[34673]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:15.051 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:14 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:15.051 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:14 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:15.051 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:14 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:15.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:14 smithi016 ceph-mon[30375]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:15.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:14 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:15.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:14 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:15.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:14 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:15.996 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:40:15 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:40:15.828+0000 7f73b855a640 -1 osd.7 0 log_to_monitors true 2024-09-21T03:40:17.053 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:16 smithi123 ceph-mon[34673]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:17.053 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:16 smithi123 ceph-mon[34673]: from='osd.7 [v2:172.21.15.123:6816/1463412739,v1:172.21.15.123:6817/1463412739]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:40:17.053 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:16 smithi123 ceph-mon[34673]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:40:17.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:16 smithi134 ceph-mon[35031]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:17.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:16 smithi134 ceph-mon[35031]: from='osd.7 [v2:172.21.15.123:6816/1463412739,v1:172.21.15.123:6817/1463412739]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:40:17.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:16 smithi134 ceph-mon[35031]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:40:17.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:16 smithi016 ceph-mon[30375]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:17.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:16 smithi016 ceph-mon[30375]: from='osd.7 [v2:172.21.15.123:6816/1463412739,v1:172.21.15.123:6817/1463412739]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:40:17.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:16 smithi016 ceph-mon[30375]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:40:17.229 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:40:17.254 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi134:vg_nvme/lv_2 2024-09-21T03:40:17.502 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:40:17.896 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:17 smithi134 ceph-mon[35031]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:40:17.896 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:17 smithi134 ceph-mon[35031]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:40:17.896 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:17 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:40:17.896 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:17 smithi134 ceph-mon[35031]: from='osd.7 [v2:172.21.15.123:6816/1463412739,v1:172.21.15.123:6817/1463412739]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:40:17.896 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:17 smithi134 ceph-mon[35031]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:40:17.896 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:17 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:17.896 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:17 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:17.896 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:17 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:17.896 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:17 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:17.896 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:17 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:17.896 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:17 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:17.897 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:17 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:17.897 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:17 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:17.897 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:17 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:18.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.986+0000 7eff856e3640 1 -- 172.21.15.134:0/786901907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff801036d0 msgr2=0x7eff80103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:18.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.986+0000 7eff856e3640 1 --2- 172.21.15.134:0/786901907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff801036d0 0x7eff80103ae0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7eff68009900 tx=0x7eff6802a480 comp rx=0 tx=0).stop 2024-09-21T03:40:18.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.986+0000 7eff856e3640 1 -- 172.21.15.134:0/786901907 shutdown_connections 2024-09-21T03:40:18.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.986+0000 7eff856e3640 1 --2- 172.21.15.134:0/786901907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff80104070 0x7eff8010aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.986+0000 7eff856e3640 1 --2- 172.21.15.134:0/786901907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff801036d0 0x7eff80103ae0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.986+0000 7eff856e3640 1 --2- 172.21.15.134:0/786901907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff80102640 0x7eff80102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.986+0000 7eff856e3640 1 -- 172.21.15.134:0/786901907 >> 172.21.15.134:0/786901907 conn(0x7eff800fdfb0 msgr2=0x7eff80100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:18.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.987+0000 7eff856e3640 1 -- 172.21.15.134:0/786901907 shutdown_connections 2024-09-21T03:40:18.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.987+0000 7eff856e3640 1 -- 172.21.15.134:0/786901907 wait complete. 2024-09-21T03:40:18.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.987+0000 7eff856e3640 1 Processor -- start 2024-09-21T03:40:18.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff856e3640 1 -- start start 2024-09-21T03:40:18.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff856e3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff80102640 0x7eff80198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:18.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff856e3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff801036d0 0x7eff80199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:18.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff856e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff80104070 0x7eff8019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:18.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff856e3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff8010e300 con 0x7eff80104070 2024-09-21T03:40:18.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff856e3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff8010e180 con 0x7eff801036d0 2024-09-21T03:40:18.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff856e3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff8010e480 con 0x7eff80102640 2024-09-21T03:40:18.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff7effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff80102640 0x7eff80198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:18.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff7effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff80102640 0x7eff80198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51772/0 (socket says 172.21.15.134:51772) 2024-09-21T03:40:18.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff7effd640 1 -- 172.21.15.134:0/898867623 learned_addr learned my addr 172.21.15.134:0/898867623 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-21T03:40:18.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff767fc640 1 --2- 172.21.15.134:0/898867623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff801036d0 0x7eff80199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:18.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff7f7fe640 1 --2- 172.21.15.134:0/898867623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff80104070 0x7eff8019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:18.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff7effd640 1 -- 172.21.15.134:0/898867623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff801036d0 msgr2=0x7eff80199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:18.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff7effd640 1 --2- 172.21.15.134:0/898867623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff801036d0 0x7eff80199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff7effd640 1 -- 172.21.15.134:0/898867623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff80104070 msgr2=0x7eff8019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:18.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff7effd640 1 --2- 172.21.15.134:0/898867623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff80104070 0x7eff8019a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:18.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff7effd640 1 -- 172.21.15.134:0/898867623 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8019a920 con 0x7eff80102640 2024-09-21T03:40:18.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.988+0000 7eff7effd640 1 --2- 172.21.15.134:0/898867623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff80102640 0x7eff80198ef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7eff7000b4a0 tx=0x7eff7000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:18.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.989+0000 7eff7cff9640 1 -- 172.21.15.134:0/898867623 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70014070 con 0x7eff80102640 2024-09-21T03:40:18.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.989+0000 7eff856e3640 1 -- 172.21.15.134:0/898867623 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8019ac10 con 0x7eff80102640 2024-09-21T03:40:18.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.989+0000 7eff7cff9640 1 -- 172.21.15.134:0/898867623 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff70010ac0 con 0x7eff80102640 2024-09-21T03:40:18.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.989+0000 7eff7cff9640 1 -- 172.21.15.134:0/898867623 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70017c30 con 0x7eff80102640 2024-09-21T03:40:18.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.989+0000 7eff856e3640 1 -- 172.21.15.134:0/898867623 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff801a1310 con 0x7eff80102640 2024-09-21T03:40:18.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.990+0000 7eff856e3640 1 -- 172.21.15.134:0/898867623 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff80109820 con 0x7eff80102640 2024-09-21T03:40:18.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.991+0000 7eff7cff9640 1 -- 172.21.15.134:0/898867623 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7eff70017070 con 0x7eff80102640 2024-09-21T03:40:18.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.992+0000 7eff7cff9640 1 --2- 172.21.15.134:0/898867623 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7eff6c0b6510 0x7eff6c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:18.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.992+0000 7eff7cff9640 1 -- 172.21.15.134:0/898867623 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5117+0+0 (secure 0 0 0) 0x7eff700e5c60 con 0x7eff80102640 2024-09-21T03:40:18.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.992+0000 7eff767fc640 1 --2- 172.21.15.134:0/898867623 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7eff6c0b6510 0x7eff6c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:18.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.993+0000 7eff767fc640 1 --2- 172.21.15.134:0/898867623 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7eff6c0b6510 0x7eff6c0b8a20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7eff68000c00 tx=0x7eff68031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:18.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:17.995+0000 7eff7cff9640 1 -- 172.21.15.134:0/898867623 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7eff700ad170 con 0x7eff80102640 2024-09-21T03:40:18.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:17 smithi123 ceph-mon[34673]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:40:18.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:17 smithi123 ceph-mon[34673]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:40:18.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:17 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:40:18.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:17 smithi123 ceph-mon[34673]: from='osd.7 [v2:172.21.15.123:6816/1463412739,v1:172.21.15.123:6817/1463412739]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:40:18.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:17 smithi123 ceph-mon[34673]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:40:18.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:17 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:18.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:17 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:18.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:17 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:18.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:17 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:18.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:17 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:18.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:17 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:18.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:17 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:18.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:17 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:18.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:17 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:18.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:17 smithi016 ceph-mon[30375]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:40:18.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:17 smithi016 ceph-mon[30375]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:40:18.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:17 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:40:18.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:17 smithi016 ceph-mon[30375]: from='osd.7 [v2:172.21.15.123:6816/1463412739,v1:172.21.15.123:6817/1463412739]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:40:18.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:17 smithi016 ceph-mon[30375]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:40:18.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:17 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:18.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:17 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:18.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:17 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:18.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:17 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:18.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:17 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:18.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:17 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:18.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:17 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:18.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:17 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:18.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:17 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:18.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:18.183+0000 7eff856e3640 1 -- 172.21.15.134:0/898867623 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7eff80063800 con 0x7eff6c0b6510 2024-09-21T03:40:18.795 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:40:18 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:40:18.465+0000 7f73b44dc640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T03:40:18.795 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:40:18 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:40:18.475+0000 7f73ac2d2640 -1 osd.7 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:40:18.812 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:18 smithi134 ceph-mon[35031]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:18.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:18 smithi134 ceph-mon[35031]: Detected new or changed devices on smithi123 2024-09-21T03:40:18.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:18 smithi134 ceph-mon[35031]: Adjusting osd_memory_target on smithi123 to 5696M 2024-09-21T03:40:18.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:18 smithi134 ceph-mon[35031]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T03:40:18.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:18 smithi134 ceph-mon[35031]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:40:18.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:18 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:40:18.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:18 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:40:18.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:18 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:40:18.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:18 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:40:18.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:18 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:18.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:18 smithi134 ceph-mon[35031]: from='osd.7 ' entity='osd.7' 2024-09-21T03:40:19.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:18 smithi123 ceph-mon[34673]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:19.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:18 smithi123 ceph-mon[34673]: Detected new or changed devices on smithi123 2024-09-21T03:40:19.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:18 smithi123 ceph-mon[34673]: Adjusting osd_memory_target on smithi123 to 5696M 2024-09-21T03:40:19.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:18 smithi123 ceph-mon[34673]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T03:40:19.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:18 smithi123 ceph-mon[34673]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:40:19.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:18 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:40:19.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:18 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:40:19.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:18 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:40:19.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:18 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:40:19.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:18 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:19.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:18 smithi123 ceph-mon[34673]: from='osd.7 ' entity='osd.7' 2024-09-21T03:40:19.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:18 smithi016 ceph-mon[30375]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:19.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:18 smithi016 ceph-mon[30375]: Detected new or changed devices on smithi123 2024-09-21T03:40:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:18 smithi016 ceph-mon[30375]: Adjusting osd_memory_target on smithi123 to 5696M 2024-09-21T03:40:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:18 smithi016 ceph-mon[30375]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T03:40:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:18 smithi016 ceph-mon[30375]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:40:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:18 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:40:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:18 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:40:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:18 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:40:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:18 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:40:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:18 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:18 smithi016 ceph-mon[30375]: from='osd.7 ' entity='osd.7' 2024-09-21T03:40:19.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:19 smithi134 ceph-mon[35031]: purged_snaps scrub starts 2024-09-21T03:40:19.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:19 smithi134 ceph-mon[35031]: purged_snaps scrub ok 2024-09-21T03:40:19.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:19 smithi134 ceph-mon[35031]: from='client.24481 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:40:19.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:40:19.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:19 smithi134 ceph-mon[35031]: osd.7 [v2:172.21.15.123:6816/1463412739,v1:172.21.15.123:6817/1463412739] boot 2024-09-21T03:40:19.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:19 smithi134 ceph-mon[35031]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T03:40:19.843 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:40:19.843 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:19 smithi134 ceph-mon[35031]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:40:20.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:19 smithi123 ceph-mon[34673]: purged_snaps scrub starts 2024-09-21T03:40:20.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:19 smithi123 ceph-mon[34673]: purged_snaps scrub ok 2024-09-21T03:40:20.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:19 smithi123 ceph-mon[34673]: from='client.24481 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:40:20.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:40:20.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:19 smithi123 ceph-mon[34673]: osd.7 [v2:172.21.15.123:6816/1463412739,v1:172.21.15.123:6817/1463412739] boot 2024-09-21T03:40:20.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:19 smithi123 ceph-mon[34673]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T03:40:20.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:40:20.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:19 smithi123 ceph-mon[34673]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:40:20.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:19 smithi016 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T03:40:20.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:19 smithi016 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T03:40:20.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:19 smithi016 ceph-mon[30375]: from='client.24481 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:40:20.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:40:20.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:19 smithi016 ceph-mon[30375]: osd.7 [v2:172.21.15.123:6816/1463412739,v1:172.21.15.123:6817/1463412739] boot 2024-09-21T03:40:20.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:19 smithi016 ceph-mon[30375]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T03:40:20.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:40:20.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:19 smithi016 ceph-mon[30375]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:40:21.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:20 smithi134 ceph-mon[35031]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:21.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:20 smithi134 ceph-mon[35031]: from='client.? 172.21.15.134:0/2181683343' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cc00014c-eb99-49ab-984c-0bc26ef85130"} : dispatch 2024-09-21T03:40:21.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:20 smithi134 ceph-mon[35031]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cc00014c-eb99-49ab-984c-0bc26ef85130"} : dispatch 2024-09-21T03:40:21.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:20 smithi134 ceph-mon[35031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc00014c-eb99-49ab-984c-0bc26ef85130"}]': finished 2024-09-21T03:40:21.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:20 smithi134 ceph-mon[35031]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T03:40:21.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:20 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:21.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:20 smithi123 ceph-mon[34673]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:21.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:20 smithi123 ceph-mon[34673]: from='client.? 172.21.15.134:0/2181683343' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cc00014c-eb99-49ab-984c-0bc26ef85130"} : dispatch 2024-09-21T03:40:21.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:20 smithi123 ceph-mon[34673]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cc00014c-eb99-49ab-984c-0bc26ef85130"} : dispatch 2024-09-21T03:40:21.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:20 smithi123 ceph-mon[34673]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc00014c-eb99-49ab-984c-0bc26ef85130"}]': finished 2024-09-21T03:40:21.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:20 smithi123 ceph-mon[34673]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T03:40:21.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:20 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:21.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:20 smithi016 ceph-mon[30375]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:40:21.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:20 smithi016 ceph-mon[30375]: from='client.? 172.21.15.134:0/2181683343' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cc00014c-eb99-49ab-984c-0bc26ef85130"} : dispatch 2024-09-21T03:40:21.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:20 smithi016 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cc00014c-eb99-49ab-984c-0bc26ef85130"} : dispatch 2024-09-21T03:40:21.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:20 smithi016 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc00014c-eb99-49ab-984c-0bc26ef85130"}]': finished 2024-09-21T03:40:21.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:20 smithi016 ceph-mon[30375]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T03:40:21.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:20 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:22.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:21 smithi134 ceph-mon[35031]: from='client.? 172.21.15.134:0/846269951' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:40:22.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:21 smithi123 ceph-mon[34673]: from='client.? 172.21.15.134:0/846269951' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:40:22.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:21 smithi016 ceph-mon[30375]: from='client.? 172.21.15.134:0/846269951' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:40:22.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:22 smithi134 ceph-mon[35031]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:23.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:22 smithi123 ceph-mon[34673]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:23.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:22 smithi016 ceph-mon[30375]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:25.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:24 smithi134 ceph-mon[35031]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:25.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:24 smithi123 ceph-mon[34673]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:25.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:24 smithi016 ceph-mon[30375]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:27.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:26 smithi134 ceph-mon[35031]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-21T03:40:27.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:26 smithi123 ceph-mon[34673]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-21T03:40:27.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:26 smithi016 ceph-mon[30375]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-21T03:40:29.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:28 smithi134 ceph-mon[35031]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:40:29.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:28 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:40:29.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:28 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:29.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:28 smithi123 ceph-mon[34673]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:40:29.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:28 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:40:29.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:28 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:29.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:28 smithi016 ceph-mon[30375]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:40:29.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:28 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:40:29.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:28 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:30.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:29 smithi134 ceph-mon[35031]: Deploying daemon osd.8 on smithi134 2024-09-21T03:40:30.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:29 smithi123 ceph-mon[34673]: Deploying daemon osd.8 on smithi134 2024-09-21T03:40:30.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:29 smithi016 ceph-mon[30375]: Deploying daemon osd.8 on smithi134 2024-09-21T03:40:30.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:30 smithi134 ceph-mon[35031]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:40:31.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:30 smithi123 ceph-mon[34673]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:40:31.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:30 smithi016 ceph-mon[30375]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:40:33.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:32 smithi134 ceph-mon[35031]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-21T03:40:33.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:32 smithi123 ceph-mon[34673]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-21T03:40:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:32 smithi016 ceph-mon[30375]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-21T03:40:35.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:34 smithi134 ceph-mon[35031]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:40:35.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:34 smithi123 ceph-mon[34673]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:40:35.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:34 smithi016 ceph-mon[30375]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:40:36.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:35 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:36.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:35 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:36.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:35 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:36.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:35 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:36.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:35 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:36.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:35 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:36.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:35 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:36.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:35 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:36.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:35 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:37.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:36 smithi134 ceph-mon[35031]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:40:37.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:36 smithi123 ceph-mon[34673]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:40:37.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:36 smithi016 ceph-mon[30375]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:40:38.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:38.888+0000 7eff7cff9640 1 -- 172.21.15.134:0/898867623 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7eff80063800 con 0x7eff6c0b6510 2024-09-21T03:40:38.890 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 8 on host 'smithi134' 2024-09-21T03:40:38.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:38.890+0000 7eff856e3640 1 -- 172.21.15.134:0/898867623 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7eff6c0b6510 msgr2=0x7eff6c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:38.890+0000 7eff856e3640 1 --2- 172.21.15.134:0/898867623 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7eff6c0b6510 0x7eff6c0b8a20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7eff68000c00 tx=0x7eff68031400 comp rx=0 tx=0).stop 2024-09-21T03:40:38.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:38.890+0000 7eff856e3640 1 -- 172.21.15.134:0/898867623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff80102640 msgr2=0x7eff80198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:38.890+0000 7eff856e3640 1 --2- 172.21.15.134:0/898867623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff80102640 0x7eff80198ef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7eff7000b4a0 tx=0x7eff7000b960 comp rx=0 tx=0).stop 2024-09-21T03:40:38.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:38.891+0000 7eff856e3640 1 -- 172.21.15.134:0/898867623 shutdown_connections 2024-09-21T03:40:38.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:38.891+0000 7eff856e3640 1 --2- 172.21.15.134:0/898867623 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7eff6c0b6510 0x7eff6c0b8a20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:38.891+0000 7eff856e3640 1 --2- 172.21.15.134:0/898867623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff80104070 0x7eff8019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:38.891+0000 7eff856e3640 1 --2- 172.21.15.134:0/898867623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff801036d0 0x7eff80199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:38.891+0000 7eff856e3640 1 --2- 172.21.15.134:0/898867623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff80102640 0x7eff80198ef0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:38.891+0000 7eff856e3640 1 -- 172.21.15.134:0/898867623 >> 172.21.15.134:0/898867623 conn(0x7eff800fdfb0 msgr2=0x7eff801069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:38.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:38.891+0000 7eff856e3640 1 -- 172.21.15.134:0/898867623 shutdown_connections 2024-09-21T03:40:38.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:40:38.891+0000 7eff856e3640 1 -- 172.21.15.134:0/898867623 wait complete. 2024-09-21T03:40:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:38 smithi134 ceph-mon[35031]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:38 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:38 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:38 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:38 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:38 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:39.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:38 smithi123 ceph-mon[34673]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:39.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:38 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:39.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:38 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:39.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:38 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:39.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:38 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:39.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:38 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:39.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:38 smithi016 ceph-mon[30375]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:39.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:38 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:39.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:38 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:39.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:38 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:39.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:38 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:39.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:38 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:39.524 DEBUG:teuthology.orchestra.run.smithi134:osd.8> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T03:40:39.526 INFO:tasks.cephadm:Deploying osd.9 on smithi016 with /dev/vg_nvme/lv_1... 2024-09-21T03:40:39.526 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:40:39.918 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:40.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:39 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:40.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:39 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:40.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:39 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:40.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:39 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:40.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:39 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:40.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:39 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:40.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:39 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:40.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:39 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:40.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:39 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:40.991 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:40 smithi134 ceph-mon[35031]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:41.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:40 smithi123 ceph-mon[34673]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:41.656 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:40:41 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:40:41.115+0000 7f59c88bc640 -1 osd.8 0 log_to_monitors true 2024-09-21T03:40:41.656 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:40 smithi016 ceph-mon[30375]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:41.949 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:41 smithi016 ceph-mon[30375]: from='osd.8 [v2:172.21.15.134:6816/2644021450,v1:172.21.15.134:6817/2644021450]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:40:41.949 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:41 smithi016 ceph-mon[30375]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:40:42.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:41 smithi134 ceph-mon[35031]: from='osd.8 [v2:172.21.15.134:6816/2644021450,v1:172.21.15.134:6817/2644021450]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:40:42.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:41 smithi134 ceph-mon[35031]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:40:42.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:41 smithi123 ceph-mon[34673]: from='osd.8 [v2:172.21.15.134:6816/2644021450,v1:172.21.15.134:6817/2644021450]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:40:42.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:41 smithi123 ceph-mon[34673]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:40:42.922 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:42 smithi123 ceph-mon[34673]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:42.922 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:42 smithi123 ceph-mon[34673]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:40:42.922 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:42 smithi123 ceph-mon[34673]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:40:42.922 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:42 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:42.922 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:42 smithi123 ceph-mon[34673]: from='osd.8 [v2:172.21.15.134:6816/2644021450,v1:172.21.15.134:6817/2644021450]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:40:42.922 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:42 smithi123 ceph-mon[34673]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:40:42.922 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:42 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:42.923 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:42 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:42.923 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:42 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:42.923 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:42 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:42.923 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:42 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:42.923 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:42 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:42.923 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:42 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:42.923 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:42 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:42.923 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:42 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:42 smithi016 ceph-mon[30375]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:42 smithi016 ceph-mon[30375]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:40:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:42 smithi016 ceph-mon[30375]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:40:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:42 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:42 smithi016 ceph-mon[30375]: from='osd.8 [v2:172.21.15.134:6816/2644021450,v1:172.21.15.134:6817/2644021450]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:40:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:42 smithi016 ceph-mon[30375]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:40:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:42 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:42 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:42 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:43.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:42 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:43.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:42 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:43.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:42 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:43.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:42 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:43.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:42 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:43.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:42 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:43.352 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:40:42 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:40:42.912+0000 7f59c483e640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T03:40:43.352 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:40:42 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:40:42.921+0000 7f59c063c640 -1 osd.8 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:40:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:42 smithi134 ceph-mon[35031]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:42 smithi134 ceph-mon[35031]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:40:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:42 smithi134 ceph-mon[35031]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:40:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:42 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:42 smithi134 ceph-mon[35031]: from='osd.8 [v2:172.21.15.134:6816/2644021450,v1:172.21.15.134:6817/2644021450]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:40:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:42 smithi134 ceph-mon[35031]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:40:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:42 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:42 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:42 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:42 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:42 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:40:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:42 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:42 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:42 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:42 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:40:43.364 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:40:43.399 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi016:vg_nvme/lv_1 2024-09-21T03:40:43.791 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:44.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:43 smithi123 ceph-mon[34673]: Detected new or changed devices on smithi134 2024-09-21T03:40:44.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:43 smithi123 ceph-mon[34673]: Adjusting osd_memory_target on smithi134 to 5696M 2024-09-21T03:40:44.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:43 smithi123 ceph-mon[34673]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-21T03:40:44.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:43 smithi123 ceph-mon[34673]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:40:44.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:43 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:44.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:43 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:44.158 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:43 smithi016 ceph-mon[30375]: Detected new or changed devices on smithi134 2024-09-21T03:40:44.158 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:43 smithi016 ceph-mon[30375]: Adjusting osd_memory_target on smithi134 to 5696M 2024-09-21T03:40:44.158 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:43 smithi016 ceph-mon[30375]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-21T03:40:44.158 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:43 smithi016 ceph-mon[30375]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:40:44.158 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:43 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:44.158 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:43 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:43 smithi134 ceph-mon[35031]: Detected new or changed devices on smithi134 2024-09-21T03:40:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:43 smithi134 ceph-mon[35031]: Adjusting osd_memory_target on smithi134 to 5696M 2024-09-21T03:40:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:43 smithi134 ceph-mon[35031]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-21T03:40:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:43 smithi134 ceph-mon[35031]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:40:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:43 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:43 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.484+0000 7fcea4200640 1 -- 172.21.15.16:0/4248357645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce9c10ae20 msgr2=0x7fce9c10b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.484+0000 7fcea4200640 1 --2- 172.21.15.16:0/4248357645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce9c10ae20 0x7fce9c10b230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce88009900 tx=0x7fce8802a480 comp rx=0 tx=0).stop 2024-09-21T03:40:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.484+0000 7fcea4200640 1 -- 172.21.15.16:0/4248357645 shutdown_connections 2024-09-21T03:40:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.484+0000 7fcea4200640 1 --2- 172.21.15.16:0/4248357645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce9c10b7c0 0x7fce9c112160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.484+0000 7fcea4200640 1 --2- 172.21.15.16:0/4248357645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce9c10ae20 0x7fce9c10b230 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.484+0000 7fcea4200640 1 --2- 172.21.15.16:0/4248357645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c109d90 0x7fce9c10a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.484+0000 7fcea4200640 1 -- 172.21.15.16:0/4248357645 >> 172.21.15.16:0/4248357645 conn(0x7fce9c079120 msgr2=0x7fce9c1081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:44.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.484+0000 7fcea4200640 1 -- 172.21.15.16:0/4248357645 shutdown_connections 2024-09-21T03:40:44.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.484+0000 7fcea4200640 1 -- 172.21.15.16:0/4248357645 wait complete. 2024-09-21T03:40:44.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.485+0000 7fcea4200640 1 Processor -- start 2024-09-21T03:40:44.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.486+0000 7fcea4200640 1 -- start start 2024-09-21T03:40:44.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.486+0000 7fcea4200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c109d90 0x7fce9c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:44.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.486+0000 7fcea4200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce9c10ae20 0x7fce9c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:44.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.486+0000 7fcea4200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce9c10b7c0 0x7fce9c1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:44.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.486+0000 7fcea4200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c1159e0 con 0x7fce9c10b7c0 2024-09-21T03:40:44.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.486+0000 7fcea4200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c115860 con 0x7fce9c10ae20 2024-09-21T03:40:44.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.486+0000 7fcea4200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c115b60 con 0x7fce9c109d90 2024-09-21T03:40:44.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.486+0000 7fcea0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce9c10ae20 0x7fce9c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:44.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.486+0000 7fcea0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce9c10ae20 0x7fce9c1a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:45844/0 (socket says 172.21.15.16:45844) 2024-09-21T03:40:44.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.486+0000 7fcea0e00640 1 -- 172.21.15.16:0/4154200124 learned_addr learned my addr 172.21.15.16:0/4154200124 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:40:44.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.486+0000 7fcea2200640 1 --2- 172.21.15.16:0/4154200124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce9c10b7c0 0x7fce9c1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:44.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.486+0000 7fcea1800640 1 --2- 172.21.15.16:0/4154200124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c109d90 0x7fce9c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:44.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.487+0000 7fcea0e00640 1 -- 172.21.15.16:0/4154200124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c109d90 msgr2=0x7fce9c1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:44.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.487+0000 7fcea0e00640 1 --2- 172.21.15.16:0/4154200124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c109d90 0x7fce9c1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.487+0000 7fcea0e00640 1 -- 172.21.15.16:0/4154200124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce9c10b7c0 msgr2=0x7fce9c1a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.487+0000 7fcea0e00640 1 --2- 172.21.15.16:0/4154200124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce9c10b7c0 0x7fce9c1a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.487+0000 7fcea0e00640 1 -- 172.21.15.16:0/4154200124 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9c1a2070 con 0x7fce9c10ae20 2024-09-21T03:40:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.487+0000 7fcea0e00640 1 --2- 172.21.15.16:0/4154200124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce9c10ae20 0x7fce9c1a0cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fce88006010 tx=0x7fce880042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.487+0000 7fce9a000640 1 -- 172.21.15.16:0/4154200124 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8803f030 con 0x7fce9c10ae20 2024-09-21T03:40:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.487+0000 7fce9a000640 1 -- 172.21.15.16:0/4154200124 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce88031910 con 0x7fce9c10ae20 2024-09-21T03:40:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.487+0000 7fce9a000640 1 -- 172.21.15.16:0/4154200124 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88035070 con 0x7fce9c10ae20 2024-09-21T03:40:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.487+0000 7fcea4200640 1 -- 172.21.15.16:0/4154200124 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9c1a2300 con 0x7fce9c10ae20 2024-09-21T03:40:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.488+0000 7fcea4200640 1 -- 172.21.15.16:0/4154200124 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce9c1a8a60 con 0x7fce9c10ae20 2024-09-21T03:40:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.489+0000 7fcea4200640 1 -- 172.21.15.16:0/4154200124 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce9c110f70 con 0x7fce9c10ae20 2024-09-21T03:40:44.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.491+0000 7fce9a000640 1 -- 172.21.15.16:0/4154200124 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fce88042030 con 0x7fce9c10ae20 2024-09-21T03:40:44.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.492+0000 7fce9a000640 1 --2- 172.21.15.16:0/4154200124 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fce840b6590 0x7fce840b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:44.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.493+0000 7fce9a000640 1 -- 172.21.15.16:0/4154200124 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5829+0+0 (secure 0 0 0) 0x7fce88107150 con 0x7fce9c10ae20 2024-09-21T03:40:44.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.493+0000 7fcea1800640 1 --2- 172.21.15.16:0/4154200124 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fce840b6590 0x7fce840b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:44.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.493+0000 7fcea1800640 1 --2- 172.21.15.16:0/4154200124 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fce840b6590 0x7fce840b8aa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fce90009bd0 tx=0x7fce90005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.494+0000 7fce9a000640 1 -- 172.21.15.16:0/4154200124 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fce880ce370 con 0x7fce9c10ae20 2024-09-21T03:40:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:40:44.702+0000 7fcea4200640 1 -- 172.21.15.16:0/4154200124 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fce9c063800 con 0x7fce840b6590 2024-09-21T03:40:45.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:44 smithi016 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T03:40:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:44 smithi016 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T03:40:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:44 smithi016 ceph-mon[30375]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:44 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:44 smithi016 ceph-mon[30375]: osd.8 [v2:172.21.15.134:6816/2644021450,v1:172.21.15.134:6817/2644021450] boot 2024-09-21T03:40:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:44 smithi016 ceph-mon[30375]: osdmap e51: 9 total, 9 up, 9 in 2024-09-21T03:40:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:44 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:44 smithi016 ceph-mon[30375]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T03:40:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:44 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:40:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:44 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:40:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:44 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:45.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:44 smithi134 ceph-mon[35031]: purged_snaps scrub starts 2024-09-21T03:40:45.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:44 smithi134 ceph-mon[35031]: purged_snaps scrub ok 2024-09-21T03:40:45.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:44 smithi134 ceph-mon[35031]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:45.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:44 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:45.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:44 smithi134 ceph-mon[35031]: osd.8 [v2:172.21.15.134:6816/2644021450,v1:172.21.15.134:6817/2644021450] boot 2024-09-21T03:40:45.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:44 smithi134 ceph-mon[35031]: osdmap e51: 9 total, 9 up, 9 in 2024-09-21T03:40:45.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:44 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:45.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:44 smithi134 ceph-mon[35031]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T03:40:45.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:44 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:40:45.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:44 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:40:45.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:44 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:45.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:44 smithi123 ceph-mon[34673]: purged_snaps scrub starts 2024-09-21T03:40:45.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:44 smithi123 ceph-mon[34673]: purged_snaps scrub ok 2024-09-21T03:40:45.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:44 smithi123 ceph-mon[34673]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:40:45.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:44 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:45.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:44 smithi123 ceph-mon[34673]: osd.8 [v2:172.21.15.134:6816/2644021450,v1:172.21.15.134:6817/2644021450] boot 2024-09-21T03:40:45.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:44 smithi123 ceph-mon[34673]: osdmap e51: 9 total, 9 up, 9 in 2024-09-21T03:40:45.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:44 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:40:45.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:44 smithi123 ceph-mon[34673]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T03:40:45.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:44 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:40:45.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:44 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:40:45.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:44 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:46.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:45 smithi016 ceph-mon[30375]: from='client.24512 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:40:46.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:45 smithi016 ceph-mon[30375]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:40:46.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:45 smithi134 ceph-mon[35031]: from='client.24512 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:40:46.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:45 smithi134 ceph-mon[35031]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:40:46.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:45 smithi123 ceph-mon[34673]: from='client.24512 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:40:46.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:45 smithi123 ceph-mon[34673]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:40:47.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:46 smithi016 ceph-mon[30375]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:40:47.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:46 smithi134 ceph-mon[35031]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:40:47.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:46 smithi123 ceph-mon[34673]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:40:49.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:48 smithi016 ceph-mon[30375]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:40:49.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:48 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3105354698' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3cbfff74-31ff-4ffd-844b-54ec8563ba32"} : dispatch 2024-09-21T03:40:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:48 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3105354698' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cbfff74-31ff-4ffd-844b-54ec8563ba32"}]': finished 2024-09-21T03:40:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:48 smithi016 ceph-mon[30375]: osdmap e54: 10 total, 9 up, 10 in 2024-09-21T03:40:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:48 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:40:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:48 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3602767991' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:40:49.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:48 smithi134 ceph-mon[35031]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:40:49.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:48 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3105354698' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3cbfff74-31ff-4ffd-844b-54ec8563ba32"} : dispatch 2024-09-21T03:40:49.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:48 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3105354698' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cbfff74-31ff-4ffd-844b-54ec8563ba32"}]': finished 2024-09-21T03:40:49.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:48 smithi134 ceph-mon[35031]: osdmap e54: 10 total, 9 up, 10 in 2024-09-21T03:40:49.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:48 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:40:49.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:48 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3602767991' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:40:49.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:48 smithi123 ceph-mon[34673]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:40:49.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:48 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3105354698' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3cbfff74-31ff-4ffd-844b-54ec8563ba32"} : dispatch 2024-09-21T03:40:49.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:48 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3105354698' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cbfff74-31ff-4ffd-844b-54ec8563ba32"}]': finished 2024-09-21T03:40:49.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:48 smithi123 ceph-mon[34673]: osdmap e54: 10 total, 9 up, 10 in 2024-09-21T03:40:49.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:48 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:40:49.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:48 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3602767991' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:40:51.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:50 smithi134 ceph-mon[35031]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:40:51.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:50 smithi123 ceph-mon[34673]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:40:51.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:50 smithi016 ceph-mon[30375]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:40:53.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:52 smithi134 ceph-mon[35031]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-21T03:40:53.391 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:52 smithi123 ceph-mon[34673]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-21T03:40:53.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:52 smithi016 ceph-mon[30375]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-21T03:40:55.116 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:54 smithi016 ceph-mon[30375]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:40:55.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:54 smithi134 ceph-mon[35031]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:40:55.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:54 smithi123 ceph-mon[34673]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:40:57.271 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:56 smithi016 ceph-mon[30375]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:40:57.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:56 smithi134 ceph-mon[35031]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:40:57.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:56 smithi123 ceph-mon[34673]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:40:58.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:40:58.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:58.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:40:58.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:58.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:40:58.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:59.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:58 smithi016 ceph-mon[30375]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:40:59.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:40:58 smithi016 ceph-mon[30375]: Deploying daemon osd.9 on smithi016 2024-09-21T03:40:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:58 smithi134 ceph-mon[35031]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:40:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:40:58 smithi134 ceph-mon[35031]: Deploying daemon osd.9 on smithi016 2024-09-21T03:40:59.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:58 smithi123 ceph-mon[34673]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:40:59.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:40:58 smithi123 ceph-mon[34673]: Deploying daemon osd.9 on smithi016 2024-09-21T03:41:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:00 smithi016 ceph-mon[30375]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:41:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:00 smithi134 ceph-mon[35031]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:41:01.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:00 smithi123 ceph-mon[34673]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:41:03.024 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:02 smithi016 ceph-mon[30375]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:41:03.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:02 smithi134 ceph-mon[35031]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:41:03.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:02 smithi123 ceph-mon[34673]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:41:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:04 smithi134 ceph-mon[35031]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:05.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:04 smithi123 ceph-mon[34673]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:05.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:04 smithi016 ceph-mon[30375]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:07.151 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:06 smithi016 ceph-mon[30375]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:07.151 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:06 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:07.151 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:06 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:07.151 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:06 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:06 smithi134 ceph-mon[35031]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:06 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:06 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:06 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:07.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:06 smithi123 ceph-mon[34673]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:07.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:06 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:07.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:06 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:07.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:06 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:09.271 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:08 smithi016 ceph-mon[30375]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:09.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:08 smithi134 ceph-mon[35031]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:09.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:08 smithi123 ceph-mon[34673]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:10.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:10 smithi016 ceph-mon[30375]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:10.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:10 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:10.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:10 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:10.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:10 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:10.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:10 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:10.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:10 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:10.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:10 smithi134 ceph-mon[35031]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:10.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:10 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:10.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:10 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:10.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:10 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:10.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:10 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:10.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:10 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:10.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:10 smithi123 ceph-mon[34673]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:10.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:10 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:10.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:10 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:10.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:10 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:10.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:10 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:10.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:10 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:11.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:41:11.325+0000 7fce9a000640 1 -- 172.21.15.16:0/4154200124 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fce9c063800 con 0x7fce840b6590 2024-09-21T03:41:11.328 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 9 on host 'smithi016' 2024-09-21T03:41:11.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:41:11.327+0000 7fcea4200640 1 -- 172.21.15.16:0/4154200124 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fce840b6590 msgr2=0x7fce840b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:41:11.327+0000 7fcea4200640 1 --2- 172.21.15.16:0/4154200124 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fce840b6590 0x7fce840b8aa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fce90009bd0 tx=0x7fce90005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:41:11.327+0000 7fcea4200640 1 -- 172.21.15.16:0/4154200124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce9c10ae20 msgr2=0x7fce9c1a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:41:11.327+0000 7fcea4200640 1 --2- 172.21.15.16:0/4154200124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce9c10ae20 0x7fce9c1a0cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fce88006010 tx=0x7fce880042f0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:41:11.328+0000 7fcea4200640 1 -- 172.21.15.16:0/4154200124 shutdown_connections 2024-09-21T03:41:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:41:11.328+0000 7fcea4200640 1 --2- 172.21.15.16:0/4154200124 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fce840b6590 0x7fce840b8aa0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:41:11.328+0000 7fcea4200640 1 --2- 172.21.15.16:0/4154200124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce9c10b7c0 0x7fce9c1a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:41:11.328+0000 7fcea4200640 1 --2- 172.21.15.16:0/4154200124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce9c10ae20 0x7fce9c1a0cb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:41:11.328+0000 7fcea4200640 1 --2- 172.21.15.16:0/4154200124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c109d90 0x7fce9c1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:41:11.328+0000 7fcea4200640 1 -- 172.21.15.16:0/4154200124 >> 172.21.15.16:0/4154200124 conn(0x7fce9c079120 msgr2=0x7fce9c10e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:41:11.328+0000 7fcea4200640 1 -- 172.21.15.16:0/4154200124 shutdown_connections 2024-09-21T03:41:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:41:11.328+0000 7fcea4200640 1 -- 172.21.15.16:0/4154200124 wait complete. 2024-09-21T03:41:11.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:11 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:11.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:11 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:11.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:11 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:11.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:11 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:11.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:11 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:11.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:11 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:12.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:11 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:12.196 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:11 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:12.196 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:11 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:12.218 DEBUG:teuthology.orchestra.run.smithi016:osd.9> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T03:41:12.223 INFO:tasks.cephadm:Deploying osd.10 on smithi123 with /dev/vg_nvme/lv_1... 2024-09-21T03:41:12.223 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:41:12.480 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:41:12.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:12 smithi016 ceph-mon[30375]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:12.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:12 smithi016 ceph-mon[30375]: from='osd.9 [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:41:12.867 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:12 smithi123 ceph-mon[34673]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:12.867 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:12 smithi123 ceph-mon[34673]: from='osd.9 [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:41:13.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:12 smithi134 ceph-mon[35031]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:13.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:12 smithi134 ceph-mon[35031]: from='osd.9 [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:41:13.887 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:13 smithi123 ceph-mon[34673]: from='osd.9 [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:41:13.887 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:13 smithi123 ceph-mon[34673]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T03:41:13.887 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:13 smithi123 ceph-mon[34673]: from='osd.9 [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:41:13.887 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:13 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:41:13.887 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:13 smithi123 ceph-mon[34673]: from='osd.9 [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T03:41:13.888 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:13 smithi123 ceph-mon[34673]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T03:41:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:13 smithi016 ceph-mon[30375]: from='osd.9 [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:41:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:13 smithi016 ceph-mon[30375]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T03:41:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:13 smithi016 ceph-mon[30375]: from='osd.9 [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:41:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:13 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:41:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:13 smithi016 ceph-mon[30375]: from='osd.9 [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T03:41:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:13 smithi016 ceph-mon[30375]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T03:41:13.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:41:13 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:41:13.626+0000 7fe940800640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T03:41:13.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:41:13 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:41:13.646+0000 7fe93b800640 -1 osd.9 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:41:14.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:13 smithi134 ceph-mon[35031]: from='osd.9 [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:41:14.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:13 smithi134 ceph-mon[35031]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T03:41:14.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:13 smithi134 ceph-mon[35031]: from='osd.9 [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T03:41:14.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:13 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:41:14.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:13 smithi134 ceph-mon[35031]: from='osd.9 [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T03:41:14.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:13 smithi134 ceph-mon[35031]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T03:41:14.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:14 smithi123 ceph-mon[34673]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:14.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:14 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:41:14.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:14 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:41:14.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:14 smithi016 ceph-mon[30375]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:14.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:14 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:41:14.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:14 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:41:15.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:14 smithi134 ceph-mon[35031]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:15.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:14 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:41:15.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:14 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:41:15.170 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T03:41:15.196 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi123:vg_nvme/lv_1 2024-09-21T03:41:15.440 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:41:15.846 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:15 smithi123 ceph-mon[34673]: purged_snaps scrub starts 2024-09-21T03:41:15.846 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:15 smithi123 ceph-mon[34673]: purged_snaps scrub ok 2024-09-21T03:41:15.846 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:15 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:41:15.846 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:15 smithi123 ceph-mon[34673]: osd.9 [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371] boot 2024-09-21T03:41:15.846 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:15 smithi123 ceph-mon[34673]: osdmap e57: 10 total, 10 up, 10 in 2024-09-21T03:41:15.846 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:15 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:41:15.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:15 smithi016 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T03:41:15.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:15 smithi016 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T03:41:15.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:15 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:41:15.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:15 smithi016 ceph-mon[30375]: osd.9 [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371] boot 2024-09-21T03:41:15.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:15 smithi016 ceph-mon[30375]: osdmap e57: 10 total, 10 up, 10 in 2024-09-21T03:41:15.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:15 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:41:15.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.937+0000 7f38e142f640 1 -- 172.21.15.123:0/2049005474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38dc1040a0 msgr2=0x7f38dc10aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.937+0000 7f38e142f640 1 --2- 172.21.15.123:0/2049005474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38dc1040a0 0x7f38dc10aa40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009ca0 tx=0x7f38d402a950 comp rx=0 tx=0).stop 2024-09-21T03:41:15.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.937+0000 7f38e142f640 1 -- 172.21.15.123:0/2049005474 shutdown_connections 2024-09-21T03:41:15.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.937+0000 7f38e142f640 1 --2- 172.21.15.123:0/2049005474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38dc1040a0 0x7f38dc10aa40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.937+0000 7f38e142f640 1 --2- 172.21.15.123:0/2049005474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38dc103700 0x7f38dc103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.937+0000 7f38e142f640 1 --2- 172.21.15.123:0/2049005474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc102670 0x7f38dc102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.937+0000 7f38e142f640 1 -- 172.21.15.123:0/2049005474 >> 172.21.15.123:0/2049005474 conn(0x7f38dc0fe000 msgr2=0x7f38dc100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:15.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.937+0000 7f38e142f640 1 -- 172.21.15.123:0/2049005474 shutdown_connections 2024-09-21T03:41:15.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.937+0000 7f38e142f640 1 -- 172.21.15.123:0/2049005474 wait complete. 2024-09-21T03:41:15.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.938+0000 7f38e142f640 1 Processor -- start 2024-09-21T03:41:15.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.938+0000 7f38e142f640 1 -- start start 2024-09-21T03:41:15.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.938+0000 7f38e142f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38dc102670 0x7f38dc06c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.938+0000 7f38e142f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc103700 0x7f38dc19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.938+0000 7f38e142f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38dc1040a0 0x7f38dc1a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.938+0000 7f38e142f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc10e2c0 con 0x7f38dc103700 2024-09-21T03:41:15.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.938+0000 7f38e142f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc10e140 con 0x7f38dc102670 2024-09-21T03:41:15.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.938+0000 7f38e142f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc10e440 con 0x7f38dc1040a0 2024-09-21T03:41:15.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.938+0000 7f38daffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38dc102670 0x7f38dc06c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.938+0000 7f38daffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38dc102670 0x7f38dc06c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42374/0 (socket says 172.21.15.123:42374) 2024-09-21T03:41:15.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.938+0000 7f38daffd640 1 -- 172.21.15.123:0/2934860692 learned_addr learned my addr 172.21.15.123:0/2934860692 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-21T03:41:15.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.938+0000 7f38d27fc640 1 --2- 172.21.15.123:0/2934860692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc103700 0x7f38dc19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.938+0000 7f38db7fe640 1 --2- 172.21.15.123:0/2934860692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38dc1040a0 0x7f38dc1a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.939+0000 7f38daffd640 1 -- 172.21.15.123:0/2934860692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38dc1040a0 msgr2=0x7f38dc1a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.939+0000 7f38daffd640 1 --2- 172.21.15.123:0/2934860692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38dc1040a0 0x7f38dc1a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.939+0000 7f38daffd640 1 -- 172.21.15.123:0/2934860692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc103700 msgr2=0x7f38dc19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:15.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.939+0000 7f38daffd640 1 --2- 172.21.15.123:0/2934860692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc103700 0x7f38dc19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:15.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.939+0000 7f38daffd640 1 -- 172.21.15.123:0/2934860692 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38dc1a1a30 con 0x7f38dc102670 2024-09-21T03:41:15.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.939+0000 7f38daffd640 1 --2- 172.21.15.123:0/2934860692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38dc102670 0x7f38dc06c320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00d950 tx=0x7f38cc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:15.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.939+0000 7f38d8ff9640 1 -- 172.21.15.123:0/2934860692 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc017070 con 0x7f38dc102670 2024-09-21T03:41:15.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.939+0000 7f38d8ff9640 1 -- 172.21.15.123:0/2934860692 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38cc009c30 con 0x7f38dc102670 2024-09-21T03:41:15.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.939+0000 7f38e142f640 1 -- 172.21.15.123:0/2934860692 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38dc1a1d20 con 0x7f38dc102670 2024-09-21T03:41:15.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.939+0000 7f38d8ff9640 1 -- 172.21.15.123:0/2934860692 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc00f620 con 0x7f38dc102670 2024-09-21T03:41:15.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.940+0000 7f38e142f640 1 -- 172.21.15.123:0/2934860692 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38dc1a21b0 con 0x7f38dc102670 2024-09-21T03:41:15.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.940+0000 7f38e142f640 1 -- 172.21.15.123:0/2934860692 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38dc109830 con 0x7f38dc102670 2024-09-21T03:41:15.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.942+0000 7f38d8ff9640 1 -- 172.21.15.123:0/2934860692 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f38cc010460 con 0x7f38dc102670 2024-09-21T03:41:15.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.942+0000 7f38d8ff9640 1 --2- 172.21.15.123:0/2934860692 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38c80b6510 0x7f38c80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:15.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.943+0000 7f38d8ff9640 1 -- 172.21.15.123:0/2934860692 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6261+0+0 (secure 0 0 0) 0x7f38cc0e67f0 con 0x7f38dc102670 2024-09-21T03:41:15.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.943+0000 7f38d27fc640 1 --2- 172.21.15.123:0/2934860692 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38c80b6510 0x7f38c80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:15.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.943+0000 7f38d27fc640 1 --2- 172.21.15.123:0/2934860692 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38c80b6510 0x7f38c80b8a20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f38c4005fd0 tx=0x7f38c4005e20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:15.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:15.947+0000 7f38d8ff9640 1 -- 172.21.15.123:0/2934860692 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f38cc0ad880 con 0x7f38dc102670 2024-09-21T03:41:16.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:15 smithi134 ceph-mon[35031]: purged_snaps scrub starts 2024-09-21T03:41:16.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:15 smithi134 ceph-mon[35031]: purged_snaps scrub ok 2024-09-21T03:41:16.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:15 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:41:16.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:15 smithi134 ceph-mon[35031]: osd.9 [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371] boot 2024-09-21T03:41:16.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:15 smithi134 ceph-mon[35031]: osdmap e57: 10 total, 10 up, 10 in 2024-09-21T03:41:16.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:15 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:41:16.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:16.134+0000 7f38e142f640 1 -- 172.21.15.123:0/2934860692 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f38dc063800 con 0x7f38c80b6510 2024-09-21T03:41:16.858 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:16 smithi123 ceph-mon[34673]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:16.858 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:16 smithi123 ceph-mon[34673]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T03:41:16.858 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:41:16.858 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:41:16.859 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:16.859 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:16.859 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:16.859 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:16.859 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:16.859 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:16.859 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:16.859 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:16.859 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:16.859 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:16.859 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:16 smithi016 ceph-mon[30375]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:16 smithi016 ceph-mon[30375]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T03:41:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:41:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:41:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:16.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:16.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:17.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:16 smithi134 ceph-mon[35031]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:41:17.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:16 smithi134 ceph-mon[35031]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T03:41:17.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:41:17.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:41:17.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:17.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:17.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:17.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:17.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:17.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:17.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:17.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:17.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:17.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:17.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:17.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:17 smithi123 ceph-mon[34673]: from='client.24539 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:41:17.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:17 smithi123 ceph-mon[34673]: Detected new or changed devices on smithi016 2024-09-21T03:41:17.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:17 smithi123 ceph-mon[34673]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-21T03:41:17.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:17 smithi016 ceph-mon[30375]: from='client.24539 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:41:17.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:17 smithi016 ceph-mon[30375]: Detected new or changed devices on smithi016 2024-09-21T03:41:17.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:17 smithi016 ceph-mon[30375]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-21T03:41:18.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:17 smithi134 ceph-mon[35031]: from='client.24539 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:41:18.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:17 smithi134 ceph-mon[35031]: Detected new or changed devices on smithi016 2024-09-21T03:41:18.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:17 smithi134 ceph-mon[35031]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-21T03:41:18.896 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:18 smithi123 ceph-mon[34673]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:18.896 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:18 smithi123 ceph-mon[34673]: from='client.? 172.21.15.123:0/784155923' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0a1caaad-70cd-4c2c-88d3-e9a5cc885655"} : dispatch 2024-09-21T03:41:18.896 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:18 smithi123 ceph-mon[34673]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0a1caaad-70cd-4c2c-88d3-e9a5cc885655"} : dispatch 2024-09-21T03:41:18.897 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:18 smithi123 ceph-mon[34673]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a1caaad-70cd-4c2c-88d3-e9a5cc885655"}]': finished 2024-09-21T03:41:18.897 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:18 smithi123 ceph-mon[34673]: osdmap e59: 11 total, 10 up, 11 in 2024-09-21T03:41:18.897 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:18 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:41:18.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:18 smithi016 ceph-mon[30375]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:18.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:18 smithi016 ceph-mon[30375]: from='client.? 172.21.15.123:0/784155923' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0a1caaad-70cd-4c2c-88d3-e9a5cc885655"} : dispatch 2024-09-21T03:41:18.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:18 smithi016 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0a1caaad-70cd-4c2c-88d3-e9a5cc885655"} : dispatch 2024-09-21T03:41:18.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:18 smithi016 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a1caaad-70cd-4c2c-88d3-e9a5cc885655"}]': finished 2024-09-21T03:41:18.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:18 smithi016 ceph-mon[30375]: osdmap e59: 11 total, 10 up, 11 in 2024-09-21T03:41:18.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:18 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:41:19.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:18 smithi134 ceph-mon[35031]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:19.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:18 smithi134 ceph-mon[35031]: from='client.? 172.21.15.123:0/784155923' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0a1caaad-70cd-4c2c-88d3-e9a5cc885655"} : dispatch 2024-09-21T03:41:19.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:18 smithi134 ceph-mon[35031]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0a1caaad-70cd-4c2c-88d3-e9a5cc885655"} : dispatch 2024-09-21T03:41:19.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:18 smithi134 ceph-mon[35031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a1caaad-70cd-4c2c-88d3-e9a5cc885655"}]': finished 2024-09-21T03:41:19.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:18 smithi134 ceph-mon[35031]: osdmap e59: 11 total, 10 up, 11 in 2024-09-21T03:41:19.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:18 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:41:19.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:19 smithi123 ceph-mon[34673]: from='client.? 172.21.15.123:0/884446618' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:41:19.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:19 smithi016 ceph-mon[30375]: from='client.? 172.21.15.123:0/884446618' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:41:20.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:19 smithi134 ceph-mon[35031]: from='client.? 172.21.15.123:0/884446618' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:41:20.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:20 smithi134 ceph-mon[35031]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:20.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:20 smithi123 ceph-mon[34673]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:20.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:20 smithi016 ceph-mon[30375]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:22.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:22 smithi123 ceph-mon[34673]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:22.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:22 smithi016 ceph-mon[30375]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:23.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:22 smithi134 ceph-mon[35031]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:24.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:24 smithi123 ceph-mon[34673]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:24.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:24 smithi016 ceph-mon[30375]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:25.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:24 smithi134 ceph-mon[35031]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:26.736 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:26 smithi123 ceph-mon[34673]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:26.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:26 smithi016 ceph-mon[30375]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:27.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:26 smithi134 ceph-mon[35031]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:27.726 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:27 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:41:27.726 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:27 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:28.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:27 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:41:28.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:27 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:28.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:27 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:41:28.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:27 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:28.791 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:28 smithi123 ceph-mon[34673]: Deploying daemon osd.10 on smithi123 2024-09-21T03:41:28.791 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:28 smithi123 ceph-mon[34673]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:29.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:28 smithi134 ceph-mon[35031]: Deploying daemon osd.10 on smithi123 2024-09-21T03:41:29.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:28 smithi134 ceph-mon[35031]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:29.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:28 smithi016 ceph-mon[30375]: Deploying daemon osd.10 on smithi123 2024-09-21T03:41:29.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:28 smithi016 ceph-mon[30375]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:30.820 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:30 smithi123 ceph-mon[34673]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:30 smithi134 ceph-mon[35031]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:31.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:30 smithi016 ceph-mon[30375]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:32.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:32 smithi123 ceph-mon[34673]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:33.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:32 smithi134 ceph-mon[35031]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:32 smithi016 ceph-mon[30375]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:34.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:33 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:34.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:33 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:34.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:33 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:34.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:33 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:34.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:33 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:34.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:33 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:34.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:33 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:34.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:33 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:34.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:33 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:35.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:34 smithi134 ceph-mon[35031]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:35.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:34 smithi123 ceph-mon[34673]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:35.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:34 smithi016 ceph-mon[30375]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:36.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:36.476+0000 7f38d8ff9640 1 -- 172.21.15.123:0/2934860692 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f38dc063800 con 0x7f38c80b6510 2024-09-21T03:41:36.479 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 10 on host 'smithi123' 2024-09-21T03:41:36.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:36.478+0000 7f38e142f640 1 -- 172.21.15.123:0/2934860692 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38c80b6510 msgr2=0x7f38c80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:36.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:36.478+0000 7f38e142f640 1 --2- 172.21.15.123:0/2934860692 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38c80b6510 0x7f38c80b8a20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f38c4005fd0 tx=0x7f38c4005e20 comp rx=0 tx=0).stop 2024-09-21T03:41:36.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:36.478+0000 7f38e142f640 1 -- 172.21.15.123:0/2934860692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38dc102670 msgr2=0x7f38dc06c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:36.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:36.478+0000 7f38e142f640 1 --2- 172.21.15.123:0/2934860692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38dc102670 0x7f38dc06c320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00d950 tx=0x7f38cc00de10 comp rx=0 tx=0).stop 2024-09-21T03:41:36.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:36.478+0000 7f38e142f640 1 -- 172.21.15.123:0/2934860692 shutdown_connections 2024-09-21T03:41:36.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:36.478+0000 7f38e142f640 1 --2- 172.21.15.123:0/2934860692 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38c80b6510 0x7f38c80b8a20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:36.478+0000 7f38e142f640 1 --2- 172.21.15.123:0/2934860692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38dc1040a0 0x7f38dc1a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:36.478+0000 7f38e142f640 1 --2- 172.21.15.123:0/2934860692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc103700 0x7f38dc19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:36.478+0000 7f38e142f640 1 --2- 172.21.15.123:0/2934860692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38dc102670 0x7f38dc06c320 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:36.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:36.478+0000 7f38e142f640 1 -- 172.21.15.123:0/2934860692 >> 172.21.15.123:0/2934860692 conn(0x7f38dc0fe000 msgr2=0x7f38dc106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:36.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:36.478+0000 7f38e142f640 1 -- 172.21.15.123:0/2934860692 shutdown_connections 2024-09-21T03:41:36.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T03:41:36.478+0000 7f38e142f640 1 -- 172.21.15.123:0/2934860692 wait complete. 2024-09-21T03:41:37.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:36 smithi134 ceph-mon[35031]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:37.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:36 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:37.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:36 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:37.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:36 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:37.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:36 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:37.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:36 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:37.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:36 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:37.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:36 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:37.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:36 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:37.106 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:36 smithi123 ceph-mon[34673]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:37.106 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:36 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:37.106 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:36 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:37.106 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:36 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:37.106 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:36 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:37.107 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:36 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:37.107 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:36 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:37.107 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:36 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:37.107 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:36 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:37.133 DEBUG:teuthology.orchestra.run.smithi123:osd.10> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T03:41:37.135 INFO:tasks.cephadm:Deploying osd.11 on smithi134 with /dev/vg_nvme/lv_1... 2024-09-21T03:41:37.135 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:41:37.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:36 smithi016 ceph-mon[30375]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:37.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:36 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:37.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:36 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:37.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:36 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:36 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:36 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:36 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:36 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:36 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:37.393 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:41:39.012 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:38 smithi123 ceph-mon[34673]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:38 smithi134 ceph-mon[35031]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:39.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:38 smithi016 ceph-mon[30375]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:39.406 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:41:39 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:41:39.009+0000 7f68e01a4640 -1 osd.10 0 log_to_monitors true 2024-09-21T03:41:40.018 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:39 smithi134 ceph-mon[35031]: from='osd.10 [v2:172.21.15.123:6824/3682234496,v1:172.21.15.123:6825/3682234496]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:41:40.018 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:39 smithi134 ceph-mon[35031]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:41:40.021 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T03:41:40.048 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi134:vg_nvme/lv_1 2024-09-21T03:41:40.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:39 smithi123 ceph-mon[34673]: from='osd.10 [v2:172.21.15.123:6824/3682234496,v1:172.21.15.123:6825/3682234496]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:41:40.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:39 smithi123 ceph-mon[34673]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:41:40.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:39 smithi016 ceph-mon[30375]: from='osd.10 [v2:172.21.15.123:6824/3682234496,v1:172.21.15.123:6825/3682234496]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:41:40.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:39 smithi016 ceph-mon[30375]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:41:40.309 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:41:40.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.822+0000 7fe612a9c640 1 -- 172.21.15.134:0/3228558298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe60c109500 msgr2=0x7fe60c10b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.822+0000 7fe612a9c640 1 --2- 172.21.15.134:0/3228558298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe60c109500 0x7fe60c10b980 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe604009ca0 tx=0x7fe60402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.822+0000 7fe612a9c640 1 -- 172.21.15.134:0/3228558298 shutdown_connections 2024-09-21T03:41:40.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.822+0000 7fe612a9c640 1 --2- 172.21.15.134:0/3228558298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe60c109500 0x7fe60c10b980 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.822+0000 7fe612a9c640 1 --2- 172.21.15.134:0/3228558298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe60c106ba0 0x7fe60c108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.822+0000 7fe612a9c640 1 --2- 172.21.15.134:0/3228558298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe60c101b00 0x7fe60c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.822+0000 7fe612a9c640 1 -- 172.21.15.134:0/3228558298 >> 172.21.15.134:0/3228558298 conn(0x7fe60c0fd770 msgr2=0x7fe60c0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:40.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.823+0000 7fe612a9c640 1 -- 172.21.15.134:0/3228558298 shutdown_connections 2024-09-21T03:41:40.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.823+0000 7fe612a9c640 1 -- 172.21.15.134:0/3228558298 wait complete. 2024-09-21T03:41:40.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.823+0000 7fe612a9c640 1 Processor -- start 2024-09-21T03:41:40.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.823+0000 7fe612a9c640 1 -- start start 2024-09-21T03:41:40.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.823+0000 7fe612a9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe60c101b00 0x7fe60c199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.823+0000 7fe612a9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe60c106ba0 0x7fe60c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.823+0000 7fe612a9c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe60c109500 0x7fe60c19b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.823+0000 7fe612a9c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c10f360 con 0x7fe60c101b00 2024-09-21T03:41:40.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.823+0000 7fe612a9c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c10f1e0 con 0x7fe60c109500 2024-09-21T03:41:40.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.823+0000 7fe612a9c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c10f4e0 con 0x7fe60c106ba0 2024-09-21T03:41:40.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.823+0000 7fe60bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe60c106ba0 0x7fe60c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.824+0000 7fe60bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe60c106ba0 0x7fe60c19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:36186/0 (socket says 172.21.15.134:36186) 2024-09-21T03:41:40.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.824+0000 7fe60bfff640 1 -- 172.21.15.134:0/1543873527 learned_addr learned my addr 172.21.15.134:0/1543873527 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-21T03:41:40.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.824+0000 7fe610838640 1 --2- 172.21.15.134:0/1543873527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe60c101b00 0x7fe60c199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.824+0000 7fe611039640 1 --2- 172.21.15.134:0/1543873527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe60c109500 0x7fe60c19b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.824+0000 7fe60bfff640 1 -- 172.21.15.134:0/1543873527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe60c109500 msgr2=0x7fe60c19b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.824+0000 7fe60bfff640 1 --2- 172.21.15.134:0/1543873527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe60c109500 0x7fe60c19b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.824+0000 7fe60bfff640 1 -- 172.21.15.134:0/1543873527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe60c101b00 msgr2=0x7fe60c199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:40.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.824+0000 7fe60bfff640 1 --2- 172.21.15.134:0/1543873527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe60c101b00 0x7fe60c199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:40.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.824+0000 7fe60bfff640 1 -- 172.21.15.134:0/1543873527 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60c19b8c0 con 0x7fe60c106ba0 2024-09-21T03:41:40.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.824+0000 7fe610838640 1 --2- 172.21.15.134:0/1543873527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe60c101b00 0x7fe60c199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:40.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.824+0000 7fe60bfff640 1 --2- 172.21.15.134:0/1543873527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe60c106ba0 0x7fe60c19a590 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800d950 tx=0x7fe5f800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:40.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.824+0000 7fe609ffb640 1 -- 172.21.15.134:0/1543873527 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8017070 con 0x7fe60c106ba0 2024-09-21T03:41:40.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.824+0000 7fe612a9c640 1 -- 172.21.15.134:0/1543873527 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60c19bbb0 con 0x7fe60c106ba0 2024-09-21T03:41:40.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.824+0000 7fe609ffb640 1 -- 172.21.15.134:0/1543873527 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5f8009c30 con 0x7fe60c106ba0 2024-09-21T03:41:40.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.824+0000 7fe609ffb640 1 -- 172.21.15.134:0/1543873527 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f800f620 con 0x7fe60c106ba0 2024-09-21T03:41:40.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.824+0000 7fe612a9c640 1 -- 172.21.15.134:0/1543873527 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe60c1a2340 con 0x7fe60c106ba0 2024-09-21T03:41:40.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.825+0000 7fe612a9c640 1 -- 172.21.15.134:0/1543873527 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe60c101f10 con 0x7fe60c106ba0 2024-09-21T03:41:40.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.827+0000 7fe609ffb640 1 -- 172.21.15.134:0/1543873527 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fe5f8010460 con 0x7fe60c106ba0 2024-09-21T03:41:40.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.828+0000 7fe609ffb640 1 --2- 172.21.15.134:0/1543873527 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5fc0b6510 0x7fe5fc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:40.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.828+0000 7fe609ffb640 1 -- 172.21.15.134:0/1543873527 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6398+0+0 (secure 0 0 0) 0x7fe5f80e6850 con 0x7fe60c106ba0 2024-09-21T03:41:40.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.828+0000 7fe610838640 1 --2- 172.21.15.134:0/1543873527 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5fc0b6510 0x7fe5fc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:40.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.829+0000 7fe610838640 1 --2- 172.21.15.134:0/1543873527 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5fc0b6510 0x7fe5fc0b8a20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe60000abe0 tx=0x7fe60000a5c0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:40.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:40.831+0000 7fe609ffb640 1 -- 172.21.15.134:0/1543873527 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe5f80ad860 con 0x7fe60c106ba0 2024-09-21T03:41:41.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:41:41.018+0000 7fe612a9c640 1 -- 172.21.15.134:0/1543873527 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe60c063800 con 0x7fe5fc0b6510 2024-09-21T03:41:41.020 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:40 smithi134 ceph-mon[35031]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:41.020 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:40 smithi134 ceph-mon[35031]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:41:41.020 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:40 smithi134 ceph-mon[35031]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T03:41:41.021 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:41:41.021 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:40 smithi134 ceph-mon[35031]: from='osd.10 [v2:172.21.15.123:6824/3682234496,v1:172.21.15.123:6825/3682234496]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:41:41.021 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:40 smithi134 ceph-mon[35031]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:41:41.021 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:41.021 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:41.021 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:41.021 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:41.021 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:41.021 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:41.021 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:41.021 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:41.021 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:41.021 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:41.156 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:41:40.846+0000 7f68dc126640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T03:41:41.156 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:41:40.861+0000 7f68d7f24640 -1 osd.10 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:41:41.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-mon[34673]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-mon[34673]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:41:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-mon[34673]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T03:41:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:41:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-mon[34673]: from='osd.10 [v2:172.21.15.123:6824/3682234496,v1:172.21.15.123:6825/3682234496]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:41:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-mon[34673]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:41:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:41.158 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:41.158 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:41.158 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:41.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:40 smithi016 ceph-mon[30375]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:41.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:40 smithi016 ceph-mon[30375]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:41:41.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:40 smithi016 ceph-mon[30375]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T03:41:41.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:41:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:40 smithi016 ceph-mon[30375]: from='osd.10 [v2:172.21.15.123:6824/3682234496,v1:172.21.15.123:6825/3682234496]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:41:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:40 smithi016 ceph-mon[30375]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T03:41:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:41:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:42.063 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:41 smithi134 ceph-mon[35031]: Detected new or changed devices on smithi123 2024-09-21T03:41:42.063 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:41 smithi134 ceph-mon[35031]: Adjusting osd_memory_target on smithi123 to 4272M 2024-09-21T03:41:42.063 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:41 smithi134 ceph-mon[35031]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T03:41:42.063 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:41 smithi134 ceph-mon[35031]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T03:41:42.063 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:41:42.063 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:41:42.063 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:41 smithi134 ceph-mon[35031]: from='client.24544 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:41:42.064 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:41:42.064 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:41:42.064 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:42.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:41 smithi123 ceph-mon[34673]: Detected new or changed devices on smithi123 2024-09-21T03:41:42.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:41 smithi123 ceph-mon[34673]: Adjusting osd_memory_target on smithi123 to 4272M 2024-09-21T03:41:42.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:41 smithi123 ceph-mon[34673]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T03:41:42.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:41 smithi123 ceph-mon[34673]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T03:41:42.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:41 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:41:42.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:41 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:41:42.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:41 smithi123 ceph-mon[34673]: from='client.24544 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:41:42.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:41 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:41:42.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:41 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:41:42.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:41 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:42.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:41 smithi016 ceph-mon[30375]: Detected new or changed devices on smithi123 2024-09-21T03:41:42.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:41 smithi016 ceph-mon[30375]: Adjusting osd_memory_target on smithi123 to 4272M 2024-09-21T03:41:42.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:41 smithi016 ceph-mon[30375]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T03:41:42.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:41 smithi016 ceph-mon[30375]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T03:41:42.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:41:42.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:41:42.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:41 smithi016 ceph-mon[30375]: from='client.24544 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:41:42.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:41:42.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:41:42.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:43.010 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:42 smithi134 ceph-mon[35031]: purged_snaps scrub starts 2024-09-21T03:41:43.010 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:42 smithi134 ceph-mon[35031]: purged_snaps scrub ok 2024-09-21T03:41:43.010 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:42 smithi134 ceph-mon[35031]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:43.011 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:42 smithi134 ceph-mon[35031]: osd.10 [v2:172.21.15.123:6824/3682234496,v1:172.21.15.123:6825/3682234496] boot 2024-09-21T03:41:43.011 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:42 smithi134 ceph-mon[35031]: osdmap e62: 11 total, 11 up, 11 in 2024-09-21T03:41:43.011 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:42 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:41:43.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:42 smithi123 ceph-mon[34673]: purged_snaps scrub starts 2024-09-21T03:41:43.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:42 smithi123 ceph-mon[34673]: purged_snaps scrub ok 2024-09-21T03:41:43.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:42 smithi123 ceph-mon[34673]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:43.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:42 smithi123 ceph-mon[34673]: osd.10 [v2:172.21.15.123:6824/3682234496,v1:172.21.15.123:6825/3682234496] boot 2024-09-21T03:41:43.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:42 smithi123 ceph-mon[34673]: osdmap e62: 11 total, 11 up, 11 in 2024-09-21T03:41:43.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:42 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:41:43.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:42 smithi016 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T03:41:43.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:42 smithi016 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T03:41:43.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:42 smithi016 ceph-mon[30375]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:41:43.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:42 smithi016 ceph-mon[30375]: osd.10 [v2:172.21.15.123:6824/3682234496,v1:172.21.15.123:6825/3682234496] boot 2024-09-21T03:41:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:42 smithi016 ceph-mon[30375]: osdmap e62: 11 total, 11 up, 11 in 2024-09-21T03:41:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:42 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:41:44.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:43 smithi123 ceph-mon[34673]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T03:41:44.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:43 smithi123 ceph-mon[34673]: from='client.? 172.21.15.134:0/1787526918' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3aff3258-5a38-41ff-aac3-4a082feffdae"} : dispatch 2024-09-21T03:41:44.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:43 smithi123 ceph-mon[34673]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3aff3258-5a38-41ff-aac3-4a082feffdae"} : dispatch 2024-09-21T03:41:44.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:43 smithi123 ceph-mon[34673]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3aff3258-5a38-41ff-aac3-4a082feffdae"}]': finished 2024-09-21T03:41:44.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:43 smithi123 ceph-mon[34673]: osdmap e64: 12 total, 11 up, 12 in 2024-09-21T03:41:44.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:43 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:41:44.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:43 smithi016 ceph-mon[30375]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T03:41:44.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:43 smithi016 ceph-mon[30375]: from='client.? 172.21.15.134:0/1787526918' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3aff3258-5a38-41ff-aac3-4a082feffdae"} : dispatch 2024-09-21T03:41:44.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:43 smithi016 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3aff3258-5a38-41ff-aac3-4a082feffdae"} : dispatch 2024-09-21T03:41:44.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:43 smithi016 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3aff3258-5a38-41ff-aac3-4a082feffdae"}]': finished 2024-09-21T03:41:44.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:43 smithi016 ceph-mon[30375]: osdmap e64: 12 total, 11 up, 12 in 2024-09-21T03:41:44.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:43 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:41:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:43 smithi134 ceph-mon[35031]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T03:41:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:43 smithi134 ceph-mon[35031]: from='client.? 172.21.15.134:0/1787526918' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3aff3258-5a38-41ff-aac3-4a082feffdae"} : dispatch 2024-09-21T03:41:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:43 smithi134 ceph-mon[35031]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3aff3258-5a38-41ff-aac3-4a082feffdae"} : dispatch 2024-09-21T03:41:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:43 smithi134 ceph-mon[35031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3aff3258-5a38-41ff-aac3-4a082feffdae"}]': finished 2024-09-21T03:41:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:43 smithi134 ceph-mon[35031]: osdmap e64: 12 total, 11 up, 12 in 2024-09-21T03:41:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:43 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:41:45.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:44 smithi123 ceph-mon[34673]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:41:45.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:44 smithi123 ceph-mon[34673]: from='client.? 172.21.15.134:0/202637353' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:41:45.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:44 smithi016 ceph-mon[30375]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:41:45.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:44 smithi016 ceph-mon[30375]: from='client.? 172.21.15.134:0/202637353' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:41:45.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:44 smithi134 ceph-mon[35031]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:41:45.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:44 smithi134 ceph-mon[35031]: from='client.? 172.21.15.134:0/202637353' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:41:47.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:46 smithi123 ceph-mon[34673]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-09-21T03:41:47.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:46 smithi016 ceph-mon[30375]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-09-21T03:41:47.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:46 smithi134 ceph-mon[35031]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-09-21T03:41:49.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:48 smithi123 ceph-mon[34673]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:41:49.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:48 smithi016 ceph-mon[30375]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:41:49.247 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:48 smithi134 ceph-mon[35031]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:41:51.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:50 smithi123 ceph-mon[34673]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-21T03:41:51.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:50 smithi016 ceph-mon[30375]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-21T03:41:51.229 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:50 smithi134 ceph-mon[35031]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-21T03:41:53.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:52 smithi123 ceph-mon[34673]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T03:41:53.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:41:53.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:53.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:52 smithi123 ceph-mon[34673]: Deploying daemon osd.11 on smithi134 2024-09-21T03:41:53.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:52 smithi016 ceph-mon[30375]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T03:41:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:41:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:52 smithi016 ceph-mon[30375]: Deploying daemon osd.11 on smithi134 2024-09-21T03:41:53.346 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:52 smithi134 ceph-mon[35031]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T03:41:53.346 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:41:53.346 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:53.346 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:52 smithi134 ceph-mon[35031]: Deploying daemon osd.11 on smithi134 2024-09-21T03:41:55.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:54 smithi123 ceph-mon[34673]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:41:55.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:54 smithi016 ceph-mon[30375]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:41:55.282 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:54 smithi134 ceph-mon[35031]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:41:57.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:56 smithi123 ceph-mon[34673]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:41:57.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:56 smithi016 ceph-mon[30375]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:41:57.213 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:56 smithi134 ceph-mon[35031]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:41:58.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:58.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:58.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:58.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:58.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:58.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:58.211 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:58.212 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:58.212 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:41:59.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:41:58 smithi123 ceph-mon[34673]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:41:59.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:41:58 smithi016 ceph-mon[30375]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:41:59.218 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:41:58 smithi134 ceph-mon[35031]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:42:01.035 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:00 smithi134 ceph-mon[35031]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:42:01.035 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:00 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:01.035 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:00 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:01.035 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:00 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:01.035 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:00 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:01.035 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:00 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:01.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:00 smithi123 ceph-mon[34673]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:42:01.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:00 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:01.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:00 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:01.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:00 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:01.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:00 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:01.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:00 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:00 smithi016 ceph-mon[30375]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:42:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:00 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:00 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:00 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:00 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:01.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:00 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:01.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:01.353+0000 7fe609ffb640 1 -- 172.21.15.134:0/1543873527 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe60c063800 con 0x7fe5fc0b6510 2024-09-21T03:42:01.356 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 11 on host 'smithi134' 2024-09-21T03:42:01.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:01.356+0000 7fe612a9c640 1 -- 172.21.15.134:0/1543873527 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5fc0b6510 msgr2=0x7fe5fc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:01.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:01.356+0000 7fe612a9c640 1 --2- 172.21.15.134:0/1543873527 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5fc0b6510 0x7fe5fc0b8a20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe60000abe0 tx=0x7fe60000a5c0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:01.356+0000 7fe612a9c640 1 -- 172.21.15.134:0/1543873527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe60c106ba0 msgr2=0x7fe60c19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:01.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:01.356+0000 7fe612a9c640 1 --2- 172.21.15.134:0/1543873527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe60c106ba0 0x7fe60c19a590 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800d950 tx=0x7fe5f800de10 comp rx=0 tx=0).stop 2024-09-21T03:42:01.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:01.356+0000 7fe612a9c640 1 -- 172.21.15.134:0/1543873527 shutdown_connections 2024-09-21T03:42:01.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:01.356+0000 7fe612a9c640 1 --2- 172.21.15.134:0/1543873527 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5fc0b6510 0x7fe5fc0b8a20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:01.356+0000 7fe612a9c640 1 --2- 172.21.15.134:0/1543873527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe60c109500 0x7fe60c19b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:01.356+0000 7fe612a9c640 1 --2- 172.21.15.134:0/1543873527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe60c106ba0 0x7fe60c19a590 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:01.356+0000 7fe612a9c640 1 --2- 172.21.15.134:0/1543873527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe60c101b00 0x7fe60c199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:01.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:01.356+0000 7fe612a9c640 1 -- 172.21.15.134:0/1543873527 >> 172.21.15.134:0/1543873527 conn(0x7fe60c0fd770 msgr2=0x7fe60c107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:01.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:01.356+0000 7fe612a9c640 1 -- 172.21.15.134:0/1543873527 shutdown_connections 2024-09-21T03:42:01.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:01.356+0000 7fe612a9c640 1 -- 172.21.15.134:0/1543873527 wait complete. 2024-09-21T03:42:02.016 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:01 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:02.016 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:01 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:02.016 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:01 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:02.046 DEBUG:teuthology.orchestra.run.smithi134:osd.11> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T03:42:02.048 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T03:42:02.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:42:02.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:01 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:02.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:01 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:02.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:01 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:02.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:01 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:02.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:01 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:02.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:01 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.052+0000 7f249b200640 1 -- 172.21.15.16:0/7898055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c104fe0 msgr2=0x7f249c10b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.052+0000 7f249b200640 1 --2- 172.21.15.16:0/7898055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c104fe0 0x7f249c10b980 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2494009ca0 tx=0x7f249402a950 comp rx=0 tx=0).stop 2024-09-21T03:42:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.052+0000 7f249b200640 1 -- 172.21.15.16:0/7898055 shutdown_connections 2024-09-21T03:42:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.052+0000 7f249b200640 1 --2- 172.21.15.16:0/7898055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c104fe0 0x7f249c10b980 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.052+0000 7f249b200640 1 --2- 172.21.15.16:0/7898055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f249c104640 0x7f249c104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.052+0000 7f249b200640 1 --2- 172.21.15.16:0/7898055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f249c1035b0 0x7f249c103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.052+0000 7f249b200640 1 -- 172.21.15.16:0/7898055 >> 172.21.15.16:0/7898055 conn(0x7f249c0fef20 msgr2=0x7f249c1013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.052+0000 7f249b200640 1 -- 172.21.15.16:0/7898055 shutdown_connections 2024-09-21T03:42:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.052+0000 7f249b200640 1 -- 172.21.15.16:0/7898055 wait complete. 2024-09-21T03:42:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.054+0000 7f249b200640 1 Processor -- start 2024-09-21T03:42:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.054+0000 7f249b200640 1 -- start start 2024-09-21T03:42:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.054+0000 7f249b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c1035b0 0x7f249c19bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.054+0000 7f249b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f249c104640 0x7f249c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.054+0000 7f249b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f249c104fe0 0x7f249c19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.054+0000 7f249b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f249c10f210 con 0x7f249c1035b0 2024-09-21T03:42:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.054+0000 7f249b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f249c10f090 con 0x7f249c104fe0 2024-09-21T03:42:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.054+0000 7f249b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f249c10f390 con 0x7f249c104640 2024-09-21T03:42:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.054+0000 7f2499e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c1035b0 0x7f249c19bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.054+0000 7f2499e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c1035b0 0x7f249c19bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45672/0 (socket says 172.21.15.16:45672) 2024-09-21T03:42:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.054+0000 7f2499e00640 1 -- 172.21.15.16:0/3896640405 learned_addr learned my addr 172.21.15.16:0/3896640405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:42:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.054+0000 7f2493e00640 1 --2- 172.21.15.16:0/3896640405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f249c104640 0x7f249c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.055+0000 7f249a800640 1 --2- 172.21.15.16:0/3896640405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f249c104fe0 0x7f249c19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.055+0000 7f2499e00640 1 -- 172.21.15.16:0/3896640405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f249c104640 msgr2=0x7f249c19c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.055+0000 7f2499e00640 1 --2- 172.21.15.16:0/3896640405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f249c104640 0x7f249c19c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.055+0000 7f2499e00640 1 -- 172.21.15.16:0/3896640405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f249c104fe0 msgr2=0x7f249c19d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.055+0000 7f2499e00640 1 --2- 172.21.15.16:0/3896640405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f249c104fe0 0x7f249c19d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.055+0000 7f2499e00640 1 -- 172.21.15.16:0/3896640405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249c19da10 con 0x7f249c1035b0 2024-09-21T03:42:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.055+0000 7f2499e00640 1 --2- 172.21.15.16:0/3896640405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c1035b0 0x7f249c19bfe0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f248c00e9f0 tx=0x7f248c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.055+0000 7f2492a00640 1 -- 172.21.15.16:0/3896640405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c00cd80 con 0x7f249c1035b0 2024-09-21T03:42:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.056+0000 7f2492a00640 1 -- 172.21.15.16:0/3896640405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f248c0054d0 con 0x7f249c1035b0 2024-09-21T03:42:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.056+0000 7f2492a00640 1 -- 172.21.15.16:0/3896640405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c010690 con 0x7f249c1035b0 2024-09-21T03:42:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.056+0000 7f249b200640 1 -- 172.21.15.16:0/3896640405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f249c19dd00 con 0x7f249c1035b0 2024-09-21T03:42:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.056+0000 7f249b200640 1 -- 172.21.15.16:0/3896640405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f249c1a4400 con 0x7f249c1035b0 2024-09-21T03:42:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.057+0000 7f249b200640 1 -- 172.21.15.16:0/3896640405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f249c1a45d0 con 0x7f249c1035b0 2024-09-21T03:42:03.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.060+0000 7f2492a00640 1 -- 172.21.15.16:0/3896640405 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f248c004e30 con 0x7f249c1035b0 2024-09-21T03:42:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.061+0000 7f2492a00640 1 --2- 172.21.15.16:0/3896640405 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24800b6590 0x7f24800b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.061+0000 7f2492a00640 1 -- 172.21.15.16:0/3896640405 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6809+0+0 (secure 0 0 0) 0x7f248c0e72f0 con 0x7f249c1035b0 2024-09-21T03:42:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.061+0000 7f2493e00640 1 --2- 172.21.15.16:0/3896640405 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24800b6590 0x7f24800b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.062+0000 7f2493e00640 1 --2- 172.21.15.16:0/3896640405 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24800b6590 0x7f24800b8aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2484009bd0 tx=0x7f2484005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.063+0000 7f2492a00640 1 -- 172.21.15.16:0/3896640405 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f248c0ae0c0 con 0x7f249c1035b0 2024-09-21T03:42:03.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:02 smithi016 ceph-mon[30375]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:42:03.246 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:02 smithi134 ceph-mon[35031]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:42:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.263+0000 7f249b200640 1 -- 172.21.15.16:0/3896640405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f249c1a47a0 con 0x7f249c1035b0 2024-09-21T03:42:03.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.263+0000 7f2492a00640 1 -- 172.21.15.16:0/3896640405 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v64) ==== 74+0+133 (secure 0 0 0) 0x7f248c0b24a0 con 0x7f249c1035b0 2024-09-21T03:42:03.265 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:42:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.266+0000 7f249b200640 1 -- 172.21.15.16:0/3896640405 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24800b6590 msgr2=0x7f24800b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.266+0000 7f249b200640 1 --2- 172.21.15.16:0/3896640405 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24800b6590 0x7f24800b8aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2484009bd0 tx=0x7f2484005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.266+0000 7f249b200640 1 -- 172.21.15.16:0/3896640405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c1035b0 msgr2=0x7f249c19bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.266+0000 7f249b200640 1 --2- 172.21.15.16:0/3896640405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c1035b0 0x7f249c19bfe0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f248c00e9f0 tx=0x7f248c00edf0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.266+0000 7f249b200640 1 -- 172.21.15.16:0/3896640405 shutdown_connections 2024-09-21T03:42:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.266+0000 7f249b200640 1 --2- 172.21.15.16:0/3896640405 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24800b6590 0x7f24800b8aa0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.266+0000 7f249b200640 1 --2- 172.21.15.16:0/3896640405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f249c104fe0 0x7f249c19d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.266+0000 7f249b200640 1 --2- 172.21.15.16:0/3896640405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f249c104640 0x7f249c19c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.266+0000 7f249b200640 1 --2- 172.21.15.16:0/3896640405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c1035b0 0x7f249c19bfe0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.266+0000 7f249b200640 1 -- 172.21.15.16:0/3896640405 >> 172.21.15.16:0/3896640405 conn(0x7f249c0fef20 msgr2=0x7f249c0755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.266+0000 7f249b200640 1 -- 172.21.15.16:0/3896640405 shutdown_connections 2024-09-21T03:42:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:03.266+0000 7f249b200640 1 -- 172.21.15.16:0/3896640405 wait complete. 2024-09-21T03:42:03.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:02 smithi123 ceph-mon[34673]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:42:04.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:03 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3896640405' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:42:04.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:03 smithi016 ceph-mon[30375]: from='osd.11 [v2:172.21.15.134:6824/584758685,v1:172.21.15.134:6825/584758685]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:42:04.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:03 smithi016 ceph-mon[30375]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:42:04.234 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:03 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3896640405' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:42:04.234 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:03 smithi134 ceph-mon[35031]: from='osd.11 [v2:172.21.15.134:6824/584758685,v1:172.21.15.134:6825/584758685]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:42:04.234 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:03 smithi134 ceph-mon[35031]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:42:04.234 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:42:03 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:42:03.873+0000 7f4bb79f6640 -1 osd.11 0 log_to_monitors true 2024-09-21T03:42:04.311 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":64,"num_osds":12,"num_up_osds":11,"osd_up_since":1726890101,"num_in_osds":12,"osd_in_since":1726890103,"num_remapped_pgs":0} 2024-09-21T03:42:04.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:03 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3896640405' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:42:04.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:03 smithi123 ceph-mon[34673]: from='osd.11 [v2:172.21.15.134:6824/584758685,v1:172.21.15.134:6825/584758685]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:42:04.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:03 smithi123 ceph-mon[34673]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:42:05.312 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:42:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:04 smithi134 ceph-mon[35031]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:42:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:04 smithi134 ceph-mon[35031]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:42:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:04 smithi134 ceph-mon[35031]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T03:42:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:04 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:42:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:04 smithi134 ceph-mon[35031]: from='osd.11 [v2:172.21.15.134:6824/584758685,v1:172.21.15.134:6825/584758685]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:42:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:04 smithi134 ceph-mon[35031]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:42:05.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:04 smithi016 ceph-mon[30375]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:42:05.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:04 smithi016 ceph-mon[30375]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:42:05.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:04 smithi016 ceph-mon[30375]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T03:42:05.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:04 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:42:05.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:04 smithi016 ceph-mon[30375]: from='osd.11 [v2:172.21.15.134:6824/584758685,v1:172.21.15.134:6825/584758685]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:42:05.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:04 smithi016 ceph-mon[30375]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:42:05.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:04 smithi123 ceph-mon[34673]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:42:05.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:04 smithi123 ceph-mon[34673]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:42:05.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:04 smithi123 ceph-mon[34673]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T03:42:05.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:04 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:42:05.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:04 smithi123 ceph-mon[34673]: from='osd.11 [v2:172.21.15.134:6824/584758685,v1:172.21.15.134:6825/584758685]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:42:05.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:04 smithi123 ceph-mon[34673]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]} : dispatch 2024-09-21T03:42:05.712 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:05.921 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:42:05.611+0000 7f4bb3978640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T03:42:05.921 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:42:05.622+0000 7f4bab76e640 -1 osd.11 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:42:06.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:05 smithi016 ceph-mon[30375]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-21T03:42:06.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:05 smithi016 ceph-mon[30375]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T03:42:06.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:42:06.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:42:06.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:05 smithi016 ceph-mon[30375]: Detected new or changed devices on smithi134 2024-09-21T03:42:06.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:06.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:06.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:06.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:06.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:06.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:06.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:05 smithi016 ceph-mon[30375]: from='osd.11 ' entity='osd.11' 2024-09-21T03:42:06.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-mon[35031]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-21T03:42:06.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-mon[35031]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T03:42:06.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:42:06.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:42:06.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-mon[35031]: Detected new or changed devices on smithi134 2024-09-21T03:42:06.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:06.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:06.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:06.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:06.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:06.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:06.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:05 smithi134 ceph-mon[35031]: from='osd.11 ' entity='osd.11' 2024-09-21T03:42:06.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.358+0000 7fe8e5a00640 1 -- 172.21.15.16:0/3180097724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e0105180 msgr2=0x7fe8e010bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.358+0000 7fe8e5a00640 1 --2- 172.21.15.16:0/3180097724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e0105180 0x7fe8e010bb20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009ca0 tx=0x7fe8d802a950 comp rx=0 tx=0).stop 2024-09-21T03:42:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.359+0000 7fe8e5a00640 1 -- 172.21.15.16:0/3180097724 shutdown_connections 2024-09-21T03:42:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.359+0000 7fe8e5a00640 1 --2- 172.21.15.16:0/3180097724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e0105180 0x7fe8e010bb20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.359+0000 7fe8e5a00640 1 --2- 172.21.15.16:0/3180097724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e01047e0 0x7fe8e0104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.359+0000 7fe8e5a00640 1 --2- 172.21.15.16:0/3180097724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8e0103750 0x7fe8e0103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.359+0000 7fe8e5a00640 1 -- 172.21.15.16:0/3180097724 >> 172.21.15.16:0/3180097724 conn(0x7fe8e00ff0c0 msgr2=0x7fe8e0101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.359+0000 7fe8e5a00640 1 -- 172.21.15.16:0/3180097724 shutdown_connections 2024-09-21T03:42:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.359+0000 7fe8e5a00640 1 -- 172.21.15.16:0/3180097724 wait complete. 2024-09-21T03:42:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.360+0000 7fe8e5a00640 1 Processor -- start 2024-09-21T03:42:06.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.360+0000 7fe8e5a00640 1 -- start start 2024-09-21T03:42:06.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.360+0000 7fe8e5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e0103750 0x7fe8e0076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.360+0000 7fe8e5a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e01047e0 0x7fe8e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.360+0000 7fe8e5a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8e0105180 0x7fe8e0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.360+0000 7fe8e5a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e010f3d0 con 0x7fe8e0103750 2024-09-21T03:42:06.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.360+0000 7fe8e5a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e010f250 con 0x7fe8e01047e0 2024-09-21T03:42:06.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.360+0000 7fe8e5a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e010f550 con 0x7fe8e0105180 2024-09-21T03:42:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.361+0000 7fe8df400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8e0105180 0x7fe8e0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.361+0000 7fe8df400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8e0105180 0x7fe8e0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:54756/0 (socket says 172.21.15.16:54756) 2024-09-21T03:42:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.361+0000 7fe8df400640 1 -- 172.21.15.16:0/2191790572 learned_addr learned my addr 172.21.15.16:0/2191790572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:42:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.361+0000 7fe8dea00640 1 --2- 172.21.15.16:0/2191790572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e0103750 0x7fe8e0076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.361+0000 7fe8de000640 1 --2- 172.21.15.16:0/2191790572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e01047e0 0x7fe8e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.362+0000 7fe8df400640 1 -- 172.21.15.16:0/2191790572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e01047e0 msgr2=0x7fe8e0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.362+0000 7fe8df400640 1 --2- 172.21.15.16:0/2191790572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e01047e0 0x7fe8e0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.362+0000 7fe8df400640 1 -- 172.21.15.16:0/2191790572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e0103750 msgr2=0x7fe8e0076e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.362+0000 7fe8df400640 1 --2- 172.21.15.16:0/2191790572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e0103750 0x7fe8e0076e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.362+0000 7fe8df400640 1 -- 172.21.15.16:0/2191790572 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e0076350 con 0x7fe8e0105180 2024-09-21T03:42:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.362+0000 7fe8dea00640 1 --2- 172.21.15.16:0/2191790572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e0103750 0x7fe8e0076e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.362+0000 7fe8df400640 1 --2- 172.21.15.16:0/2191790572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8e0105180 0x7fe8e0075c50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe8d802f810 tx=0x7fe8d802f5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.362+0000 7fe8d7400640 1 -- 172.21.15.16:0/2191790572 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8034040 con 0x7fe8e0105180 2024-09-21T03:42:06.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.362+0000 7fe8d7400640 1 -- 172.21.15.16:0/2191790572 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8d802fe20 con 0x7fe8e0105180 2024-09-21T03:42:06.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.362+0000 7fe8e5a00640 1 -- 172.21.15.16:0/2191790572 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e01a3fd0 con 0x7fe8e0105180 2024-09-21T03:42:06.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.362+0000 7fe8d7400640 1 -- 172.21.15.16:0/2191790572 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8004720 con 0x7fe8e0105180 2024-09-21T03:42:06.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.362+0000 7fe8e5a00640 1 -- 172.21.15.16:0/2191790572 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e01a4370 con 0x7fe8e0105180 2024-09-21T03:42:06.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.363+0000 7fe8e5a00640 1 -- 172.21.15.16:0/2191790572 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e010a910 con 0x7fe8e0105180 2024-09-21T03:42:06.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.366+0000 7fe8d7400640 1 -- 172.21.15.16:0/2191790572 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fe8d8037020 con 0x7fe8e0105180 2024-09-21T03:42:06.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.368+0000 7fe8d7400640 1 --2- 172.21.15.16:0/2191790572 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe8c40b6510 0x7fe8c40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:06.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.368+0000 7fe8d7400640 1 -- 172.21.15.16:0/2191790572 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6847+0+0 (secure 0 0 0) 0x7fe8d8107cb0 con 0x7fe8e0105180 2024-09-21T03:42:06.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.368+0000 7fe8dea00640 1 --2- 172.21.15.16:0/2191790572 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe8c40b6510 0x7fe8c40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:06.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.369+0000 7fe8dea00640 1 --2- 172.21.15.16:0/2191790572 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe8c40b6510 0x7fe8c40b8a20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009fc0 tx=0x7fe8d0005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:06.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.369+0000 7fe8d7400640 1 -- 172.21.15.16:0/2191790572 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe8d80ceaf0 con 0x7fe8e0105180 2024-09-21T03:42:06.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:05 smithi123 ceph-mon[34673]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-21T03:42:06.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:05 smithi123 ceph-mon[34673]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T03:42:06.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:42:06.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:42:06.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:05 smithi123 ceph-mon[34673]: Detected new or changed devices on smithi134 2024-09-21T03:42:06.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:06.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:06.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:42:06.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:06.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:06.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:06.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:06.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:05 smithi123 ceph-mon[34673]: from='osd.11 ' entity='osd.11' 2024-09-21T03:42:06.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.569+0000 7fe8e5a00640 1 -- 172.21.15.16:0/2191790572 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fe8e0063fc0 con 0x7fe8e0105180 2024-09-21T03:42:06.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.570+0000 7fe8d7400640 1 -- 172.21.15.16:0/2191790572 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v66) ==== 74+0+133 (secure 0 0 0) 0x7fe8d80d2ed0 con 0x7fe8e0105180 2024-09-21T03:42:06.572 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:42:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.573+0000 7fe8e5a00640 1 -- 172.21.15.16:0/2191790572 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe8c40b6510 msgr2=0x7fe8c40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.573+0000 7fe8e5a00640 1 --2- 172.21.15.16:0/2191790572 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe8c40b6510 0x7fe8c40b8a20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009fc0 tx=0x7fe8d0005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.573+0000 7fe8e5a00640 1 -- 172.21.15.16:0/2191790572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8e0105180 msgr2=0x7fe8e0075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.573+0000 7fe8e5a00640 1 --2- 172.21.15.16:0/2191790572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8e0105180 0x7fe8e0075c50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe8d802f810 tx=0x7fe8d802f5d0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.573+0000 7fe8e5a00640 1 -- 172.21.15.16:0/2191790572 shutdown_connections 2024-09-21T03:42:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.573+0000 7fe8e5a00640 1 --2- 172.21.15.16:0/2191790572 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe8c40b6510 0x7fe8c40b8a20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.573+0000 7fe8e5a00640 1 --2- 172.21.15.16:0/2191790572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8e0105180 0x7fe8e0075c50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.573+0000 7fe8e5a00640 1 --2- 172.21.15.16:0/2191790572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e01047e0 0x7fe8e0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.573+0000 7fe8e5a00640 1 --2- 172.21.15.16:0/2191790572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e0103750 0x7fe8e0076e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.573+0000 7fe8e5a00640 1 -- 172.21.15.16:0/2191790572 >> 172.21.15.16:0/2191790572 conn(0x7fe8e00ff0c0 msgr2=0x7fe8e0107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.573+0000 7fe8e5a00640 1 -- 172.21.15.16:0/2191790572 shutdown_connections 2024-09-21T03:42:06.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:06.573+0000 7fe8e5a00640 1 -- 172.21.15.16:0/2191790572 wait complete. 2024-09-21T03:42:07.187 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:06 smithi016 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T03:42:07.187 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:06 smithi016 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T03:42:07.187 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:06 smithi016 ceph-mon[30375]: Adjusting osd_memory_target on smithi134 to 4272M 2024-09-21T03:42:07.187 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:06 smithi016 ceph-mon[30375]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:42:07.187 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:06 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:42:07.187 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:06 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2191790572' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:42:07.187 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:06 smithi016 ceph-mon[30375]: osd.11 [v2:172.21.15.134:6824/584758685,v1:172.21.15.134:6825/584758685] boot 2024-09-21T03:42:07.187 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:06 smithi016 ceph-mon[30375]: osdmap e67: 12 total, 12 up, 12 in 2024-09-21T03:42:07.188 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:06 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:42:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:06 smithi134 ceph-mon[35031]: purged_snaps scrub starts 2024-09-21T03:42:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:06 smithi134 ceph-mon[35031]: purged_snaps scrub ok 2024-09-21T03:42:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:06 smithi134 ceph-mon[35031]: Adjusting osd_memory_target on smithi134 to 4272M 2024-09-21T03:42:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:06 smithi134 ceph-mon[35031]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:42:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:06 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:42:07.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:06 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2191790572' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:42:07.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:06 smithi134 ceph-mon[35031]: osd.11 [v2:172.21.15.134:6824/584758685,v1:172.21.15.134:6825/584758685] boot 2024-09-21T03:42:07.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:06 smithi134 ceph-mon[35031]: osdmap e67: 12 total, 12 up, 12 in 2024-09-21T03:42:07.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:06 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:42:07.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:06 smithi123 ceph-mon[34673]: purged_snaps scrub starts 2024-09-21T03:42:07.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:06 smithi123 ceph-mon[34673]: purged_snaps scrub ok 2024-09-21T03:42:07.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:06 smithi123 ceph-mon[34673]: Adjusting osd_memory_target on smithi134 to 4272M 2024-09-21T03:42:07.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:06 smithi123 ceph-mon[34673]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:42:07.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:06 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:42:07.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:06 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2191790572' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:42:07.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:06 smithi123 ceph-mon[34673]: osd.11 [v2:172.21.15.134:6824/584758685,v1:172.21.15.134:6825/584758685] boot 2024-09-21T03:42:07.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:06 smithi123 ceph-mon[34673]: osdmap e67: 12 total, 12 up, 12 in 2024-09-21T03:42:07.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:06 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:42:07.639 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":66,"num_osds":12,"num_up_osds":11,"osd_up_since":1726890101,"num_in_osds":12,"osd_in_since":1726890103,"num_remapped_pgs":0} 2024-09-21T03:42:08.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:42:08.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:08 smithi123 ceph-mon[34673]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:08.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:08 smithi123 ceph-mon[34673]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T03:42:08.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:08 smithi016 ceph-mon[30375]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:08.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:08 smithi016 ceph-mon[30375]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T03:42:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:09.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:08 smithi134 ceph-mon[35031]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:09.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:08 smithi134 ceph-mon[35031]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T03:42:09.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.663+0000 7fbe92400640 1 -- 172.21.15.16:0/3260275193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe8c105180 msgr2=0x7fbe8c10bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.663+0000 7fbe92400640 1 --2- 172.21.15.16:0/3260275193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe8c105180 0x7fbe8c10bb20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009be0 tx=0x7fbe8402a750 comp rx=0 tx=0).stop 2024-09-21T03:42:09.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.663+0000 7fbe92400640 1 -- 172.21.15.16:0/3260275193 shutdown_connections 2024-09-21T03:42:09.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.663+0000 7fbe92400640 1 --2- 172.21.15.16:0/3260275193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe8c105180 0x7fbe8c10bb20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.663+0000 7fbe92400640 1 --2- 172.21.15.16:0/3260275193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe8c1047e0 0x7fbe8c104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.663+0000 7fbe92400640 1 --2- 172.21.15.16:0/3260275193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c103750 0x7fbe8c103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.663+0000 7fbe92400640 1 -- 172.21.15.16:0/3260275193 >> 172.21.15.16:0/3260275193 conn(0x7fbe8c0ff0c0 msgr2=0x7fbe8c101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:09.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.663+0000 7fbe92400640 1 -- 172.21.15.16:0/3260275193 shutdown_connections 2024-09-21T03:42:09.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.663+0000 7fbe92400640 1 -- 172.21.15.16:0/3260275193 wait complete. 2024-09-21T03:42:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.664+0000 7fbe92400640 1 Processor -- start 2024-09-21T03:42:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.664+0000 7fbe92400640 1 -- start start 2024-09-21T03:42:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.665+0000 7fbe92400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe8c103750 0x7fbe8c19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.665+0000 7fbe92400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c1047e0 0x7fbe8c19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.665+0000 7fbe92400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe8c105180 0x7fbe8c19d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.665+0000 7fbe92400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c10f3a0 con 0x7fbe8c1047e0 2024-09-21T03:42:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.665+0000 7fbe92400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c10f220 con 0x7fbe8c105180 2024-09-21T03:42:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.665+0000 7fbe92400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c10f520 con 0x7fbe8c103750 2024-09-21T03:42:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.665+0000 7fbe8b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe8c103750 0x7fbe8c19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.665+0000 7fbe8b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe8c103750 0x7fbe8c19c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:54764/0 (socket says 172.21.15.16:54764) 2024-09-21T03:42:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.665+0000 7fbe8b400640 1 -- 172.21.15.16:0/1302226701 learned_addr learned my addr 172.21.15.16:0/1302226701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:42:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.665+0000 7fbe82a00640 1 --2- 172.21.15.16:0/1302226701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c1047e0 0x7fbe8c19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.665+0000 7fbe8be00640 1 --2- 172.21.15.16:0/1302226701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe8c105180 0x7fbe8c19d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.665+0000 7fbe8b400640 1 -- 172.21.15.16:0/1302226701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe8c105180 msgr2=0x7fbe8c19d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.665+0000 7fbe8b400640 1 --2- 172.21.15.16:0/1302226701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe8c105180 0x7fbe8c19d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.665+0000 7fbe8b400640 1 -- 172.21.15.16:0/1302226701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c1047e0 msgr2=0x7fbe8c19c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.665+0000 7fbe8b400640 1 --2- 172.21.15.16:0/1302226701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c1047e0 0x7fbe8c19c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.665+0000 7fbe8b400640 1 -- 172.21.15.16:0/1302226701 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8c19dba0 con 0x7fbe8c103750 2024-09-21T03:42:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.665+0000 7fbe8be00640 1 --2- 172.21.15.16:0/1302226701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe8c105180 0x7fbe8c19d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.666+0000 7fbe8b400640 1 --2- 172.21.15.16:0/1302226701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe8c103750 0x7fbe8c19c1e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00e9c0 tx=0x7fbe7c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.666+0000 7fbe88c00640 1 -- 172.21.15.16:0/1302226701 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c00ccc0 con 0x7fbe8c103750 2024-09-21T03:42:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.666+0000 7fbe92400640 1 -- 172.21.15.16:0/1302226701 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe8c19de90 con 0x7fbe8c103750 2024-09-21T03:42:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.666+0000 7fbe88c00640 1 -- 172.21.15.16:0/1302226701 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe7c0054d0 con 0x7fbe8c103750 2024-09-21T03:42:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.666+0000 7fbe88c00640 1 -- 172.21.15.16:0/1302226701 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c010430 con 0x7fbe8c103750 2024-09-21T03:42:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.666+0000 7fbe92400640 1 -- 172.21.15.16:0/1302226701 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe8c1a4590 con 0x7fbe8c103750 2024-09-21T03:42:09.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.667+0000 7fbe92400640 1 -- 172.21.15.16:0/1302226701 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe8c1a4760 con 0x7fbe8c103750 2024-09-21T03:42:09.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.670+0000 7fbe88c00640 1 -- 172.21.15.16:0/1302226701 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fbe7c004f80 con 0x7fbe8c103750 2024-09-21T03:42:09.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.671+0000 7fbe88c00640 1 --2- 172.21.15.16:0/1302226701 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbe700b6510 0x7fbe700b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.672+0000 7fbe88c00640 1 -- 172.21.15.16:0/1302226701 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7fbe7c0e76f0 con 0x7fbe8c103750 2024-09-21T03:42:09.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.672+0000 7fbe82a00640 1 --2- 172.21.15.16:0/1302226701 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbe700b6510 0x7fbe700b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.672+0000 7fbe82a00640 1 --2- 172.21.15.16:0/1302226701 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbe700b6510 0x7fbe700b8a20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009bd0 tx=0x7fbe74005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.677+0000 7fbe88c00640 1 -- 172.21.15.16:0/1302226701 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbe7c0ae380 con 0x7fbe8c103750 2024-09-21T03:42:09.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.878+0000 7fbe92400640 1 -- 172.21.15.16:0/1302226701 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fbe8c1a4930 con 0x7fbe8c103750 2024-09-21T03:42:09.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.879+0000 7fbe88c00640 1 -- 172.21.15.16:0/1302226701 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7fbe7c0b2760 con 0x7fbe8c103750 2024-09-21T03:42:09.882 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:42:09.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.881+0000 7fbe92400640 1 -- 172.21.15.16:0/1302226701 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbe700b6510 msgr2=0x7fbe700b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.881+0000 7fbe92400640 1 --2- 172.21.15.16:0/1302226701 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbe700b6510 0x7fbe700b8a20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009bd0 tx=0x7fbe74005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:09.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.881+0000 7fbe92400640 1 -- 172.21.15.16:0/1302226701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe8c103750 msgr2=0x7fbe8c19c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.881+0000 7fbe92400640 1 --2- 172.21.15.16:0/1302226701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe8c103750 0x7fbe8c19c1e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00e9c0 tx=0x7fbe7c00ee80 comp rx=0 tx=0).stop 2024-09-21T03:42:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.881+0000 7fbe92400640 1 -- 172.21.15.16:0/1302226701 shutdown_connections 2024-09-21T03:42:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.881+0000 7fbe92400640 1 --2- 172.21.15.16:0/1302226701 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbe700b6510 0x7fbe700b8a20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.881+0000 7fbe92400640 1 --2- 172.21.15.16:0/1302226701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe8c105180 0x7fbe8c19d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.881+0000 7fbe92400640 1 --2- 172.21.15.16:0/1302226701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c1047e0 0x7fbe8c19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.881+0000 7fbe92400640 1 --2- 172.21.15.16:0/1302226701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe8c103750 0x7fbe8c19c1e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.881+0000 7fbe92400640 1 -- 172.21.15.16:0/1302226701 >> 172.21.15.16:0/1302226701 conn(0x7fbe8c0ff0c0 msgr2=0x7fbe8c0755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.882+0000 7fbe92400640 1 -- 172.21.15.16:0/1302226701 shutdown_connections 2024-09-21T03:42:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:09.882+0000 7fbe92400640 1 -- 172.21.15.16:0/1302226701 wait complete. 2024-09-21T03:42:10.628 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":68,"num_osds":12,"num_up_osds":12,"osd_up_since":1726890126,"num_in_osds":12,"osd_in_since":1726890103,"num_remapped_pgs":0} 2024-09-21T03:42:10.628 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:42:10.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:10 smithi123 ceph-mon[34673]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:10.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:10 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1302226701' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:42:10.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:10 smithi016 ceph-mon[30375]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:10.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:10 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1302226701' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:42:11.001 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:11.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:10 smithi134 ceph-mon[35031]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:11.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:10 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1302226701' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:42:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.662+0000 7fd2a7a00640 1 -- 172.21.15.16:0/753756507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a010b6c0 msgr2=0x7fd2a010bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.662+0000 7fd2a7a00640 1 --2- 172.21.15.16:0/753756507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a010b6c0 0x7fd2a010bad0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd288009900 tx=0x7fd28802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.662+0000 7fd2a7a00640 1 -- 172.21.15.16:0/753756507 shutdown_connections 2024-09-21T03:42:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.662+0000 7fd2a7a00640 1 --2- 172.21.15.16:0/753756507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a010c010 0x7fd2a0112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.662+0000 7fd2a7a00640 1 --2- 172.21.15.16:0/753756507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a010b6c0 0x7fd2a010bad0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.662+0000 7fd2a7a00640 1 --2- 172.21.15.16:0/753756507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a0105d80 0x7fd2a0106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.662+0000 7fd2a7a00640 1 -- 172.21.15.16:0/753756507 >> 172.21.15.16:0/753756507 conn(0x7fd2a0101810 msgr2=0x7fd2a0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.663+0000 7fd2a7a00640 1 -- 172.21.15.16:0/753756507 shutdown_connections 2024-09-21T03:42:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.663+0000 7fd2a7a00640 1 -- 172.21.15.16:0/753756507 wait complete. 2024-09-21T03:42:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.664+0000 7fd2a7a00640 1 Processor -- start 2024-09-21T03:42:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.664+0000 7fd2a7a00640 1 -- start start 2024-09-21T03:42:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.665+0000 7fd2a7a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a0105d80 0x7fd2a019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.665+0000 7fd2a7a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a010b6c0 0x7fd2a019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.665+0000 7fd2a7a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a010c010 0x7fd2a01a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.665+0000 7fd2a7a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a0115d20 con 0x7fd2a0105d80 2024-09-21T03:42:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.665+0000 7fd2a7a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a0115ba0 con 0x7fd2a010c010 2024-09-21T03:42:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.665+0000 7fd2a7a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a0115ea0 con 0x7fd2a010b6c0 2024-09-21T03:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.665+0000 7fd2a5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a0105d80 0x7fd2a019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.665+0000 7fd2a5a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a010c010 0x7fd2a01a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.665+0000 7fd2a5a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a010c010 0x7fd2a01a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:46502/0 (socket says 172.21.15.16:46502) 2024-09-21T03:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.665+0000 7fd2a5a00640 1 -- 172.21.15.16:0/2125713115 learned_addr learned my addr 172.21.15.16:0/2125713115 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.665+0000 7fd2a5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a0105d80 0x7fd2a019c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45714/0 (socket says 172.21.15.16:45714) 2024-09-21T03:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.665+0000 7fd297e00640 1 --2- 172.21.15.16:0/2125713115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a010b6c0 0x7fd2a019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.666+0000 7fd2a5a00640 1 -- 172.21.15.16:0/2125713115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a010b6c0 msgr2=0x7fd2a019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.666+0000 7fd2a5a00640 1 --2- 172.21.15.16:0/2125713115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a010b6c0 0x7fd2a019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.666+0000 7fd2a5a00640 1 -- 172.21.15.16:0/2125713115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a0105d80 msgr2=0x7fd2a019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.666+0000 7fd2a5a00640 1 --2- 172.21.15.16:0/2125713115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a0105d80 0x7fd2a019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.666+0000 7fd2a5a00640 1 -- 172.21.15.16:0/2125713115 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a01a2640 con 0x7fd2a010c010 2024-09-21T03:42:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.666+0000 7fd2a5a00640 1 --2- 172.21.15.16:0/2125713115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a010c010 0x7fd2a01a1f10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd29800eb10 tx=0x7fd29800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.666+0000 7fd29e000640 1 -- 172.21.15.16:0/2125713115 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298004b20 con 0x7fd2a010c010 2024-09-21T03:42:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.666+0000 7fd29e000640 1 -- 172.21.15.16:0/2125713115 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd298004cc0 con 0x7fd2a010c010 2024-09-21T03:42:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.666+0000 7fd29e000640 1 -- 172.21.15.16:0/2125713115 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298010670 con 0x7fd2a010c010 2024-09-21T03:42:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.666+0000 7fd2a7a00640 1 -- 172.21.15.16:0/2125713115 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a01a2930 con 0x7fd2a010c010 2024-09-21T03:42:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.666+0000 7fd2a7a00640 1 -- 172.21.15.16:0/2125713115 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a01a9090 con 0x7fd2a010c010 2024-09-21T03:42:11.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.667+0000 7fd2a7a00640 1 -- 172.21.15.16:0/2125713115 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a0106190 con 0x7fd2a010c010 2024-09-21T03:42:11.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.670+0000 7fd29e000640 1 -- 172.21.15.16:0/2125713115 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fd298005020 con 0x7fd2a010c010 2024-09-21T03:42:11.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.671+0000 7fd29e000640 1 --2- 172.21.15.16:0/2125713115 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd2840b6590 0x7fd2840b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.671+0000 7fd29e000640 1 -- 172.21.15.16:0/2125713115 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7fd2980e76c0 con 0x7fd2a010c010 2024-09-21T03:42:11.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.672+0000 7fd2a5000640 1 --2- 172.21.15.16:0/2125713115 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd2840b6590 0x7fd2840b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.672+0000 7fd2a5000640 1 --2- 172.21.15.16:0/2125713115 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd2840b6590 0x7fd2840b8aa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd290005d00 tx=0x7fd29000b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:11.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.673+0000 7fd29e000640 1 -- 172.21.15.16:0/2125713115 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd2980ae350 con 0x7fd2a010c010 2024-09-21T03:42:11.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.875+0000 7fd2a7a00640 1 -- 172.21.15.16:0/2125713115 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2a019d800 con 0x7fd2a010c010 2024-09-21T03:42:11.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.877+0000 7fd29e000640 1 -- 172.21.15.16:0/2125713115 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+18488 (secure 0 0 0) 0x7fd2a019d800 con 0x7fd2a010c010 2024-09-21T03:42:11.880 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:42:11.880 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":68,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:33:53.220358+0000","modified":"2024-09-21T03:42:07.616080+0000","last_up_change":"2024-09-21T03:42:06.614247+0000","last_in_change":"2024-09-21T03:41:43.379404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:37:55.708915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a7e4b9e-078f-4ae9-ab9e-37be995e4deb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6803","nonce":311974551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6805","nonce":311974551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6809","nonce":311974551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6807","nonce":311974551}]},"public_addr":"172.21.15.16:6803/311974551","cluster_addr":"172.21.15.16:6805/311974551","heartbeat_back_addr":"172.21.15.16:6809/311974551","heartbeat_front_addr":"172.21.15.16:6807/311974551","state":["exists","up"]},{"osd":1,"uuid":"3a497a64-e1a8-43ef-8509-0a79515da0aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6801","nonce":2152486571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6803","nonce":2152486571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6807","nonce":2152486571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6805","nonce":2152486571}]},"public_addr":"172.21.15.123:6801/2152486571","cluster_addr":"172.21.15.123:6803/2152486571","heartbeat_back_addr":"172.21.15.123:6807/2152486571","heartbeat_front_addr":"172.21.15.123:6805/2152486571","state":["exists","up"]},{"osd":2,"uuid":"1c39518e-6cce-4a0f-9ee8-55c60ad325f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6801","nonce":2812351279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6803","nonce":2812351279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6807","nonce":2812351279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6805","nonce":2812351279}]},"public_addr":"172.21.15.134:6801/2812351279","cluster_addr":"172.21.15.134:6803/2812351279","heartbeat_back_addr":"172.21.15.134:6807/2812351279","heartbeat_front_addr":"172.21.15.134:6805/2812351279","state":["exists","up"]},{"osd":3,"uuid":"0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6811","nonce":2366216026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6813","nonce":2366216026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6817","nonce":2366216026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6815","nonce":2366216026}]},"public_addr":"172.21.15.16:6811/2366216026","cluster_addr":"172.21.15.16:6813/2366216026","heartbeat_back_addr":"172.21.15.16:6817/2366216026","heartbeat_front_addr":"172.21.15.16:6815/2366216026","state":["exists","up"]},{"osd":4,"uuid":"ad5e6d0e-31e8-4779-9e74-f215032777ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6809","nonce":2354279116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6811","nonce":2354279116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6815","nonce":2354279116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6813","nonce":2354279116}]},"public_addr":"172.21.15.123:6809/2354279116","cluster_addr":"172.21.15.123:6811/2354279116","heartbeat_back_addr":"172.21.15.123:6815/2354279116","heartbeat_front_addr":"172.21.15.123:6813/2354279116","state":["exists","up"]},{"osd":5,"uuid":"ca09bf88-1772-4dc0-8711-cbf9a5b31977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6809","nonce":2678416182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6811","nonce":2678416182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6815","nonce":2678416182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6813","nonce":2678416182}]},"public_addr":"172.21.15.134:6809/2678416182","cluster_addr":"172.21.15.134:6811/2678416182","heartbeat_back_addr":"172.21.15.134:6815/2678416182","heartbeat_front_addr":"172.21.15.134:6813/2678416182","state":["exists","up"]},{"osd":6,"uuid":"0d41ed4e-8699-4d88-9254-12d3b84941e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6819","nonce":3377807744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6821","nonce":3377807744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6825","nonce":3377807744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6823","nonce":3377807744}]},"public_addr":"172.21.15.16:6819/3377807744","cluster_addr":"172.21.15.16:6821/3377807744","heartbeat_back_addr":"172.21.15.16:6825/3377807744","heartbeat_front_addr":"172.21.15.16:6823/3377807744","state":["exists","up"]},{"osd":7,"uuid":"1541a272-c524-4f12-9b94-763f69bb1c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6817","nonce":1463412739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6819","nonce":1463412739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6823","nonce":1463412739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6821","nonce":1463412739}]},"public_addr":"172.21.15.123:6817/1463412739","cluster_addr":"172.21.15.123:6819/1463412739","heartbeat_back_addr":"172.21.15.123:6823/1463412739","heartbeat_front_addr":"172.21.15.123:6821/1463412739","state":["exists","up"]},{"osd":8,"uuid":"cc00014c-eb99-49ab-984c-0bc26ef85130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6817","nonce":2644021450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6819","nonce":2644021450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6823","nonce":2644021450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6821","nonce":2644021450}]},"public_addr":"172.21.15.134:6817/2644021450","cluster_addr":"172.21.15.134:6819/2644021450","heartbeat_back_addr":"172.21.15.134:6823/2644021450","heartbeat_front_addr":"172.21.15.134:6821/2644021450","state":["exists","up"]},{"osd":9,"uuid":"3cbfff74-31ff-4ffd-844b-54ec8563ba32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6827","nonce":1653164371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6829","nonce":1653164371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6833","nonce":1653164371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6831","nonce":1653164371}]},"public_addr":"172.21.15.16:6827/1653164371","cluster_addr":"172.21.15.16:6829/1653164371","heartbeat_back_addr":"172.21.15.16:6833/1653164371","heartbeat_front_addr":"172.21.15.16:6831/1653164371","state":["exists","up"]},{"osd":10,"uuid":"0a1caaad-70cd-4c2c-88d3-e9a5cc885655","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6825","nonce":3682234496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6827","nonce":3682234496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6831","nonce":3682234496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6829","nonce":3682234496}]},"public_addr":"172.21.15.123:6825/3682234496","cluster_addr":"172.21.15.123:6827/3682234496","heartbeat_back_addr":"172.21.15.123:6831/3682234496","heartbeat_front_addr":"172.21.15.123:6829/3682234496","state":["exists","up"]},{"osd":11,"uuid":"3aff3258-5a38-41ff-aac3-4a082feffdae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6825","nonce":584758685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6827","nonce":584758685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6831","nonce":584758685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6829","nonce":584758685}]},"public_addr":"172.21.15.134:6825/584758685","cluster_addr":"172.21.15.134:6827/584758685","heartbeat_back_addr":"172.21.15.134:6831/584758685","heartbeat_front_addr":"172.21.15.134:6829/584758685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:02.177731+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:28.847562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:54.272384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:24.338596+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:52.430371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:17.832737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:50.397890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:16.842245+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:42.102933+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:13.074732+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:40.057905+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:04.898190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/557742783":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/3238676548":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6801/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/2357336233":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6801/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6800/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/3806844703":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/2351581621":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6800/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/355731724":"2024-09-22T03:34:22.549156+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:42:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.879+0000 7fd2a7a00640 1 -- 172.21.15.16:0/2125713115 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd2840b6590 msgr2=0x7fd2840b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.879+0000 7fd2a7a00640 1 --2- 172.21.15.16:0/2125713115 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd2840b6590 0x7fd2840b8aa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd290005d00 tx=0x7fd29000b410 comp rx=0 tx=0).stop 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.879+0000 7fd2a7a00640 1 -- 172.21.15.16:0/2125713115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a010c010 msgr2=0x7fd2a01a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.879+0000 7fd2a7a00640 1 --2- 172.21.15.16:0/2125713115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a010c010 0x7fd2a01a1f10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd29800eb10 tx=0x7fd29800c2d0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.880+0000 7fd2a7a00640 1 -- 172.21.15.16:0/2125713115 shutdown_connections 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.880+0000 7fd2a7a00640 1 --2- 172.21.15.16:0/2125713115 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd2840b6590 0x7fd2840b8aa0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.880+0000 7fd2a7a00640 1 --2- 172.21.15.16:0/2125713115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a010c010 0x7fd2a01a1f10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.880+0000 7fd2a7a00640 1 --2- 172.21.15.16:0/2125713115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a010b6c0 0x7fd2a019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.880+0000 7fd2a7a00640 1 --2- 172.21.15.16:0/2125713115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a0105d80 0x7fd2a019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.880+0000 7fd2a7a00640 1 -- 172.21.15.16:0/2125713115 >> 172.21.15.16:0/2125713115 conn(0x7fd2a0101810 msgr2=0x7fd2a0103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.880+0000 7fd2a7a00640 1 -- 172.21.15.16:0/2125713115 shutdown_connections 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:11.880+0000 7fd2a7a00640 1 -- 172.21.15.16:0/2125713115 wait complete. 2024-09-21T03:42:12.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:12 smithi016 ceph-mon[30375]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:12.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:12 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2125713115' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:42:12.772 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:37:55.708915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:42:12.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T03:42:12.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:12 smithi123 ceph-mon[34673]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:12.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:12 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2125713115' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:42:13.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:12 smithi134 ceph-mon[35031]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:13.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:12 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2125713115' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:42:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.809+0000 7f53eda00640 1 -- 172.21.15.16:0/1206956595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e81092f0 msgr2=0x7f53e8109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.809+0000 7f53eda00640 1 --2- 172.21.15.16:0/1206956595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e81092f0 0x7f53e8109700 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009900 tx=0x7f53d402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.809+0000 7f53eda00640 1 -- 172.21.15.16:0/1206956595 shutdown_connections 2024-09-21T03:42:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.809+0000 7f53eda00640 1 --2- 172.21.15.16:0/1206956595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e8109c40 0x7f53e8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.809+0000 7f53eda00640 1 --2- 172.21.15.16:0/1206956595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e81092f0 0x7f53e8109700 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.809+0000 7f53eda00640 1 --2- 172.21.15.16:0/1206956595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53e81039b0 0x7f53e8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.809+0000 7f53eda00640 1 -- 172.21.15.16:0/1206956595 >> 172.21.15.16:0/1206956595 conn(0x7f53e80ff440 msgr2=0x7f53e8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.809+0000 7f53eda00640 1 -- 172.21.15.16:0/1206956595 shutdown_connections 2024-09-21T03:42:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.809+0000 7f53eda00640 1 -- 172.21.15.16:0/1206956595 wait complete. 2024-09-21T03:42:13.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.810+0000 7f53eda00640 1 Processor -- start 2024-09-21T03:42:13.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.811+0000 7f53eda00640 1 -- start start 2024-09-21T03:42:13.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.811+0000 7f53eda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53e81039b0 0x7f53e819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.811+0000 7f53eda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e81092f0 0x7f53e819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.811+0000 7f53eda00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e8109c40 0x7f53e81a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.811+0000 7f53eda00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8113950 con 0x7f53e81092f0 2024-09-21T03:42:13.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.811+0000 7f53eda00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81137d0 con 0x7f53e8109c40 2024-09-21T03:42:13.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.811+0000 7f53eda00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8113ad0 con 0x7f53e81039b0 2024-09-21T03:42:13.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.811+0000 7f53e7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53e81039b0 0x7f53e819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.811+0000 7f53e7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53e81039b0 0x7f53e819c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:40220/0 (socket says 172.21.15.16:40220) 2024-09-21T03:42:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.811+0000 7f53e7e00640 1 -- 172.21.15.16:0/1183602739 learned_addr learned my addr 172.21.15.16:0/1183602739 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:42:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.811+0000 7f53e7400640 1 --2- 172.21.15.16:0/1183602739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e81092f0 0x7f53e819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.811+0000 7f53ed000640 1 --2- 172.21.15.16:0/1183602739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e8109c40 0x7f53e81a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.812+0000 7f53e7e00640 1 -- 172.21.15.16:0/1183602739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e8109c40 msgr2=0x7f53e81a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.812+0000 7f53e7e00640 1 --2- 172.21.15.16:0/1183602739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e8109c40 0x7f53e81a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.812+0000 7f53e7e00640 1 -- 172.21.15.16:0/1183602739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e81092f0 msgr2=0x7f53e819cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.812+0000 7f53e7e00640 1 --2- 172.21.15.16:0/1183602739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e81092f0 0x7f53e819cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.812+0000 7f53e7e00640 1 -- 172.21.15.16:0/1183602739 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e81a24b0 con 0x7f53e81039b0 2024-09-21T03:42:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.812+0000 7f53e7e00640 1 --2- 172.21.15.16:0/1183602739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53e81039b0 0x7f53e819c540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f53d800d950 tx=0x7f53d800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.812+0000 7f53e4c00640 1 -- 172.21.15.16:0/1183602739 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8017070 con 0x7f53e81039b0 2024-09-21T03:42:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.812+0000 7f53e4c00640 1 -- 172.21.15.16:0/1183602739 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53d8009c30 con 0x7f53e81039b0 2024-09-21T03:42:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.812+0000 7f53eda00640 1 -- 172.21.15.16:0/1183602739 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e81a27a0 con 0x7f53e81039b0 2024-09-21T03:42:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.813+0000 7f53e4c00640 1 -- 172.21.15.16:0/1183602739 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d800f620 con 0x7f53e81039b0 2024-09-21T03:42:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.813+0000 7f53eda00640 1 -- 172.21.15.16:0/1183602739 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e81a8f00 con 0x7f53e81039b0 2024-09-21T03:42:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.814+0000 7f53eda00640 1 -- 172.21.15.16:0/1183602739 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e8103dc0 con 0x7f53e81039b0 2024-09-21T03:42:13.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.816+0000 7f53e4c00640 1 -- 172.21.15.16:0/1183602739 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f53d8005030 con 0x7f53e81039b0 2024-09-21T03:42:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.818+0000 7f53e4c00640 1 --2- 172.21.15.16:0/1183602739 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f53cc0b6510 0x7f53cc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.818+0000 7f53e4c00640 1 -- 172.21.15.16:0/1183602739 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f53d80e6a10 con 0x7f53e81039b0 2024-09-21T03:42:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.818+0000 7f53e7400640 1 --2- 172.21.15.16:0/1183602739 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f53cc0b6510 0x7f53cc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.819+0000 7f53e7400640 1 --2- 172.21.15.16:0/1183602739 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f53cc0b6510 0x7f53cc0b8a20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f53d402a9e0 tx=0x7f53d4005a20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:13.820+0000 7f53e4c00640 1 -- 172.21.15.16:0/1183602739 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f53d8009de0 con 0x7f53e81039b0 2024-09-21T03:42:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:14.016+0000 7f53eda00640 1 -- 172.21.15.16:0/1183602739 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f53e81a9160 con 0x7f53e81039b0 2024-09-21T03:42:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:14.017+0000 7f53e4c00640 1 -- 172.21.15.16:0/1183602739 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) ==== 93+0+10 (secure 0 0 0) 0x7f53d80b1a80 con 0x7f53e81039b0 2024-09-21T03:42:14.019 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-21T03:42:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:14.019+0000 7f53eda00640 1 -- 172.21.15.16:0/1183602739 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f53cc0b6510 msgr2=0x7f53cc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:14.019+0000 7f53eda00640 1 --2- 172.21.15.16:0/1183602739 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f53cc0b6510 0x7f53cc0b8a20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f53d402a9e0 tx=0x7f53d4005a20 comp rx=0 tx=0).stop 2024-09-21T03:42:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:14.019+0000 7f53eda00640 1 -- 172.21.15.16:0/1183602739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53e81039b0 msgr2=0x7f53e819c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:14.019+0000 7f53eda00640 1 --2- 172.21.15.16:0/1183602739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53e81039b0 0x7f53e819c540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f53d800d950 tx=0x7f53d800de10 comp rx=0 tx=0).stop 2024-09-21T03:42:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:14.019+0000 7f53eda00640 1 -- 172.21.15.16:0/1183602739 shutdown_connections 2024-09-21T03:42:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:14.019+0000 7f53eda00640 1 --2- 172.21.15.16:0/1183602739 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f53cc0b6510 0x7f53cc0b8a20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:14.019+0000 7f53eda00640 1 --2- 172.21.15.16:0/1183602739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e8109c40 0x7f53e81a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:14.019+0000 7f53eda00640 1 --2- 172.21.15.16:0/1183602739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e81092f0 0x7f53e819cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:14.019+0000 7f53eda00640 1 --2- 172.21.15.16:0/1183602739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53e81039b0 0x7f53e819c540 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:14.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:14.019+0000 7f53eda00640 1 -- 172.21.15.16:0/1183602739 >> 172.21.15.16:0/1183602739 conn(0x7f53e80ff440 msgr2=0x7f53e810e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:14.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:14.019+0000 7f53eda00640 1 -- 172.21.15.16:0/1183602739 shutdown_connections 2024-09-21T03:42:14.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:14.019+0000 7f53eda00640 1 -- 172.21.15.16:0/1183602739 wait complete. 2024-09-21T03:42:14.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:14 smithi016 ceph-mon[30375]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:14.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:14 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1183602739' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:42:14.800 INFO:tasks.cephadm:Adding mds.a on smithi016 2024-09-21T03:42:14.800 INFO:tasks.cephadm:Adding mds.d on smithi016 2024-09-21T03:42:14.800 INFO:tasks.cephadm:Adding mds.g on smithi016 2024-09-21T03:42:14.800 INFO:tasks.cephadm:Adding mds.j on smithi016 2024-09-21T03:42:14.800 INFO:tasks.cephadm:Adding mds.b on smithi123 2024-09-21T03:42:14.800 INFO:tasks.cephadm:Adding mds.e on smithi123 2024-09-21T03:42:14.800 INFO:tasks.cephadm:Adding mds.h on smithi123 2024-09-21T03:42:14.801 INFO:tasks.cephadm:Adding mds.k on smithi123 2024-09-21T03:42:14.801 INFO:tasks.cephadm:Adding mds.c on smithi134 2024-09-21T03:42:14.801 INFO:tasks.cephadm:Adding mds.f on smithi134 2024-09-21T03:42:14.801 INFO:tasks.cephadm:Adding mds.i on smithi134 2024-09-21T03:42:14.801 INFO:tasks.cephadm:Adding mds.l on smithi134 2024-09-21T03:42:14.801 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l' 2024-09-21T03:42:14.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:14 smithi123 ceph-mon[34673]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:14.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:14 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1183602739' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:42:15.064 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:42:15.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:14 smithi134 ceph-mon[35031]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:15.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:14 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1183602739' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:42:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.582+0000 7f86aac30640 1 -- 172.21.15.134:0/950072763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86a4104080 msgr2=0x7f86a410aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.582+0000 7f86aac30640 1 --2- 172.21.15.134:0/950072763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86a4104080 0x7f86a410aa20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f869c009ca0 tx=0x7f869c02a950 comp rx=0 tx=0).stop 2024-09-21T03:42:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.582+0000 7f86aac30640 1 -- 172.21.15.134:0/950072763 shutdown_connections 2024-09-21T03:42:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.582+0000 7f86aac30640 1 --2- 172.21.15.134:0/950072763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86a4104080 0x7f86a410aa20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.582+0000 7f86aac30640 1 --2- 172.21.15.134:0/950072763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86a41036e0 0x7f86a4103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.582+0000 7f86aac30640 1 --2- 172.21.15.134:0/950072763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a4102650 0x7f86a4102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.582+0000 7f86aac30640 1 -- 172.21.15.134:0/950072763 >> 172.21.15.134:0/950072763 conn(0x7f86a40fdfe0 msgr2=0x7f86a4100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.582+0000 7f86aac30640 1 -- 172.21.15.134:0/950072763 shutdown_connections 2024-09-21T03:42:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.582+0000 7f86aac30640 1 -- 172.21.15.134:0/950072763 wait complete. 2024-09-21T03:42:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.583+0000 7f86aac30640 1 Processor -- start 2024-09-21T03:42:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.583+0000 7f86aac30640 1 -- start start 2024-09-21T03:42:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.583+0000 7f86aac30640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a4102650 0x7f86a4199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.583+0000 7f86aac30640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86a41036e0 0x7f86a419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.583+0000 7f86aac30640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86a4104080 0x7f86a419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.583+0000 7f86aac30640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86a410e280 con 0x7f86a41036e0 2024-09-21T03:42:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.583+0000 7f86aac30640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f86a410e100 con 0x7f86a4102650 2024-09-21T03:42:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.583+0000 7f86aac30640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86a410e400 con 0x7f86a4104080 2024-09-21T03:42:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.583+0000 7f86a91cd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86a4104080 0x7f86a419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.583+0000 7f86a91cd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86a4104080 0x7f86a419b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:60478/0 (socket says 172.21.15.134:60478) 2024-09-21T03:42:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.583+0000 7f86a91cd640 1 -- 172.21.15.134:0/1372858860 learned_addr learned my addr 172.21.15.134:0/1372858860 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-21T03:42:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.583+0000 7f86a89cc640 1 --2- 172.21.15.134:0/1372858860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a4102650 0x7f86a4199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.583+0000 7f869bfff640 1 --2- 172.21.15.134:0/1372858860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86a41036e0 0x7f86a419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.584+0000 7f86a91cd640 1 -- 172.21.15.134:0/1372858860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a4102650 msgr2=0x7f86a4199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.584+0000 7f86a91cd640 1 --2- 172.21.15.134:0/1372858860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a4102650 0x7f86a4199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.584+0000 7f86a91cd640 1 -- 172.21.15.134:0/1372858860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86a41036e0 msgr2=0x7f86a419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.584+0000 7f86a91cd640 1 --2- 172.21.15.134:0/1372858860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86a41036e0 0x7f86a419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.584+0000 7f86a91cd640 1 -- 172.21.15.134:0/1372858860 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a419b9a0 con 0x7f86a4104080 2024-09-21T03:42:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.584+0000 7f86a91cd640 1 --2- 172.21.15.134:0/1372858860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86a4104080 0x7f86a419b180 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f869c006010 tx=0x7f869c02fc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.584+0000 7f8699ffb640 1 -- 172.21.15.134:0/1372858860 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c034040 con 0x7f86a4104080 2024-09-21T03:42:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.584+0000 7f86aac30640 1 -- 172.21.15.134:0/1372858860 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a419bc30 con 0x7f86a4104080 2024-09-21T03:42:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.584+0000 7f8699ffb640 1 -- 172.21.15.134:0/1372858860 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f869c0308f0 con 0x7f86a4104080 2024-09-21T03:42:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.584+0000 7f8699ffb640 1 -- 172.21.15.134:0/1372858860 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c039450 con 0x7f86a4104080 2024-09-21T03:42:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.584+0000 7f86aac30640 1 -- 172.21.15.134:0/1372858860 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a41a21c0 con 0x7f86a4104080 2024-09-21T03:42:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.585+0000 7f86aac30640 1 -- 172.21.15.134:0/1372858860 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86a41a2420 con 0x7f86a4104080 2024-09-21T03:42:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.587+0000 7f8699ffb640 1 -- 172.21.15.134:0/1372858860 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f869c037020 con 0x7f86a4104080 2024-09-21T03:42:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.588+0000 7f8699ffb640 1 --2- 172.21.15.134:0/1372858860 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f868c0b6510 0x7f868c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.588+0000 7f8699ffb640 1 -- 172.21.15.134:0/1372858860 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f869c1080f0 con 0x7f86a4104080 2024-09-21T03:42:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.588+0000 7f86a89cc640 1 --2- 172.21.15.134:0/1372858860 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f868c0b6510 0x7f868c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.589+0000 7f86a89cc640 1 --2- 172.21.15.134:0/1372858860 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f868c0b6510 0x7f868c0b8a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f86a000bbe0 tx=0x7f86a000b390 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.591+0000 7f8699ffb640 1 -- 172.21.15.134:0/1372858860 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f869c0cee20 con 0x7f86a4104080 2024-09-21T03:42:15.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.780+0000 7f86aac30640 1 -- 172.21.15.134:0/1372858860 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}) -- 0x7f86a41000b0 con 0x7f868c0b6510 2024-09-21T03:42:15.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.789+0000 7f8699ffb640 1 -- 172.21.15.134:0/1372858860 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f86a41000b0 con 0x7f868c0b6510 2024-09-21T03:42:15.791 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mds.all update... 2024-09-21T03:42:15.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.792+0000 7f86aac30640 1 -- 172.21.15.134:0/1372858860 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f868c0b6510 msgr2=0x7f868c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:15.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.792+0000 7f86aac30640 1 --2- 172.21.15.134:0/1372858860 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f868c0b6510 0x7f868c0b8a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f86a000bbe0 tx=0x7f86a000b390 comp rx=0 tx=0).stop 2024-09-21T03:42:15.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.792+0000 7f86aac30640 1 -- 172.21.15.134:0/1372858860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86a4104080 msgr2=0x7f86a419b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:15.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.792+0000 7f86aac30640 1 --2- 172.21.15.134:0/1372858860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86a4104080 0x7f86a419b180 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f869c006010 tx=0x7f869c02fc00 comp rx=0 tx=0).stop 2024-09-21T03:42:15.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.792+0000 7f86aac30640 1 -- 172.21.15.134:0/1372858860 shutdown_connections 2024-09-21T03:42:15.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.792+0000 7f86aac30640 1 --2- 172.21.15.134:0/1372858860 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f868c0b6510 0x7f868c0b8a20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.792+0000 7f86aac30640 1 --2- 172.21.15.134:0/1372858860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86a4104080 0x7f86a419b180 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.792+0000 7f86aac30640 1 --2- 172.21.15.134:0/1372858860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86a41036e0 0x7f86a419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.792+0000 7f86aac30640 1 --2- 172.21.15.134:0/1372858860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a4102650 0x7f86a4199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.792+0000 7f86aac30640 1 -- 172.21.15.134:0/1372858860 >> 172.21.15.134:0/1372858860 conn(0x7f86a40fdfe0 msgr2=0x7f86a40ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:15.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.792+0000 7f86aac30640 1 -- 172.21.15.134:0/1372858860 shutdown_connections 2024-09-21T03:42:15.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T03:42:15.792+0000 7f86aac30640 1 -- 172.21.15.134:0/1372858860 wait complete. 2024-09-21T03:42:16.521 DEBUG:teuthology.orchestra.run.smithi016:mds.a> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T03:42:16.525 DEBUG:teuthology.orchestra.run.smithi016:mds.d> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T03:42:16.570 DEBUG:teuthology.orchestra.run.smithi016:mds.g> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T03:42:16.615 DEBUG:teuthology.orchestra.run.smithi016:mds.j> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T03:42:16.660 DEBUG:teuthology.orchestra.run.smithi123:mds.b> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T03:42:16.663 DEBUG:teuthology.orchestra.run.smithi123:mds.e> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T03:42:16.706 DEBUG:teuthology.orchestra.run.smithi123:mds.h> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T03:42:16.748 DEBUG:teuthology.orchestra.run.smithi123:mds.k> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T03:42:16.791 DEBUG:teuthology.orchestra.run.smithi134:mds.c> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T03:42:16.793 DEBUG:teuthology.orchestra.run.smithi134:mds.f> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T03:42:16.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:16 smithi016 ceph-mon[30375]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:16.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:16 smithi016 ceph-mon[30375]: from='client.24601 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:42:16.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:16 smithi016 ceph-mon[30375]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-09-21T03:42:16.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:16.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:16.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:16.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:16.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:16.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:16.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:16.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:16.838 DEBUG:teuthology.orchestra.run.smithi134:mds.i> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T03:42:16.881 DEBUG:teuthology.orchestra.run.smithi134:mds.l> sudo journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T03:42:16.923 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T03:42:16.924 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:17.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:16 smithi134 ceph-mon[35031]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:17.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:16 smithi134 ceph-mon[35031]: from='client.24601 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:42:17.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:16 smithi134 ceph-mon[35031]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-09-21T03:42:17.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:17.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:17.087 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:17.087 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:17.087 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:17.087 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:17.087 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:17.087 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:17.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:16 smithi123 ceph-mon[34673]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:17.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:16 smithi123 ceph-mon[34673]: from='client.24601 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:42:17.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:16 smithi123 ceph-mon[34673]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-09-21T03:42:17.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:17.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:17.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:17.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:17.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:17.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:17.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:17.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:17.295 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.942+0000 7f7310e00640 1 -- 172.21.15.16:0/2572363277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c10be50 msgr2=0x7f730c112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.942+0000 7f7310e00640 1 --2- 172.21.15.16:0/2572363277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c10be50 0x7f730c112680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7304009ca0 tx=0x7f730402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.942+0000 7f7310e00640 1 -- 172.21.15.16:0/2572363277 shutdown_connections 2024-09-21T03:42:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.942+0000 7f7310e00640 1 --2- 172.21.15.16:0/2572363277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c10be50 0x7f730c112680 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.942+0000 7f7310e00640 1 --2- 172.21.15.16:0/2572363277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f730c10b500 0x7f730c10b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.942+0000 7f7310e00640 1 --2- 172.21.15.16:0/2572363277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f730c105bc0 0x7f730c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.942+0000 7f7310e00640 1 -- 172.21.15.16:0/2572363277 >> 172.21.15.16:0/2572363277 conn(0x7f730c101670 msgr2=0x7f730c103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.942+0000 7f7310e00640 1 -- 172.21.15.16:0/2572363277 shutdown_connections 2024-09-21T03:42:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.942+0000 7f7310e00640 1 -- 172.21.15.16:0/2572363277 wait complete. 2024-09-21T03:42:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.943+0000 7f7310e00640 1 Processor -- start 2024-09-21T03:42:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.943+0000 7f7310e00640 1 -- start start 2024-09-21T03:42:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.944+0000 7f7310e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f730c105bc0 0x7f730c19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.944+0000 7f7310e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f730c10b500 0x7f730c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.944+0000 7f7310e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c10be50 0x7f730c1a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.944+0000 7f7310e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f730c115b10 con 0x7f730c10be50 2024-09-21T03:42:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.944+0000 7f7310e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f730c115990 con 0x7f730c10b500 2024-09-21T03:42:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.944+0000 7f7310e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f730c115c90 con 0x7f730c105bc0 2024-09-21T03:42:17.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.944+0000 7f730be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c10be50 0x7f730c1a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.944+0000 7f7302a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f730c10b500 0x7f730c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.944+0000 7f730be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c10be50 0x7f730c1a1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50938/0 (socket says 172.21.15.16:50938) 2024-09-21T03:42:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.944+0000 7f730be00640 1 -- 172.21.15.16:0/829308092 learned_addr learned my addr 172.21.15.16:0/829308092 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:42:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.944+0000 7f730b400640 1 --2- 172.21.15.16:0/829308092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f730c105bc0 0x7f730c19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.945+0000 7f7302a00640 1 -- 172.21.15.16:0/829308092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f730c105bc0 msgr2=0x7f730c19c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.945+0000 7f7302a00640 1 --2- 172.21.15.16:0/829308092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f730c105bc0 0x7f730c19c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.945+0000 7f7302a00640 1 -- 172.21.15.16:0/829308092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c10be50 msgr2=0x7f730c1a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.945+0000 7f7302a00640 1 --2- 172.21.15.16:0/829308092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c10be50 0x7f730c1a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.945+0000 7f7302a00640 1 -- 172.21.15.16:0/829308092 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730c1a2410 con 0x7f730c10b500 2024-09-21T03:42:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.945+0000 7f7302a00640 1 --2- 172.21.15.16:0/829308092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f730c10b500 0x7f730c19ca40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f730c06c8a0 tx=0x7f72f400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.945+0000 7f7308c00640 1 -- 172.21.15.16:0/829308092 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f400cc40 con 0x7f730c10b500 2024-09-21T03:42:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.945+0000 7f7310e00640 1 -- 172.21.15.16:0/829308092 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730c1a2700 con 0x7f730c10b500 2024-09-21T03:42:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.945+0000 7f7308c00640 1 -- 172.21.15.16:0/829308092 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72f400cde0 con 0x7f730c10b500 2024-09-21T03:42:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.945+0000 7f7308c00640 1 -- 172.21.15.16:0/829308092 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f4018760 con 0x7f730c10b500 2024-09-21T03:42:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.945+0000 7f7310e00640 1 -- 172.21.15.16:0/829308092 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f730c1a8e60 con 0x7f730c10b500 2024-09-21T03:42:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.947+0000 7f7310e00640 1 -- 172.21.15.16:0/829308092 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f730c105fd0 con 0x7f730c10b500 2024-09-21T03:42:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.949+0000 7f7308c00640 1 -- 172.21.15.16:0/829308092 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f72f4010a90 con 0x7f730c10b500 2024-09-21T03:42:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.950+0000 7f7308c00640 1 --2- 172.21.15.16:0/829308092 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f72f00b6510 0x7f72f00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.951+0000 7f7308c00640 1 -- 172.21.15.16:0/829308092 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f72f40e73b0 con 0x7f730c10b500 2024-09-21T03:42:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.951+0000 7f730b400640 1 --2- 172.21.15.16:0/829308092 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f72f00b6510 0x7f72f00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.951+0000 7f730b400640 1 --2- 172.21.15.16:0/829308092 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f72f00b6510 0x7f72f00b8a20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009bd0 tx=0x7f72fc005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:17.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:17.952+0000 7f7308c00640 1 -- 172.21.15.16:0/829308092 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f72f40b2420 con 0x7f730c10b500 2024-09-21T03:42:18.097 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:17 smithi134 ceph-mon[35031]: Deploying daemon mds.l on smithi134 2024-09-21T03:42:18.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:17 smithi123 ceph-mon[34673]: Deploying daemon mds.l on smithi134 2024-09-21T03:42:18.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:17 smithi016 ceph-mon[30375]: Deploying daemon mds.l on smithi134 2024-09-21T03:42:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:18.191+0000 7f7310e00640 1 -- 172.21.15.16:0/829308092 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f730c19d5d0 con 0x7f730c10b500 2024-09-21T03:42:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:18.192+0000 7f7308c00640 1 -- 172.21.15.16:0/829308092 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f72f40b7360 con 0x7f730c10b500 2024-09-21T03:42:18.195 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:42:18.195 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":1,"btime":"2024-09-21T03:33:53:220089+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T03:42:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 1 2024-09-21T03:42:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:18.194+0000 7f7310e00640 1 -- 172.21.15.16:0/829308092 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f72f00b6510 msgr2=0x7f72f00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:18.194+0000 7f7310e00640 1 --2- 172.21.15.16:0/829308092 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f72f00b6510 0x7f72f00b8a20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009bd0 tx=0x7f72fc005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:18.194+0000 7f7310e00640 1 -- 172.21.15.16:0/829308092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f730c10b500 msgr2=0x7f730c19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:18.194+0000 7f7310e00640 1 --2- 172.21.15.16:0/829308092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f730c10b500 0x7f730c19ca40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f730c06c8a0 tx=0x7f72f400ee50 comp rx=0 tx=0).stop 2024-09-21T03:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:18.194+0000 7f7310e00640 1 -- 172.21.15.16:0/829308092 shutdown_connections 2024-09-21T03:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:18.194+0000 7f7310e00640 1 --2- 172.21.15.16:0/829308092 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f72f00b6510 0x7f72f00b8a20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:18.194+0000 7f7310e00640 1 --2- 172.21.15.16:0/829308092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c10be50 0x7f730c1a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:18.194+0000 7f7310e00640 1 --2- 172.21.15.16:0/829308092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f730c10b500 0x7f730c19ca40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:18.194+0000 7f7310e00640 1 --2- 172.21.15.16:0/829308092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f730c105bc0 0x7f730c19c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:18.195+0000 7f7310e00640 1 -- 172.21.15.16:0/829308092 >> 172.21.15.16:0/829308092 conn(0x7f730c101670 msgr2=0x7f730c108b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:18.195+0000 7f7310e00640 1 -- 172.21.15.16:0/829308092 shutdown_connections 2024-09-21T03:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:18.195+0000 7f7310e00640 1 -- 172.21.15.16:0/829308092 wait complete. 2024-09-21T03:42:18.352 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 03:42:18 smithi134 systemd[1]: Starting Ceph mds.l for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:42:18.797 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 03:42:18 smithi134 podman[62849]: 2024-09-21 03:42:18.406356915 +0000 UTC m=+0.015382128 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:42:18.797 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 03:42:18 smithi134 podman[62849]: 2024-09-21 03:42:18.525937345 +0000 UTC m=+0.134962546 container create d160eef181782272dd72429e062c2f647d1fee5de07b5e0d3294acbbed8ee520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-l, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:42:18.798 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 03:42:18 smithi134 podman[62849]: 2024-09-21 03:42:18.795892962 +0000 UTC m=+0.404918164 container init d160eef181782272dd72429e062c2f647d1fee5de07b5e0d3294acbbed8ee520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-l, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:42:18.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:18 smithi016 ceph-mon[30375]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:18.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:18 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/829308092' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:19.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:18 smithi134 ceph-mon[35031]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:19.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:18 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/829308092' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:19.102 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 03:42:18 smithi134 podman[62849]: 2024-09-21 03:42:18.799013275 +0000 UTC m=+0.408038478 container start d160eef181782272dd72429e062c2f647d1fee5de07b5e0d3294acbbed8ee520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-l, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T03:42:19.103 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 03:42:18 smithi134 ceph-mds[62868]: -- 172.21.15.134:0/2499862028 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559eb45a5860 con 0x559eb874da80 2024-09-21T03:42:19.103 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 03:42:18 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-l[62864]: starting mds.l at 2024-09-21T03:42:19.103 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 03:42:18 smithi134 bash[62849]: d160eef181782272dd72429e062c2f647d1fee5de07b5e0d3294acbbed8ee520 2024-09-21T03:42:19.103 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 03:42:18 smithi134 systemd[1]: Started Ceph mds.l for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:42:19.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:18 smithi123 ceph-mon[34673]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:19.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:18 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/829308092' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:19.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:19 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.134:6832/2239407349,v1:172.21.15.134:6833/2239407349] up:boot 2024-09-21T03:42:19.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:19 smithi123 ceph-mon[34673]: fsmap 1 up:standby 2024-09-21T03:42:19.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:42:19.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:19.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:19.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:19.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:19.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:19.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:19.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:19 smithi123 ceph-mon[34673]: Deploying daemon mds.k on smithi123 2024-09-21T03:42:20.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:19 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.134:6832/2239407349,v1:172.21.15.134:6833/2239407349] up:boot 2024-09-21T03:42:20.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:19 smithi134 ceph-mon[35031]: fsmap 1 up:standby 2024-09-21T03:42:20.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:42:20.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:20.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:20.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:20.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:20.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:20.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:20.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:19 smithi134 ceph-mon[35031]: Deploying daemon mds.k on smithi123 2024-09-21T03:42:20.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:19 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.134:6832/2239407349,v1:172.21.15.134:6833/2239407349] up:boot 2024-09-21T03:42:20.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:19 smithi016 ceph-mon[30375]: fsmap 1 up:standby 2024-09-21T03:42:20.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:42:20.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:20.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:20.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:20.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:20.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:20.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:20.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:19 smithi016 ceph-mon[30375]: Deploying daemon mds.k on smithi123 2024-09-21T03:42:21.026 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:21.143 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:20 smithi123 ceph-mon[34673]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:21.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:20 smithi134 ceph-mon[35031]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:21.408 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:21.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:20 smithi016 ceph-mon[30375]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:21.656 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 21 03:42:21 smithi123 systemd[1]: Started Ceph mds.k for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:42:22.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.061+0000 7f9a60000640 1 -- 172.21.15.16:0/3711270398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a58103b80 msgr2=0x7f9a58103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.061+0000 7f9a60000640 1 --2- 172.21.15.16:0/3711270398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a58103b80 0x7f9a58103f90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009900 tx=0x7f9a4c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.061+0000 7f9a60000640 1 -- 172.21.15.16:0/3711270398 shutdown_connections 2024-09-21T03:42:22.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.061+0000 7f9a60000640 1 --2- 172.21.15.16:0/3711270398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a58109e10 0x7f9a58110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.061+0000 7f9a60000640 1 --2- 172.21.15.16:0/3711270398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a581094c0 0x7f9a581098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.061+0000 7f9a60000640 1 --2- 172.21.15.16:0/3711270398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a58103b80 0x7f9a58103f90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.061+0000 7f9a60000640 1 -- 172.21.15.16:0/3711270398 >> 172.21.15.16:0/3711270398 conn(0x7f9a580ff590 msgr2=0x7f9a581019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:22.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.061+0000 7f9a60000640 1 -- 172.21.15.16:0/3711270398 shutdown_connections 2024-09-21T03:42:22.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.061+0000 7f9a60000640 1 -- 172.21.15.16:0/3711270398 wait complete. 2024-09-21T03:42:22.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.062+0000 7f9a60000640 1 Processor -- start 2024-09-21T03:42:22.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.062+0000 7f9a60000640 1 -- start start 2024-09-21T03:42:22.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.062+0000 7f9a60000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a58103b80 0x7f9a58077120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.062+0000 7f9a60000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a581094c0 0x7f9a58075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.063+0000 7f9a60000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a58109e10 0x7f9a58075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.063+0000 7f9a60000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a58113b20 con 0x7f9a58109e10 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.063+0000 7f9a60000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581139a0 con 0x7f9a58103b80 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.063+0000 7f9a60000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9a58113ca0 con 0x7f9a581094c0 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.063+0000 7f9a5e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a58109e10 0x7f9a58075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.063+0000 7f9a5e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a58109e10 0x7f9a58075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36748/0 (socket says 172.21.15.16:36748) 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.063+0000 7f9a5cc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a581094c0 0x7f9a58075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.063+0000 7f9a5e000640 1 -- 172.21.15.16:0/1217329524 learned_addr learned my addr 172.21.15.16:0/1217329524 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.063+0000 7f9a5d600640 1 --2- 172.21.15.16:0/1217329524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a58103b80 0x7f9a58077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.063+0000 7f9a5e000640 1 -- 172.21.15.16:0/1217329524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a581094c0 msgr2=0x7f9a58075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.063+0000 7f9a5e000640 1 --2- 172.21.15.16:0/1217329524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a581094c0 0x7f9a58075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.063+0000 7f9a5e000640 1 -- 172.21.15.16:0/1217329524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a58103b80 msgr2=0x7f9a58077120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.063+0000 7f9a5e000640 1 --2- 172.21.15.16:0/1217329524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a58103b80 0x7f9a58077120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.063+0000 7f9a5e000640 1 -- 172.21.15.16:0/1217329524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a58076470 con 0x7f9a58109e10 2024-09-21T03:42:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.064+0000 7f9a5d600640 1 --2- 172.21.15.16:0/1217329524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a58103b80 0x7f9a58077120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.063+0000 7f9a5cc00640 1 --2- 172.21.15.16:0/1217329524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a581094c0 0x7f9a58075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.064+0000 7f9a5e000640 1 --2- 172.21.15.16:0/1217329524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a58109e10 0x7f9a58075c50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000eb10 tx=0x7f9a5000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.064+0000 7f9a56000640 1 -- 172.21.15.16:0/1217329524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50004b20 con 0x7f9a58109e10 2024-09-21T03:42:22.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.064+0000 7f9a56000640 1 -- 172.21.15.16:0/1217329524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a50004cc0 con 0x7f9a58109e10 2024-09-21T03:42:22.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.064+0000 7f9a60000640 1 -- 172.21.15.16:0/1217329524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a581acf30 con 0x7f9a58109e10 2024-09-21T03:42:22.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.064+0000 7f9a56000640 1 -- 172.21.15.16:0/1217329524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50010670 con 0x7f9a58109e10 2024-09-21T03:42:22.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.064+0000 7f9a60000640 1 -- 172.21.15.16:0/1217329524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a581ad390 con 0x7f9a58109e10 2024-09-21T03:42:22.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.066+0000 7f9a60000640 1 -- 172.21.15.16:0/1217329524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a58103f90 con 0x7f9a58109e10 2024-09-21T03:42:22.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.070+0000 7f9a56000640 1 -- 172.21.15.16:0/1217329524 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f9a50005000 con 0x7f9a58109e10 2024-09-21T03:42:22.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.072+0000 7f9a56000640 1 --2- 172.21.15.16:0/1217329524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9a400b6510 0x7f9a400b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.072+0000 7f9a56000640 1 -- 172.21.15.16:0/1217329524 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f9a500e7530 con 0x7f9a58109e10 2024-09-21T03:42:22.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.072+0000 7f9a5d600640 1 --2- 172.21.15.16:0/1217329524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9a400b6510 0x7f9a400b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.073+0000 7f9a5d600640 1 --2- 172.21.15.16:0/1217329524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9a400b6510 0x7f9a400b8a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c005ab0 tx=0x7f9a4c005a20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.074+0000 7f9a56000640 1 -- 172.21.15.16:0/1217329524 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9a50056710 con 0x7f9a58109e10 2024-09-21T03:42:22.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.313+0000 7f9a60000640 1 -- 172.21.15.16:0/1217329524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a5802e640 con 0x7f9a58109e10 2024-09-21T03:42:22.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.314+0000 7f9a56000640 1 -- 172.21.15.16:0/1217329524 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7f9a500ae260 con 0x7f9a58109e10 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"btime":"2024-09-21T03:42:21:956788+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.316+0000 7f9a60000640 1 -- 172.21.15.16:0/1217329524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9a400b6510 msgr2=0x7f9a400b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.316+0000 7f9a60000640 1 --2- 172.21.15.16:0/1217329524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9a400b6510 0x7f9a400b8a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c005ab0 tx=0x7f9a4c005a20 comp rx=0 tx=0).stop 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.316+0000 7f9a60000640 1 -- 172.21.15.16:0/1217329524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a58109e10 msgr2=0x7f9a58075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.316+0000 7f9a60000640 1 --2- 172.21.15.16:0/1217329524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a58109e10 0x7f9a58075c50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000eb10 tx=0x7f9a5000c2d0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.316+0000 7f9a60000640 1 -- 172.21.15.16:0/1217329524 shutdown_connections 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.316+0000 7f9a60000640 1 --2- 172.21.15.16:0/1217329524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9a400b6510 0x7f9a400b8a20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.316+0000 7f9a60000640 1 --2- 172.21.15.16:0/1217329524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a58109e10 0x7f9a58075c50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.316+0000 7f9a60000640 1 --2- 172.21.15.16:0/1217329524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a581094c0 0x7f9a58075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.316+0000 7f9a60000640 1 --2- 172.21.15.16:0/1217329524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a58103b80 0x7f9a58077120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.316+0000 7f9a60000640 1 -- 172.21.15.16:0/1217329524 >> 172.21.15.16:0/1217329524 conn(0x7f9a580ff590 msgr2=0x7f9a581028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.317+0000 7f9a60000640 1 -- 172.21.15.16:0/1217329524 shutdown_connections 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:22.317+0000 7f9a60000640 1 -- 172.21.15.16:0/1217329524 wait complete. 2024-09-21T03:42:22.725 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:22 smithi016 ceph-mon[30375]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:22.726 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:22.726 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:22.726 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:22.726 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:22.726 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:22.726 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:22.726 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:22 smithi016 ceph-mon[30375]: Deploying daemon mds.j on smithi016 2024-09-21T03:42:22.726 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:22 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.123:6832/3581513383,v1:172.21.15.123:6833/3581513383] up:boot 2024-09-21T03:42:22.726 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:22 smithi016 ceph-mon[30375]: fsmap 2 up:standby 2024-09-21T03:42:22.726 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:42:22.726 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:22 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1217329524' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:22.826 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:22 smithi123 ceph-mon[34673]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:22.827 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:22.827 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:22.827 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:22.827 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:22.827 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:22.827 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:22.827 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:22 smithi123 ceph-mon[34673]: Deploying daemon mds.j on smithi016 2024-09-21T03:42:22.827 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:22 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.123:6832/3581513383,v1:172.21.15.123:6833/3581513383] up:boot 2024-09-21T03:42:22.827 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:22 smithi123 ceph-mon[34673]: fsmap 2 up:standby 2024-09-21T03:42:22.827 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:42:22.827 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:22 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1217329524' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:22.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:22 smithi134 ceph-mon[35031]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:22.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:22.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:22.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:22.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:22.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:22.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:22.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:22 smithi134 ceph-mon[35031]: Deploying daemon mds.j on smithi016 2024-09-21T03:42:22.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:22 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.123:6832/3581513383,v1:172.21.15.123:6833/3581513383] up:boot 2024-09-21T03:42:22.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:22 smithi134 ceph-mon[35031]: fsmap 2 up:standby 2024-09-21T03:42:22.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:42:22.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:22 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1217329524' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:24.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:24 smithi016 ceph-mon[30375]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:24.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:24 smithi134 ceph-mon[35031]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:24.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:24 smithi123 ceph-mon[34673]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:25.080 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 03:42:24 smithi016 systemd[1]: Starting Ceph mds.j for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:42:25.340 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 03:42:25 smithi016 podman[70483]: 2024-09-21 03:42:24.982575948 +0000 UTC m=+0.027526618 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:42:25.340 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 03:42:25 smithi016 podman[70483]: 2024-09-21 03:42:25.118895258 +0000 UTC m=+0.163845906 container create 89ba4424df0bd0a1df472bfaa724018dcb1e3aaec8824b56737ce8b8da4f9024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-j, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T03:42:25.340 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 03:42:25 smithi016 podman[70483]: 2024-09-21 03:42:25.338340108 +0000 UTC m=+0.383290779 container init 89ba4424df0bd0a1df472bfaa724018dcb1e3aaec8824b56737ce8b8da4f9024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-j, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-21T03:42:25.662 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 03:42:25 smithi016 podman[70483]: 2024-09-21 03:42:25.344806269 +0000 UTC m=+0.389756920 container start 89ba4424df0bd0a1df472bfaa724018dcb1e3aaec8824b56737ce8b8da4f9024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-j, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:42:25.663 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 03:42:25 smithi016 ceph-mds[70501]: -- 172.21.15.16:0/244907846 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55be9afbd860 con 0x55be9f165a80 2024-09-21T03:42:25.663 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 03:42:25 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-j[70497]: starting mds.j at 2024-09-21T03:42:25.663 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 03:42:25 smithi016 bash[70483]: 89ba4424df0bd0a1df472bfaa724018dcb1e3aaec8824b56737ce8b8da4f9024 2024-09-21T03:42:25.663 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 03:42:25 smithi016 systemd[1]: Started Ceph mds.j for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:42:25.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:26.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:26 smithi134 ceph-mon[35031]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:26.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:26 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:26.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:26 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:26.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:26 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:26.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:26 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:26.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:26 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:26.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:26 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:26.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:26 smithi134 ceph-mon[35031]: Deploying daemon mds.d on smithi016 2024-09-21T03:42:26.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:26 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.16:6834/2280740148,v1:172.21.15.16:6835/2280740148] up:boot 2024-09-21T03:42:26.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:26 smithi134 ceph-mon[35031]: fsmap 3 up:standby 2024-09-21T03:42:26.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:26 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:42:26.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:26 smithi123 ceph-mon[34673]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:26.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:26 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:26.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:26 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:26.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:26 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:26.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:26 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:26.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:26 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:26.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:26 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:26.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:26 smithi123 ceph-mon[34673]: Deploying daemon mds.d on smithi016 2024-09-21T03:42:26.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:26 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.16:6834/2280740148,v1:172.21.15.16:6835/2280740148] up:boot 2024-09-21T03:42:26.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:26 smithi123 ceph-mon[34673]: fsmap 3 up:standby 2024-09-21T03:42:26.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:26 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:42:26.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:26 smithi016 ceph-mon[30375]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:26.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:26 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:26.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:26 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:26.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:26 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:26.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:26 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:26.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:26 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:26.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:26 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:26.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:26 smithi016 ceph-mon[30375]: Deploying daemon mds.d on smithi016 2024-09-21T03:42:26.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:26 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.16:6834/2280740148,v1:172.21.15.16:6835/2280740148] up:boot 2024-09-21T03:42:26.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:26 smithi016 ceph-mon[30375]: fsmap 3 up:standby 2024-09-21T03:42:26.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:26 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:42:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.987+0000 7f4009600640 1 -- 172.21.15.16:0/745105560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4004078290 msgr2=0x7f4004114210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.987+0000 7f4009600640 1 --2- 172.21.15.16:0/745105560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4004078290 0x7f4004114210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009900 tx=0x7f3ff002a410 comp rx=0 tx=0).stop 2024-09-21T03:42:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.988+0000 7f4009600640 1 -- 172.21.15.16:0/745105560 shutdown_connections 2024-09-21T03:42:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.988+0000 7f4009600640 1 --2- 172.21.15.16:0/745105560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4004114750 0x7f4004116b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.988+0000 7f4009600640 1 --2- 172.21.15.16:0/745105560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4004078290 0x7f4004114210 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.988+0000 7f4009600640 1 --2- 172.21.15.16:0/745105560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4004077940 0x7f4004077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.988+0000 7f4009600640 1 -- 172.21.15.16:0/745105560 >> 172.21.15.16:0/745105560 conn(0x7f4004101860 msgr2=0x7f4004103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.988+0000 7f4009600640 1 -- 172.21.15.16:0/745105560 shutdown_connections 2024-09-21T03:42:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.988+0000 7f4009600640 1 -- 172.21.15.16:0/745105560 wait complete. 2024-09-21T03:42:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.989+0000 7f4009600640 1 Processor -- start 2024-09-21T03:42:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.989+0000 7f4009600640 1 -- start start 2024-09-21T03:42:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.990+0000 7f4009600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4004077940 0x7f40041a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.990+0000 7f4009600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4004078290 0x7f40041a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.990+0000 7f4009600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4004114750 0x7f40041a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.990+0000 7f4009600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f400411a0e0 con 0x7f4004114750 2024-09-21T03:42:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.990+0000 7f4009600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4004119f60 con 0x7f4004077940 2024-09-21T03:42:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.990+0000 7f4009600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f400411a260 con 0x7f4004078290 2024-09-21T03:42:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.990+0000 7f4003400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4004114750 0x7f40041a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.990+0000 7f4003400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4004114750 0x7f40041a6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36786/0 (socket says 172.21.15.16:36786) 2024-09-21T03:42:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.990+0000 7f4003400640 1 -- 172.21.15.16:0/332078620 learned_addr learned my addr 172.21.15.16:0/332078620 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:42:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.990+0000 7f4002a00640 1 --2- 172.21.15.16:0/332078620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4004077940 0x7f40041a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.990+0000 7f4002000640 1 --2- 172.21.15.16:0/332078620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4004078290 0x7f40041a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.990+0000 7f4003400640 1 -- 172.21.15.16:0/332078620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4004078290 msgr2=0x7f40041a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.990+0000 7f4003400640 1 --2- 172.21.15.16:0/332078620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4004078290 0x7f40041a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.990+0000 7f4003400640 1 -- 172.21.15.16:0/332078620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4004077940 msgr2=0x7f40041a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.990+0000 7f4003400640 1 --2- 172.21.15.16:0/332078620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4004077940 0x7f40041a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.990+0000 7f4003400640 1 -- 172.21.15.16:0/332078620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40041a6a60 con 0x7f4004114750 2024-09-21T03:42:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.991+0000 7f4003400640 1 --2- 172.21.15.16:0/332078620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4004114750 0x7f40041a6330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00eb10 tx=0x7f3ffc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.991+0000 7f3ffb400640 1 -- 172.21.15.16:0/332078620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc004b20 con 0x7f4004114750 2024-09-21T03:42:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.991+0000 7f3ffb400640 1 -- 172.21.15.16:0/332078620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ffc004cc0 con 0x7f4004114750 2024-09-21T03:42:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.991+0000 7f3ffb400640 1 -- 172.21.15.16:0/332078620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc010670 con 0x7f4004114750 2024-09-21T03:42:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.991+0000 7f4009600640 1 -- 172.21.15.16:0/332078620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40041a6d50 con 0x7f4004114750 2024-09-21T03:42:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.991+0000 7f4009600640 1 -- 172.21.15.16:0/332078620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40041ad4b0 con 0x7f4004114750 2024-09-21T03:42:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.993+0000 7f4009600640 1 -- 172.21.15.16:0/332078620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4004077d50 con 0x7f4004114750 2024-09-21T03:42:27.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.995+0000 7f3ffb400640 1 -- 172.21.15.16:0/332078620 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f3ffc005000 con 0x7f4004114750 2024-09-21T03:42:28.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.997+0000 7f3ffb400640 1 --2- 172.21.15.16:0/332078620 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fe80b6590 0x7f3fe80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:28.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.997+0000 7f3ffb400640 1 -- 172.21.15.16:0/332078620 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f3ffc0e75d0 con 0x7f4004114750 2024-09-21T03:42:28.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.997+0000 7f4002a00640 1 --2- 172.21.15.16:0/332078620 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fe80b6590 0x7f3fe80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:28.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.998+0000 7f4002a00640 1 --2- 172.21.15.16:0/332078620 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fe80b6590 0x7f3fe80b8aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009bd0 tx=0x7f3ff4005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:28.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:27.999+0000 7f3ffb400640 1 -- 172.21.15.16:0/332078620 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3ffc056710 con 0x7f4004114750 2024-09-21T03:42:28.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:28.253+0000 7f4009600640 1 -- 172.21.15.16:0/332078620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40041a1c20 con 0x7f4004114750 2024-09-21T03:42:28.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:28.254+0000 7f3ffb400640 1 -- 172.21.15.16:0/332078620 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7f40041a1c20 con 0x7f4004114750 2024-09-21T03:42:28.257 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:42:28.257 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":4,"btime":"2024-09-21T03:42:25:563144+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T03:42:28.257 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 4 2024-09-21T03:42:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:28.256+0000 7f4009600640 1 -- 172.21.15.16:0/332078620 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fe80b6590 msgr2=0x7f3fe80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:28.256+0000 7f4009600640 1 --2- 172.21.15.16:0/332078620 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fe80b6590 0x7f3fe80b8aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009bd0 tx=0x7f3ff4005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:28.256+0000 7f4009600640 1 -- 172.21.15.16:0/332078620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4004114750 msgr2=0x7f40041a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:28.256+0000 7f4009600640 1 --2- 172.21.15.16:0/332078620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4004114750 0x7f40041a6330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00eb10 tx=0x7f3ffc00c2d0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:28.256+0000 7f4009600640 1 -- 172.21.15.16:0/332078620 shutdown_connections 2024-09-21T03:42:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:28.256+0000 7f4009600640 1 --2- 172.21.15.16:0/332078620 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fe80b6590 0x7f3fe80b8aa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:28.256+0000 7f4009600640 1 --2- 172.21.15.16:0/332078620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4004114750 0x7f40041a6330 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:28.256+0000 7f4009600640 1 --2- 172.21.15.16:0/332078620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4004078290 0x7f40041a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:28.256+0000 7f4009600640 1 --2- 172.21.15.16:0/332078620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4004077940 0x7f40041a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:28.256+0000 7f4009600640 1 -- 172.21.15.16:0/332078620 >> 172.21.15.16:0/332078620 conn(0x7f4004101860 msgr2=0x7f40041167e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:28.257+0000 7f4009600640 1 -- 172.21.15.16:0/332078620 shutdown_connections 2024-09-21T03:42:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:28.257+0000 7f4009600640 1 -- 172.21.15.16:0/332078620 wait complete. 2024-09-21T03:42:28.552 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 03:42:28 smithi016 systemd[1]: Starting Ceph mds.d for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:42:28.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:28 smithi134 ceph-mon[35031]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:28.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:28 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/332078620' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:28.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:28 smithi123 ceph-mon[34673]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:28.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:28 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/332078620' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:28.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:28 smithi016 ceph-mon[30375]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:28.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:28 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/332078620' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:29.305 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 03:42:29 smithi016 podman[70872]: 2024-09-21 03:42:29.151179534 +0000 UTC m=+0.027857294 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:42:29.306 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 03:42:29 smithi016 podman[70872]: 2024-09-21 03:42:29.304149299 +0000 UTC m=+0.180827037 container create e609d21e660b31849c85d5df9259978a9820954d025c046410b5a6fadf8911e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-d, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:42:29.912 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 03:42:29 smithi016 podman[70872]: 2024-09-21 03:42:29.565408689 +0000 UTC m=+0.442086450 container init e609d21e660b31849c85d5df9259978a9820954d025c046410b5a6fadf8911e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:42:29.913 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 03:42:29 smithi016 podman[70872]: 2024-09-21 03:42:29.57183679 +0000 UTC m=+0.448514530 container start e609d21e660b31849c85d5df9259978a9820954d025c046410b5a6fadf8911e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-d, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T03:42:29.913 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 03:42:29 smithi016 bash[70872]: e609d21e660b31849c85d5df9259978a9820954d025c046410b5a6fadf8911e6 2024-09-21T03:42:29.913 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 03:42:29 smithi016 ceph-mds[70891]: -- 172.21.15.16:0/3232181696 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561226bab860 con 0x56122ad53180 2024-09-21T03:42:29.913 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 03:42:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-d[70887]: starting mds.d at 2024-09-21T03:42:29.913 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 03:42:29 smithi016 systemd[1]: Started Ceph mds.d for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:42:31.079 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:30 smithi134 ceph-mon[35031]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:30 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:30 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:30 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:30 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:30 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:31.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:30 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:31.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:30 smithi134 ceph-mon[35031]: Deploying daemon mds.e on smithi123 2024-09-21T03:42:31.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:30 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.16:6836/1667623391,v1:172.21.15.16:6837/1667623391] up:boot 2024-09-21T03:42:31.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:30 smithi134 ceph-mon[35031]: fsmap 4 up:standby 2024-09-21T03:42:31.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:30 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:42:31.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:30 smithi016 ceph-mon[30375]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:31.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:30 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:31.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:30 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:31.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:30 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:31.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:30 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:31.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:30 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:31.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:30 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:31.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:30 smithi016 ceph-mon[30375]: Deploying daemon mds.e on smithi123 2024-09-21T03:42:31.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:30 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.16:6836/1667623391,v1:172.21.15.16:6837/1667623391] up:boot 2024-09-21T03:42:31.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:30 smithi016 ceph-mon[30375]: fsmap 4 up:standby 2024-09-21T03:42:31.136 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:30 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:42:31.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:30 smithi123 ceph-mon[34673]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:31.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:30 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:31.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:30 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:31.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:30 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:31.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:30 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:31.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:30 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:31.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:30 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:31.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:30 smithi123 ceph-mon[34673]: Deploying daemon mds.e on smithi123 2024-09-21T03:42:31.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:30 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.16:6836/1667623391,v1:172.21.15.16:6837/1667623391] up:boot 2024-09-21T03:42:31.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:30 smithi123 ceph-mon[34673]: fsmap 4 up:standby 2024-09-21T03:42:31.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:30 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:42:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:31.906 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 03:42:31 smithi123 systemd[1]: Starting Ceph mds.e for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:42:32.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.098+0000 7f42e1c00640 1 -- 172.21.15.16:0/3692794461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42dc109c40 msgr2=0x7f42dc110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:32.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.098+0000 7f42e1c00640 1 --2- 172.21.15.16:0/3692794461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42dc109c40 0x7f42dc110470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009be0 tx=0x7f42d402a6c0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.098+0000 7f42e1c00640 1 -- 172.21.15.16:0/3692794461 shutdown_connections 2024-09-21T03:42:32.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.098+0000 7f42e1c00640 1 --2- 172.21.15.16:0/3692794461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42dc109c40 0x7f42dc110470 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.098+0000 7f42e1c00640 1 --2- 172.21.15.16:0/3692794461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42dc1092f0 0x7f42dc109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.098+0000 7f42e1c00640 1 --2- 172.21.15.16:0/3692794461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc1039b0 0x7f42dc103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.098+0000 7f42e1c00640 1 -- 172.21.15.16:0/3692794461 >> 172.21.15.16:0/3692794461 conn(0x7f42dc0ff400 msgr2=0x7f42dc101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:32.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.098+0000 7f42e1c00640 1 -- 172.21.15.16:0/3692794461 shutdown_connections 2024-09-21T03:42:32.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.098+0000 7f42e1c00640 1 -- 172.21.15.16:0/3692794461 wait complete. 2024-09-21T03:42:32.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.099+0000 7f42e1c00640 1 Processor -- start 2024-09-21T03:42:32.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.100+0000 7f42e1c00640 1 -- start start 2024-09-21T03:42:32.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.100+0000 7f42e1c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42dc1039b0 0x7f42dc19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.100+0000 7f42e1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc1092f0 0x7f42dc19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.100+0000 7f42e1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42dc109c40 0x7f42dc19fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.100+0000 7f42e1c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc113900 con 0x7f42dc1092f0 2024-09-21T03:42:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.100+0000 7f42e1c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc113780 con 0x7f42dc1039b0 2024-09-21T03:42:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.100+0000 7f42e1c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc113a80 con 0x7f42dc109c40 2024-09-21T03:42:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.100+0000 7f42dbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42dc1039b0 0x7f42dc19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.100+0000 7f42db400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc1092f0 0x7f42dc19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.100+0000 7f42dbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42dc1039b0 0x7f42dc19a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34760/0 (socket says 172.21.15.16:34760) 2024-09-21T03:42:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.100+0000 7f42dbe00640 1 -- 172.21.15.16:0/937752845 learned_addr learned my addr 172.21.15.16:0/937752845 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:42:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.101+0000 7f42e1200640 1 --2- 172.21.15.16:0/937752845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42dc109c40 0x7f42dc19fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.101+0000 7f42dbe00640 1 -- 172.21.15.16:0/937752845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42dc109c40 msgr2=0x7f42dc19fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.101+0000 7f42dbe00640 1 --2- 172.21.15.16:0/937752845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42dc109c40 0x7f42dc19fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.101+0000 7f42dbe00640 1 -- 172.21.15.16:0/937752845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc1092f0 msgr2=0x7f42dc19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:32.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.101+0000 7f42dbe00640 1 --2- 172.21.15.16:0/937752845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc1092f0 0x7f42dc19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.101+0000 7f42dbe00640 1 -- 172.21.15.16:0/937752845 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42dc1a01e0 con 0x7f42dc1039b0 2024-09-21T03:42:32.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.101+0000 7f42e1200640 1 --2- 172.21.15.16:0/937752845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42dc109c40 0x7f42dc19fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:32.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.101+0000 7f42dbe00640 1 --2- 172.21.15.16:0/937752845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42dc1039b0 0x7f42dc19a270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00e9c0 tx=0x7f42cc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:32.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.101+0000 7f42d8c00640 1 -- 172.21.15.16:0/937752845 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc00ccc0 con 0x7f42dc1039b0 2024-09-21T03:42:32.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.102+0000 7f42d8c00640 1 -- 172.21.15.16:0/937752845 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42cc0054d0 con 0x7f42dc1039b0 2024-09-21T03:42:32.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.102+0000 7f42e1c00640 1 -- 172.21.15.16:0/937752845 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42dc1a04d0 con 0x7f42dc1039b0 2024-09-21T03:42:32.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.102+0000 7f42d8c00640 1 -- 172.21.15.16:0/937752845 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc010430 con 0x7f42dc1039b0 2024-09-21T03:42:32.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.102+0000 7f42e1c00640 1 -- 172.21.15.16:0/937752845 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42dc1a6c30 con 0x7f42dc1039b0 2024-09-21T03:42:32.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.103+0000 7f42e1c00640 1 -- 172.21.15.16:0/937752845 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42dc103dc0 con 0x7f42dc1039b0 2024-09-21T03:42:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.105+0000 7f42d8c00640 1 -- 172.21.15.16:0/937752845 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f42cc004f80 con 0x7f42dc1039b0 2024-09-21T03:42:32.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.106+0000 7f42d8c00640 1 --2- 172.21.15.16:0/937752845 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42c00b6590 0x7f42c00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:32.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.107+0000 7f42d8c00640 1 -- 172.21.15.16:0/937752845 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f42cc0e72f0 con 0x7f42dc1039b0 2024-09-21T03:42:32.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.107+0000 7f42db400640 1 --2- 172.21.15.16:0/937752845 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42c00b6590 0x7f42c00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:32.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.107+0000 7f42db400640 1 --2- 172.21.15.16:0/937752845 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42c00b6590 0x7f42c00b8aa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f42dc108dd0 tx=0x7f42c8005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:32.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.110+0000 7f42d8c00640 1 -- 172.21.15.16:0/937752845 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f42cc0adf80 con 0x7f42dc1039b0 2024-09-21T03:42:32.215 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 03:42:31 smithi123 podman[61905]: 2024-09-21 03:42:31.861561826 +0000 UTC m=+0.015685009 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:42:32.215 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 03:42:31 smithi123 podman[61905]: 2024-09-21 03:42:31.961160596 +0000 UTC m=+0.115283812 container create 28797bd39437d78cce128c2a378877a73f02d158a982ef63d2819c00d4cd8cdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-e, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T03:42:32.215 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 03:42:32 smithi123 podman[61905]: 2024-09-21 03:42:32.213793142 +0000 UTC m=+0.367916316 container init 28797bd39437d78cce128c2a378877a73f02d158a982ef63d2819c00d4cd8cdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T03:42:32.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.346+0000 7f42e1c00640 1 -- 172.21.15.16:0/937752845 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42dc19b3a0 con 0x7f42dc1039b0 2024-09-21T03:42:32.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.347+0000 7f42d8c00640 1 -- 172.21.15.16:0/937752845 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3916 (secure 0 0 0) 0x7f42cc0b2360 con 0x7f42dc1039b0 2024-09-21T03:42:32.350 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:42:32.350 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"btime":"2024-09-21T03:42:30:552234+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T03:42:32.350 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 5 2024-09-21T03:42:32.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.349+0000 7f42e1c00640 1 -- 172.21.15.16:0/937752845 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42c00b6590 msgr2=0x7f42c00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.349+0000 7f42e1c00640 1 --2- 172.21.15.16:0/937752845 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42c00b6590 0x7f42c00b8aa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f42dc108dd0 tx=0x7f42c8005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.349+0000 7f42e1c00640 1 -- 172.21.15.16:0/937752845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42dc1039b0 msgr2=0x7f42dc19a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.349+0000 7f42e1c00640 1 --2- 172.21.15.16:0/937752845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42dc1039b0 0x7f42dc19a270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00e9c0 tx=0x7f42cc00ee80 comp rx=0 tx=0).stop 2024-09-21T03:42:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.350+0000 7f42e1c00640 1 -- 172.21.15.16:0/937752845 shutdown_connections 2024-09-21T03:42:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.350+0000 7f42e1c00640 1 --2- 172.21.15.16:0/937752845 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42c00b6590 0x7f42c00b8aa0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.350+0000 7f42e1c00640 1 --2- 172.21.15.16:0/937752845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42dc109c40 0x7f42dc19fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.350+0000 7f42e1c00640 1 --2- 172.21.15.16:0/937752845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc1092f0 0x7f42dc19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.350+0000 7f42e1c00640 1 --2- 172.21.15.16:0/937752845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42dc1039b0 0x7f42dc19a270 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.350+0000 7f42e1c00640 1 -- 172.21.15.16:0/937752845 >> 172.21.15.16:0/937752845 conn(0x7f42dc0ff400 msgr2=0x7f42dc101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.350+0000 7f42e1c00640 1 -- 172.21.15.16:0/937752845 shutdown_connections 2024-09-21T03:42:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:32.350+0000 7f42e1c00640 1 -- 172.21.15.16:0/937752845 wait complete. 2024-09-21T03:42:32.656 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 03:42:32 smithi123 podman[61905]: 2024-09-21 03:42:32.216797631 +0000 UTC m=+0.370920816 container start 28797bd39437d78cce128c2a378877a73f02d158a982ef63d2819c00d4cd8cdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-e, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T03:42:32.656 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 03:42:32 smithi123 ceph-mds[61924]: -- 172.21.15.123:0/1106951186 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55617acc7860 con 0x55617ee6f600 2024-09-21T03:42:32.657 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 03:42:32 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-e[61920]: starting mds.e at 2024-09-21T03:42:32.657 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 03:42:32 smithi123 bash[61905]: 28797bd39437d78cce128c2a378877a73f02d158a982ef63d2819c00d4cd8cdc 2024-09-21T03:42:32.657 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 03:42:32 smithi123 systemd[1]: Started Ceph mds.e for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:42:33.042 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:32 smithi134 ceph-mon[35031]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:33.042 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:32 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/937752845' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:33.042 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:32 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:33.042 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:32 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:33.042 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:32 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:33.042 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:32 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:33.042 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:32 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:33.042 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:32 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:33.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:32 smithi123 ceph-mon[34673]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:33.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:32 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/937752845' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:33.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:32 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:33.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:32 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:33.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:32 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:33.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:32 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:33.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:32 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:33.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:32 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:32 smithi016 ceph-mon[30375]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:32 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/937752845' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:32 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:32 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:32 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:32 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:32 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:32 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:34.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:33 smithi134 ceph-mon[35031]: Deploying daemon mds.f on smithi134 2024-09-21T03:42:34.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:33 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] up:boot 2024-09-21T03:42:34.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:33 smithi134 ceph-mon[35031]: fsmap 5 up:standby 2024-09-21T03:42:34.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:33 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:42:34.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:33 smithi123 ceph-mon[34673]: Deploying daemon mds.f on smithi134 2024-09-21T03:42:34.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:33 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] up:boot 2024-09-21T03:42:34.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:33 smithi123 ceph-mon[34673]: fsmap 5 up:standby 2024-09-21T03:42:34.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:33 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:42:34.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:33 smithi016 ceph-mon[30375]: Deploying daemon mds.f on smithi134 2024-09-21T03:42:34.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:33 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] up:boot 2024-09-21T03:42:34.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:33 smithi016 ceph-mon[30375]: fsmap 5 up:standby 2024-09-21T03:42:34.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:33 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:42:34.537 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 03:42:34 smithi134 systemd[1]: Starting Ceph mds.f for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:42:34.794 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:34 smithi134 ceph-mon[35031]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:34.795 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 03:42:34 smithi134 podman[63117]: 2024-09-21 03:42:34.43783119 +0000 UTC m=+0.015544475 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:42:34.795 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 03:42:34 smithi134 podman[63117]: 2024-09-21 03:42:34.556842446 +0000 UTC m=+0.134555718 container create e9f1b37174f336e3af118238a246320c7d84cd77b2a99fb7f93f1aa7840d6dd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-f, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T03:42:34.795 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 03:42:34 smithi134 podman[63117]: 2024-09-21 03:42:34.793409678 +0000 UTC m=+0.371122950 container init e9f1b37174f336e3af118238a246320c7d84cd77b2a99fb7f93f1aa7840d6dd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-f, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T03:42:35.102 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 03:42:34 smithi134 podman[63117]: 2024-09-21 03:42:34.796601715 +0000 UTC m=+0.374315000 container start e9f1b37174f336e3af118238a246320c7d84cd77b2a99fb7f93f1aa7840d6dd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-f, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:42:35.102 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 03:42:34 smithi134 ceph-mds[63136]: -- 172.21.15.134:0/3202838481 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56506637b860 con 0x56506a523600 2024-09-21T03:42:35.103 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 03:42:34 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-f[63132]: starting mds.f at 2024-09-21T03:42:35.103 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 03:42:34 smithi134 bash[63117]: e9f1b37174f336e3af118238a246320c7d84cd77b2a99fb7f93f1aa7840d6dd9 2024-09-21T03:42:35.103 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 03:42:34 smithi134 systemd[1]: Started Ceph mds.f for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:42:35.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:34 smithi123 ceph-mon[34673]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:35.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:34 smithi016 ceph-mon[30375]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:35.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:36.089 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:35 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:36.089 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:35 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:36.089 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:35 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:36.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:35 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:36.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:35 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:36.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:35 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:36.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:35 smithi016 ceph-mon[30375]: Deploying daemon mds.c on smithi134 2024-09-21T03:42:36.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:35 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.134:6834/135285598,v1:172.21.15.134:6835/135285598] up:boot 2024-09-21T03:42:36.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:35 smithi016 ceph-mon[30375]: fsmap 6 up:standby 2024-09-21T03:42:36.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:35 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:42:36.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.266+0000 7fbfb4400640 1 -- 172.21.15.16:0/3347066007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfac109df0 msgr2=0x7fbfac110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:36.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.266+0000 7fbfb4400640 1 --2- 172.21.15.16:0/3347066007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfac109df0 0x7fbfac110620 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009ca0 tx=0x7fbfa402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.266+0000 7fbfb4400640 1 -- 172.21.15.16:0/3347066007 shutdown_connections 2024-09-21T03:42:36.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.266+0000 7fbfb4400640 1 --2- 172.21.15.16:0/3347066007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfac109df0 0x7fbfac110620 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.266+0000 7fbfb4400640 1 --2- 172.21.15.16:0/3347066007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfac1094a0 0x7fbfac1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.266+0000 7fbfb4400640 1 --2- 172.21.15.16:0/3347066007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfac103b60 0x7fbfac103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.266+0000 7fbfb4400640 1 -- 172.21.15.16:0/3347066007 >> 172.21.15.16:0/3347066007 conn(0x7fbfac07b850 msgr2=0x7fbfac07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.266+0000 7fbfb4400640 1 -- 172.21.15.16:0/3347066007 shutdown_connections 2024-09-21T03:42:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.266+0000 7fbfb4400640 1 -- 172.21.15.16:0/3347066007 wait complete. 2024-09-21T03:42:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.267+0000 7fbfb4400640 1 Processor -- start 2024-09-21T03:42:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.268+0000 7fbfb4400640 1 -- start start 2024-09-21T03:42:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.268+0000 7fbfb4400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfac103b60 0x7fbfac19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.268+0000 7fbfb4400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfac1094a0 0x7fbfac19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.268+0000 7fbfb4400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfac109df0 0x7fbfac1a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.268+0000 7fbfb4400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac113b00 con 0x7fbfac109df0 2024-09-21T03:42:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.268+0000 7fbfb4400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac113980 con 0x7fbfac1094a0 2024-09-21T03:42:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.268+0000 7fbfb4400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac113c80 con 0x7fbfac103b60 2024-09-21T03:42:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.268+0000 7fbfb1a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfac103b60 0x7fbfac19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.268+0000 7fbfb1a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfac103b60 0x7fbfac19c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:57890/0 (socket says 172.21.15.16:57890) 2024-09-21T03:42:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.268+0000 7fbfb1a00640 1 -- 172.21.15.16:0/2825271997 learned_addr learned my addr 172.21.15.16:0/2825271997 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:42:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.268+0000 7fbfb2400640 1 --2- 172.21.15.16:0/2825271997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfac109df0 0x7fbfac1a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.269+0000 7fbfb1000640 1 --2- 172.21.15.16:0/2825271997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfac1094a0 0x7fbfac19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.269+0000 7fbfb1a00640 1 -- 172.21.15.16:0/2825271997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfac1094a0 msgr2=0x7fbfac19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.269+0000 7fbfb1a00640 1 --2- 172.21.15.16:0/2825271997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfac1094a0 0x7fbfac19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.269+0000 7fbfb1a00640 1 -- 172.21.15.16:0/2825271997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfac109df0 msgr2=0x7fbfac1a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.269+0000 7fbfb1a00640 1 --2- 172.21.15.16:0/2825271997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfac109df0 0x7fbfac1a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.269+0000 7fbfb1a00640 1 -- 172.21.15.16:0/2825271997 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfac1a25e0 con 0x7fbfac103b60 2024-09-21T03:42:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.269+0000 7fbfb1a00640 1 --2- 172.21.15.16:0/2825271997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfac103b60 0x7fbfac19c670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbfa000e9f0 tx=0x7fbfa000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.269+0000 7fbfaa000640 1 -- 172.21.15.16:0/2825271997 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa000cd80 con 0x7fbfac103b60 2024-09-21T03:42:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.269+0000 7fbfb4400640 1 -- 172.21.15.16:0/2825271997 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfac1a28d0 con 0x7fbfac103b60 2024-09-21T03:42:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.269+0000 7fbfaa000640 1 -- 172.21.15.16:0/2825271997 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa00108e0 con 0x7fbfac103b60 2024-09-21T03:42:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.270+0000 7fbfaa000640 1 -- 172.21.15.16:0/2825271997 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0018980 con 0x7fbfac103b60 2024-09-21T03:42:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.270+0000 7fbfb4400640 1 -- 172.21.15.16:0/2825271997 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfac1a9030 con 0x7fbfac103b60 2024-09-21T03:42:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.271+0000 7fbfb4400640 1 -- 172.21.15.16:0/2825271997 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfac103f70 con 0x7fbfac103b60 2024-09-21T03:42:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.273+0000 7fbfaa000640 1 -- 172.21.15.16:0/2825271997 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fbfa0010430 con 0x7fbfac103b60 2024-09-21T03:42:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.275+0000 7fbfaa000640 1 --2- 172.21.15.16:0/2825271997 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbf940b6590 0x7fbf940b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.275+0000 7fbfaa000640 1 -- 172.21.15.16:0/2825271997 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7fbfa00e7620 con 0x7fbfac103b60 2024-09-21T03:42:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.275+0000 7fbfb1000640 1 --2- 172.21.15.16:0/2825271997 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbf940b6590 0x7fbf940b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.276+0000 7fbfb1000640 1 --2- 172.21.15.16:0/2825271997 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbf940b6590 0x7fbf940b8aa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbfac1073c0 tx=0x7fbf98005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.276+0000 7fbfaa000640 1 -- 172.21.15.16:0/2825271997 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbfa00b2690 con 0x7fbfac103b60 2024-09-21T03:42:36.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:35 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:36.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:35 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:36.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:35 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:36.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:35 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:36.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:35 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:36.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:35 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:36.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:35 smithi134 ceph-mon[35031]: Deploying daemon mds.c on smithi134 2024-09-21T03:42:36.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:35 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.134:6834/135285598,v1:172.21.15.134:6835/135285598] up:boot 2024-09-21T03:42:36.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:35 smithi134 ceph-mon[35031]: fsmap 6 up:standby 2024-09-21T03:42:36.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:35 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:42:36.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:35 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:36.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:35 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:36.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:35 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:36.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:35 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:36.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:35 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:36.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:35 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:36.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:35 smithi123 ceph-mon[34673]: Deploying daemon mds.c on smithi134 2024-09-21T03:42:36.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:35 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.134:6834/135285598,v1:172.21.15.134:6835/135285598] up:boot 2024-09-21T03:42:36.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:35 smithi123 ceph-mon[34673]: fsmap 6 up:standby 2024-09-21T03:42:36.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:35 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:42:36.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.514+0000 7fbfb4400640 1 -- 172.21.15.16:0/2825271997 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfac19d7a0 con 0x7fbfac103b60 2024-09-21T03:42:36.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.515+0000 7fbfaa000640 1 -- 172.21.15.16:0/2825271997 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5563 (secure 0 0 0) 0x7fbfa00b75d0 con 0x7fbfac103b60 2024-09-21T03:42:36.518 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:42:36.518 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"btime":"2024-09-21T03:42:35:769578+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24658,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24668,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-21T03:42:36.518 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2024-09-21T03:42:36.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.518+0000 7fbfb4400640 1 -- 172.21.15.16:0/2825271997 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbf940b6590 msgr2=0x7fbf940b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:36.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.518+0000 7fbfb4400640 1 --2- 172.21.15.16:0/2825271997 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbf940b6590 0x7fbf940b8aa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbfac1073c0 tx=0x7fbf98005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:36.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.518+0000 7fbfb4400640 1 -- 172.21.15.16:0/2825271997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfac103b60 msgr2=0x7fbfac19c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:36.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.518+0000 7fbfb4400640 1 --2- 172.21.15.16:0/2825271997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfac103b60 0x7fbfac19c670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbfa000e9f0 tx=0x7fbfa000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.518+0000 7fbfb4400640 1 -- 172.21.15.16:0/2825271997 shutdown_connections 2024-09-21T03:42:36.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.518+0000 7fbfb4400640 1 --2- 172.21.15.16:0/2825271997 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbf940b6590 0x7fbf940b8aa0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.518+0000 7fbfb4400640 1 --2- 172.21.15.16:0/2825271997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfac109df0 0x7fbfac1a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.518+0000 7fbfb4400640 1 --2- 172.21.15.16:0/2825271997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfac1094a0 0x7fbfac19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.518+0000 7fbfb4400640 1 --2- 172.21.15.16:0/2825271997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfac103b60 0x7fbfac19c670 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:36.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.518+0000 7fbfb4400640 1 -- 172.21.15.16:0/2825271997 >> 172.21.15.16:0/2825271997 conn(0x7fbfac07b850 msgr2=0x7fbfac105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:36.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.518+0000 7fbfb4400640 1 -- 172.21.15.16:0/2825271997 shutdown_connections 2024-09-21T03:42:36.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:36.518+0000 7fbfb4400640 1 -- 172.21.15.16:0/2825271997 wait complete. 2024-09-21T03:42:36.968 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 03:42:36 smithi134 systemd[1]: Starting Ceph mds.c for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:42:37.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:36 smithi016 ceph-mon[30375]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:36 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2825271997' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:37.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:36 smithi134 ceph-mon[35031]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:37.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:36 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2825271997' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:37.318 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 03:42:37 smithi134 podman[63387]: 2024-09-21 03:42:36.945003027 +0000 UTC m=+0.016318475 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:42:37.318 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 03:42:37 smithi134 podman[63387]: 2024-09-21 03:42:37.063874276 +0000 UTC m=+0.135189713 container create ca791862f45302e9e9e8c31973d43b844cdbc045b70a2b7014a965df94839e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:42:37.318 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 03:42:37 smithi134 podman[63387]: 2024-09-21 03:42:37.316584651 +0000 UTC m=+0.387900099 container init ca791862f45302e9e9e8c31973d43b844cdbc045b70a2b7014a965df94839e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T03:42:37.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:36 smithi123 ceph-mon[34673]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:37.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:36 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2825271997' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:37.602 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 03:42:37 smithi134 podman[63387]: 2024-09-21 03:42:37.319503182 +0000 UTC m=+0.390818619 container start ca791862f45302e9e9e8c31973d43b844cdbc045b70a2b7014a965df94839e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T03:42:37.602 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 03:42:37 smithi134 ceph-mds[63406]: -- 172.21.15.134:0/3792322234 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b5a431c20 con 0x564b5e5d9a80 2024-09-21T03:42:37.602 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 03:42:37 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-c[63402]: starting mds.c at 2024-09-21T03:42:37.602 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 03:42:37 smithi134 bash[63387]: ca791862f45302e9e9e8c31973d43b844cdbc045b70a2b7014a965df94839e00 2024-09-21T03:42:37.602 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 03:42:37 smithi134 systemd[1]: Started Ceph mds.c for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:42:38.580 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:38 smithi123 ceph-mon[34673]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:38.581 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:38 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:38.581 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:38 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:38.581 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:38 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:38.581 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:38 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:38.581 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:38 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:38.581 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:38 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:38.581 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:38 smithi123 ceph-mon[34673]: Deploying daemon mds.h on smithi123 2024-09-21T03:42:38.581 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:38 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] up:boot 2024-09-21T03:42:38.581 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:38 smithi123 ceph-mon[34673]: fsmap 7 up:standby 2024-09-21T03:42:38.581 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:38 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:42:38.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:38 smithi134 ceph-mon[35031]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:38.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:38 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:38.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:38 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:38.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:38 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:38.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:38 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:38.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:38 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:38.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:38 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:38.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:38 smithi134 ceph-mon[35031]: Deploying daemon mds.h on smithi123 2024-09-21T03:42:38.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:38 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] up:boot 2024-09-21T03:42:38.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:38 smithi134 ceph-mon[35031]: fsmap 7 up:standby 2024-09-21T03:42:38.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:38 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:42:38.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:38 smithi016 ceph-mon[30375]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:38 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:38 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:38 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:38 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:38 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:38 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:38 smithi016 ceph-mon[30375]: Deploying daemon mds.h on smithi123 2024-09-21T03:42:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:38 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] up:boot 2024-09-21T03:42:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:38 smithi016 ceph-mon[30375]: fsmap 7 up:standby 2024-09-21T03:42:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:38 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:42:39.275 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:39.586 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 03:42:39 smithi123 systemd[1]: Starting Ceph mds.h for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:42:39.675 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:39.860 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 03:42:39 smithi123 podman[62175]: 2024-09-21 03:42:39.487009748 +0000 UTC m=+0.015692849 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:42:39.860 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 03:42:39 smithi123 podman[62175]: 2024-09-21 03:42:39.606280514 +0000 UTC m=+0.134963615 container create 1059a271b1f20b5be8041698dfa6b9d1e8fe4df234050310c2437765e9325733 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-h, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:42:40.156 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 03:42:39 smithi123 podman[62175]: 2024-09-21 03:42:39.85943398 +0000 UTC m=+0.388117098 container init 1059a271b1f20b5be8041698dfa6b9d1e8fe4df234050310c2437765e9325733 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-h, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T03:42:40.156 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 03:42:39 smithi123 podman[62175]: 2024-09-21 03:42:39.862549092 +0000 UTC m=+0.391232213 container start 1059a271b1f20b5be8041698dfa6b9d1e8fe4df234050310c2437765e9325733 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-h, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:42:40.157 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 03:42:39 smithi123 ceph-mds[62194]: -- 172.21.15.123:0/3983281013 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558738c11860 con 0x55873cdb9a80 2024-09-21T03:42:40.157 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 03:42:39 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-h[62190]: starting mds.h at 2024-09-21T03:42:40.157 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 03:42:39 smithi123 bash[62175]: 1059a271b1f20b5be8041698dfa6b9d1e8fe4df234050310c2437765e9325733 2024-09-21T03:42:40.157 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 03:42:39 smithi123 systemd[1]: Started Ceph mds.h for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:42:40.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.329+0000 7f02bae00640 1 -- 172.21.15.16:0/2817970874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b41023e0 msgr2=0x7f02b41027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:40.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.329+0000 7f02bae00640 1 --2- 172.21.15.16:0/2817970874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b41023e0 0x7f02b41027f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009860 tx=0x7f02a802a3d0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.329+0000 7f02bae00640 1 -- 172.21.15.16:0/2817970874 shutdown_connections 2024-09-21T03:42:40.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.329+0000 7f02bae00640 1 --2- 172.21.15.16:0/2817970874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b410ff90 0x7f02b41123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.329+0000 7f02bae00640 1 --2- 172.21.15.16:0/2817970874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02b4102d30 0x7f02b410fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.329+0000 7f02bae00640 1 --2- 172.21.15.16:0/2817970874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b41023e0 0x7f02b41027f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.329+0000 7f02bae00640 1 -- 172.21.15.16:0/2817970874 >> 172.21.15.16:0/2817970874 conn(0x7f02b40fe2c0 msgr2=0x7f02b4100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.329+0000 7f02bae00640 1 -- 172.21.15.16:0/2817970874 shutdown_connections 2024-09-21T03:42:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.329+0000 7f02bae00640 1 -- 172.21.15.16:0/2817970874 wait complete. 2024-09-21T03:42:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.330+0000 7f02bae00640 1 Processor -- start 2024-09-21T03:42:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.330+0000 7f02bae00640 1 -- start start 2024-09-21T03:42:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.331+0000 7f02bae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b41023e0 0x7f02b419c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.331+0000 7f02bae00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b4102d30 0x7f02b419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.331+0000 7f02bae00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02b410ff90 0x7f02b41a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.331+0000 7f02bae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02b4115ad0 con 0x7f02b41023e0 2024-09-21T03:42:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.331+0000 7f02bae00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f02b4115950 con 0x7f02b4102d30 2024-09-21T03:42:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.331+0000 7f02bae00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f02b4115c50 con 0x7f02b410ff90 2024-09-21T03:42:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.331+0000 7f02b9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b4102d30 0x7f02b419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.331+0000 7f02b9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b41023e0 0x7f02b419c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.331+0000 7f02b9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b4102d30 0x7f02b419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34788/0 (socket says 172.21.15.16:34788) 2024-09-21T03:42:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.331+0000 7f02b9000640 1 -- 172.21.15.16:0/1875463702 learned_addr learned my addr 172.21.15.16:0/1875463702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:42:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.331+0000 7f02ba400640 1 --2- 172.21.15.16:0/1875463702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02b410ff90 0x7f02b41a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.331+0000 7f02b9000640 1 -- 172.21.15.16:0/1875463702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02b410ff90 msgr2=0x7f02b41a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.331+0000 7f02b9000640 1 --2- 172.21.15.16:0/1875463702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02b410ff90 0x7f02b41a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.331+0000 7f02b9000640 1 -- 172.21.15.16:0/1875463702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b41023e0 msgr2=0x7f02b419c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.331+0000 7f02b9000640 1 --2- 172.21.15.16:0/1875463702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b41023e0 0x7f02b419c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.331+0000 7f02b9000640 1 -- 172.21.15.16:0/1875463702 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b41a2410 con 0x7f02b4102d30 2024-09-21T03:42:40.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.332+0000 7f02b9000640 1 --2- 172.21.15.16:0/1875463702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b4102d30 0x7f02b419ca40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f02a000e9c0 tx=0x7f02a000edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:40.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.332+0000 7f02b2000640 1 -- 172.21.15.16:0/1875463702 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a000ccc0 con 0x7f02b4102d30 2024-09-21T03:42:40.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.332+0000 7f02bae00640 1 -- 172.21.15.16:0/1875463702 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b41a2700 con 0x7f02b4102d30 2024-09-21T03:42:40.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.332+0000 7f02b2000640 1 -- 172.21.15.16:0/1875463702 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02a00054d0 con 0x7f02b4102d30 2024-09-21T03:42:40.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.332+0000 7f02b2000640 1 -- 172.21.15.16:0/1875463702 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0010430 con 0x7f02b4102d30 2024-09-21T03:42:40.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.332+0000 7f02bae00640 1 -- 172.21.15.16:0/1875463702 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b41a8e60 con 0x7f02b4102d30 2024-09-21T03:42:40.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.333+0000 7f02bae00640 1 -- 172.21.15.16:0/1875463702 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b41027f0 con 0x7f02b4102d30 2024-09-21T03:42:40.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.336+0000 7f02b2000640 1 -- 172.21.15.16:0/1875463702 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f02a0004f20 con 0x7f02b4102d30 2024-09-21T03:42:40.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.338+0000 7f02b2000640 1 --2- 172.21.15.16:0/1875463702 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f029c0b6590 0x7f029c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:40.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.338+0000 7f02b2000640 1 -- 172.21.15.16:0/1875463702 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f02a00e7370 con 0x7f02b4102d30 2024-09-21T03:42:40.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.338+0000 7f02b9a00640 1 --2- 172.21.15.16:0/1875463702 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f029c0b6590 0x7f029c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:40.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.339+0000 7f02b9a00640 1 --2- 172.21.15.16:0/1875463702 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f029c0b6590 0x7f029c0b8aa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f02a80095f0 tx=0x7f02a8005c90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:40.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.341+0000 7f02b2000640 1 -- 172.21.15.16:0/1875463702 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f02a00ae000 con 0x7f02b4102d30 2024-09-21T03:42:40.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.580+0000 7f02bae00640 1 -- 172.21.15.16:0/1875463702 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02b402e640 con 0x7f02b4102d30 2024-09-21T03:42:40.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.581+0000 7f02b2000640 1 -- 172.21.15.16:0/1875463702 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7219 (secure 0 0 0) 0x7f02a00b23e0 con 0x7f02b4102d30 2024-09-21T03:42:40.583 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:42:40.583 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"btime":"2024-09-21T03:42:40:460056+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24658,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24668,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24689,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T03:42:40.584 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-09-21T03:42:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.584+0000 7f02bae00640 1 -- 172.21.15.16:0/1875463702 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f029c0b6590 msgr2=0x7f029c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.584+0000 7f02bae00640 1 --2- 172.21.15.16:0/1875463702 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f029c0b6590 0x7f029c0b8aa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f02a80095f0 tx=0x7f02a8005c90 comp rx=0 tx=0).stop 2024-09-21T03:42:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.584+0000 7f02bae00640 1 -- 172.21.15.16:0/1875463702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b4102d30 msgr2=0x7f02b419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.584+0000 7f02bae00640 1 --2- 172.21.15.16:0/1875463702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b4102d30 0x7f02b419ca40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f02a000e9c0 tx=0x7f02a000edc0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.584+0000 7f02bae00640 1 -- 172.21.15.16:0/1875463702 shutdown_connections 2024-09-21T03:42:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.584+0000 7f02bae00640 1 --2- 172.21.15.16:0/1875463702 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f029c0b6590 0x7f029c0b8aa0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.584+0000 7f02bae00640 1 --2- 172.21.15.16:0/1875463702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02b410ff90 0x7f02b41a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.584+0000 7f02bae00640 1 --2- 172.21.15.16:0/1875463702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02b4102d30 0x7f02b419ca40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.584+0000 7f02bae00640 1 --2- 172.21.15.16:0/1875463702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b41023e0 0x7f02b419c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.584+0000 7f02bae00640 1 -- 172.21.15.16:0/1875463702 >> 172.21.15.16:0/1875463702 conn(0x7f02b40fe2c0 msgr2=0x7f02b4112120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:40.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.584+0000 7f02bae00640 1 -- 172.21.15.16:0/1875463702 shutdown_connections 2024-09-21T03:42:40.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:40.584+0000 7f02bae00640 1 -- 172.21.15.16:0/1875463702 wait complete. 2024-09-21T03:42:41.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:41 smithi134 ceph-mon[35031]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:41.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:41.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:41.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:41.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:41.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:41.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:41.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:41 smithi134 ceph-mon[35031]: Deploying daemon mds.i on smithi134 2024-09-21T03:42:41.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:41 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] up:boot 2024-09-21T03:42:41.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:41 smithi134 ceph-mon[35031]: fsmap 8 up:standby 2024-09-21T03:42:41.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:41 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:42:41.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:41 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1875463702' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:41.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:41 smithi016 ceph-mon[30375]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:41.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:41.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:41.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:41.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:41.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:41.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:41.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:41 smithi016 ceph-mon[30375]: Deploying daemon mds.i on smithi134 2024-09-21T03:42:41.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:41 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] up:boot 2024-09-21T03:42:41.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:41 smithi016 ceph-mon[30375]: fsmap 8 up:standby 2024-09-21T03:42:41.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:41 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:42:41.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:41 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1875463702' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:41.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:41 smithi123 ceph-mon[34673]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:41.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:41 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:41.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:41 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:41 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:41 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:41 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:41 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:41 smithi123 ceph-mon[34673]: Deploying daemon mds.i on smithi134 2024-09-21T03:42:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:41 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] up:boot 2024-09-21T03:42:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:41 smithi123 ceph-mon[34673]: fsmap 8 up:standby 2024-09-21T03:42:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:41 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:42:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:41 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1875463702' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:42.102 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 03:42:42 smithi134 systemd[1]: Starting Ceph mds.i for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:42:42.567 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 03:42:42 smithi134 podman[63667]: 2024-09-21 03:42:42.169625821 +0000 UTC m=+0.015419838 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:42:42.567 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 03:42:42 smithi134 podman[63667]: 2024-09-21 03:42:42.329353208 +0000 UTC m=+0.175147227 container create 5ec8b8f8c6014950e81fac8c07afcd9908f0b667a00cb3efb04bec2115d7cd8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-i, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T03:42:42.567 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 03:42:42 smithi134 podman[63667]: 2024-09-21 03:42:42.565852299 +0000 UTC m=+0.411646323 container init 5ec8b8f8c6014950e81fac8c07afcd9908f0b667a00cb3efb04bec2115d7cd8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:42:42.852 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 03:42:42 smithi134 podman[63667]: 2024-09-21 03:42:42.568607365 +0000 UTC m=+0.414401382 container start 5ec8b8f8c6014950e81fac8c07afcd9908f0b667a00cb3efb04bec2115d7cd8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-i, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T03:42:42.852 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 03:42:42 smithi134 ceph-mds[63686]: -- 172.21.15.134:0/1287406699 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56287bfc9860 con 0x562880171a80 2024-09-21T03:42:42.852 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 03:42:42 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-i[63682]: starting mds.i at 2024-09-21T03:42:42.852 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 03:42:42 smithi134 bash[63667]: 5ec8b8f8c6014950e81fac8c07afcd9908f0b667a00cb3efb04bec2115d7cd8a 2024-09-21T03:42:42.852 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 03:42:42 smithi134 systemd[1]: Started Ceph mds.i for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:42:43.294 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:43 smithi016 ceph-mon[30375]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:43.294 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:43 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:43.294 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:43 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:43.294 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:43 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:43.294 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:43 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:43.294 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:43 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:43.294 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:43 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:43 smithi134 ceph-mon[35031]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:43 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:43 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:43 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:43 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:43 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:43 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:43.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:43.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:43 smithi123 ceph-mon[34673]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:43.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:43 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:43.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:43 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:43.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:43 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:43.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:43 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:43.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:43 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:43.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:43 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:44 smithi134 ceph-mon[35031]: Deploying daemon mds.g on smithi016 2024-09-21T03:42:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:44 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] up:boot 2024-09-21T03:42:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:44 smithi134 ceph-mon[35031]: fsmap 9 up:standby 2024-09-21T03:42:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:44 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:42:44.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:44 smithi016 ceph-mon[30375]: Deploying daemon mds.g on smithi016 2024-09-21T03:42:44.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:44 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] up:boot 2024-09-21T03:42:44.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:44 smithi016 ceph-mon[30375]: fsmap 9 up:standby 2024-09-21T03:42:44.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:44 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:42:44.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:44 smithi123 ceph-mon[34673]: Deploying daemon mds.g on smithi016 2024-09-21T03:42:44.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:44 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] up:boot 2024-09-21T03:42:44.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:44 smithi123 ceph-mon[34673]: fsmap 9 up:standby 2024-09-21T03:42:44.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:44 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:42:44.584 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:45.282 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:42:45 smithi016 systemd[1]: Starting Ceph mds.g for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:42:45.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:45 smithi016 ceph-mon[30375]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.278+0000 7f3500000640 1 -- 172.21.15.16:0/868733271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34f81094b0 msgr2=0x7f34f81098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.278+0000 7f3500000640 1 --2- 172.21.15.16:0/868733271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34f81094b0 0x7f34f81098c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009900 tx=0x7f34e402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.278+0000 7f3500000640 1 -- 172.21.15.16:0/868733271 shutdown_connections 2024-09-21T03:42:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.278+0000 7f3500000640 1 --2- 172.21.15.16:0/868733271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f8109e00 0x7f34f8110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.278+0000 7f3500000640 1 --2- 172.21.15.16:0/868733271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34f81094b0 0x7f34f81098c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.278+0000 7f3500000640 1 --2- 172.21.15.16:0/868733271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8103b70 0x7f34f8103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.278+0000 7f3500000640 1 -- 172.21.15.16:0/868733271 >> 172.21.15.16:0/868733271 conn(0x7f34f80ff5e0 msgr2=0x7f34f8101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.279+0000 7f3500000640 1 -- 172.21.15.16:0/868733271 shutdown_connections 2024-09-21T03:42:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.279+0000 7f3500000640 1 -- 172.21.15.16:0/868733271 wait complete. 2024-09-21T03:42:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.280+0000 7f3500000640 1 Processor -- start 2024-09-21T03:42:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.280+0000 7f3500000640 1 -- start start 2024-09-21T03:42:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.281+0000 7f3500000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8103b70 0x7f34f819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.281+0000 7f3500000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f81094b0 0x7f34f819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.281+0000 7f3500000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34f8109e00 0x7f34f819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.281+0000 7f3500000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34f8113ac0 con 0x7f34f8103b70 2024-09-21T03:42:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.281+0000 7f3500000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f34f8113940 con 0x7f34f8109e00 2024-09-21T03:42:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.281+0000 7f3500000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34f8113c40 con 0x7f34f81094b0 2024-09-21T03:42:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.281+0000 7f34fcc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f81094b0 0x7f34f819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.281+0000 7f34fe000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34f8109e00 0x7f34f819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.281+0000 7f34fd600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8103b70 0x7f34f819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.281+0000 7f34fe000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34f8109e00 0x7f34f819fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:48134/0 (socket says 172.21.15.16:48134) 2024-09-21T03:42:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.281+0000 7f34fe000640 1 -- 172.21.15.16:0/4012968662 learned_addr learned my addr 172.21.15.16:0/4012968662 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:42:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.282+0000 7f34fe000640 1 -- 172.21.15.16:0/4012968662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f81094b0 msgr2=0x7f34f819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.282+0000 7f34fe000640 1 --2- 172.21.15.16:0/4012968662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f81094b0 0x7f34f819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.282+0000 7f34fe000640 1 -- 172.21.15.16:0/4012968662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8103b70 msgr2=0x7f34f819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.282+0000 7f34fe000640 1 --2- 172.21.15.16:0/4012968662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8103b70 0x7f34f819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.282+0000 7f34fe000640 1 -- 172.21.15.16:0/4012968662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f81a0400 con 0x7f34f8109e00 2024-09-21T03:42:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.282+0000 7f34fe000640 1 --2- 172.21.15.16:0/4012968662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34f8109e00 0x7f34f819fcd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f34f000eb10 tx=0x7f34f000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.282+0000 7f34f6000640 1 -- 172.21.15.16:0/4012968662 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0004b20 con 0x7f34f8109e00 2024-09-21T03:42:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.282+0000 7f34f6000640 1 -- 172.21.15.16:0/4012968662 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34f0004cc0 con 0x7f34f8109e00 2024-09-21T03:42:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.282+0000 7f34f6000640 1 -- 172.21.15.16:0/4012968662 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0010670 con 0x7f34f8109e00 2024-09-21T03:42:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.282+0000 7f3500000640 1 -- 172.21.15.16:0/4012968662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f81a06f0 con 0x7f34f8109e00 2024-09-21T03:42:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.283+0000 7f3500000640 1 -- 172.21.15.16:0/4012968662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f81a6e50 con 0x7f34f8109e00 2024-09-21T03:42:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.284+0000 7f3500000640 1 -- 172.21.15.16:0/4012968662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f8103f80 con 0x7f34f8109e00 2024-09-21T03:42:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.286+0000 7f34f6000640 1 -- 172.21.15.16:0/4012968662 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f34f0005020 con 0x7f34f8109e00 2024-09-21T03:42:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.287+0000 7f34f6000640 1 --2- 172.21.15.16:0/4012968662 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f34e00b6590 0x7f34e00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.288+0000 7f34f6000640 1 -- 172.21.15.16:0/4012968662 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f34f00e7730 con 0x7f34f8109e00 2024-09-21T03:42:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.288+0000 7f34fd600640 1 --2- 172.21.15.16:0/4012968662 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f34e00b6590 0x7f34e00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.289+0000 7f34fd600640 1 --2- 172.21.15.16:0/4012968662 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f34e00b6590 0x7f34e00b8aa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0095f0 tx=0x7f34ec005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.289+0000 7f34f6000640 1 -- 172.21.15.16:0/4012968662 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f34f00ae3c0 con 0x7f34f8109e00 2024-09-21T03:42:45.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:45 smithi134 ceph-mon[35031]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:45.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:45 smithi123 ceph-mon[34673]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:45.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.540+0000 7f3500000640 1 -- 172.21.15.16:0/4012968662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34f810bc50 con 0x7f34f8109e00 2024-09-21T03:42:45.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.541+0000 7f34f6000640 1 -- 172.21.15.16:0/4012968662 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8046 (secure 0 0 0) 0x7f34f00b27a0 con 0x7f34f8109e00 2024-09-21T03:42:45.543 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:42:45.543 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"btime":"2024-09-21T03:42:43:011812+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24658,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24668,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24682,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24689,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T03:42:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 10 2024-09-21T03:42:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.544+0000 7f3500000640 1 -- 172.21.15.16:0/4012968662 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f34e00b6590 msgr2=0x7f34e00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.544+0000 7f3500000640 1 --2- 172.21.15.16:0/4012968662 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f34e00b6590 0x7f34e00b8aa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0095f0 tx=0x7f34ec005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.544+0000 7f3500000640 1 -- 172.21.15.16:0/4012968662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34f8109e00 msgr2=0x7f34f819fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.544+0000 7f3500000640 1 --2- 172.21.15.16:0/4012968662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34f8109e00 0x7f34f819fcd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f34f000eb10 tx=0x7f34f000c2d0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.544+0000 7f3500000640 1 -- 172.21.15.16:0/4012968662 shutdown_connections 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.544+0000 7f3500000640 1 --2- 172.21.15.16:0/4012968662 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f34e00b6590 0x7f34e00b8aa0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.544+0000 7f3500000640 1 --2- 172.21.15.16:0/4012968662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34f8109e00 0x7f34f819fcd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.544+0000 7f3500000640 1 --2- 172.21.15.16:0/4012968662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f81094b0 0x7f34f819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.544+0000 7f3500000640 1 --2- 172.21.15.16:0/4012968662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8103b70 0x7f34f819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.544+0000 7f3500000640 1 -- 172.21.15.16:0/4012968662 >> 172.21.15.16:0/4012968662 conn(0x7f34f80ff5e0 msgr2=0x7f34f8101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.544+0000 7f3500000640 1 -- 172.21.15.16:0/4012968662 shutdown_connections 2024-09-21T03:42:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:45.544+0000 7f3500000640 1 -- 172.21.15.16:0/4012968662 wait complete. 2024-09-21T03:42:45.913 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:42:45 smithi016 podman[71586]: 2024-09-21 03:42:45.501655678 +0000 UTC m=+0.027210052 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:42:45.913 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:42:45 smithi016 podman[71586]: 2024-09-21 03:42:45.629017413 +0000 UTC m=+0.154571776 container create f21e48a25ecddee71dce7e861f357887d74a165e75328dd05b9341550c3c3928 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-g, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T03:42:46.322 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:46 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/4012968662' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:46.322 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:42:46 smithi016 podman[71586]: 2024-09-21 03:42:46.210087391 +0000 UTC m=+0.735641784 container init f21e48a25ecddee71dce7e861f357887d74a165e75328dd05b9341550c3c3928 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-g, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T03:42:46.322 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:42:46 smithi016 podman[71586]: 2024-09-21 03:42:46.216539413 +0000 UTC m=+0.742093780 container start f21e48a25ecddee71dce7e861f357887d74a165e75328dd05b9341550c3c3928 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-g, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS) 2024-09-21T03:42:46.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:46 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/4012968662' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:46.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:46 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/4012968662' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:46.640 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:42:46 smithi016 ceph-mds[71621]: -- 172.21.15.16:0/2238180480 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f843ddf860 con 0x55f847f87600 2024-09-21T03:42:46.640 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:42:46 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-g[71616]: starting mds.g at 2024-09-21T03:42:46.912 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:42:46 smithi016 bash[71586]: f21e48a25ecddee71dce7e861f357887d74a165e75328dd05b9341550c3c3928 2024-09-21T03:42:46.912 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:42:46 smithi016 systemd[1]: Started Ceph mds.g for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:42:47.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:47 smithi016 ceph-mon[30375]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:47.287 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:47 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:47.287 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:47 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:47.287 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:47 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:47.287 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:47 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:47.287 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:47 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:47.287 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:47 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:47.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:47 smithi134 ceph-mon[35031]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:47.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:47 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:47.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:47 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:47.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:47 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:47.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:47 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:47.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:47 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:47.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:47 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:47.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:47 smithi123 ceph-mon[34673]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:47.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:47 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:47.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:47 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:47.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:47 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:47.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:47 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:47.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:47 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:47.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:47 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:48.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:48 smithi134 ceph-mon[35031]: Deploying daemon mds.a on smithi016 2024-09-21T03:42:48.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:48 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.16:6838/89613887,v1:172.21.15.16:6839/89613887] up:boot 2024-09-21T03:42:48.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:48 smithi134 ceph-mon[35031]: fsmap 10 up:standby 2024-09-21T03:42:48.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:48 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:42:48.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:48 smithi123 ceph-mon[34673]: Deploying daemon mds.a on smithi016 2024-09-21T03:42:48.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:48 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.16:6838/89613887,v1:172.21.15.16:6839/89613887] up:boot 2024-09-21T03:42:48.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:48 smithi123 ceph-mon[34673]: fsmap 10 up:standby 2024-09-21T03:42:48.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:48 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:42:48.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:48 smithi016 ceph-mon[30375]: Deploying daemon mds.a on smithi016 2024-09-21T03:42:48.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:48 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.16:6838/89613887,v1:172.21.15.16:6839/89613887] up:boot 2024-09-21T03:42:48.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:48 smithi016 ceph-mon[30375]: fsmap 10 up:standby 2024-09-21T03:42:48.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:48 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:42:48.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:49.240 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:49 smithi016 ceph-mon[30375]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:49.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:49 smithi134 ceph-mon[35031]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:49.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:49 smithi123 ceph-mon[34673]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:49.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.648+0000 7f1f48e00640 1 -- 172.21.15.16:0/2181867461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f441092f0 msgr2=0x7f1f44109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:49.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.648+0000 7f1f48e00640 1 --2- 172.21.15.16:0/2181867461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f441092f0 0x7f1f44109700 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009900 tx=0x7f1f3002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.649+0000 7f1f48e00640 1 -- 172.21.15.16:0/2181867461 shutdown_connections 2024-09-21T03:42:49.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.649+0000 7f1f48e00640 1 --2- 172.21.15.16:0/2181867461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f44109c40 0x7f1f44110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.649+0000 7f1f48e00640 1 --2- 172.21.15.16:0/2181867461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f441092f0 0x7f1f44109700 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.649+0000 7f1f48e00640 1 --2- 172.21.15.16:0/2181867461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f441039b0 0x7f1f44103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.649+0000 7f1f48e00640 1 -- 172.21.15.16:0/2181867461 >> 172.21.15.16:0/2181867461 conn(0x7f1f440ff440 msgr2=0x7f1f44101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:49.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.649+0000 7f1f48e00640 1 -- 172.21.15.16:0/2181867461 shutdown_connections 2024-09-21T03:42:49.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.649+0000 7f1f48e00640 1 -- 172.21.15.16:0/2181867461 wait complete. 2024-09-21T03:42:49.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.650+0000 7f1f48e00640 1 Processor -- start 2024-09-21T03:42:49.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.650+0000 7f1f48e00640 1 -- start start 2024-09-21T03:42:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.651+0000 7f1f48e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f441039b0 0x7f1f4419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.651+0000 7f1f48e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f441092f0 0x7f1f4419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.651+0000 7f1f48e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f44109c40 0x7f1f4419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.651+0000 7f1f48e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f44113900 con 0x7f1f441092f0 2024-09-21T03:42:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.651+0000 7f1f48e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f44113780 con 0x7f1f441039b0 2024-09-21T03:42:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.651+0000 7f1f48e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1f44113a80 con 0x7f1f44109c40 2024-09-21T03:42:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.651+0000 7f1f42a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f441092f0 0x7f1f4419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.651+0000 7f1f43400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f441039b0 0x7f1f4419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.651+0000 7f1f42a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f441092f0 0x7f1f4419a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57702/0 (socket says 172.21.15.16:57702) 2024-09-21T03:42:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.651+0000 7f1f42a00640 1 -- 172.21.15.16:0/1853652294 learned_addr learned my addr 172.21.15.16:0/1853652294 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:42:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.651+0000 7f1f43e00640 1 --2- 172.21.15.16:0/1853652294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f44109c40 0x7f1f4419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.651+0000 7f1f43400640 1 -- 172.21.15.16:0/1853652294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f44109c40 msgr2=0x7f1f4419fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.651+0000 7f1f43400640 1 --2- 172.21.15.16:0/1853652294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f44109c40 0x7f1f4419fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.651+0000 7f1f43400640 1 -- 172.21.15.16:0/1853652294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f441092f0 msgr2=0x7f1f4419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.651+0000 7f1f43400640 1 --2- 172.21.15.16:0/1853652294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f441092f0 0x7f1f4419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.651+0000 7f1f43400640 1 -- 172.21.15.16:0/1853652294 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f441a0260 con 0x7f1f441039b0 2024-09-21T03:42:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.652+0000 7f1f43400640 1 --2- 172.21.15.16:0/1853652294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f441039b0 0x7f1f4419a2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400e9f0 tx=0x7f1f3400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.652+0000 7f1f3be00640 1 -- 172.21.15.16:0/1853652294 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400ccc0 con 0x7f1f441039b0 2024-09-21T03:42:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.652+0000 7f1f48e00640 1 -- 172.21.15.16:0/1853652294 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f441a0550 con 0x7f1f441039b0 2024-09-21T03:42:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.652+0000 7f1f3be00640 1 -- 172.21.15.16:0/1853652294 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f34005020 con 0x7f1f441039b0 2024-09-21T03:42:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.652+0000 7f1f3be00640 1 -- 172.21.15.16:0/1853652294 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34004a00 con 0x7f1f441039b0 2024-09-21T03:42:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.652+0000 7f1f48e00640 1 -- 172.21.15.16:0/1853652294 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f440755f0 con 0x7f1f441039b0 2024-09-21T03:42:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.654+0000 7f1f48e00640 1 -- 172.21.15.16:0/1853652294 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f240052c0 con 0x7f1f441039b0 2024-09-21T03:42:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.656+0000 7f1f3be00640 1 -- 172.21.15.16:0/1853652294 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f1f34004ba0 con 0x7f1f441039b0 2024-09-21T03:42:49.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.658+0000 7f1f3be00640 1 --2- 172.21.15.16:0/1853652294 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1f280b6510 0x7f1f280b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:49.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.658+0000 7f1f3be00640 1 -- 172.21.15.16:0/1853652294 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f1f340e6ca0 con 0x7f1f441039b0 2024-09-21T03:42:49.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.658+0000 7f1f42a00640 1 --2- 172.21.15.16:0/1853652294 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1f280b6510 0x7f1f280b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:49.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.659+0000 7f1f42a00640 1 --2- 172.21.15.16:0/1853652294 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1f280b6510 0x7f1f280b8a20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1f44108e80 tx=0x7f1f30009560 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:49.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.662+0000 7f1f3be00640 1 -- 172.21.15.16:0/1853652294 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1f340ad930 con 0x7f1f441039b0 2024-09-21T03:42:49.832 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:42:49 smithi016 podman[71963]: 2024-09-21 03:42:49.707753034 +0000 UTC m=+0.040067912 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:42:49.833 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:42:49 smithi016 podman[71963]: 2024-09-21 03:42:49.830865487 +0000 UTC m=+0.163180355 container create fc21c0a5e14e3283e3db6ab69deab2bb6a57c61614a84268dc600d646beea00c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-21T03:42:49.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.930+0000 7f1f48e00640 1 -- 172.21.15.16:0/1853652294 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f240055b0 con 0x7f1f441039b0 2024-09-21T03:42:49.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.931+0000 7f1f3be00640 1 -- 172.21.15.16:0/1853652294 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8866 (secure 0 0 0) 0x7f1f340b1d10 con 0x7f1f441039b0 2024-09-21T03:42:49.933 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:42:49.933 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":11,"btime":"2024-09-21T03:42:47:033845+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14700,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24658,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24668,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24682,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24689,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T03:42:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 11 2024-09-21T03:42:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.934+0000 7f1f48e00640 1 -- 172.21.15.16:0/1853652294 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1f280b6510 msgr2=0x7f1f280b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.934+0000 7f1f48e00640 1 --2- 172.21.15.16:0/1853652294 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1f280b6510 0x7f1f280b8a20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1f44108e80 tx=0x7f1f30009560 comp rx=0 tx=0).stop 2024-09-21T03:42:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.934+0000 7f1f48e00640 1 -- 172.21.15.16:0/1853652294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f441039b0 msgr2=0x7f1f4419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.934+0000 7f1f48e00640 1 --2- 172.21.15.16:0/1853652294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f441039b0 0x7f1f4419a2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400e9f0 tx=0x7f1f3400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.934+0000 7f1f48e00640 1 -- 172.21.15.16:0/1853652294 shutdown_connections 2024-09-21T03:42:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.934+0000 7f1f48e00640 1 --2- 172.21.15.16:0/1853652294 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1f280b6510 0x7f1f280b8a20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.934+0000 7f1f48e00640 1 --2- 172.21.15.16:0/1853652294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f44109c40 0x7f1f4419fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.934+0000 7f1f48e00640 1 --2- 172.21.15.16:0/1853652294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f441092f0 0x7f1f4419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.934+0000 7f1f48e00640 1 --2- 172.21.15.16:0/1853652294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f441039b0 0x7f1f4419a2f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.934+0000 7f1f48e00640 1 -- 172.21.15.16:0/1853652294 >> 172.21.15.16:0/1853652294 conn(0x7f1f440ff440 msgr2=0x7f1f44101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.934+0000 7f1f48e00640 1 -- 172.21.15.16:0/1853652294 shutdown_connections 2024-09-21T03:42:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:49.934+0000 7f1f48e00640 1 -- 172.21.15.16:0/1853652294 wait complete. 2024-09-21T03:42:50.136 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:50 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1853652294' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:50.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:50 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1853652294' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:50.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:50 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1853652294' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:51.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:51 smithi016 ceph-mon[30375]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:51.162 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:42:50 smithi016 podman[71963]: 2024-09-21 03:42:50.975146692 +0000 UTC m=+1.307461554 container init fc21c0a5e14e3283e3db6ab69deab2bb6a57c61614a84268dc600d646beea00c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T03:42:51.163 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:42:50 smithi016 podman[71963]: 2024-09-21 03:42:50.981303406 +0000 UTC m=+1.313618268 container start fc21c0a5e14e3283e3db6ab69deab2bb6a57c61614a84268dc600d646beea00c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-21T03:42:51.163 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:42:51 smithi016 bash[71963]: fc21c0a5e14e3283e3db6ab69deab2bb6a57c61614a84268dc600d646beea00c 2024-09-21T03:42:51.163 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:42:51 smithi016 ceph-mds[71998]: -- 172.21.15.16:0/3716050445 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bd05eb9860 con 0x55bd0a061a80 2024-09-21T03:42:51.163 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:42:51 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-a[71991]: starting mds.a at 2024-09-21T03:42:51.163 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:42:51 smithi016 systemd[1]: Started Ceph mds.a for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:42:51.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:51 smithi134 ceph-mon[35031]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:51.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:51 smithi123 ceph-mon[34673]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:52.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:52.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:52.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:52.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:52.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:52.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:52.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:52 smithi134 ceph-mon[35031]: Deploying daemon mds.b on smithi123 2024-09-21T03:42:52.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:52 smithi134 ceph-mon[35031]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:52.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:52 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.16:6840/1546097467,v1:172.21.15.16:6841/1546097467] up:boot 2024-09-21T03:42:52.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:52 smithi134 ceph-mon[35031]: fsmap 11 up:standby 2024-09-21T03:42:52.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:42:52.602 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:52.602 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:52.602 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:52.602 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:52.602 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:52.602 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:52.602 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:52 smithi123 ceph-mon[34673]: Deploying daemon mds.b on smithi123 2024-09-21T03:42:52.602 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:52 smithi123 ceph-mon[34673]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:52.602 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:52 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.16:6840/1546097467,v1:172.21.15.16:6841/1546097467] up:boot 2024-09-21T03:42:52.602 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:52 smithi123 ceph-mon[34673]: fsmap 11 up:standby 2024-09-21T03:42:52.603 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:42:52.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:52.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:52.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:52.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:42:52.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:42:52.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:52.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:52 smithi016 ceph-mon[30375]: Deploying daemon mds.b on smithi123 2024-09-21T03:42:52.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:52 smithi016 ceph-mon[30375]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:52.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:52 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.16:6840/1546097467,v1:172.21.15.16:6841/1546097467] up:boot 2024-09-21T03:42:52.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:52 smithi016 ceph-mon[30375]: fsmap 11 up:standby 2024-09-21T03:42:52.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:42:52.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:53.156 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 03:42:53 smithi123 systemd[1]: Starting Ceph mds.b for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:42:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:53.566 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 03:42:53 smithi123 podman[62455]: 2024-09-21 03:42:53.195363605 +0000 UTC m=+0.015538575 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:42:53.566 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 03:42:53 smithi123 podman[62455]: 2024-09-21 03:42:53.311349783 +0000 UTC m=+0.131524789 container create f4db202a12ddd9b3b3f43013301c5f892718588fb1098b2bba39591061e73a6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:42:53.566 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 03:42:53 smithi123 podman[62455]: 2024-09-21 03:42:53.5644162 +0000 UTC m=+0.384591160 container init f4db202a12ddd9b3b3f43013301c5f892718588fb1098b2bba39591061e73a6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T03:42:53.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.900+0000 7f42d2c00640 1 -- 172.21.15.16:0/1548673926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc1049d0 msgr2=0x7f42cc104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.900+0000 7f42d2c00640 1 --2- 172.21.15.16:0/1548673926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc1049d0 0x7f42cc104de0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009900 tx=0x7f42bc02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.900+0000 7f42d2c00640 1 -- 172.21.15.16:0/1548673926 shutdown_connections 2024-09-21T03:42:53.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.900+0000 7f42d2c00640 1 --2- 172.21.15.16:0/1548673926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42cc10de30 0x7f42cc110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.900+0000 7f42d2c00640 1 --2- 172.21.15.16:0/1548673926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc105320 0x7f42cc10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.900+0000 7f42d2c00640 1 --2- 172.21.15.16:0/1548673926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc1049d0 0x7f42cc104de0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.900+0000 7f42d2c00640 1 -- 172.21.15.16:0/1548673926 >> 172.21.15.16:0/1548673926 conn(0x7f42cc079740 msgr2=0x7f42cc079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:53.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.900+0000 7f42d2c00640 1 -- 172.21.15.16:0/1548673926 shutdown_connections 2024-09-21T03:42:53.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.900+0000 7f42d2c00640 1 -- 172.21.15.16:0/1548673926 wait complete. 2024-09-21T03:42:53.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.901+0000 7f42d2c00640 1 Processor -- start 2024-09-21T03:42:53.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.901+0000 7f42d2c00640 1 -- start start 2024-09-21T03:42:53.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.902+0000 7f42d2c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc1049d0 0x7f42cc102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.902+0000 7f42d2c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc105320 0x7f42cc102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.902+0000 7f42d2c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42cc10de30 0x7f42cc103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.902+0000 7f42d2c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc113ae0 con 0x7f42cc1049d0 2024-09-21T03:42:53.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.902+0000 7f42d2c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc113960 con 0x7f42cc105320 2024-09-21T03:42:53.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.902+0000 7f42d2c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc113c60 con 0x7f42cc10de30 2024-09-21T03:42:53.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.902+0000 7f42d0c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42cc10de30 0x7f42cc103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.902+0000 7f42c3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc105320 0x7f42cc102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.902+0000 7f42cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc1049d0 0x7f42cc102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.902+0000 7f42c3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc105320 0x7f42cc102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:45114/0 (socket says 172.21.15.16:45114) 2024-09-21T03:42:53.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.902+0000 7f42c3e00640 1 -- 172.21.15.16:0/540741827 learned_addr learned my addr 172.21.15.16:0/540741827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:42:53.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.902+0000 7f42d0c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42cc10de30 0x7f42cc103190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:45998/0 (socket says 172.21.15.16:45998) 2024-09-21T03:42:53.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.902+0000 7f42c3e00640 1 -- 172.21.15.16:0/540741827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42cc10de30 msgr2=0x7f42cc103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.902+0000 7f42c3e00640 1 --2- 172.21.15.16:0/540741827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42cc10de30 0x7f42cc103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.902+0000 7f42c3e00640 1 -- 172.21.15.16:0/540741827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc1049d0 msgr2=0x7f42cc102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.902+0000 7f42c3e00640 1 --2- 172.21.15.16:0/540741827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc1049d0 0x7f42cc102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.902+0000 7f42c3e00640 1 -- 172.21.15.16:0/540741827 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42cc1a8750 con 0x7f42cc105320 2024-09-21T03:42:53.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.903+0000 7f42c3e00640 1 --2- 172.21.15.16:0/540741827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc105320 0x7f42cc102c50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42b400e970 tx=0x7f42b400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:53.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.903+0000 7f42c9600640 1 -- 172.21.15.16:0/540741827 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b400cd80 con 0x7f42cc105320 2024-09-21T03:42:53.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.903+0000 7f42c9600640 1 -- 172.21.15.16:0/540741827 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b40035f0 con 0x7f42cc105320 2024-09-21T03:42:53.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.903+0000 7f42d2c00640 1 -- 172.21.15.16:0/540741827 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42cc1a8a40 con 0x7f42cc105320 2024-09-21T03:42:53.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.903+0000 7f42c9600640 1 -- 172.21.15.16:0/540741827 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4010690 con 0x7f42cc105320 2024-09-21T03:42:53.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.903+0000 7f42d2c00640 1 -- 172.21.15.16:0/540741827 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42cc1a8f50 con 0x7f42cc105320 2024-09-21T03:42:53.908 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 03:42:53 smithi123 podman[62455]: 2024-09-21 03:42:53.567699199 +0000 UTC m=+0.387874162 container start f4db202a12ddd9b3b3f43013301c5f892718588fb1098b2bba39591061e73a6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:42:53.908 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 03:42:53 smithi123 ceph-mds[62474]: -- 172.21.15.123:0/153019349 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556e4e1f5860 con 0x556e5239d180 2024-09-21T03:42:53.909 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 03:42:53 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-b[62470]: starting mds.b at 2024-09-21T03:42:53.909 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 03:42:53 smithi123 bash[62455]: f4db202a12ddd9b3b3f43013301c5f892718588fb1098b2bba39591061e73a6b 2024-09-21T03:42:53.909 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 03:42:53 smithi123 systemd[1]: Started Ceph mds.b for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T03:42:53.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.904+0000 7f42d2c00640 1 -- 172.21.15.16:0/540741827 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42cc06c240 con 0x7f42cc105320 2024-09-21T03:42:53.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.907+0000 7f42c9600640 1 -- 172.21.15.16:0/540741827 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f42b4003aa0 con 0x7f42cc105320 2024-09-21T03:42:53.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.909+0000 7f42c9600640 1 --2- 172.21.15.16:0/540741827 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42b00b6510 0x7f42b00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.909+0000 7f42c9600640 1 -- 172.21.15.16:0/540741827 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f42b40e7150 con 0x7f42cc105320 2024-09-21T03:42:53.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.909+0000 7f42cbe00640 1 --2- 172.21.15.16:0/540741827 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42b00b6510 0x7f42b00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.910+0000 7f42cbe00640 1 --2- 172.21.15.16:0/540741827 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42b00b6510 0x7f42b00b8a20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f42bc031440 tx=0x7f42bc005a20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:53.916+0000 7f42c9600640 1 -- 172.21.15.16:0/540741827 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f42b40adde0 con 0x7f42cc105320 2024-09-21T03:42:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:54.165+0000 7f42d2c00640 1 -- 172.21.15.16:0/540741827 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42cc103a10 con 0x7f42cc105320 2024-09-21T03:42:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:54.166+0000 7f42c9600640 1 -- 172.21.15.16:0/540741827 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9692 (secure 0 0 0) 0x7f42cc103a10 con 0x7f42cc105320 2024-09-21T03:42:54.168 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:42:54.168 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":12,"btime":"2024-09-21T03:42:52:055627+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14700,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24658,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24668,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24682,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24689,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T03:42:54.169 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 12 2024-09-21T03:42:54.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:54.168+0000 7f42d2c00640 1 -- 172.21.15.16:0/540741827 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42b00b6510 msgr2=0x7f42b00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:54.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:54.168+0000 7f42d2c00640 1 --2- 172.21.15.16:0/540741827 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42b00b6510 0x7f42b00b8a20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f42bc031440 tx=0x7f42bc005a20 comp rx=0 tx=0).stop 2024-09-21T03:42:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:54.168+0000 7f42d2c00640 1 -- 172.21.15.16:0/540741827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc105320 msgr2=0x7f42cc102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:54.168+0000 7f42d2c00640 1 --2- 172.21.15.16:0/540741827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc105320 0x7f42cc102c50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42b400e970 tx=0x7f42b400ee30 comp rx=0 tx=0).stop 2024-09-21T03:42:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:54.168+0000 7f42d2c00640 1 -- 172.21.15.16:0/540741827 shutdown_connections 2024-09-21T03:42:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:54.168+0000 7f42d2c00640 1 --2- 172.21.15.16:0/540741827 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42b00b6510 0x7f42b00b8a20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:54.168+0000 7f42d2c00640 1 --2- 172.21.15.16:0/540741827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42cc10de30 0x7f42cc103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:54.168+0000 7f42d2c00640 1 --2- 172.21.15.16:0/540741827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc105320 0x7f42cc102c50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:54.168+0000 7f42d2c00640 1 --2- 172.21.15.16:0/540741827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc1049d0 0x7f42cc102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:54.168+0000 7f42d2c00640 1 -- 172.21.15.16:0/540741827 >> 172.21.15.16:0/540741827 conn(0x7f42cc079740 msgr2=0x7f42cc0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:54.169+0000 7f42d2c00640 1 -- 172.21.15.16:0/540741827 shutdown_connections 2024-09-21T03:42:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:54.169+0000 7f42d2c00640 1 -- 172.21.15.16:0/540741827 wait complete. 2024-09-21T03:42:54.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:54 smithi123 ceph-mon[34673]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:54.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:54.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:54 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/540741827' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:54 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] up:boot 2024-09-21T03:42:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:54 smithi123 ceph-mon[34673]: fsmap 12 up:standby 2024-09-21T03:42:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:42:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:54 smithi134 ceph-mon[35031]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:54 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/540741827' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:55.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:54 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] up:boot 2024-09-21T03:42:55.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:54 smithi134 ceph-mon[35031]: fsmap 12 up:standby 2024-09-21T03:42:55.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:42:55.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:54 smithi016 ceph-mon[30375]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:55.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:54 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/540741827' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:54 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] up:boot 2024-09-21T03:42:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:54 smithi016 ceph-mon[30375]: fsmap 12 up:standby 2024-09-21T03:42:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:42:56.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:56 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:56.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:56 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:56.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:56 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:56.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:56 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:56.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:56 smithi134 ceph-mon[35031]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:56.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:56 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:56.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:56 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:56.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:56 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:56.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:56 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:56.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:56 smithi123 ceph-mon[34673]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:56.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:56 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:56.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:56 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:56.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:56 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:56.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:56 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:56.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:56 smithi016 ceph-mon[30375]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:57.298 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:58.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:58.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:58.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:58.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:58.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:58.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:57 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:58.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:58.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:58.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:58.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:58.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:58.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:57 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:58.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:58.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:58.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:58.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:58.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:58.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:57 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:59.031 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:58 smithi016 ceph-mon[30375]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:59.031 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:59.031 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:59.031 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:59.031 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:59.031 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:58 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:59.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.088+0000 7f83d7200640 1 -- 172.21.15.16:0/3257234973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d0105320 msgr2=0x7f83d010d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:59.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.088+0000 7f83d7200640 1 --2- 172.21.15.16:0/3257234973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d0105320 0x7f83d010d870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009860 tx=0x7f83bc02a340 comp rx=0 tx=0).stop 2024-09-21T03:42:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.088+0000 7f83d7200640 1 -- 172.21.15.16:0/3257234973 shutdown_connections 2024-09-21T03:42:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.088+0000 7f83d7200640 1 --2- 172.21.15.16:0/3257234973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d010de30 0x7f83d0110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.088+0000 7f83d7200640 1 --2- 172.21.15.16:0/3257234973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d0105320 0x7f83d010d870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.088+0000 7f83d7200640 1 --2- 172.21.15.16:0/3257234973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d01049d0 0x7f83d0104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.088+0000 7f83d7200640 1 -- 172.21.15.16:0/3257234973 >> 172.21.15.16:0/3257234973 conn(0x7f83d0079740 msgr2=0x7f83d0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.088+0000 7f83d7200640 1 -- 172.21.15.16:0/3257234973 shutdown_connections 2024-09-21T03:42:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.088+0000 7f83d7200640 1 -- 172.21.15.16:0/3257234973 wait complete. 2024-09-21T03:42:59.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.089+0000 7f83d7200640 1 Processor -- start 2024-09-21T03:42:59.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.089+0000 7f83d7200640 1 -- start start 2024-09-21T03:42:59.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.090+0000 7f83d7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d01049d0 0x7f83d019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:59.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.090+0000 7f83d7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d0105320 0x7f83d019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:59.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.090+0000 7f83d7200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d010de30 0x7f83d019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:59.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.090+0000 7f83d7200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83d0113a90 con 0x7f83d010de30 2024-09-21T03:42:59.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.090+0000 7f83d7200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83d0113910 con 0x7f83d0105320 2024-09-21T03:42:59.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.090+0000 7f83d7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83d0113c10 con 0x7f83d01049d0 2024-09-21T03:42:59.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.090+0000 7f83cfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d01049d0 0x7f83d019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:59.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.090+0000 7f83cfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d01049d0 0x7f83d019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:46014/0 (socket says 172.21.15.16:46014) 2024-09-21T03:42:59.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.090+0000 7f83cfe00640 1 -- 172.21.15.16:0/4232493095 learned_addr learned my addr 172.21.15.16:0/4232493095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:42:59.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.090+0000 7f83cf400640 1 --2- 172.21.15.16:0/4232493095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d0105320 0x7f83d019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:59.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.090+0000 7f83d5200640 1 --2- 172.21.15.16:0/4232493095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d010de30 0x7f83d019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:59.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.091+0000 7f83cfe00640 1 -- 172.21.15.16:0/4232493095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d0105320 msgr2=0x7f83d019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:59.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.091+0000 7f83cfe00640 1 --2- 172.21.15.16:0/4232493095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d0105320 0x7f83d019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.091+0000 7f83cfe00640 1 -- 172.21.15.16:0/4232493095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d010de30 msgr2=0x7f83d019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:59.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.091+0000 7f83cfe00640 1 --2- 172.21.15.16:0/4232493095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d010de30 0x7f83d019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.091+0000 7f83cfe00640 1 -- 172.21.15.16:0/4232493095 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d01a0430 con 0x7f83d01049d0 2024-09-21T03:42:59.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.091+0000 7f83cfe00640 1 --2- 172.21.15.16:0/4232493095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d01049d0 0x7f83d019a4c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f83c000ebc0 tx=0x7f83c000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:59.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.091+0000 7f83ccc00640 1 -- 172.21.15.16:0/4232493095 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0003b70 con 0x7f83d01049d0 2024-09-21T03:42:59.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.091+0000 7f83ccc00640 1 -- 172.21.15.16:0/4232493095 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c0003d10 con 0x7f83d01049d0 2024-09-21T03:42:59.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.091+0000 7f83ccc00640 1 -- 172.21.15.16:0/4232493095 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0019800 con 0x7f83d01049d0 2024-09-21T03:42:59.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.091+0000 7f83d7200640 1 -- 172.21.15.16:0/4232493095 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d01a0720 con 0x7f83d01049d0 2024-09-21T03:42:59.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.092+0000 7f83d7200640 1 -- 172.21.15.16:0/4232493095 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d01a6e80 con 0x7f83d01049d0 2024-09-21T03:42:59.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.093+0000 7f83d7200640 1 -- 172.21.15.16:0/4232493095 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d006c240 con 0x7f83d01049d0 2024-09-21T03:42:59.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.095+0000 7f83ccc00640 1 -- 172.21.15.16:0/4232493095 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f83c00199a0 con 0x7f83d01049d0 2024-09-21T03:42:59.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.097+0000 7f83ccc00640 1 --2- 172.21.15.16:0/4232493095 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f83b40b6590 0x7f83b40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:59.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.097+0000 7f83ccc00640 1 -- 172.21.15.16:0/4232493095 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f83c00e7760 con 0x7f83d01049d0 2024-09-21T03:42:59.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.097+0000 7f83cf400640 1 --2- 172.21.15.16:0/4232493095 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f83b40b6590 0x7f83b40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:59.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.098+0000 7f83cf400640 1 --2- 172.21.15.16:0/4232493095 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f83b40b6590 0x7f83b40b8aa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f83bc035040 tx=0x7f83bc030340 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.098+0000 7f83ccc00640 1 -- 172.21.15.16:0/4232493095 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f83c00ae480 con 0x7f83d01049d0 2024-09-21T03:42:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:58 smithi134 ceph-mon[35031]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:58 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:59.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.360+0000 7f83d7200640 1 -- 172.21.15.16:0/4232493095 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83d0104de0 con 0x7f83d01049d0 2024-09-21T03:42:59.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.362+0000 7f83ccc00640 1 -- 172.21.15.16:0/4232493095 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10518 (secure 0 0 0) 0x7f83c00b2860 con 0x7f83d01049d0 2024-09-21T03:42:59.364 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:42:59.364 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 13 2024-09-21T03:42:59.364 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":13,"btime":"2024-09-21T03:42:54:374656+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14700,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24658,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24668,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24682,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24689,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24731,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-21T03:42:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.364+0000 7f83d7200640 1 -- 172.21.15.16:0/4232493095 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f83b40b6590 msgr2=0x7f83b40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.364+0000 7f83d7200640 1 --2- 172.21.15.16:0/4232493095 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f83b40b6590 0x7f83b40b8aa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f83bc035040 tx=0x7f83bc030340 comp rx=0 tx=0).stop 2024-09-21T03:42:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.364+0000 7f83d7200640 1 -- 172.21.15.16:0/4232493095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d01049d0 msgr2=0x7f83d019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.364+0000 7f83d7200640 1 --2- 172.21.15.16:0/4232493095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d01049d0 0x7f83d019a4c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f83c000ebc0 tx=0x7f83c000c6a0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.364+0000 7f83d7200640 1 -- 172.21.15.16:0/4232493095 shutdown_connections 2024-09-21T03:42:59.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.364+0000 7f83d7200640 1 --2- 172.21.15.16:0/4232493095 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f83b40b6590 0x7f83b40b8aa0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.364+0000 7f83d7200640 1 --2- 172.21.15.16:0/4232493095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d010de30 0x7f83d019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.364+0000 7f83d7200640 1 --2- 172.21.15.16:0/4232493095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d0105320 0x7f83d019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.364+0000 7f83d7200640 1 --2- 172.21.15.16:0/4232493095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83d01049d0 0x7f83d019a4c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.364+0000 7f83d7200640 1 -- 172.21.15.16:0/4232493095 >> 172.21.15.16:0/4232493095 conn(0x7f83d0079740 msgr2=0x7f83d0107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:59.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.365+0000 7f83d7200640 1 -- 172.21.15.16:0/4232493095 shutdown_connections 2024-09-21T03:42:59.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:42:59.365+0000 7f83d7200640 1 -- 172.21.15.16:0/4232493095 wait complete. 2024-09-21T03:42:59.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:58 smithi123 ceph-mon[34673]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:42:59.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:59.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:59.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:59.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:42:59.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:58 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:43:00.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:42:59 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/4232493095' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:00.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T03:43:00.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:42:59 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/4232493095' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:00.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:42:59 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/4232493095' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:00.678 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:00 smithi016 ceph-mon[30375]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:00 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:43:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:00 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:43:01.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:00 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:01.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:00 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:01.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:00 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:43:01.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.343+0000 7f1103400640 1 -- 172.21.15.16:0/1414927225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc105d80 msgr2=0x7f10fc106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.343+0000 7f1103400640 1 --2- 172.21.15.16:0/1414927225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc105d80 0x7f10fc106190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009860 tx=0x7f10ec02a340 comp rx=0 tx=0).stop 2024-09-21T03:43:01.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.343+0000 7f1103400640 1 -- 172.21.15.16:0/1414927225 shutdown_connections 2024-09-21T03:43:01.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.343+0000 7f1103400640 1 --2- 172.21.15.16:0/1414927225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10fc10c010 0x7f10fc112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.343+0000 7f1103400640 1 --2- 172.21.15.16:0/1414927225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10fc10b6c0 0x7f10fc10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.343+0000 7f1103400640 1 --2- 172.21.15.16:0/1414927225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc105d80 0x7f10fc106190 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.343+0000 7f1103400640 1 -- 172.21.15.16:0/1414927225 >> 172.21.15.16:0/1414927225 conn(0x7f10fc101810 msgr2=0x7f10fc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:01.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.343+0000 7f1103400640 1 -- 172.21.15.16:0/1414927225 shutdown_connections 2024-09-21T03:43:01.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.343+0000 7f1103400640 1 -- 172.21.15.16:0/1414927225 wait complete. 2024-09-21T03:43:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.344+0000 7f1103400640 1 Processor -- start 2024-09-21T03:43:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.344+0000 7f1103400640 1 -- start start 2024-09-21T03:43:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.345+0000 7f1103400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10fc105d80 0x7f10fc19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.345+0000 7f1103400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10fc10b6c0 0x7f10fc19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.345+0000 7f1103400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc10c010 0x7f10fc1a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.345+0000 7f1103400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc115cd0 con 0x7f10fc10b6c0 2024-09-21T03:43:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.345+0000 7f1103400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc115b50 con 0x7f10fc105d80 2024-09-21T03:43:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.345+0000 7f1103400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc115e50 con 0x7f10fc10c010 2024-09-21T03:43:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.345+0000 7f1100a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10fc105d80 0x7f10fc19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.345+0000 7f10f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10fc10b6c0 0x7f10fc19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.345+0000 7f1100a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10fc105d80 0x7f10fc19c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:45158/0 (socket says 172.21.15.16:45158) 2024-09-21T03:43:01.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.345+0000 7f1100a00640 1 -- 172.21.15.16:0/2994815070 learned_addr learned my addr 172.21.15.16:0/2994815070 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:01.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.345+0000 7f1101400640 1 --2- 172.21.15.16:0/2994815070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc10c010 0x7f10fc1a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.345+0000 7f1100a00640 1 -- 172.21.15.16:0/2994815070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc10c010 msgr2=0x7f10fc1a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.345+0000 7f1100a00640 1 --2- 172.21.15.16:0/2994815070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc10c010 0x7f10fc1a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.345+0000 7f1100a00640 1 -- 172.21.15.16:0/2994815070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10fc10b6c0 msgr2=0x7f10fc19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.345+0000 7f1100a00640 1 --2- 172.21.15.16:0/2994815070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10fc10b6c0 0x7f10fc19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.346+0000 7f1100a00640 1 -- 172.21.15.16:0/2994815070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10fc1a2650 con 0x7f10fc105d80 2024-09-21T03:43:01.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.346+0000 7f1100a00640 1 --2- 172.21.15.16:0/2994815070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10fc105d80 0x7f10fc19c6e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0303f0 tx=0x7f10ec032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:01.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.346+0000 7f10fa000640 1 -- 172.21.15.16:0/2994815070 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec040020 con 0x7f10fc105d80 2024-09-21T03:43:01.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.346+0000 7f1103400640 1 -- 172.21.15.16:0/2994815070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10fc1a28e0 con 0x7f10fc105d80 2024-09-21T03:43:01.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.346+0000 7f10fa000640 1 -- 172.21.15.16:0/2994815070 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ec030dd0 con 0x7f10fc105d80 2024-09-21T03:43:01.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.346+0000 7f10fa000640 1 -- 172.21.15.16:0/2994815070 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec037040 con 0x7f10fc105d80 2024-09-21T03:43:01.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.346+0000 7f1103400640 1 -- 172.21.15.16:0/2994815070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10fc1a90a0 con 0x7f10fc105d80 2024-09-21T03:43:01.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.347+0000 7f1103400640 1 -- 172.21.15.16:0/2994815070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10fc106190 con 0x7f10fc105d80 2024-09-21T03:43:01.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.350+0000 7f10fa000640 1 -- 172.21.15.16:0/2994815070 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f10ec043020 con 0x7f10fc105d80 2024-09-21T03:43:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:00 smithi134 ceph-mon[35031]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:00 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:43:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:00 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:43:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:00 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:00 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:00 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:43:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.351+0000 7f10fa000640 1 --2- 172.21.15.16:0/2994815070 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f10e00b6510 0x7f10e00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.351+0000 7f10fa000640 1 -- 172.21.15.16:0/2994815070 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f10ec107180 con 0x7f10fc105d80 2024-09-21T03:43:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.351+0000 7f10f3e00640 1 --2- 172.21.15.16:0/2994815070 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f10e00b6510 0x7f10e00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.352+0000 7f10f3e00640 1 --2- 172.21.15.16:0/2994815070 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f10e00b6510 0x7f10e00b8a20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f10fc10b1f0 tx=0x7f10e4005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:01.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.353+0000 7f10fa000640 1 -- 172.21.15.16:0/2994815070 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f10ec04a020 con 0x7f10fc105d80 2024-09-21T03:43:01.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:00 smithi123 ceph-mon[34673]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:01.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:00 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:43:01.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:00 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:43:01.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:00 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:01.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:00 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:01.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:00 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:43:01.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.556+0000 7f1103400640 1 -- 172.21.15.16:0/2994815070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f10fc19d810 con 0x7f10fc105d80 2024-09-21T03:43:01.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.557+0000 7f10fa000640 1 -- 172.21.15.16:0/2994815070 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) ==== 84+0+65 (secure 0 0 0) 0x7f10ec0328c0 con 0x7f10fc105d80 2024-09-21T03:43:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:43:01.559 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-21T03:43:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:43:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-21T03:43:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-21T03:43:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:43:01.559 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-21T03:43:01.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.560+0000 7f1103400640 1 -- 172.21.15.16:0/2994815070 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f10e00b6510 msgr2=0x7f10e00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.560+0000 7f1103400640 1 --2- 172.21.15.16:0/2994815070 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f10e00b6510 0x7f10e00b8a20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f10fc10b1f0 tx=0x7f10e4005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:01.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.560+0000 7f1103400640 1 -- 172.21.15.16:0/2994815070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10fc105d80 msgr2=0x7f10fc19c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.560+0000 7f1103400640 1 --2- 172.21.15.16:0/2994815070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10fc105d80 0x7f10fc19c6e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0303f0 tx=0x7f10ec032da0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.560+0000 7f1103400640 1 -- 172.21.15.16:0/2994815070 shutdown_connections 2024-09-21T03:43:01.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.560+0000 7f1103400640 1 --2- 172.21.15.16:0/2994815070 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f10e00b6510 0x7f10e00b8a20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.560+0000 7f1103400640 1 --2- 172.21.15.16:0/2994815070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc10c010 0x7f10fc1a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.560+0000 7f1103400640 1 --2- 172.21.15.16:0/2994815070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10fc10b6c0 0x7f10fc19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.560+0000 7f1103400640 1 --2- 172.21.15.16:0/2994815070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10fc105d80 0x7f10fc19c6e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.560+0000 7f1103400640 1 -- 172.21.15.16:0/2994815070 >> 172.21.15.16:0/2994815070 conn(0x7f10fc101810 msgr2=0x7f10fc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.560+0000 7f1103400640 1 -- 172.21.15.16:0/2994815070 shutdown_connections 2024-09-21T03:43:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:01.560+0000 7f1103400640 1 -- 172.21.15.16:0/2994815070 wait complete. 2024-09-21T03:43:02.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:01 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2994815070' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:43:02.320 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T03:43:02.320 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T03:43:02.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:01 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2994815070' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:43:02.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:01 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2994815070' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:43:02.699 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:03.161 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:02 smithi016 ceph-mon[30375]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.342+0000 7ff30e000640 1 -- 172.21.15.16:0/1339401729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff308109c20 msgr2=0x7ff308110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.342+0000 7ff30e000640 1 --2- 172.21.15.16:0/1339401729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff308109c20 0x7ff308110450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff300009ca0 tx=0x7ff30002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.342+0000 7ff30e000640 1 -- 172.21.15.16:0/1339401729 shutdown_connections 2024-09-21T03:43:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.342+0000 7ff30e000640 1 --2- 172.21.15.16:0/1339401729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff308109c20 0x7ff308110450 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.342+0000 7ff30e000640 1 --2- 172.21.15.16:0/1339401729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3081092d0 0x7ff3081096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.342+0000 7ff30e000640 1 --2- 172.21.15.16:0/1339401729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff308103990 0x7ff308103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.342+0000 7ff30e000640 1 -- 172.21.15.16:0/1339401729 >> 172.21.15.16:0/1339401729 conn(0x7ff3080ff440 msgr2=0x7ff308101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.342+0000 7ff30e000640 1 -- 172.21.15.16:0/1339401729 shutdown_connections 2024-09-21T03:43:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.342+0000 7ff30e000640 1 -- 172.21.15.16:0/1339401729 wait complete. 2024-09-21T03:43:03.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.343+0000 7ff30e000640 1 Processor -- start 2024-09-21T03:43:03.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.343+0000 7ff30e000640 1 -- start start 2024-09-21T03:43:03.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.343+0000 7ff30e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff308103990 0x7ff30819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:03.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.344+0000 7ff30e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3081092d0 0x7ff30819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:03.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.344+0000 7ff30e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff308109c20 0x7ff3081a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:03.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.344+0000 7ff30e000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3081138e0 con 0x7ff3081092d0 2024-09-21T03:43:03.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.344+0000 7ff30e000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff308113760 con 0x7ff308109c20 2024-09-21T03:43:03.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.344+0000 7ff30e000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff308113a60 con 0x7ff308103990 2024-09-21T03:43:03.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.344+0000 7ff307e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3081092d0 0x7ff30819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:03.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.344+0000 7ff307e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3081092d0 0x7ff30819ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45082/0 (socket says 172.21.15.16:45082) 2024-09-21T03:43:03.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.344+0000 7ff307e00640 1 -- 172.21.15.16:0/454413921 learned_addr learned my addr 172.21.15.16:0/454413921 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:03.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.344+0000 7ff30cc00640 1 --2- 172.21.15.16:0/454413921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff308103990 0x7ff30819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:03.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.344+0000 7ff30d600640 1 --2- 172.21.15.16:0/454413921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff308109c20 0x7ff3081a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:03.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.344+0000 7ff307e00640 1 -- 172.21.15.16:0/454413921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff308103990 msgr2=0x7ff30819c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.344+0000 7ff307e00640 1 --2- 172.21.15.16:0/454413921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff308103990 0x7ff30819c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.344+0000 7ff307e00640 1 -- 172.21.15.16:0/454413921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff308109c20 msgr2=0x7ff3081a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.344+0000 7ff307e00640 1 --2- 172.21.15.16:0/454413921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff308109c20 0x7ff3081a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.344+0000 7ff307e00640 1 -- 172.21.15.16:0/454413921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3081a2400 con 0x7ff3081092d0 2024-09-21T03:43:03.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.345+0000 7ff307e00640 1 --2- 172.21.15.16:0/454413921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3081092d0 0x7ff30819ca30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400e9f0 tx=0x7ff2f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:03.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.345+0000 7ff305600640 1 -- 172.21.15.16:0/454413921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f400ccc0 con 0x7ff3081092d0 2024-09-21T03:43:03.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.345+0000 7ff305600640 1 -- 172.21.15.16:0/454413921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f40108e0 con 0x7ff3081092d0 2024-09-21T03:43:03.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.345+0000 7ff30e000640 1 -- 172.21.15.16:0/454413921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3081a26f0 con 0x7ff3081092d0 2024-09-21T03:43:03.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.345+0000 7ff305600640 1 -- 172.21.15.16:0/454413921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4018860 con 0x7ff3081092d0 2024-09-21T03:43:03.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.345+0000 7ff30e000640 1 -- 172.21.15.16:0/454413921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3081a8e50 con 0x7ff3081092d0 2024-09-21T03:43:03.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.346+0000 7ff30e000640 1 -- 172.21.15.16:0/454413921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff308103da0 con 0x7ff3081092d0 2024-09-21T03:43:03.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.349+0000 7ff305600640 1 -- 172.21.15.16:0/454413921 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7ff2f4010430 con 0x7ff3081092d0 2024-09-21T03:43:03.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:02 smithi134 ceph-mon[35031]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:03.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.351+0000 7ff305600640 1 --2- 172.21.15.16:0/454413921 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff2f00b6510 0x7ff2f00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:03.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.352+0000 7ff305600640 1 -- 172.21.15.16:0/454413921 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7ff2f40e7600 con 0x7ff3081092d0 2024-09-21T03:43:03.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.352+0000 7ff30cc00640 1 --2- 172.21.15.16:0/454413921 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff2f00b6510 0x7ff2f00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:03.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.352+0000 7ff30cc00640 1 --2- 172.21.15.16:0/454413921 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff2f00b6510 0x7ff2f00b8a20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009bd0 tx=0x7ff2fc005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:03.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.354+0000 7ff305600640 1 -- 172.21.15.16:0/454413921 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff2f40b2670 con 0x7ff3081092d0 2024-09-21T03:43:03.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:02 smithi123 ceph-mon[34673]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.554+0000 7ff30e000640 1 -- 172.21.15.16:0/454413921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff30819d5c0 con 0x7ff3081092d0 2024-09-21T03:43:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.952+0000 7ff305600640 1 -- 172.21.15.16:0/454413921 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) ==== 138+0+0 (secure 0 0 0) 0x7ff30819d5c0 con 0x7ff3081092d0 2024-09-21T03:43:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-21T03:43:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.955+0000 7ff30e000640 1 -- 172.21.15.16:0/454413921 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff2f00b6510 msgr2=0x7ff2f00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.955+0000 7ff30e000640 1 --2- 172.21.15.16:0/454413921 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff2f00b6510 0x7ff2f00b8a20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009bd0 tx=0x7ff2fc005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.955+0000 7ff30e000640 1 -- 172.21.15.16:0/454413921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3081092d0 msgr2=0x7ff30819ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.955+0000 7ff30e000640 1 --2- 172.21.15.16:0/454413921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3081092d0 0x7ff30819ca30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400e9f0 tx=0x7ff2f400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.955+0000 7ff30e000640 1 -- 172.21.15.16:0/454413921 shutdown_connections 2024-09-21T03:43:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.955+0000 7ff30e000640 1 --2- 172.21.15.16:0/454413921 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff2f00b6510 0x7ff2f00b8a20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.955+0000 7ff30e000640 1 --2- 172.21.15.16:0/454413921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff308109c20 0x7ff3081a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.955+0000 7ff30e000640 1 --2- 172.21.15.16:0/454413921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3081092d0 0x7ff30819ca30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.955+0000 7ff30e000640 1 --2- 172.21.15.16:0/454413921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff308103990 0x7ff30819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.955+0000 7ff30e000640 1 -- 172.21.15.16:0/454413921 >> 172.21.15.16:0/454413921 conn(0x7ff3080ff440 msgr2=0x7ff3081068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.955+0000 7ff30e000640 1 -- 172.21.15.16:0/454413921 shutdown_connections 2024-09-21T03:43:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:03.955+0000 7ff30e000640 1 -- 172.21.15.16:0/454413921 wait complete. 2024-09-21T03:43:04.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:03 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/454413921' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:43:04.375 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:03 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/454413921' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:43:04.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:03 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/454413921' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:43:04.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-21T03:43:05.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:04 smithi016 ceph-mon[30375]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:04 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/454413921' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:43:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:04 smithi016 ceph-mon[30375]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T03:43:05.163 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:04 smithi134 ceph-mon[35031]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:04 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/454413921' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:43:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:04 smithi134 ceph-mon[35031]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T03:43:05.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:04 smithi123 ceph-mon[34673]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:05.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:04 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/454413921' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:43:05.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:04 smithi123 ceph-mon[34673]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T03:43:05.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.815+0000 7f8938200640 1 -- 172.21.15.16:0/3137429969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8930109df0 msgr2=0x7f8930110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.815+0000 7f8938200640 1 --2- 172.21.15.16:0/3137429969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8930109df0 0x7f8930110620 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8928009ca0 tx=0x7f892802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.816+0000 7f8938200640 1 -- 172.21.15.16:0/3137429969 shutdown_connections 2024-09-21T03:43:05.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.816+0000 7f8938200640 1 --2- 172.21.15.16:0/3137429969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8930109df0 0x7f8930110620 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.816+0000 7f8938200640 1 --2- 172.21.15.16:0/3137429969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89301094a0 0x7f89301098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.816+0000 7f8938200640 1 --2- 172.21.15.16:0/3137429969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8930103b60 0x7f8930103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.816+0000 7f8938200640 1 -- 172.21.15.16:0/3137429969 >> 172.21.15.16:0/3137429969 conn(0x7f893007b850 msgr2=0x7f893007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:05.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.816+0000 7f8938200640 1 -- 172.21.15.16:0/3137429969 shutdown_connections 2024-09-21T03:43:05.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.816+0000 7f8938200640 1 -- 172.21.15.16:0/3137429969 wait complete. 2024-09-21T03:43:05.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.817+0000 7f8938200640 1 Processor -- start 2024-09-21T03:43:05.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.817+0000 7f8938200640 1 -- start start 2024-09-21T03:43:05.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.817+0000 7f8938200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8930103b60 0x7f893019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.817+0000 7f8938200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89301094a0 0x7f893019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.817+0000 7f8938200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8930109df0 0x7f893019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.817+0000 7f8938200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8930113b00 con 0x7f8930103b60 2024-09-21T03:43:05.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.817+0000 7f8938200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8930113980 con 0x7f8930109df0 2024-09-21T03:43:05.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.817+0000 7f8938200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8930113c80 con 0x7f89301094a0 2024-09-21T03:43:05.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.818+0000 7f8935800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8930103b60 0x7f893019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.818+0000 7f8934e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89301094a0 0x7f893019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.818+0000 7f8935800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8930103b60 0x7f893019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45106/0 (socket says 172.21.15.16:45106) 2024-09-21T03:43:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.818+0000 7f8934e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89301094a0 0x7f893019aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:60876/0 (socket says 172.21.15.16:60876) 2024-09-21T03:43:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.818+0000 7f8935800640 1 -- 172.21.15.16:0/2269817594 learned_addr learned my addr 172.21.15.16:0/2269817594 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.818+0000 7f8936200640 1 --2- 172.21.15.16:0/2269817594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8930109df0 0x7f893019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.818+0000 7f8934e00640 1 -- 172.21.15.16:0/2269817594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8930109df0 msgr2=0x7f893019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.818+0000 7f8934e00640 1 --2- 172.21.15.16:0/2269817594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8930109df0 0x7f893019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.818+0000 7f8934e00640 1 -- 172.21.15.16:0/2269817594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8930103b60 msgr2=0x7f893019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.818+0000 7f8934e00640 1 --2- 172.21.15.16:0/2269817594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8930103b60 0x7f893019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.818+0000 7f8934e00640 1 -- 172.21.15.16:0/2269817594 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89301a0460 con 0x7f89301094a0 2024-09-21T03:43:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.818+0000 7f8935800640 1 --2- 172.21.15.16:0/2269817594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8930103b60 0x7f893019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.819+0000 7f8934e00640 1 --2- 172.21.15.16:0/2269817594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89301094a0 0x7f893019aa90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f893006c9f0 tx=0x7f891c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.819+0000 7f892e000640 1 -- 172.21.15.16:0/2269817594 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c00cc40 con 0x7f89301094a0 2024-09-21T03:43:05.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.819+0000 7f892e000640 1 -- 172.21.15.16:0/2269817594 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f891c00cde0 con 0x7f89301094a0 2024-09-21T03:43:05.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.819+0000 7f8938200640 1 -- 172.21.15.16:0/2269817594 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89301a0750 con 0x7f89301094a0 2024-09-21T03:43:05.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.819+0000 7f892e000640 1 -- 172.21.15.16:0/2269817594 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c004a00 con 0x7f89301094a0 2024-09-21T03:43:05.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.819+0000 7f8938200640 1 -- 172.21.15.16:0/2269817594 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89301a6eb0 con 0x7f89301094a0 2024-09-21T03:43:05.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.820+0000 7f8938200640 1 -- 172.21.15.16:0/2269817594 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8930103f70 con 0x7f89301094a0 2024-09-21T03:43:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.823+0000 7f892e000640 1 -- 172.21.15.16:0/2269817594 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f891c004ba0 con 0x7f89301094a0 2024-09-21T03:43:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.825+0000 7f892e000640 1 --2- 172.21.15.16:0/2269817594 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f89180b6590 0x7f89180b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.826+0000 7f892e000640 1 -- 172.21.15.16:0/2269817594 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7497+0+0 (secure 0 0 0) 0x7f891c0e6f70 con 0x7f89301094a0 2024-09-21T03:43:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.826+0000 7f8935800640 1 --2- 172.21.15.16:0/2269817594 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f89180b6590 0x7f89180b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:05.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.826+0000 7f892e000640 1 -- 172.21.15.16:0/2269817594 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f891c0adb20 con 0x7f89301094a0 2024-09-21T03:43:05.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:05.826+0000 7f8935800640 1 --2- 172.21.15.16:0/2269817594 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f89180b6590 0x7f89180b8aa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8924005fd0 tx=0x7f892400b4a0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:06.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:06.034+0000 7f8938200640 1 -- 172.21.15.16:0/2269817594 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f893010bc40 con 0x7f89301094a0 2024-09-21T03:43:06.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:05 smithi134 ceph-mon[35031]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T03:43:06.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:05 smithi123 ceph-mon[34673]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T03:43:06.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:05 smithi016 ceph-mon[30375]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T03:43:06.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:06.983+0000 7f892e000640 1 -- 172.21.15.16:0/2269817594 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v72) ==== 171+0+0 (secure 0 0 0) 0x7f891c0b1f00 con 0x7f89301094a0 2024-09-21T03:43:06.986 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-21T03:43:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:06.990+0000 7f8938200640 1 -- 172.21.15.16:0/2269817594 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f89180b6590 msgr2=0x7f89180b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:06.990+0000 7f8938200640 1 --2- 172.21.15.16:0/2269817594 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f89180b6590 0x7f89180b8aa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8924005fd0 tx=0x7f892400b4a0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:06.990+0000 7f8938200640 1 -- 172.21.15.16:0/2269817594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89301094a0 msgr2=0x7f893019aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:06.990+0000 7f8938200640 1 --2- 172.21.15.16:0/2269817594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89301094a0 0x7f893019aa90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f893006c9f0 tx=0x7f891c00ee50 comp rx=0 tx=0).stop 2024-09-21T03:43:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:06.991+0000 7f8938200640 1 -- 172.21.15.16:0/2269817594 shutdown_connections 2024-09-21T03:43:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:06.991+0000 7f8938200640 1 --2- 172.21.15.16:0/2269817594 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f89180b6590 0x7f89180b8aa0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:06.991+0000 7f8938200640 1 --2- 172.21.15.16:0/2269817594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8930109df0 0x7f893019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:06.991+0000 7f8938200640 1 --2- 172.21.15.16:0/2269817594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89301094a0 0x7f893019aa90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:06.991+0000 7f8938200640 1 --2- 172.21.15.16:0/2269817594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8930103b60 0x7f893019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:06.991+0000 7f8938200640 1 -- 172.21.15.16:0/2269817594 >> 172.21.15.16:0/2269817594 conn(0x7f893007b850 msgr2=0x7f8930107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:06.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:06.991+0000 7f8938200640 1 -- 172.21.15.16:0/2269817594 shutdown_connections 2024-09-21T03:43:06.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:06.992+0000 7f8938200640 1 -- 172.21.15.16:0/2269817594 wait complete. 2024-09-21T03:43:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:06 smithi134 ceph-mon[35031]: pgmap v306: 33 pgs: 17 creating+peering, 1 active+clean, 15 unknown; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:06 smithi134 ceph-mon[35031]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:43:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:06 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2269817594' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T03:43:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:06 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T03:43:07.379 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:06 smithi016 ceph-mon[30375]: pgmap v306: 33 pgs: 17 creating+peering, 1 active+clean, 15 unknown; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:07.379 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:06 smithi016 ceph-mon[30375]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:43:07.379 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:06 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2269817594' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T03:43:07.379 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:06 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T03:43:07.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:06 smithi123 ceph-mon[34673]: pgmap v306: 33 pgs: 17 creating+peering, 1 active+clean, 15 unknown; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:07.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:06 smithi123 ceph-mon[34673]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:43:07.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:06 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2269817594' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T03:43:07.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:06 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T03:43:07.875 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T03:43:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:08.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:07 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T03:43:08.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:07 smithi134 ceph-mon[35031]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:43:08.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:07 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T03:43:08.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:07 smithi123 ceph-mon[34673]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:43:08.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:07 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T03:43:08.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:07 smithi016 ceph-mon[30375]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:43:09.002 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:08 smithi016 ceph-mon[30375]: pgmap v309: 97 pgs: 17 creating+peering, 8 active+clean, 72 unknown; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:09.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:08 smithi016 ceph-mon[30375]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:43:09.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.080+0000 7eff55200640 1 -- 172.21.15.16:0/3912755799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff50114740 msgr2=0x7eff50116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.080+0000 7eff55200640 1 --2- 172.21.15.16:0/3912755799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff50114740 0x7eff50116b60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7eff48009ca0 tx=0x7eff4802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.080+0000 7eff55200640 1 -- 172.21.15.16:0/3912755799 shutdown_connections 2024-09-21T03:43:09.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.080+0000 7eff55200640 1 --2- 172.21.15.16:0/3912755799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff50114740 0x7eff50116b60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.080+0000 7eff55200640 1 --2- 172.21.15.16:0/3912755799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff50078290 0x7eff50114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.080+0000 7eff55200640 1 --2- 172.21.15.16:0/3912755799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50077940 0x7eff50077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.080+0000 7eff55200640 1 -- 172.21.15.16:0/3912755799 >> 172.21.15.16:0/3912755799 conn(0x7eff50101810 msgr2=0x7eff50103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:09.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.080+0000 7eff55200640 1 -- 172.21.15.16:0/3912755799 shutdown_connections 2024-09-21T03:43:09.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.080+0000 7eff55200640 1 -- 172.21.15.16:0/3912755799 wait complete. 2024-09-21T03:43:09.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.081+0000 7eff55200640 1 Processor -- start 2024-09-21T03:43:09.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.082+0000 7eff55200640 1 -- start start 2024-09-21T03:43:09.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.082+0000 7eff55200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50077940 0x7eff501a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.082+0000 7eff55200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff50078290 0x7eff501a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.082+0000 7eff55200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff50114740 0x7eff501a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.082+0000 7eff55200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff5011a100 con 0x7eff50078290 2024-09-21T03:43:09.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.082+0000 7eff55200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff50119f80 con 0x7eff50077940 2024-09-21T03:43:09.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.082+0000 7eff55200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff5011a280 con 0x7eff50114740 2024-09-21T03:43:09.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.082+0000 7eff47e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff50078290 0x7eff501a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.082+0000 7eff47e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff50078290 0x7eff501a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45112/0 (socket says 172.21.15.16:45112) 2024-09-21T03:43:09.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.082+0000 7eff47e00640 1 -- 172.21.15.16:0/4047238767 learned_addr learned my addr 172.21.15.16:0/4047238767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:09.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.082+0000 7eff4de00640 1 --2- 172.21.15.16:0/4047238767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50077940 0x7eff501a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.082+0000 7eff4e800640 1 --2- 172.21.15.16:0/4047238767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff50114740 0x7eff501a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.083+0000 7eff47e00640 1 -- 172.21.15.16:0/4047238767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff50114740 msgr2=0x7eff501a6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.083+0000 7eff47e00640 1 --2- 172.21.15.16:0/4047238767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff50114740 0x7eff501a6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.083+0000 7eff47e00640 1 -- 172.21.15.16:0/4047238767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50077940 msgr2=0x7eff501a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:09.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.083+0000 7eff47e00640 1 --2- 172.21.15.16:0/4047238767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50077940 0x7eff501a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:09.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.083+0000 7eff47e00640 1 -- 172.21.15.16:0/4047238767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff501a69a0 con 0x7eff50078290 2024-09-21T03:43:09.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.083+0000 7eff47e00640 1 --2- 172.21.15.16:0/4047238767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff50078290 0x7eff501a0fd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7eff3800e9f0 tx=0x7eff3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:09.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.083+0000 7eff46a00640 1 -- 172.21.15.16:0/4047238767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3800ccc0 con 0x7eff50078290 2024-09-21T03:43:09.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.083+0000 7eff55200640 1 -- 172.21.15.16:0/4047238767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff501a6c90 con 0x7eff50078290 2024-09-21T03:43:09.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.083+0000 7eff46a00640 1 -- 172.21.15.16:0/4047238767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff380108e0 con 0x7eff50078290 2024-09-21T03:43:09.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.083+0000 7eff46a00640 1 -- 172.21.15.16:0/4047238767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38018860 con 0x7eff50078290 2024-09-21T03:43:09.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.084+0000 7eff55200640 1 -- 172.21.15.16:0/4047238767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff501ad3f0 con 0x7eff50078290 2024-09-21T03:43:09.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.085+0000 7eff55200640 1 -- 172.21.15.16:0/4047238767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff50077d50 con 0x7eff50078290 2024-09-21T03:43:09.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.087+0000 7eff46a00640 1 -- 172.21.15.16:0/4047238767 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7eff38010430 con 0x7eff50078290 2024-09-21T03:43:09.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.089+0000 7eff46a00640 1 --2- 172.21.15.16:0/4047238767 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7eff340b6510 0x7eff340b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:09.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.090+0000 7eff46a00640 1 -- 172.21.15.16:0/4047238767 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7879+0+0 (secure 0 0 0) 0x7eff380e7900 con 0x7eff50078290 2024-09-21T03:43:09.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.090+0000 7eff4de00640 1 --2- 172.21.15.16:0/4047238767 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7eff340b6510 0x7eff340b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:09.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.091+0000 7eff4de00640 1 --2- 172.21.15.16:0/4047238767 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7eff340b6510 0x7eff340b8a20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7eff40009bd0 tx=0x7eff40005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:09.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.091+0000 7eff46a00640 1 -- 172.21.15.16:0/4047238767 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7eff380ae340 con 0x7eff50078290 2024-09-21T03:43:09.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:09.331+0000 7eff55200640 1 -- 172.21.15.16:0/4047238767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7eff501a1b60 con 0x7eff50078290 2024-09-21T03:43:09.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:08 smithi134 ceph-mon[35031]: pgmap v309: 97 pgs: 17 creating+peering, 8 active+clean, 72 unknown; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:09.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:08 smithi134 ceph-mon[35031]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:43:09.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:08 smithi123 ceph-mon[34673]: pgmap v309: 97 pgs: 17 creating+peering, 8 active+clean, 72 unknown; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:09.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:08 smithi123 ceph-mon[34673]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:43:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:10.004+0000 7eff46a00640 1 -- 172.21.15.16:0/4047238767 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7eff501a1b60 con 0x7eff50078290 2024-09-21T03:43:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T03:43:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:10.007+0000 7eff55200640 1 -- 172.21.15.16:0/4047238767 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7eff340b6510 msgr2=0x7eff340b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:10.007+0000 7eff55200640 1 --2- 172.21.15.16:0/4047238767 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7eff340b6510 0x7eff340b8a20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7eff40009bd0 tx=0x7eff40005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:10.007+0000 7eff55200640 1 -- 172.21.15.16:0/4047238767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff50078290 msgr2=0x7eff501a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:10.007+0000 7eff55200640 1 --2- 172.21.15.16:0/4047238767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff50078290 0x7eff501a0fd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7eff3800e9f0 tx=0x7eff3800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:10.007+0000 7eff55200640 1 -- 172.21.15.16:0/4047238767 shutdown_connections 2024-09-21T03:43:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:10.007+0000 7eff55200640 1 --2- 172.21.15.16:0/4047238767 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7eff340b6510 0x7eff340b8a20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:10.007+0000 7eff55200640 1 --2- 172.21.15.16:0/4047238767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff50114740 0x7eff501a6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:10.007+0000 7eff55200640 1 --2- 172.21.15.16:0/4047238767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff50078290 0x7eff501a0fd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:10.007+0000 7eff55200640 1 --2- 172.21.15.16:0/4047238767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff50077940 0x7eff501a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:10.007+0000 7eff55200640 1 -- 172.21.15.16:0/4047238767 >> 172.21.15.16:0/4047238767 conn(0x7eff50101810 msgr2=0x7eff50116720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:10.007+0000 7eff55200640 1 -- 172.21.15.16:0/4047238767 shutdown_connections 2024-09-21T03:43:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:10.008+0000 7eff55200640 1 -- 172.21.15.16:0/4047238767 wait complete. 2024-09-21T03:43:10.136 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:10 smithi016 ceph-mon[30375]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:43:10.136 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:10 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/4047238767' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:43:10.136 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:10 smithi016 ceph-mon[30375]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T03:43:10.267 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:10 smithi134 ceph-mon[35031]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:43:10.267 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:10 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/4047238767' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:43:10.267 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:10 smithi134 ceph-mon[35031]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T03:43:10.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:10 smithi123 ceph-mon[34673]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:43:10.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:10 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/4047238767' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:43:10.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:10 smithi123 ceph-mon[34673]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T03:43:10.923 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-21T03:43:10.924 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-21T03:43:11.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:11 smithi016 ceph-mon[30375]: pgmap v312: 97 pgs: 30 creating+peering, 34 active+clean, 33 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:11.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:11 smithi016 ceph-mon[30375]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:43:11.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:11 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/4047238767' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:43:11.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:11 smithi016 ceph-mon[30375]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T03:43:11.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:11 smithi016 ceph-mon[30375]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T03:43:11.314 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:11.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:11 smithi134 ceph-mon[35031]: pgmap v312: 97 pgs: 30 creating+peering, 34 active+clean, 33 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:11.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:11 smithi134 ceph-mon[35031]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:43:11.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:11 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/4047238767' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:43:11.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:11 smithi134 ceph-mon[35031]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T03:43:11.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:11 smithi134 ceph-mon[35031]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T03:43:11.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:11 smithi123 ceph-mon[34673]: pgmap v312: 97 pgs: 30 creating+peering, 34 active+clean, 33 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:11.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:11 smithi123 ceph-mon[34673]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:43:11.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:11 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/4047238767' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:43:11.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:11 smithi123 ceph-mon[34673]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T03:43:11.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:11 smithi123 ceph-mon[34673]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T03:43:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:11.998+0000 7f453e400640 1 -- 172.21.15.16:0/1839862411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f453810b4f0 msgr2=0x7f453810b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:11.998+0000 7f453e400640 1 --2- 172.21.15.16:0/1839862411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f453810b4f0 0x7f453810b900 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4520009860 tx=0x7f452002a340 comp rx=0 tx=0).stop 2024-09-21T03:43:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:11.998+0000 7f453e400640 1 -- 172.21.15.16:0/1839862411 shutdown_connections 2024-09-21T03:43:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:11.998+0000 7f453e400640 1 --2- 172.21.15.16:0/1839862411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f453810be40 0x7f4538112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:11.998+0000 7f453e400640 1 --2- 172.21.15.16:0/1839862411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f453810b4f0 0x7f453810b900 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:11.998+0000 7f453e400640 1 --2- 172.21.15.16:0/1839862411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4538105bb0 0x7f4538105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:11.998+0000 7f453e400640 1 -- 172.21.15.16:0/1839862411 >> 172.21.15.16:0/1839862411 conn(0x7f4538101680 msgr2=0x7f4538103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:11.998+0000 7f453e400640 1 -- 172.21.15.16:0/1839862411 shutdown_connections 2024-09-21T03:43:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:11.998+0000 7f453e400640 1 -- 172.21.15.16:0/1839862411 wait complete. 2024-09-21T03:43:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:11.999+0000 7f453e400640 1 Processor -- start 2024-09-21T03:43:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.000+0000 7f453e400640 1 -- start start 2024-09-21T03:43:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.000+0000 7f453e400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4538105bb0 0x7f453819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.000+0000 7f453e400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f453810b4f0 0x7f453819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.000+0000 7f453e400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f453810be40 0x7f45381a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.000+0000 7f453e400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4538115b00 con 0x7f453810be40 2024-09-21T03:43:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.000+0000 7f453e400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4538115980 con 0x7f453810b4f0 2024-09-21T03:43:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.000+0000 7f453e400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4538115c80 con 0x7f4538105bb0 2024-09-21T03:43:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.000+0000 7f453d000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4538105bb0 0x7f453819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.000+0000 7f453da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f453810be40 0x7f45381a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.000+0000 7f453d000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4538105bb0 0x7f453819c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:38816/0 (socket says 172.21.15.16:38816) 2024-09-21T03:43:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.000+0000 7f453d000640 1 -- 172.21.15.16:0/1269956728 learned_addr learned my addr 172.21.15.16:0/1269956728 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.000+0000 7f452fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f453810b4f0 0x7f453819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.001+0000 7f453d000640 1 -- 172.21.15.16:0/1269956728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f453810b4f0 msgr2=0x7f453819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.001+0000 7f453d000640 1 --2- 172.21.15.16:0/1269956728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f453810b4f0 0x7f453819ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.001+0000 7f453d000640 1 -- 172.21.15.16:0/1269956728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f453810be40 msgr2=0x7f45381a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.001+0000 7f453d000640 1 --2- 172.21.15.16:0/1269956728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f453810be40 0x7f45381a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.001+0000 7f453d000640 1 -- 172.21.15.16:0/1269956728 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45381a2440 con 0x7f4538105bb0 2024-09-21T03:43:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.001+0000 7f452fe00640 1 --2- 172.21.15.16:0/1269956728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f453810b4f0 0x7f453819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.001+0000 7f453da00640 1 --2- 172.21.15.16:0/1269956728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f453810be40 0x7f45381a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:12.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.001+0000 7f453d000640 1 --2- 172.21.15.16:0/1269956728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4538105bb0 0x7f453819c4d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f452800e9c0 tx=0x7f452800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:12.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.001+0000 7f4536000640 1 -- 172.21.15.16:0/1269956728 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452800cde0 con 0x7f4538105bb0 2024-09-21T03:43:12.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.001+0000 7f4536000640 1 -- 172.21.15.16:0/1269956728 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45280035f0 con 0x7f4538105bb0 2024-09-21T03:43:12.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.002+0000 7f4536000640 1 -- 172.21.15.16:0/1269956728 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528010690 con 0x7f4538105bb0 2024-09-21T03:43:12.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.002+0000 7f453e400640 1 -- 172.21.15.16:0/1269956728 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45381a2730 con 0x7f4538105bb0 2024-09-21T03:43:12.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.002+0000 7f453e400640 1 -- 172.21.15.16:0/1269956728 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45381a8e90 con 0x7f4538105bb0 2024-09-21T03:43:12.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.003+0000 7f453e400640 1 -- 172.21.15.16:0/1269956728 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4538105fc0 con 0x7f4538105bb0 2024-09-21T03:43:12.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.005+0000 7f4536000640 1 -- 172.21.15.16:0/1269956728 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f4528004000 con 0x7f4538105bb0 2024-09-21T03:43:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.006+0000 7f4536000640 1 --2- 172.21.15.16:0/1269956728 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f451c0b6590 0x7f451c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.007+0000 7f4536000640 1 -- 172.21.15.16:0/1269956728 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f45280e75a0 con 0x7f4538105bb0 2024-09-21T03:43:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.007+0000 7f452fe00640 1 --2- 172.21.15.16:0/1269956728 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f451c0b6590 0x7f451c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.007+0000 7f452fe00640 1 --2- 172.21.15.16:0/1269956728 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f451c0b6590 0x7f451c0b8aa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4520005940 tx=0x7f45200058d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:12.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.009+0000 7f4536000640 1 -- 172.21.15.16:0/1269956728 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f45280adf70 con 0x7f4538105bb0 2024-09-21T03:43:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:12.217+0000 7f453e400640 1 -- 172.21.15.16:0/1269956728 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f453802e640 con 0x7f4538105bb0 2024-09-21T03:43:12.220 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:12 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] up:active 2024-09-21T03:43:12.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:12 smithi016 ceph-mon[30375]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:43:12.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:12 smithi016 ceph-mon[30375]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:43:12.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:12 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] up:active 2024-09-21T03:43:12.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:12 smithi134 ceph-mon[35031]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:43:12.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:12 smithi134 ceph-mon[35031]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:43:12.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:12 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] up:active 2024-09-21T03:43:12.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:12 smithi123 ceph-mon[34673]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:43:12.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:12 smithi123 ceph-mon[34673]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:43:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:13.031+0000 7f4536000640 1 -- 172.21.15.16:0/1269956728 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v76) ==== 157+0+0 (secure 0 0 0) 0x7f45280b2350 con 0x7f4538105bb0 2024-09-21T03:43:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:13.035+0000 7f453e400640 1 -- 172.21.15.16:0/1269956728 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f451c0b6590 msgr2=0x7f451c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:13.035+0000 7f453e400640 1 --2- 172.21.15.16:0/1269956728 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f451c0b6590 0x7f451c0b8aa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4520005940 tx=0x7f45200058d0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:13.035+0000 7f453e400640 1 -- 172.21.15.16:0/1269956728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4538105bb0 msgr2=0x7f453819c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:13.035+0000 7f453e400640 1 --2- 172.21.15.16:0/1269956728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4538105bb0 0x7f453819c4d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f452800e9c0 tx=0x7f452800ee80 comp rx=0 tx=0).stop 2024-09-21T03:43:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:13.035+0000 7f453e400640 1 -- 172.21.15.16:0/1269956728 shutdown_connections 2024-09-21T03:43:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:13.035+0000 7f453e400640 1 --2- 172.21.15.16:0/1269956728 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f451c0b6590 0x7f451c0b8aa0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:13.035+0000 7f453e400640 1 --2- 172.21.15.16:0/1269956728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f453810be40 0x7f45381a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:13.035+0000 7f453e400640 1 --2- 172.21.15.16:0/1269956728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f453810b4f0 0x7f453819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:13.035+0000 7f453e400640 1 --2- 172.21.15.16:0/1269956728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4538105bb0 0x7f453819c4d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:13.035+0000 7f453e400640 1 -- 172.21.15.16:0/1269956728 >> 172.21.15.16:0/1269956728 conn(0x7f4538101680 msgr2=0x7f453810aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:13.035+0000 7f453e400640 1 -- 172.21.15.16:0/1269956728 shutdown_connections 2024-09-21T03:43:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:13.035+0000 7f453e400640 1 -- 172.21.15.16:0/1269956728 wait complete. 2024-09-21T03:43:13.307 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:13 smithi123 ceph-mon[34673]: pgmap v314: 97 pgs: 18 creating+peering, 62 active+clean, 17 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 235 B/s wr, 0 op/s 2024-09-21T03:43:13.307 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:13 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1269956728' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T03:43:13.307 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:13 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T03:43:13.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:13 smithi134 ceph-mon[35031]: pgmap v314: 97 pgs: 18 creating+peering, 62 active+clean, 17 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 235 B/s wr, 0 op/s 2024-09-21T03:43:13.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:13 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1269956728' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T03:43:13.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:13 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T03:43:13.378 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:13 smithi016 ceph-mon[30375]: pgmap v314: 97 pgs: 18 creating+peering, 62 active+clean, 17 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 235 B/s wr, 0 op/s 2024-09-21T03:43:13.378 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:13 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1269956728' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T03:43:13.378 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:13 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T03:43:13.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T03:43:14.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:14 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T03:43:14.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:14 smithi016 ceph-mon[30375]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:43:14.280 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:14.319 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:14 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T03:43:14.319 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:14 smithi123 ceph-mon[34673]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:43:14.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:14 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T03:43:14.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:14 smithi134 ceph-mon[35031]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:43:14.573 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 03:43:14 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-b[62470]: 2024-09-21T03:43:14.386+0000 7f4e89757640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:43:14.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.909+0000 7f2d6f200640 1 -- 172.21.15.16:0/1323746326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d681049f0 msgr2=0x7f2d68104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:14.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.909+0000 7f2d6f200640 1 --2- 172.21.15.16:0/1323746326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d681049f0 0x7f2d68104e00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009900 tx=0x7f2d5802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:43:14.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.909+0000 7f2d6f200640 1 -- 172.21.15.16:0/1323746326 shutdown_connections 2024-09-21T03:43:14.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.909+0000 7f2d6f200640 1 --2- 172.21.15.16:0/1323746326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d6810de50 0x7f2d68110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:14.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.909+0000 7f2d6f200640 1 --2- 172.21.15.16:0/1323746326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d68105340 0x7f2d6810d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:14.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.909+0000 7f2d6f200640 1 --2- 172.21.15.16:0/1323746326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d681049f0 0x7f2d68104e00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:14.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.909+0000 7f2d6f200640 1 -- 172.21.15.16:0/1323746326 >> 172.21.15.16:0/1323746326 conn(0x7f2d68079790 msgr2=0x7f2d68079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:14.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.909+0000 7f2d6f200640 1 -- 172.21.15.16:0/1323746326 shutdown_connections 2024-09-21T03:43:14.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.909+0000 7f2d6f200640 1 -- 172.21.15.16:0/1323746326 wait complete. 2024-09-21T03:43:14.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.910+0000 7f2d6f200640 1 Processor -- start 2024-09-21T03:43:14.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.910+0000 7f2d6f200640 1 -- start start 2024-09-21T03:43:14.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.911+0000 7f2d6f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d681049f0 0x7f2d6819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:14.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.911+0000 7f2d6f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d68105340 0x7f2d6819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:14.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.911+0000 7f2d6f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d6810de50 0x7f2d6819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:14.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.911+0000 7f2d6f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d68113b20 con 0x7f2d681049f0 2024-09-21T03:43:14.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.911+0000 7f2d6f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d681139a0 con 0x7f2d68105340 2024-09-21T03:43:14.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.911+0000 7f2d6f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2d68113ca0 con 0x7f2d6810de50 2024-09-21T03:43:14.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.911+0000 7f2d5fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d68105340 0x7f2d6819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:14.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.911+0000 7f2d5fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d68105340 0x7f2d6819aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42978/0 (socket says 172.21.15.16:42978) 2024-09-21T03:43:14.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.911+0000 7f2d67e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d681049f0 0x7f2d6819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:14.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.911+0000 7f2d5fe00640 1 -- 172.21.15.16:0/3399997012 learned_addr learned my addr 172.21.15.16:0/3399997012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:14.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.911+0000 7f2d6d200640 1 --2- 172.21.15.16:0/3399997012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d6810de50 0x7f2d6819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:14.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.911+0000 7f2d5fe00640 1 -- 172.21.15.16:0/3399997012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d6810de50 msgr2=0x7f2d6819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:14.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.911+0000 7f2d5fe00640 1 --2- 172.21.15.16:0/3399997012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d6810de50 0x7f2d6819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:14.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.911+0000 7f2d5fe00640 1 -- 172.21.15.16:0/3399997012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d681049f0 msgr2=0x7f2d6819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:14.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.911+0000 7f2d5fe00640 1 --2- 172.21.15.16:0/3399997012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d681049f0 0x7f2d6819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:14.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.911+0000 7f2d5fe00640 1 -- 172.21.15.16:0/3399997012 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d681a0400 con 0x7f2d68105340 2024-09-21T03:43:14.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.912+0000 7f2d5fe00640 1 --2- 172.21.15.16:0/3399997012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d68105340 0x7f2d6819aa30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000e9f0 tx=0x7f2d5000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:14.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.912+0000 7f2d65600640 1 -- 172.21.15.16:0/3399997012 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5000ccc0 con 0x7f2d68105340 2024-09-21T03:43:14.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.912+0000 7f2d65600640 1 -- 172.21.15.16:0/3399997012 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d500108e0 con 0x7f2d68105340 2024-09-21T03:43:14.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.912+0000 7f2d65600640 1 -- 172.21.15.16:0/3399997012 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d50018860 con 0x7f2d68105340 2024-09-21T03:43:14.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.912+0000 7f2d6f200640 1 -- 172.21.15.16:0/3399997012 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d681a06f0 con 0x7f2d68105340 2024-09-21T03:43:14.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.912+0000 7f2d6f200640 1 -- 172.21.15.16:0/3399997012 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d681a6e50 con 0x7f2d68105340 2024-09-21T03:43:14.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.913+0000 7f2d6f200640 1 -- 172.21.15.16:0/3399997012 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d68104e00 con 0x7f2d68105340 2024-09-21T03:43:14.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.915+0000 7f2d65600640 1 -- 172.21.15.16:0/3399997012 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f2d50010430 con 0x7f2d68105340 2024-09-21T03:43:14.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.917+0000 7f2d65600640 1 --2- 172.21.15.16:0/3399997012 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2d4c0b6590 0x7f2d4c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:14.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.917+0000 7f2d65600640 1 -- 172.21.15.16:0/3399997012 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8277+0+0 (secure 0 0 0) 0x7f2d500e7a90 con 0x7f2d68105340 2024-09-21T03:43:14.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.917+0000 7f2d67e00640 1 --2- 172.21.15.16:0/3399997012 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2d4c0b6590 0x7f2d4c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:14.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.918+0000 7f2d67e00640 1 --2- 172.21.15.16:0/3399997012 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2d4c0b6590 0x7f2d4c0b8aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2d58031440 tx=0x7f2d58005a20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:14.919+0000 7f2d65600640 1 -- 172.21.15.16:0/3399997012 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2d500b2670 con 0x7f2d68105340 2024-09-21T03:43:15.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:15.120+0000 7f2d6f200640 1 -- 172.21.15.16:0/3399997012 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2d681a7080 con 0x7f2d68105340 2024-09-21T03:43:15.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:15 smithi134 ceph-mon[35031]: pgmap v316: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 2 op/s 2024-09-21T03:43:15.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:15 smithi123 ceph-mon[34673]: pgmap v316: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 2 op/s 2024-09-21T03:43:15.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:15 smithi016 ceph-mon[30375]: pgmap v316: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 2 op/s 2024-09-21T03:43:16.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:16 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3399997012' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:43:16.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:16 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:43:16.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:16 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3399997012' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:43:16.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:16 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:43:16.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:16 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3399997012' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:43:16.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:16 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:43:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:17.058+0000 7f2d65600640 1 -- 172.21.15.16:0/3399997012 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v78) ==== 228+0+0 (secure 0 0 0) 0x7f2d500b75b0 con 0x7f2d68105340 2024-09-21T03:43:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' created 2024-09-21T03:43:17.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:17.066+0000 7f2d6f200640 1 -- 172.21.15.16:0/3399997012 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2d4c0b6590 msgr2=0x7f2d4c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:17.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:17.066+0000 7f2d6f200640 1 --2- 172.21.15.16:0/3399997012 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2d4c0b6590 0x7f2d4c0b8aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2d58031440 tx=0x7f2d58005a20 comp rx=0 tx=0).stop 2024-09-21T03:43:17.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:17.067+0000 7f2d6f200640 1 -- 172.21.15.16:0/3399997012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d68105340 msgr2=0x7f2d6819aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:17.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:17.067+0000 7f2d6f200640 1 --2- 172.21.15.16:0/3399997012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d68105340 0x7f2d6819aa30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000e9f0 tx=0x7f2d5000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:17.067+0000 7f2d6f200640 1 -- 172.21.15.16:0/3399997012 shutdown_connections 2024-09-21T03:43:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:17.067+0000 7f2d6f200640 1 --2- 172.21.15.16:0/3399997012 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2d4c0b6590 0x7f2d4c0b8aa0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:17.067+0000 7f2d6f200640 1 --2- 172.21.15.16:0/3399997012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d6810de50 0x7f2d6819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:17.067+0000 7f2d6f200640 1 --2- 172.21.15.16:0/3399997012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d68105340 0x7f2d6819aa30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:17.067+0000 7f2d6f200640 1 --2- 172.21.15.16:0/3399997012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d681049f0 0x7f2d6819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:17.067+0000 7f2d6f200640 1 -- 172.21.15.16:0/3399997012 >> 172.21.15.16:0/3399997012 conn(0x7f2d68079790 msgr2=0x7f2d68107950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:17.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:17.068+0000 7f2d6f200640 1 -- 172.21.15.16:0/3399997012 shutdown_connections 2024-09-21T03:43:17.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:17.068+0000 7f2d6f200640 1 -- 172.21.15.16:0/3399997012 wait complete. 2024-09-21T03:43:17.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:17 smithi134 ceph-mon[35031]: pgmap v317: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-21T03:43:17.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:17 smithi134 ceph-mon[35031]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:43:17.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:17 smithi123 ceph-mon[34673]: pgmap v317: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-21T03:43:17.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:17 smithi123 ceph-mon[34673]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:43:17.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:17 smithi016 ceph-mon[30375]: pgmap v317: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-21T03:43:17.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:17 smithi016 ceph-mon[30375]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:43:17.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-21T03:43:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:18.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:18 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:43:18.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:18 smithi134 ceph-mon[35031]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:43:18.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:18 smithi134 ceph-mon[35031]: pgmap v320: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-21T03:43:18.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:18 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:43:18.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:18 smithi123 ceph-mon[34673]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:43:18.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:18 smithi123 ceph-mon[34673]: pgmap v320: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-21T03:43:18.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:18 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:43:18.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:18 smithi016 ceph-mon[30375]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:43:18.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:18 smithi016 ceph-mon[30375]: pgmap v320: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-21T03:43:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.966+0000 7f94e4c00640 1 -- 172.21.15.16:0/2861554569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94e010b4f0 msgr2=0x7f94e010b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.966+0000 7f94e4c00640 1 --2- 172.21.15.16:0/2861554569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94e010b4f0 0x7f94e010b900 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009900 tx=0x7f94c802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.966+0000 7f94e4c00640 1 -- 172.21.15.16:0/2861554569 shutdown_connections 2024-09-21T03:43:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.966+0000 7f94e4c00640 1 --2- 172.21.15.16:0/2861554569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94e010be40 0x7f94e0112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.966+0000 7f94e4c00640 1 --2- 172.21.15.16:0/2861554569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94e010b4f0 0x7f94e010b900 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.966+0000 7f94e4c00640 1 --2- 172.21.15.16:0/2861554569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e0105bb0 0x7f94e0105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.966+0000 7f94e4c00640 1 -- 172.21.15.16:0/2861554569 >> 172.21.15.16:0/2861554569 conn(0x7f94e0101680 msgr2=0x7f94e0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.966+0000 7f94e4c00640 1 -- 172.21.15.16:0/2861554569 shutdown_connections 2024-09-21T03:43:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.966+0000 7f94e4c00640 1 -- 172.21.15.16:0/2861554569 wait complete. 2024-09-21T03:43:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.967+0000 7f94e4c00640 1 Processor -- start 2024-09-21T03:43:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.967+0000 7f94e4c00640 1 -- start start 2024-09-21T03:43:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.968+0000 7f94e4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94e0105bb0 0x7f94e019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.968+0000 7f94e4c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e010b4f0 0x7f94e019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.968+0000 7f94e4c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94e010be40 0x7f94e01a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.968+0000 7f94e4c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f94e0115b00 con 0x7f94e0105bb0 2024-09-21T03:43:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.968+0000 7f94e4c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94e0115980 con 0x7f94e010be40 2024-09-21T03:43:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.968+0000 7f94e4c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f94e0115c80 con 0x7f94e010b4f0 2024-09-21T03:43:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.968+0000 7f94df400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94e0105bb0 0x7f94e019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.968+0000 7f94df400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94e0105bb0 0x7f94e019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51384/0 (socket says 172.21.15.16:51384) 2024-09-21T03:43:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.968+0000 7f94dfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94e010be40 0x7f94e01a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.968+0000 7f94df400640 1 -- 172.21.15.16:0/1142163271 learned_addr learned my addr 172.21.15.16:0/1142163271 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.968+0000 7f94d7e00640 1 --2- 172.21.15.16:0/1142163271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e010b4f0 0x7f94e019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.969+0000 7f94dfe00640 1 -- 172.21.15.16:0/1142163271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e010b4f0 msgr2=0x7f94e019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.969+0000 7f94dfe00640 1 --2- 172.21.15.16:0/1142163271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e010b4f0 0x7f94e019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.969+0000 7f94dfe00640 1 -- 172.21.15.16:0/1142163271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94e0105bb0 msgr2=0x7f94e019c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.969+0000 7f94dfe00640 1 --2- 172.21.15.16:0/1142163271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94e0105bb0 0x7f94e019c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.969+0000 7f94dfe00640 1 -- 172.21.15.16:0/1142163271 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e01a2460 con 0x7f94e010be40 2024-09-21T03:43:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.969+0000 7f94dfe00640 1 --2- 172.21.15.16:0/1142163271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94e010be40 0x7f94e01a1d30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f94d800da10 tx=0x7f94d800ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.969+0000 7f94dcc00640 1 -- 172.21.15.16:0/1142163271 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d800bd70 con 0x7f94e010be40 2024-09-21T03:43:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.969+0000 7f94e4c00640 1 -- 172.21.15.16:0/1142163271 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e01a2750 con 0x7f94e010be40 2024-09-21T03:43:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.969+0000 7f94dcc00640 1 -- 172.21.15.16:0/1142163271 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94d8003e20 con 0x7f94e010be40 2024-09-21T03:43:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.969+0000 7f94dcc00640 1 -- 172.21.15.16:0/1142163271 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d80106c0 con 0x7f94e010be40 2024-09-21T03:43:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.969+0000 7f94e4c00640 1 -- 172.21.15.16:0/1142163271 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e01a8eb0 con 0x7f94e010be40 2024-09-21T03:43:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.970+0000 7f94e4c00640 1 -- 172.21.15.16:0/1142163271 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94e0105fc0 con 0x7f94e010be40 2024-09-21T03:43:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.973+0000 7f94dcc00640 1 -- 172.21.15.16:0/1142163271 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f94d8004270 con 0x7f94e010be40 2024-09-21T03:43:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.975+0000 7f94dcc00640 1 --2- 172.21.15.16:0/1142163271 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f94c40b6590 0x7f94c40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.975+0000 7f94dcc00640 1 -- 172.21.15.16:0/1142163271 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8770+0+0 (secure 0 0 0) 0x7f94d80e7c00 con 0x7f94e010be40 2024-09-21T03:43:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.975+0000 7f94df400640 1 --2- 172.21.15.16:0/1142163271 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f94c40b6590 0x7f94c40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.976+0000 7f94df400640 1 --2- 172.21.15.16:0/1142163271 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f94c40b6590 0x7f94c40b8aa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009cd0 tx=0x7f94d0005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:18.977+0000 7f94dcc00640 1 -- 172.21.15.16:0/1142163271 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f94d80ae290 con 0x7f94e010be40 2024-09-21T03:43:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:19.175+0000 7f94e4c00640 1 -- 172.21.15.16:0/1142163271 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f94e019d620 con 0x7f94e010be40 2024-09-21T03:43:19.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:19 smithi134 ceph-mon[35031]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:43:19.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:19 smithi123 ceph-mon[34673]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:43:19.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:19 smithi016 ceph-mon[30375]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:43:20.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:20.080+0000 7f94dcc00640 1 -- 172.21.15.16:0/1142163271 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v81) ==== 169+0+0 (secure 0 0 0) 0x7f94e019d620 con 0x7f94e010be40 2024-09-21T03:43:20.083 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 4 allow_ec_overwrites to true 2024-09-21T03:43:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:20.083+0000 7f94e4c00640 1 -- 172.21.15.16:0/1142163271 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f94c40b6590 msgr2=0x7f94c40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:20.083+0000 7f94e4c00640 1 --2- 172.21.15.16:0/1142163271 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f94c40b6590 0x7f94c40b8aa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009cd0 tx=0x7f94d0005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:20.083+0000 7f94e4c00640 1 -- 172.21.15.16:0/1142163271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94e010be40 msgr2=0x7f94e01a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:20.083+0000 7f94e4c00640 1 --2- 172.21.15.16:0/1142163271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94e010be40 0x7f94e01a1d30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f94d800da10 tx=0x7f94d800ded0 comp rx=0 tx=0).stop 2024-09-21T03:43:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:20.083+0000 7f94e4c00640 1 -- 172.21.15.16:0/1142163271 shutdown_connections 2024-09-21T03:43:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:20.083+0000 7f94e4c00640 1 --2- 172.21.15.16:0/1142163271 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f94c40b6590 0x7f94c40b8aa0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:20.083+0000 7f94e4c00640 1 --2- 172.21.15.16:0/1142163271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94e010be40 0x7f94e01a1d30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:20.083+0000 7f94e4c00640 1 --2- 172.21.15.16:0/1142163271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e010b4f0 0x7f94e019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:20.083+0000 7f94e4c00640 1 --2- 172.21.15.16:0/1142163271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94e0105bb0 0x7f94e019c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:20.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:20.083+0000 7f94e4c00640 1 -- 172.21.15.16:0/1142163271 >> 172.21.15.16:0/1142163271 conn(0x7f94e0101680 msgr2=0x7f94e0103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:20.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:20.083+0000 7f94e4c00640 1 -- 172.21.15.16:0/1142163271 shutdown_connections 2024-09-21T03:43:20.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:20.083+0000 7f94e4c00640 1 -- 172.21.15.16:0/1142163271 wait complete. 2024-09-21T03:43:20.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:20 smithi134 ceph-mon[35031]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:43:20.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:20 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1142163271' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T03:43:20.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:20 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T03:43:20.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:20 smithi134 ceph-mon[35031]: pgmap v323: 129 pgs: 5 creating+peering, 21 unknown, 103 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-21T03:43:20.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:20 smithi123 ceph-mon[34673]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:43:20.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:20 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1142163271' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T03:43:20.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:20 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T03:43:20.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:20 smithi123 ceph-mon[34673]: pgmap v323: 129 pgs: 5 creating+peering, 21 unknown, 103 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-21T03:43:20.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:20 smithi016 ceph-mon[30375]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:43:20.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:20 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1142163271' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T03:43:20.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:20 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T03:43:20.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:20 smithi016 ceph-mon[30375]: pgmap v323: 129 pgs: 5 creating+peering, 21 unknown, 103 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-21T03:43:20.984 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-21T03:43:21.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:21 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T03:43:21.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:21 smithi134 ceph-mon[35031]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:43:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:21.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:21 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T03:43:21.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:21 smithi123 ceph-mon[34673]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:43:21.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:21 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T03:43:21.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:21 smithi016 ceph-mon[30375]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:43:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.990+0000 7f49a6e00640 1 -- 172.21.15.16:0/3290370722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a0109c20 msgr2=0x7f49a0110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.990+0000 7f49a6e00640 1 --2- 172.21.15.16:0/3290370722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a0109c20 0x7f49a0110450 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4998009ca0 tx=0x7f499802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:43:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.990+0000 7f49a6e00640 1 -- 172.21.15.16:0/3290370722 shutdown_connections 2024-09-21T03:43:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.990+0000 7f49a6e00640 1 --2- 172.21.15.16:0/3290370722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a0109c20 0x7f49a0110450 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.990+0000 7f49a6e00640 1 --2- 172.21.15.16:0/3290370722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49a01092d0 0x7f49a01096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.990+0000 7f49a6e00640 1 --2- 172.21.15.16:0/3290370722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a0103990 0x7f49a0103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.990+0000 7f49a6e00640 1 -- 172.21.15.16:0/3290370722 >> 172.21.15.16:0/3290370722 conn(0x7f49a00ff3e0 msgr2=0x7f49a0101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.991+0000 7f49a6e00640 1 -- 172.21.15.16:0/3290370722 shutdown_connections 2024-09-21T03:43:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.991+0000 7f49a6e00640 1 -- 172.21.15.16:0/3290370722 wait complete. 2024-09-21T03:43:21.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.992+0000 7f49a6e00640 1 Processor -- start 2024-09-21T03:43:21.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.992+0000 7f49a6e00640 1 -- start start 2024-09-21T03:43:21.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.992+0000 7f49a6e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a0103990 0x7f49a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:21.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.992+0000 7f49a6e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a01092d0 0x7f49a0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:21.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.992+0000 7f49a6e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49a0109c20 0x7f49a0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:21.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.992+0000 7f49a6e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f49a0113930 con 0x7f49a0103990 2024-09-21T03:43:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.992+0000 7f49a6e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f49a01137b0 con 0x7f49a0109c20 2024-09-21T03:43:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.992+0000 7f49a6e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f49a0113ab0 con 0x7f49a01092d0 2024-09-21T03:43:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.992+0000 7f49a5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a0103990 0x7f49a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.992+0000 7f49a5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a0103990 0x7f49a0075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41766/0 (socket says 172.21.15.16:41766) 2024-09-21T03:43:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.992+0000 7f49a5a00640 1 -- 172.21.15.16:0/2585214414 learned_addr learned my addr 172.21.15.16:0/2585214414 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.993+0000 7f49a5000640 1 --2- 172.21.15.16:0/2585214414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a01092d0 0x7f49a0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.993+0000 7f49a6400640 1 --2- 172.21.15.16:0/2585214414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49a0109c20 0x7f49a0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.993+0000 7f49a5a00640 1 -- 172.21.15.16:0/2585214414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a01092d0 msgr2=0x7f49a0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.993+0000 7f49a5a00640 1 --2- 172.21.15.16:0/2585214414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a01092d0 0x7f49a0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.993+0000 7f49a5a00640 1 -- 172.21.15.16:0/2585214414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49a0109c20 msgr2=0x7f49a0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.993+0000 7f49a5a00640 1 --2- 172.21.15.16:0/2585214414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49a0109c20 0x7f49a0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.993+0000 7f49a5a00640 1 -- 172.21.15.16:0/2585214414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a01a8580 con 0x7f49a0103990 2024-09-21T03:43:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.993+0000 7f49a5a00640 1 --2- 172.21.15.16:0/2585214414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a0103990 0x7f49a0075710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f499400e9f0 tx=0x7f499400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.994+0000 7f499e000640 1 -- 172.21.15.16:0/2585214414 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499400cd80 con 0x7f49a0103990 2024-09-21T03:43:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.994+0000 7f499e000640 1 -- 172.21.15.16:0/2585214414 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49940054d0 con 0x7f49a0103990 2024-09-21T03:43:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.994+0000 7f499e000640 1 -- 172.21.15.16:0/2585214414 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994010690 con 0x7f49a0103990 2024-09-21T03:43:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.994+0000 7f49a6e00640 1 -- 172.21.15.16:0/2585214414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a01a8870 con 0x7f49a0103990 2024-09-21T03:43:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.994+0000 7f49a6e00640 1 -- 172.21.15.16:0/2585214414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a01a8d60 con 0x7f49a0103990 2024-09-21T03:43:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.995+0000 7f49a6e00640 1 -- 172.21.15.16:0/2585214414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a0103da0 con 0x7f49a0103990 2024-09-21T03:43:22.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.998+0000 7f499e000640 1 -- 172.21.15.16:0/2585214414 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f4994004e30 con 0x7f49a0103990 2024-09-21T03:43:22.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.999+0000 7f499e000640 1 --2- 172.21.15.16:0/2585214414 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f49880b6590 0x7f49880b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:22.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.999+0000 7f499e000640 1 -- 172.21.15.16:0/2585214414 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8770+0+0 (secure 0 0 0) 0x7f49940e7a50 con 0x7f49a0103990 2024-09-21T03:43:22.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:21.999+0000 7f49a5000640 1 --2- 172.21.15.16:0/2585214414 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f49880b6590 0x7f49880b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:22.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:22.000+0000 7f49a5000640 1 --2- 172.21.15.16:0/2585214414 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f49880b6590 0x7f49880b8aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f49a0106d50 tx=0x7f498c005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:22.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:22.002+0000 7f499e000640 1 -- 172.21.15.16:0/2585214414 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f49940ae0c0 con 0x7f49a0103990 2024-09-21T03:43:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:22.244+0000 7f49a6e00640 1 -- 172.21.15.16:0/2585214414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f49a002e640 con 0x7f49a0103990 2024-09-21T03:43:22.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:22 smithi134 ceph-mon[35031]: pgmap v325: 129 pgs: 5 creating+peering, 8 unknown, 116 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:22.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:22 smithi123 ceph-mon[34673]: pgmap v325: 129 pgs: 5 creating+peering, 8 unknown, 116 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:22.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:22 smithi016 ceph-mon[30375]: pgmap v325: 129 pgs: 5 creating+peering, 8 unknown, 116 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:23.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:23.090+0000 7f499e000640 1 -- 172.21.15.16:0/2585214414 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f49940b24a0 con 0x7f49a0103990 2024-09-21T03:43:23.093 INFO:teuthology.orchestra.run.smithi016.stderr:added data pool 4 to fsmap 2024-09-21T03:43:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:23.093+0000 7f49a6e00640 1 -- 172.21.15.16:0/2585214414 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f49880b6590 msgr2=0x7f49880b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:23.093+0000 7f49a6e00640 1 --2- 172.21.15.16:0/2585214414 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f49880b6590 0x7f49880b8aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f49a0106d50 tx=0x7f498c005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:23.093+0000 7f49a6e00640 1 -- 172.21.15.16:0/2585214414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a0103990 msgr2=0x7f49a0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:23.093+0000 7f49a6e00640 1 --2- 172.21.15.16:0/2585214414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a0103990 0x7f49a0075710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f499400e9f0 tx=0x7f499400edf0 comp rx=0 tx=0).stop 2024-09-21T03:43:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:23.094+0000 7f49a6e00640 1 -- 172.21.15.16:0/2585214414 shutdown_connections 2024-09-21T03:43:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:23.094+0000 7f49a6e00640 1 --2- 172.21.15.16:0/2585214414 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f49880b6590 0x7f49880b8aa0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:23.094+0000 7f49a6e00640 1 --2- 172.21.15.16:0/2585214414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49a0109c20 0x7f49a0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:23.094+0000 7f49a6e00640 1 --2- 172.21.15.16:0/2585214414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a01092d0 0x7f49a0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:23.094+0000 7f49a6e00640 1 --2- 172.21.15.16:0/2585214414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a0103990 0x7f49a0075710 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:23.094+0000 7f49a6e00640 1 -- 172.21.15.16:0/2585214414 >> 172.21.15.16:0/2585214414 conn(0x7f49a00ff3e0 msgr2=0x7f49a0101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:23.094+0000 7f49a6e00640 1 -- 172.21.15.16:0/2585214414 shutdown_connections 2024-09-21T03:43:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:23.094+0000 7f49a6e00640 1 -- 172.21.15.16:0/2585214414 wait complete. 2024-09-21T03:43:23.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:23 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2585214414' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T03:43:23.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:23 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2585214414' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T03:43:23.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:23 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2585214414' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T03:43:24.009 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:43:24.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:24 smithi134 ceph-mon[35031]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:43:24.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:24 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2585214414' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T03:43:24.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:24 smithi134 ceph-mon[35031]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:43:24.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:24 smithi134 ceph-mon[35031]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:24.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:24 smithi123 ceph-mon[34673]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:43:24.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:24 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2585214414' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T03:43:24.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:24 smithi123 ceph-mon[34673]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:43:24.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:24 smithi123 ceph-mon[34673]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:24.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:24 smithi016 ceph-mon[30375]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:43:24.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:24 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2585214414' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T03:43:24.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:24 smithi016 ceph-mon[30375]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:43:24.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:24 smithi016 ceph-mon[30375]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:25.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.031+0000 7f59afe00640 1 -- 172.21.15.16:0/37750341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59b010b4f0 msgr2=0x7f59b010b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.031+0000 7f59afe00640 1 --2- 172.21.15.16:0/37750341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59b010b4f0 0x7f59b010b900 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5998009900 tx=0x7f599802a410 comp rx=0 tx=0).stop 2024-09-21T03:43:25.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.031+0000 7f59afe00640 1 -- 172.21.15.16:0/37750341 shutdown_connections 2024-09-21T03:43:25.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.032+0000 7f59afe00640 1 --2- 172.21.15.16:0/37750341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59b010be40 0x7f59b0112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.032+0000 7f59afe00640 1 --2- 172.21.15.16:0/37750341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59b010b4f0 0x7f59b010b900 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.032+0000 7f59afe00640 1 --2- 172.21.15.16:0/37750341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59b0105bb0 0x7f59b0105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.032+0000 7f59afe00640 1 -- 172.21.15.16:0/37750341 >> 172.21.15.16:0/37750341 conn(0x7f59b0101680 msgr2=0x7f59b0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:25.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.032+0000 7f59afe00640 1 -- 172.21.15.16:0/37750341 shutdown_connections 2024-09-21T03:43:25.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.032+0000 7f59afe00640 1 -- 172.21.15.16:0/37750341 wait complete. 2024-09-21T03:43:25.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.033+0000 7f59afe00640 1 Processor -- start 2024-09-21T03:43:25.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.033+0000 7f59afe00640 1 -- start start 2024-09-21T03:43:25.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.033+0000 7f59afe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59b0105bb0 0x7f59b019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.033+0000 7f59afe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59b010b4f0 0x7f59b019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.033+0000 7f59afe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59b010be40 0x7f59b01a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.033+0000 7f59afe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f59b0115b00 con 0x7f59b010b4f0 2024-09-21T03:43:25.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.033+0000 7f59afe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f59b0115980 con 0x7f59b010be40 2024-09-21T03:43:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.033+0000 7f59afe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59b0115c80 con 0x7f59b0105bb0 2024-09-21T03:43:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.033+0000 7f59a6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59b010b4f0 0x7f59b019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.034+0000 7f59af400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59b010be40 0x7f59b01a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.034+0000 7f59aea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59b0105bb0 0x7f59b019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.034+0000 7f59af400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59b010be40 0x7f59b01a1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:47894/0 (socket says 172.21.15.16:47894) 2024-09-21T03:43:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.034+0000 7f59aea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59b0105bb0 0x7f59b019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:42034/0 (socket says 172.21.15.16:42034) 2024-09-21T03:43:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.034+0000 7f59af400640 1 -- 172.21.15.16:0/2186619715 learned_addr learned my addr 172.21.15.16:0/2186619715 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.034+0000 7f59af400640 1 -- 172.21.15.16:0/2186619715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59b0105bb0 msgr2=0x7f59b019c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.034+0000 7f59af400640 1 --2- 172.21.15.16:0/2186619715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59b0105bb0 0x7f59b019c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.034+0000 7f59af400640 1 -- 172.21.15.16:0/2186619715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59b010b4f0 msgr2=0x7f59b019cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.034+0000 7f59af400640 1 --2- 172.21.15.16:0/2186619715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59b010b4f0 0x7f59b019cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.034+0000 7f59af400640 1 -- 172.21.15.16:0/2186619715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b01a2480 con 0x7f59b010be40 2024-09-21T03:43:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.034+0000 7f59aea00640 1 --2- 172.21.15.16:0/2186619715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59b0105bb0 0x7f59b019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:25.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.034+0000 7f59a6000640 1 --2- 172.21.15.16:0/2186619715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59b010b4f0 0x7f59b019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:25.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.034+0000 7f59af400640 1 --2- 172.21.15.16:0/2186619715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59b010be40 0x7f59b01a1d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f59a800eb10 tx=0x7f59a800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:25.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.035+0000 7f59a7e00640 1 -- 172.21.15.16:0/2186619715 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8004b20 con 0x7f59b010be40 2024-09-21T03:43:25.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.035+0000 7f59a7e00640 1 -- 172.21.15.16:0/2186619715 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59a8004cc0 con 0x7f59b010be40 2024-09-21T03:43:25.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.035+0000 7f59afe00640 1 -- 172.21.15.16:0/2186619715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b01a2770 con 0x7f59b010be40 2024-09-21T03:43:25.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.035+0000 7f59a7e00640 1 -- 172.21.15.16:0/2186619715 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8010670 con 0x7f59b010be40 2024-09-21T03:43:25.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.035+0000 7f59afe00640 1 -- 172.21.15.16:0/2186619715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b01a8ed0 con 0x7f59b010be40 2024-09-21T03:43:25.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.036+0000 7f59afe00640 1 -- 172.21.15.16:0/2186619715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b0105fc0 con 0x7f59b010be40 2024-09-21T03:43:25.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.038+0000 7f59a7e00640 1 -- 172.21.15.16:0/2186619715 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f59a8005000 con 0x7f59b010be40 2024-09-21T03:43:25.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.040+0000 7f59a7e00640 1 --2- 172.21.15.16:0/2186619715 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f59940b6510 0x7f59940b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.040+0000 7f59a7e00640 1 -- 172.21.15.16:0/2186619715 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8802+0+0 (secure 0 0 0) 0x7f59a80e7ce0 con 0x7f59b010be40 2024-09-21T03:43:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.040+0000 7f59aea00640 1 --2- 172.21.15.16:0/2186619715 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f59940b6510 0x7f59940b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.041+0000 7f59aea00640 1 --2- 172.21.15.16:0/2186619715 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f59940b6510 0x7f59940b8a20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009fc0 tx=0x7f59a0005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:25.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.043+0000 7f59a7e00640 1 -- 172.21.15.16:0/2186619715 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f59a8056710 con 0x7f59b010be40 2024-09-21T03:43:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.280+0000 7f59afe00640 1 -- 172.21.15.16:0/2186619715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59b019d640 con 0x7f59b010be40 2024-09-21T03:43:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.281+0000 7f59a7e00640 1 -- 172.21.15.16:0/2186619715 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11836 (secure 0 0 0) 0x7f59b019d640 con 0x7f59b010be40 2024-09-21T03:43:25.284 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:43:25.284 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"btime":"2024-09-21T03:43:23:082019+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14700,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24658,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24668,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24682,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24689,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:43:22.247004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24731},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24731,"qdb_cluster":[24731]},"id":1}]} 2024-09-21T03:43:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-09-21T03:43:25.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.283+0000 7f59afe00640 1 -- 172.21.15.16:0/2186619715 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f59940b6510 msgr2=0x7f59940b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.283+0000 7f59afe00640 1 --2- 172.21.15.16:0/2186619715 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f59940b6510 0x7f59940b8a20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009fc0 tx=0x7f59a0005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.283+0000 7f59afe00640 1 -- 172.21.15.16:0/2186619715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59b010be40 msgr2=0x7f59b01a1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.283+0000 7f59afe00640 1 --2- 172.21.15.16:0/2186619715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59b010be40 0x7f59b01a1d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f59a800eb10 tx=0x7f59a800c2d0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.284+0000 7f59afe00640 1 -- 172.21.15.16:0/2186619715 shutdown_connections 2024-09-21T03:43:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.284+0000 7f59afe00640 1 --2- 172.21.15.16:0/2186619715 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f59940b6510 0x7f59940b8a20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.284+0000 7f59afe00640 1 --2- 172.21.15.16:0/2186619715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59b010be40 0x7f59b01a1d50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.284+0000 7f59afe00640 1 --2- 172.21.15.16:0/2186619715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59b010b4f0 0x7f59b019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.284+0000 7f59afe00640 1 --2- 172.21.15.16:0/2186619715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59b0105bb0 0x7f59b019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.284+0000 7f59afe00640 1 -- 172.21.15.16:0/2186619715 >> 172.21.15.16:0/2186619715 conn(0x7f59b0101680 msgr2=0x7f59b0103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.284+0000 7f59afe00640 1 -- 172.21.15.16:0/2186619715 shutdown_connections 2024-09-21T03:43:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:25.284+0000 7f59afe00640 1 -- 172.21.15.16:0/2186619715 wait complete. 2024-09-21T03:43:25.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:25 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2186619715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:25.606 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:25 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2186619715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:25.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:25 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2186619715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:26.362 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:43:26.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:26 smithi016 ceph-mon[30375]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:26.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:26 smithi123 ceph-mon[34673]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:26.743 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:26.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:26 smithi134 ceph-mon[35031]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:27.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.430+0000 7f1238a00640 1 -- 172.21.15.16:0/2210745588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1234109c40 msgr2=0x7f1234110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.430+0000 7f1238a00640 1 --2- 172.21.15.16:0/2210745588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1234109c40 0x7f1234110470 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f122c009ca0 tx=0x7f122c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.431+0000 7f1238a00640 1 -- 172.21.15.16:0/2210745588 shutdown_connections 2024-09-21T03:43:27.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.431+0000 7f1238a00640 1 --2- 172.21.15.16:0/2210745588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1234109c40 0x7f1234110470 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.431+0000 7f1238a00640 1 --2- 172.21.15.16:0/2210745588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12341092f0 0x7f1234109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.431+0000 7f1238a00640 1 --2- 172.21.15.16:0/2210745588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12341039b0 0x7f1234103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.431+0000 7f1238a00640 1 -- 172.21.15.16:0/2210745588 >> 172.21.15.16:0/2210745588 conn(0x7f12340ff440 msgr2=0x7f1234101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:27.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.431+0000 7f1238a00640 1 -- 172.21.15.16:0/2210745588 shutdown_connections 2024-09-21T03:43:27.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.431+0000 7f1238a00640 1 -- 172.21.15.16:0/2210745588 wait complete. 2024-09-21T03:43:27.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.432+0000 7f1238a00640 1 Processor -- start 2024-09-21T03:43:27.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.432+0000 7f1238a00640 1 -- start start 2024-09-21T03:43:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.433+0000 7f1238a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12341039b0 0x7f123419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.433+0000 7f1238a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12341092f0 0x7f123419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.433+0000 7f1238a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1234109c40 0x7f123419fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.433+0000 7f1238a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1234113900 con 0x7f12341092f0 2024-09-21T03:43:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.433+0000 7f1238a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1234113780 con 0x7f1234109c40 2024-09-21T03:43:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.433+0000 7f1238a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1234113a80 con 0x7f12341039b0 2024-09-21T03:43:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.433+0000 7f1231e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12341092f0 0x7f123419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.433+0000 7f1232800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12341039b0 0x7f123419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.433+0000 7f1231e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12341092f0 0x7f123419a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41802/0 (socket says 172.21.15.16:41802) 2024-09-21T03:43:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.433+0000 7f1231e00640 1 -- 172.21.15.16:0/97006596 learned_addr learned my addr 172.21.15.16:0/97006596 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.433+0000 7f1233200640 1 --2- 172.21.15.16:0/97006596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1234109c40 0x7f123419fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.434+0000 7f1232800640 1 -- 172.21.15.16:0/97006596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1234109c40 msgr2=0x7f123419fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.434+0000 7f1232800640 1 --2- 172.21.15.16:0/97006596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1234109c40 0x7f123419fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.434+0000 7f1232800640 1 -- 172.21.15.16:0/97006596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12341092f0 msgr2=0x7f123419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.434+0000 7f1232800640 1 --2- 172.21.15.16:0/97006596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12341092f0 0x7f123419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.434+0000 7f1232800640 1 -- 172.21.15.16:0/97006596 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12341a0200 con 0x7f12341039b0 2024-09-21T03:43:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.434+0000 7f1232800640 1 --2- 172.21.15.16:0/97006596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12341039b0 0x7f123419a290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f122400e9f0 tx=0x7f122400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.434+0000 7f122b400640 1 -- 172.21.15.16:0/97006596 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122400ccc0 con 0x7f12341039b0 2024-09-21T03:43:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.434+0000 7f122b400640 1 -- 172.21.15.16:0/97006596 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12240108e0 con 0x7f12341039b0 2024-09-21T03:43:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.434+0000 7f1238a00640 1 -- 172.21.15.16:0/97006596 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12341a04f0 con 0x7f12341039b0 2024-09-21T03:43:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.435+0000 7f122b400640 1 -- 172.21.15.16:0/97006596 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224018880 con 0x7f12341039b0 2024-09-21T03:43:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.435+0000 7f1238a00640 1 -- 172.21.15.16:0/97006596 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12341a6c50 con 0x7f12341039b0 2024-09-21T03:43:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.436+0000 7f1238a00640 1 -- 172.21.15.16:0/97006596 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1234103dc0 con 0x7f12341039b0 2024-09-21T03:43:27.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.438+0000 7f122b400640 1 -- 172.21.15.16:0/97006596 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f1224010430 con 0x7f12341039b0 2024-09-21T03:43:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.439+0000 7f122b400640 1 --2- 172.21.15.16:0/97006596 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f12180b6510 0x7f12180b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.440+0000 7f122b400640 1 -- 172.21.15.16:0/97006596 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8802+0+0 (secure 0 0 0) 0x7f12240e7b10 con 0x7f12341039b0 2024-09-21T03:43:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.440+0000 7f1231e00640 1 --2- 172.21.15.16:0/97006596 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f12180b6510 0x7f12180b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.441+0000 7f1231e00640 1 --2- 172.21.15.16:0/97006596 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f12180b6510 0x7f12180b8a20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1234107dd0 tx=0x7f1220005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.441+0000 7f122b400640 1 -- 172.21.15.16:0/97006596 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f12240b24e0 con 0x7f12341039b0 2024-09-21T03:43:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.641+0000 7f1238a00640 1 -- 172.21.15.16:0/97006596 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f123402e640 con 0x7f12341039b0 2024-09-21T03:43:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.643+0000 7f122b400640 1 -- 172.21.15.16:0/97006596 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+24232 (secure 0 0 0) 0x7f12240a9020 con 0x7f12341039b0 2024-09-21T03:43:27.647 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:43:27.647 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:33:53.220358+0000","modified":"2024-09-21T03:43:22.246847+0000","last_up_change":"2024-09-21T03:42:06.614247+0000","last_in_change":"2024-09-21T03:41:43.379404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:37:55.708915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:43:03.567522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:43:06.048083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:43:16.061356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6a7e4b9e-078f-4ae9-ab9e-37be995e4deb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6803","nonce":311974551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6805","nonce":311974551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6809","nonce":311974551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6807","nonce":311974551}]},"public_addr":"172.21.15.16:6803/311974551","cluster_addr":"172.21.15.16:6805/311974551","heartbeat_back_addr":"172.21.15.16:6809/311974551","heartbeat_front_addr":"172.21.15.16:6807/311974551","state":["exists","up"]},{"osd":1,"uuid":"3a497a64-e1a8-43ef-8509-0a79515da0aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6801","nonce":2152486571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6803","nonce":2152486571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6807","nonce":2152486571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6805","nonce":2152486571}]},"public_addr":"172.21.15.123:6801/2152486571","cluster_addr":"172.21.15.123:6803/2152486571","heartbeat_back_addr":"172.21.15.123:6807/2152486571","heartbeat_front_addr":"172.21.15.123:6805/2152486571","state":["exists","up"]},{"osd":2,"uuid":"1c39518e-6cce-4a0f-9ee8-55c60ad325f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6801","nonce":2812351279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6803","nonce":2812351279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6807","nonce":2812351279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6805","nonce":2812351279}]},"public_addr":"172.21.15.134:6801/2812351279","cluster_addr":"172.21.15.134:6803/2812351279","heartbeat_back_addr":"172.21.15.134:6807/2812351279","heartbeat_front_addr":"172.21.15.134:6805/2812351279","state":["exists","up"]},{"osd":3,"uuid":"0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6811","nonce":2366216026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6813","nonce":2366216026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6817","nonce":2366216026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6815","nonce":2366216026}]},"public_addr":"172.21.15.16:6811/2366216026","cluster_addr":"172.21.15.16:6813/2366216026","heartbeat_back_addr":"172.21.15.16:6817/2366216026","heartbeat_front_addr":"172.21.15.16:6815/2366216026","state":["exists","up"]},{"osd":4,"uuid":"ad5e6d0e-31e8-4779-9e74-f215032777ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6809","nonce":2354279116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6811","nonce":2354279116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6815","nonce":2354279116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6813","nonce":2354279116}]},"public_addr":"172.21.15.123:6809/2354279116","cluster_addr":"172.21.15.123:6811/2354279116","heartbeat_back_addr":"172.21.15.123:6815/2354279116","heartbeat_front_addr":"172.21.15.123:6813/2354279116","state":["exists","up"]},{"osd":5,"uuid":"ca09bf88-1772-4dc0-8711-cbf9a5b31977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6809","nonce":2678416182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6811","nonce":2678416182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6815","nonce":2678416182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6813","nonce":2678416182}]},"public_addr":"172.21.15.134:6809/2678416182","cluster_addr":"172.21.15.134:6811/2678416182","heartbeat_back_addr":"172.21.15.134:6815/2678416182","heartbeat_front_addr":"172.21.15.134:6813/2678416182","state":["exists","up"]},{"osd":6,"uuid":"0d41ed4e-8699-4d88-9254-12d3b84941e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6819","nonce":3377807744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6821","nonce":3377807744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6825","nonce":3377807744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6823","nonce":3377807744}]},"public_addr":"172.21.15.16:6819/3377807744","cluster_addr":"172.21.15.16:6821/3377807744","heartbeat_back_addr":"172.21.15.16:6825/3377807744","heartbeat_front_addr":"172.21.15.16:6823/3377807744","state":["exists","up"]},{"osd":7,"uuid":"1541a272-c524-4f12-9b94-763f69bb1c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6817","nonce":1463412739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6819","nonce":1463412739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6823","nonce":1463412739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6821","nonce":1463412739}]},"public_addr":"172.21.15.123:6817/1463412739","cluster_addr":"172.21.15.123:6819/1463412739","heartbeat_back_addr":"172.21.15.123:6823/1463412739","heartbeat_front_addr":"172.21.15.123:6821/1463412739","state":["exists","up"]},{"osd":8,"uuid":"cc00014c-eb99-49ab-984c-0bc26ef85130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6817","nonce":2644021450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6819","nonce":2644021450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6823","nonce":2644021450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6821","nonce":2644021450}]},"public_addr":"172.21.15.134:6817/2644021450","cluster_addr":"172.21.15.134:6819/2644021450","heartbeat_back_addr":"172.21.15.134:6823/2644021450","heartbeat_front_addr":"172.21.15.134:6821/2644021450","state":["exists","up"]},{"osd":9,"uuid":"3cbfff74-31ff-4ffd-844b-54ec8563ba32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6827","nonce":1653164371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6829","nonce":1653164371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6833","nonce":1653164371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6831","nonce":1653164371}]},"public_addr":"172.21.15.16:6827/1653164371","cluster_addr":"172.21.15.16:6829/1653164371","heartbeat_back_addr":"172.21.15.16:6833/1653164371","heartbeat_front_addr":"172.21.15.16:6831/1653164371","state":["exists","up"]},{"osd":10,"uuid":"0a1caaad-70cd-4c2c-88d3-e9a5cc885655","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6825","nonce":3682234496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6827","nonce":3682234496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6831","nonce":3682234496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6829","nonce":3682234496}]},"public_addr":"172.21.15.123:6825/3682234496","cluster_addr":"172.21.15.123:6827/3682234496","heartbeat_back_addr":"172.21.15.123:6831/3682234496","heartbeat_front_addr":"172.21.15.123:6829/3682234496","state":["exists","up"]},{"osd":11,"uuid":"3aff3258-5a38-41ff-aac3-4a082feffdae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6825","nonce":584758685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6827","nonce":584758685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6831","nonce":584758685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6829","nonce":584758685}]},"public_addr":"172.21.15.134:6825/584758685","cluster_addr":"172.21.15.134:6827/584758685","heartbeat_back_addr":"172.21.15.134:6831/584758685","heartbeat_front_addr":"172.21.15.134:6829/584758685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:02.177731+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:28.847562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:54.272384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:24.338596+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:52.430371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:17.832737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:50.397890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:16.842245+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:42.102933+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:13.074732+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:40.057905+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:04.898190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/557742783":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/3238676548":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6801/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/2357336233":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6801/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6800/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/3806844703":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/2351581621":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6800/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/355731724":"2024-09-22T03:34:22.549156+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:43:27.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.646+0000 7f1238a00640 1 -- 172.21.15.16:0/97006596 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f12180b6510 msgr2=0x7f12180b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.646+0000 7f1238a00640 1 --2- 172.21.15.16:0/97006596 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f12180b6510 0x7f12180b8a20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1234107dd0 tx=0x7f1220005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:27.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.646+0000 7f1238a00640 1 -- 172.21.15.16:0/97006596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12341039b0 msgr2=0x7f123419a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:27.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.646+0000 7f1238a00640 1 --2- 172.21.15.16:0/97006596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12341039b0 0x7f123419a290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f122400e9f0 tx=0x7f122400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.646+0000 7f1238a00640 1 -- 172.21.15.16:0/97006596 shutdown_connections 2024-09-21T03:43:27.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.646+0000 7f1238a00640 1 --2- 172.21.15.16:0/97006596 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f12180b6510 0x7f12180b8a20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.646+0000 7f1238a00640 1 --2- 172.21.15.16:0/97006596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1234109c40 0x7f123419fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.646+0000 7f1238a00640 1 --2- 172.21.15.16:0/97006596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12341092f0 0x7f123419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.646+0000 7f1238a00640 1 --2- 172.21.15.16:0/97006596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12341039b0 0x7f123419a290 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:27.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.646+0000 7f1238a00640 1 -- 172.21.15.16:0/97006596 >> 172.21.15.16:0/97006596 conn(0x7f12340ff440 msgr2=0x7f1234101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:27.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.647+0000 7f1238a00640 1 -- 172.21.15.16:0/97006596 shutdown_connections 2024-09-21T03:43:27.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:27.647+0000 7f1238a00640 1 -- 172.21.15.16:0/97006596 wait complete. 2024-09-21T03:43:28.572 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:43:28.652 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:28 smithi016 ceph-mon[30375]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:28.652 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:28 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/97006596' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:28.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:28 smithi123 ceph-mon[34673]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:28.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:28 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/97006596' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:28.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:28 smithi134 ceph-mon[35031]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:28.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:28 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/97006596' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:28.957 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:29.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.638+0000 7fd1bd600640 1 -- 172.21.15.16:0/1200728305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b81092d0 msgr2=0x7fd1b81096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.638+0000 7fd1bd600640 1 --2- 172.21.15.16:0/1200728305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b81092d0 0x7fd1b81096e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009900 tx=0x7fd1a402a410 comp rx=0 tx=0).stop 2024-09-21T03:43:29.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.639+0000 7fd1bd600640 1 -- 172.21.15.16:0/1200728305 shutdown_connections 2024-09-21T03:43:29.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.639+0000 7fd1bd600640 1 --2- 172.21.15.16:0/1200728305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b8109c20 0x7fd1b8110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.639+0000 7fd1bd600640 1 --2- 172.21.15.16:0/1200728305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b81092d0 0x7fd1b81096e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.639+0000 7fd1bd600640 1 --2- 172.21.15.16:0/1200728305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1b8103990 0x7fd1b8103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.639+0000 7fd1bd600640 1 -- 172.21.15.16:0/1200728305 >> 172.21.15.16:0/1200728305 conn(0x7fd1b80ff3e0 msgr2=0x7fd1b8101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:29.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.639+0000 7fd1bd600640 1 -- 172.21.15.16:0/1200728305 shutdown_connections 2024-09-21T03:43:29.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.639+0000 7fd1bd600640 1 -- 172.21.15.16:0/1200728305 wait complete. 2024-09-21T03:43:29.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.640+0000 7fd1bd600640 1 Processor -- start 2024-09-21T03:43:29.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.640+0000 7fd1bd600640 1 -- start start 2024-09-21T03:43:29.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.640+0000 7fd1bd600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b8103990 0x7fd1b819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.640+0000 7fd1bd600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b81092d0 0x7fd1b819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.640+0000 7fd1bd600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1b8109c20 0x7fd1b81a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.640+0000 7fd1bd600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b81138e0 con 0x7fd1b8103990 2024-09-21T03:43:29.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.640+0000 7fd1bd600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b8113760 con 0x7fd1b8109c20 2024-09-21T03:43:29.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.640+0000 7fd1bd600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b8113a60 con 0x7fd1b81092d0 2024-09-21T03:43:29.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.640+0000 7fd1b7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b8103990 0x7fd1b819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.641+0000 7fd1b7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b81092d0 0x7fd1b819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.641+0000 7fd1b7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b8103990 0x7fd1b819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41818/0 (socket says 172.21.15.16:41818) 2024-09-21T03:43:29.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.641+0000 7fd1b7e00640 1 -- 172.21.15.16:0/3224243038 learned_addr learned my addr 172.21.15.16:0/3224243038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:29.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.641+0000 7fd1bcc00640 1 --2- 172.21.15.16:0/3224243038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1b8109c20 0x7fd1b81a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.641+0000 7fd1b7400640 1 -- 172.21.15.16:0/3224243038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1b8109c20 msgr2=0x7fd1b81a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.641+0000 7fd1b7400640 1 --2- 172.21.15.16:0/3224243038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1b8109c20 0x7fd1b81a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.641+0000 7fd1b7400640 1 -- 172.21.15.16:0/3224243038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b8103990 msgr2=0x7fd1b819c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.641+0000 7fd1b7400640 1 --2- 172.21.15.16:0/3224243038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b8103990 0x7fd1b819c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.641+0000 7fd1b7400640 1 -- 172.21.15.16:0/3224243038 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b81a24a0 con 0x7fd1b81092d0 2024-09-21T03:43:29.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.641+0000 7fd1b7400640 1 --2- 172.21.15.16:0/3224243038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b81092d0 0x7fd1b819cad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd1a402a930 tx=0x7fd1a4031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.642+0000 7fd1b4c00640 1 -- 172.21.15.16:0/3224243038 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a403f070 con 0x7fd1b81092d0 2024-09-21T03:43:29.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.642+0000 7fd1bd600640 1 -- 172.21.15.16:0/3224243038 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b81a2730 con 0x7fd1b81092d0 2024-09-21T03:43:29.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.642+0000 7fd1bd600640 1 -- 172.21.15.16:0/3224243038 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b81a8ef0 con 0x7fd1b81092d0 2024-09-21T03:43:29.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.642+0000 7fd1b4c00640 1 -- 172.21.15.16:0/3224243038 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1a4004490 con 0x7fd1b81092d0 2024-09-21T03:43:29.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.642+0000 7fd1b4c00640 1 -- 172.21.15.16:0/3224243038 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4048060 con 0x7fd1b81092d0 2024-09-21T03:43:29.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.643+0000 7fd1bd600640 1 -- 172.21.15.16:0/3224243038 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1980052c0 con 0x7fd1b81092d0 2024-09-21T03:43:29.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.646+0000 7fd1b4c00640 1 -- 172.21.15.16:0/3224243038 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fd1a403b460 con 0x7fd1b81092d0 2024-09-21T03:43:29.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.648+0000 7fd1b4c00640 1 --2- 172.21.15.16:0/3224243038 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd19c0b6510 0x7fd19c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:29.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.648+0000 7fd1b4c00640 1 -- 172.21.15.16:0/3224243038 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8802+0+0 (secure 0 0 0) 0x7fd1a41076c0 con 0x7fd1b81092d0 2024-09-21T03:43:29.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.648+0000 7fd1b7e00640 1 --2- 172.21.15.16:0/3224243038 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd19c0b6510 0x7fd19c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:29.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.649+0000 7fd1b7e00640 1 --2- 172.21.15.16:0/3224243038 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd19c0b6510 0x7fd19c0b8a20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80095f0 tx=0x7fd1a8005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:29.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.649+0000 7fd1b4c00640 1 -- 172.21.15.16:0/3224243038 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd1a40d2090 con 0x7fd1b81092d0 2024-09-21T03:43:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.849+0000 7fd1bd600640 1 -- 172.21.15.16:0/3224243038 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd198005880 con 0x7fd1b81092d0 2024-09-21T03:43:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.850+0000 7fd1b4c00640 1 -- 172.21.15.16:0/3224243038 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+24232 (secure 0 0 0) 0x7fd1a40d6fd0 con 0x7fd1b81092d0 2024-09-21T03:43:29.854 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:43:29.854 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:33:53.220358+0000","modified":"2024-09-21T03:43:22.246847+0000","last_up_change":"2024-09-21T03:42:06.614247+0000","last_in_change":"2024-09-21T03:41:43.379404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:37:55.708915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:43:03.567522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:43:06.048083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:43:16.061356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6a7e4b9e-078f-4ae9-ab9e-37be995e4deb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6803","nonce":311974551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6805","nonce":311974551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6809","nonce":311974551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6807","nonce":311974551}]},"public_addr":"172.21.15.16:6803/311974551","cluster_addr":"172.21.15.16:6805/311974551","heartbeat_back_addr":"172.21.15.16:6809/311974551","heartbeat_front_addr":"172.21.15.16:6807/311974551","state":["exists","up"]},{"osd":1,"uuid":"3a497a64-e1a8-43ef-8509-0a79515da0aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6801","nonce":2152486571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6803","nonce":2152486571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6807","nonce":2152486571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6805","nonce":2152486571}]},"public_addr":"172.21.15.123:6801/2152486571","cluster_addr":"172.21.15.123:6803/2152486571","heartbeat_back_addr":"172.21.15.123:6807/2152486571","heartbeat_front_addr":"172.21.15.123:6805/2152486571","state":["exists","up"]},{"osd":2,"uuid":"1c39518e-6cce-4a0f-9ee8-55c60ad325f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6801","nonce":2812351279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6803","nonce":2812351279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6807","nonce":2812351279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6805","nonce":2812351279}]},"public_addr":"172.21.15.134:6801/2812351279","cluster_addr":"172.21.15.134:6803/2812351279","heartbeat_back_addr":"172.21.15.134:6807/2812351279","heartbeat_front_addr":"172.21.15.134:6805/2812351279","state":["exists","up"]},{"osd":3,"uuid":"0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6811","nonce":2366216026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6813","nonce":2366216026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6817","nonce":2366216026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6815","nonce":2366216026}]},"public_addr":"172.21.15.16:6811/2366216026","cluster_addr":"172.21.15.16:6813/2366216026","heartbeat_back_addr":"172.21.15.16:6817/2366216026","heartbeat_front_addr":"172.21.15.16:6815/2366216026","state":["exists","up"]},{"osd":4,"uuid":"ad5e6d0e-31e8-4779-9e74-f215032777ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6809","nonce":2354279116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6811","nonce":2354279116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6815","nonce":2354279116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6813","nonce":2354279116}]},"public_addr":"172.21.15.123:6809/2354279116","cluster_addr":"172.21.15.123:6811/2354279116","heartbeat_back_addr":"172.21.15.123:6815/2354279116","heartbeat_front_addr":"172.21.15.123:6813/2354279116","state":["exists","up"]},{"osd":5,"uuid":"ca09bf88-1772-4dc0-8711-cbf9a5b31977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6809","nonce":2678416182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6811","nonce":2678416182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6815","nonce":2678416182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6813","nonce":2678416182}]},"public_addr":"172.21.15.134:6809/2678416182","cluster_addr":"172.21.15.134:6811/2678416182","heartbeat_back_addr":"172.21.15.134:6815/2678416182","heartbeat_front_addr":"172.21.15.134:6813/2678416182","state":["exists","up"]},{"osd":6,"uuid":"0d41ed4e-8699-4d88-9254-12d3b84941e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6819","nonce":3377807744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6821","nonce":3377807744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6825","nonce":3377807744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6823","nonce":3377807744}]},"public_addr":"172.21.15.16:6819/3377807744","cluster_addr":"172.21.15.16:6821/3377807744","heartbeat_back_addr":"172.21.15.16:6825/3377807744","heartbeat_front_addr":"172.21.15.16:6823/3377807744","state":["exists","up"]},{"osd":7,"uuid":"1541a272-c524-4f12-9b94-763f69bb1c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6817","nonce":1463412739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6819","nonce":1463412739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6823","nonce":1463412739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6821","nonce":1463412739}]},"public_addr":"172.21.15.123:6817/1463412739","cluster_addr":"172.21.15.123:6819/1463412739","heartbeat_back_addr":"172.21.15.123:6823/1463412739","heartbeat_front_addr":"172.21.15.123:6821/1463412739","state":["exists","up"]},{"osd":8,"uuid":"cc00014c-eb99-49ab-984c-0bc26ef85130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6817","nonce":2644021450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6819","nonce":2644021450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6823","nonce":2644021450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6821","nonce":2644021450}]},"public_addr":"172.21.15.134:6817/2644021450","cluster_addr":"172.21.15.134:6819/2644021450","heartbeat_back_addr":"172.21.15.134:6823/2644021450","heartbeat_front_addr":"172.21.15.134:6821/2644021450","state":["exists","up"]},{"osd":9,"uuid":"3cbfff74-31ff-4ffd-844b-54ec8563ba32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6827","nonce":1653164371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6829","nonce":1653164371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6833","nonce":1653164371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6831","nonce":1653164371}]},"public_addr":"172.21.15.16:6827/1653164371","cluster_addr":"172.21.15.16:6829/1653164371","heartbeat_back_addr":"172.21.15.16:6833/1653164371","heartbeat_front_addr":"172.21.15.16:6831/1653164371","state":["exists","up"]},{"osd":10,"uuid":"0a1caaad-70cd-4c2c-88d3-e9a5cc885655","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6825","nonce":3682234496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6827","nonce":3682234496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6831","nonce":3682234496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6829","nonce":3682234496}]},"public_addr":"172.21.15.123:6825/3682234496","cluster_addr":"172.21.15.123:6827/3682234496","heartbeat_back_addr":"172.21.15.123:6831/3682234496","heartbeat_front_addr":"172.21.15.123:6829/3682234496","state":["exists","up"]},{"osd":11,"uuid":"3aff3258-5a38-41ff-aac3-4a082feffdae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6825","nonce":584758685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6827","nonce":584758685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6831","nonce":584758685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6829","nonce":584758685}]},"public_addr":"172.21.15.134:6825/584758685","cluster_addr":"172.21.15.134:6827/584758685","heartbeat_back_addr":"172.21.15.134:6831/584758685","heartbeat_front_addr":"172.21.15.134:6829/584758685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:02.177731+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:28.847562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:54.272384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:24.338596+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:52.430371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:17.832737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:50.397890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:16.842245+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:42.102933+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:13.074732+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:40.057905+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:04.898190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/557742783":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/3238676548":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6801/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/2357336233":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6801/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6800/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/3806844703":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/2351581621":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6800/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/355731724":"2024-09-22T03:34:22.549156+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:43:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.853+0000 7fd1bd600640 1 -- 172.21.15.16:0/3224243038 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd19c0b6510 msgr2=0x7fd19c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.853+0000 7fd1bd600640 1 --2- 172.21.15.16:0/3224243038 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd19c0b6510 0x7fd19c0b8a20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80095f0 tx=0x7fd1a8005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.853+0000 7fd1bd600640 1 -- 172.21.15.16:0/3224243038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b81092d0 msgr2=0x7fd1b819cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.853+0000 7fd1bd600640 1 --2- 172.21.15.16:0/3224243038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b81092d0 0x7fd1b819cad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd1a402a930 tx=0x7fd1a4031a20 comp rx=0 tx=0).stop 2024-09-21T03:43:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.853+0000 7fd1bd600640 1 -- 172.21.15.16:0/3224243038 shutdown_connections 2024-09-21T03:43:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.853+0000 7fd1bd600640 1 --2- 172.21.15.16:0/3224243038 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd19c0b6510 0x7fd19c0b8a20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.853+0000 7fd1bd600640 1 --2- 172.21.15.16:0/3224243038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1b8109c20 0x7fd1b81a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.853+0000 7fd1bd600640 1 --2- 172.21.15.16:0/3224243038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b81092d0 0x7fd1b819cad0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.853+0000 7fd1bd600640 1 --2- 172.21.15.16:0/3224243038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b8103990 0x7fd1b819c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.853+0000 7fd1bd600640 1 -- 172.21.15.16:0/3224243038 >> 172.21.15.16:0/3224243038 conn(0x7fd1b80ff3e0 msgr2=0x7fd1b8108a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.853+0000 7fd1bd600640 1 -- 172.21.15.16:0/3224243038 shutdown_connections 2024-09-21T03:43:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:29.853+0000 7fd1bd600640 1 -- 172.21.15.16:0/3224243038 wait complete. 2024-09-21T03:43:30.566 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:30 smithi016 ceph-mon[30375]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:30.566 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:30 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3224243038' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:30.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:43:30.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:30 smithi123 ceph-mon[34673]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:30.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:30 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3224243038' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:30.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:30 smithi134 ceph-mon[35031]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:30.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:30 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3224243038' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:31.000 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.656+0000 7f92d3200640 1 -- 172.21.15.16:0/4075064811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92cc103b80 msgr2=0x7f92cc103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.656+0000 7f92d3200640 1 --2- 172.21.15.16:0/4075064811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92cc103b80 0x7f92cc103f90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009860 tx=0x7f92bc02a340 comp rx=0 tx=0).stop 2024-09-21T03:43:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.656+0000 7f92d3200640 1 -- 172.21.15.16:0/4075064811 shutdown_connections 2024-09-21T03:43:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.656+0000 7f92d3200640 1 --2- 172.21.15.16:0/4075064811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92cc109e10 0x7f92cc110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.656+0000 7f92d3200640 1 --2- 172.21.15.16:0/4075064811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92cc1094c0 0x7f92cc1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.656+0000 7f92d3200640 1 --2- 172.21.15.16:0/4075064811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92cc103b80 0x7f92cc103f90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.656+0000 7f92d3200640 1 -- 172.21.15.16:0/4075064811 >> 172.21.15.16:0/4075064811 conn(0x7f92cc0ff5d0 msgr2=0x7f92cc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.656+0000 7f92d3200640 1 -- 172.21.15.16:0/4075064811 shutdown_connections 2024-09-21T03:43:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.656+0000 7f92d3200640 1 -- 172.21.15.16:0/4075064811 wait complete. 2024-09-21T03:43:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.657+0000 7f92d3200640 1 Processor -- start 2024-09-21T03:43:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.657+0000 7f92d3200640 1 -- start start 2024-09-21T03:43:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.657+0000 7f92d3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92cc103b80 0x7f92cc19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.657+0000 7f92d3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92cc1094c0 0x7f92cc19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.657+0000 7f92d3200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92cc109e10 0x7f92cc1a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.657+0000 7f92d3200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc113ad0 con 0x7f92cc103b80 2024-09-21T03:43:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.657+0000 7f92d3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc113950 con 0x7f92cc1094c0 2024-09-21T03:43:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.657+0000 7f92d3200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc113c50 con 0x7f92cc109e10 2024-09-21T03:43:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.658+0000 7f92cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92cc103b80 0x7f92cc19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.658+0000 7f92cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92cc103b80 0x7f92cc19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41830/0 (socket says 172.21.15.16:41830) 2024-09-21T03:43:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.658+0000 7f92cbe00640 1 -- 172.21.15.16:0/840980428 learned_addr learned my addr 172.21.15.16:0/840980428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.658+0000 7f92d1200640 1 --2- 172.21.15.16:0/840980428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92cc109e10 0x7f92cc1a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.658+0000 7f92c3400640 1 --2- 172.21.15.16:0/840980428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92cc1094c0 0x7f92cc19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.658+0000 7f92cbe00640 1 -- 172.21.15.16:0/840980428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92cc109e10 msgr2=0x7f92cc1a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.658+0000 7f92cbe00640 1 --2- 172.21.15.16:0/840980428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92cc109e10 0x7f92cc1a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.658+0000 7f92cbe00640 1 -- 172.21.15.16:0/840980428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92cc1094c0 msgr2=0x7f92cc19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.658+0000 7f92cbe00640 1 --2- 172.21.15.16:0/840980428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92cc1094c0 0x7f92cc19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.658+0000 7f92cbe00640 1 -- 172.21.15.16:0/840980428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92cc1a2630 con 0x7f92cc103b80 2024-09-21T03:43:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.658+0000 7f92d1200640 1 --2- 172.21.15.16:0/840980428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92cc109e10 0x7f92cc1a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.659+0000 7f92cbe00640 1 --2- 172.21.15.16:0/840980428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92cc103b80 0x7f92cc19c6c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0303f0 tx=0x7f92bc032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.659+0000 7f92c9600640 1 -- 172.21.15.16:0/840980428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc040020 con 0x7f92cc103b80 2024-09-21T03:43:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.659+0000 7f92c9600640 1 -- 172.21.15.16:0/840980428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92bc030dd0 con 0x7f92cc103b80 2024-09-21T03:43:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.659+0000 7f92d3200640 1 -- 172.21.15.16:0/840980428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92cc1a28c0 con 0x7f92cc103b80 2024-09-21T03:43:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.659+0000 7f92c9600640 1 -- 172.21.15.16:0/840980428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc037040 con 0x7f92cc103b80 2024-09-21T03:43:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.659+0000 7f92d3200640 1 -- 172.21.15.16:0/840980428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92cc1a9080 con 0x7f92cc103b80 2024-09-21T03:43:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.660+0000 7f92d3200640 1 -- 172.21.15.16:0/840980428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92cc103f90 con 0x7f92cc103b80 2024-09-21T03:43:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.664+0000 7f92c9600640 1 -- 172.21.15.16:0/840980428 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f92bc043020 con 0x7f92cc103b80 2024-09-21T03:43:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.666+0000 7f92c9600640 1 --2- 172.21.15.16:0/840980428 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f92b00b6510 0x7f92b00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.666+0000 7f92c9600640 1 -- 172.21.15.16:0/840980428 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8802+0+0 (secure 0 0 0) 0x7f92bc107860 con 0x7f92cc103b80 2024-09-21T03:43:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.666+0000 7f92c3400640 1 --2- 172.21.15.16:0/840980428 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f92b00b6510 0x7f92b00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.667+0000 7f92c3400640 1 --2- 172.21.15.16:0/840980428 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f92b00b6510 0x7f92b00b8a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f92cc105bc0 tx=0x7f92b4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:31.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.668+0000 7f92c9600640 1 -- 172.21.15.16:0/840980428 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f92bc04a020 con 0x7f92cc103b80 2024-09-21T03:43:31.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.908+0000 7f92d3200640 1 -- 172.21.15.16:0/840980428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92cc1a92e0 con 0x7f92cc103b80 2024-09-21T03:43:31.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.908+0000 7f92c9600640 1 -- 172.21.15.16:0/840980428 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11836 (secure 0 0 0) 0x7f92bc0328c0 con 0x7f92cc103b80 2024-09-21T03:43:31.911 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:43:31.912 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"btime":"2024-09-21T03:43:23:082019+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14700,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24658,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24668,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24682,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24689,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:43:22.247004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24731},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24731,"qdb_cluster":[24731]},"id":1}]} 2024-09-21T03:43:31.912 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-09-21T03:43:31.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.911+0000 7f92d3200640 1 -- 172.21.15.16:0/840980428 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f92b00b6510 msgr2=0x7f92b00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:31.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.911+0000 7f92d3200640 1 --2- 172.21.15.16:0/840980428 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f92b00b6510 0x7f92b00b8a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f92cc105bc0 tx=0x7f92b4009260 comp rx=0 tx=0).stop 2024-09-21T03:43:31.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.911+0000 7f92d3200640 1 -- 172.21.15.16:0/840980428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92cc103b80 msgr2=0x7f92cc19c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:31.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.911+0000 7f92d3200640 1 --2- 172.21.15.16:0/840980428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92cc103b80 0x7f92cc19c6c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0303f0 tx=0x7f92bc032da0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.911+0000 7f92d3200640 1 -- 172.21.15.16:0/840980428 shutdown_connections 2024-09-21T03:43:31.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.911+0000 7f92d3200640 1 --2- 172.21.15.16:0/840980428 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f92b00b6510 0x7f92b00b8a20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.911+0000 7f92d3200640 1 --2- 172.21.15.16:0/840980428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92cc109e10 0x7f92cc1a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.911+0000 7f92d3200640 1 --2- 172.21.15.16:0/840980428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92cc1094c0 0x7f92cc19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.911+0000 7f92d3200640 1 --2- 172.21.15.16:0/840980428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92cc103b80 0x7f92cc19c6c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:31.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.911+0000 7f92d3200640 1 -- 172.21.15.16:0/840980428 >> 172.21.15.16:0/840980428 conn(0x7f92cc0ff5d0 msgr2=0x7f92cc108b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:31.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.911+0000 7f92d3200640 1 -- 172.21.15.16:0/840980428 shutdown_connections 2024-09-21T03:43:31.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:31.911+0000 7f92d3200640 1 -- 172.21.15.16:0/840980428 wait complete. 2024-09-21T03:43:32.634 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:32 smithi016 ceph-mon[30375]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:32.634 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:32 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/840980428' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:32.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:32 smithi123 ceph-mon[34673]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:32.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:32 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/840980428' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:32.672 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:43:32.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:32 smithi134 ceph-mon[35031]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:32.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:32 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/840980428' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:33.050 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.724+0000 7f1bcb000640 1 -- 172.21.15.16:0/2625908861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bc41094c0 msgr2=0x7f1bc41098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.724+0000 7f1bcb000640 1 --2- 172.21.15.16:0/2625908861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bc41094c0 0x7f1bc41098d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009900 tx=0x7f1bac02a410 comp rx=0 tx=0).stop 2024-09-21T03:43:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.724+0000 7f1bcb000640 1 -- 172.21.15.16:0/2625908861 shutdown_connections 2024-09-21T03:43:33.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.724+0000 7f1bcb000640 1 --2- 172.21.15.16:0/2625908861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bc4109e10 0x7f1bc4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.724+0000 7f1bcb000640 1 --2- 172.21.15.16:0/2625908861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bc41094c0 0x7f1bc41098d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.724+0000 7f1bcb000640 1 --2- 172.21.15.16:0/2625908861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bc4103b80 0x7f1bc4103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.724+0000 7f1bcb000640 1 -- 172.21.15.16:0/2625908861 >> 172.21.15.16:0/2625908861 conn(0x7f1bc40ff5d0 msgr2=0x7f1bc4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:33.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.724+0000 7f1bcb000640 1 -- 172.21.15.16:0/2625908861 shutdown_connections 2024-09-21T03:43:33.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.724+0000 7f1bcb000640 1 -- 172.21.15.16:0/2625908861 wait complete. 2024-09-21T03:43:33.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.725+0000 7f1bcb000640 1 Processor -- start 2024-09-21T03:43:33.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.726+0000 7f1bcb000640 1 -- start start 2024-09-21T03:43:33.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.726+0000 7f1bcb000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bc4103b80 0x7f1bc419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:33.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.726+0000 7f1bcb000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bc41094c0 0x7f1bc419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:33.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.726+0000 7f1bcb000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bc4109e10 0x7f1bc41a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:33.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.726+0000 7f1bcb000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc4113b20 con 0x7f1bc41094c0 2024-09-21T03:43:33.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.726+0000 7f1bcb000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc41139a0 con 0x7f1bc4103b80 2024-09-21T03:43:33.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.726+0000 7f1bcb000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc4113ca0 con 0x7f1bc4109e10 2024-09-21T03:43:33.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.726+0000 7f1bbbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bc41094c0 0x7f1bc419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:33.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.726+0000 7f1bc3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bc4103b80 0x7f1bc419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:33.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.726+0000 7f1bc3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bc4103b80 0x7f1bc419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:57180/0 (socket says 172.21.15.16:57180) 2024-09-21T03:43:33.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.726+0000 7f1bc3e00640 1 -- 172.21.15.16:0/1511574572 learned_addr learned my addr 172.21.15.16:0/1511574572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:33.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.727+0000 7f1bc9000640 1 --2- 172.21.15.16:0/1511574572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bc4109e10 0x7f1bc41a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.727+0000 7f1bc3e00640 1 -- 172.21.15.16:0/1511574572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bc4109e10 msgr2=0x7f1bc41a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.727+0000 7f1bc3e00640 1 --2- 172.21.15.16:0/1511574572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bc4109e10 0x7f1bc41a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.727+0000 7f1bc3e00640 1 -- 172.21.15.16:0/1511574572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bc41094c0 msgr2=0x7f1bc419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.727+0000 7f1bc3e00640 1 --2- 172.21.15.16:0/1511574572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bc41094c0 0x7f1bc419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.727+0000 7f1bc3e00640 1 -- 172.21.15.16:0/1511574572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc41a2660 con 0x7f1bc4103b80 2024-09-21T03:43:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.727+0000 7f1bc3e00640 1 --2- 172.21.15.16:0/1511574572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bc4103b80 0x7f1bc419c6f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400e9f0 tx=0x7f1bb400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.727+0000 7f1bc1600640 1 -- 172.21.15.16:0/1511574572 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb400cd80 con 0x7f1bc4103b80 2024-09-21T03:43:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.728+0000 7f1bcb000640 1 -- 172.21.15.16:0/1511574572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc41a2950 con 0x7f1bc4103b80 2024-09-21T03:43:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.728+0000 7f1bc1600640 1 -- 172.21.15.16:0/1511574572 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bb40108e0 con 0x7f1bc4103b80 2024-09-21T03:43:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.728+0000 7f1bc1600640 1 -- 172.21.15.16:0/1511574572 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4018980 con 0x7f1bc4103b80 2024-09-21T03:43:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.728+0000 7f1bcb000640 1 -- 172.21.15.16:0/1511574572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc41a90b0 con 0x7f1bc4103b80 2024-09-21T03:43:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.729+0000 7f1bcb000640 1 -- 172.21.15.16:0/1511574572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bc4103f90 con 0x7f1bc4103b80 2024-09-21T03:43:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.731+0000 7f1bc1600640 1 -- 172.21.15.16:0/1511574572 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f1bb4010430 con 0x7f1bc4103b80 2024-09-21T03:43:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.733+0000 7f1bc1600640 1 --2- 172.21.15.16:0/1511574572 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1ba80b6510 0x7f1ba80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.733+0000 7f1bc1600640 1 -- 172.21.15.16:0/1511574572 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8802+0+0 (secure 0 0 0) 0x7f1bb40e7e00 con 0x7f1bc4103b80 2024-09-21T03:43:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.733+0000 7f1bbbe00640 1 --2- 172.21.15.16:0/1511574572 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1ba80b6510 0x7f1ba80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:33.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.734+0000 7f1bbbe00640 1 --2- 172.21.15.16:0/1511574572 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1ba80b6510 0x7f1ba80b8a20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1bac02aa00 tx=0x7f1bac005a20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:33.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.735+0000 7f1bc1600640 1 -- 172.21.15.16:0/1511574572 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1bb40ae3f0 con 0x7f1bc4103b80 2024-09-21T03:43:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.938+0000 7f1bcb000640 1 -- 172.21.15.16:0/1511574572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bc41a9450 con 0x7f1bc4103b80 2024-09-21T03:43:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.939+0000 7f1bc1600640 1 -- 172.21.15.16:0/1511574572 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+24232 (secure 0 0 0) 0x7f1bb40b27d0 con 0x7f1bc4103b80 2024-09-21T03:43:33.943 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:43:33.943 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:33:53.220358+0000","modified":"2024-09-21T03:43:22.246847+0000","last_up_change":"2024-09-21T03:42:06.614247+0000","last_in_change":"2024-09-21T03:41:43.379404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:37:55.708915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:43:03.567522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:43:06.048083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:43:16.061356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6a7e4b9e-078f-4ae9-ab9e-37be995e4deb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6803","nonce":311974551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6805","nonce":311974551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6809","nonce":311974551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6807","nonce":311974551}]},"public_addr":"172.21.15.16:6803/311974551","cluster_addr":"172.21.15.16:6805/311974551","heartbeat_back_addr":"172.21.15.16:6809/311974551","heartbeat_front_addr":"172.21.15.16:6807/311974551","state":["exists","up"]},{"osd":1,"uuid":"3a497a64-e1a8-43ef-8509-0a79515da0aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6801","nonce":2152486571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6803","nonce":2152486571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6807","nonce":2152486571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6805","nonce":2152486571}]},"public_addr":"172.21.15.123:6801/2152486571","cluster_addr":"172.21.15.123:6803/2152486571","heartbeat_back_addr":"172.21.15.123:6807/2152486571","heartbeat_front_addr":"172.21.15.123:6805/2152486571","state":["exists","up"]},{"osd":2,"uuid":"1c39518e-6cce-4a0f-9ee8-55c60ad325f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6801","nonce":2812351279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6803","nonce":2812351279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6807","nonce":2812351279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6805","nonce":2812351279}]},"public_addr":"172.21.15.134:6801/2812351279","cluster_addr":"172.21.15.134:6803/2812351279","heartbeat_back_addr":"172.21.15.134:6807/2812351279","heartbeat_front_addr":"172.21.15.134:6805/2812351279","state":["exists","up"]},{"osd":3,"uuid":"0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6811","nonce":2366216026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6813","nonce":2366216026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6817","nonce":2366216026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6815","nonce":2366216026}]},"public_addr":"172.21.15.16:6811/2366216026","cluster_addr":"172.21.15.16:6813/2366216026","heartbeat_back_addr":"172.21.15.16:6817/2366216026","heartbeat_front_addr":"172.21.15.16:6815/2366216026","state":["exists","up"]},{"osd":4,"uuid":"ad5e6d0e-31e8-4779-9e74-f215032777ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6809","nonce":2354279116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6811","nonce":2354279116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6815","nonce":2354279116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6813","nonce":2354279116}]},"public_addr":"172.21.15.123:6809/2354279116","cluster_addr":"172.21.15.123:6811/2354279116","heartbeat_back_addr":"172.21.15.123:6815/2354279116","heartbeat_front_addr":"172.21.15.123:6813/2354279116","state":["exists","up"]},{"osd":5,"uuid":"ca09bf88-1772-4dc0-8711-cbf9a5b31977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6809","nonce":2678416182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6811","nonce":2678416182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6815","nonce":2678416182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6813","nonce":2678416182}]},"public_addr":"172.21.15.134:6809/2678416182","cluster_addr":"172.21.15.134:6811/2678416182","heartbeat_back_addr":"172.21.15.134:6815/2678416182","heartbeat_front_addr":"172.21.15.134:6813/2678416182","state":["exists","up"]},{"osd":6,"uuid":"0d41ed4e-8699-4d88-9254-12d3b84941e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6819","nonce":3377807744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6821","nonce":3377807744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6825","nonce":3377807744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6823","nonce":3377807744}]},"public_addr":"172.21.15.16:6819/3377807744","cluster_addr":"172.21.15.16:6821/3377807744","heartbeat_back_addr":"172.21.15.16:6825/3377807744","heartbeat_front_addr":"172.21.15.16:6823/3377807744","state":["exists","up"]},{"osd":7,"uuid":"1541a272-c524-4f12-9b94-763f69bb1c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6817","nonce":1463412739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6819","nonce":1463412739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6823","nonce":1463412739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6821","nonce":1463412739}]},"public_addr":"172.21.15.123:6817/1463412739","cluster_addr":"172.21.15.123:6819/1463412739","heartbeat_back_addr":"172.21.15.123:6823/1463412739","heartbeat_front_addr":"172.21.15.123:6821/1463412739","state":["exists","up"]},{"osd":8,"uuid":"cc00014c-eb99-49ab-984c-0bc26ef85130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6817","nonce":2644021450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6819","nonce":2644021450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6823","nonce":2644021450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6821","nonce":2644021450}]},"public_addr":"172.21.15.134:6817/2644021450","cluster_addr":"172.21.15.134:6819/2644021450","heartbeat_back_addr":"172.21.15.134:6823/2644021450","heartbeat_front_addr":"172.21.15.134:6821/2644021450","state":["exists","up"]},{"osd":9,"uuid":"3cbfff74-31ff-4ffd-844b-54ec8563ba32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6827","nonce":1653164371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6829","nonce":1653164371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6833","nonce":1653164371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6831","nonce":1653164371}]},"public_addr":"172.21.15.16:6827/1653164371","cluster_addr":"172.21.15.16:6829/1653164371","heartbeat_back_addr":"172.21.15.16:6833/1653164371","heartbeat_front_addr":"172.21.15.16:6831/1653164371","state":["exists","up"]},{"osd":10,"uuid":"0a1caaad-70cd-4c2c-88d3-e9a5cc885655","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6825","nonce":3682234496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6827","nonce":3682234496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6831","nonce":3682234496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6829","nonce":3682234496}]},"public_addr":"172.21.15.123:6825/3682234496","cluster_addr":"172.21.15.123:6827/3682234496","heartbeat_back_addr":"172.21.15.123:6831/3682234496","heartbeat_front_addr":"172.21.15.123:6829/3682234496","state":["exists","up"]},{"osd":11,"uuid":"3aff3258-5a38-41ff-aac3-4a082feffdae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6825","nonce":584758685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6827","nonce":584758685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6831","nonce":584758685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6829","nonce":584758685}]},"public_addr":"172.21.15.134:6825/584758685","cluster_addr":"172.21.15.134:6827/584758685","heartbeat_back_addr":"172.21.15.134:6831/584758685","heartbeat_front_addr":"172.21.15.134:6829/584758685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:02.177731+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:28.847562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:54.272384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:24.338596+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:52.430371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:17.832737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:50.397890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:16.842245+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:42.102933+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:13.074732+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:40.057905+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:04.898190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/557742783":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/3238676548":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6801/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/2357336233":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6801/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6800/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/3806844703":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/2351581621":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6800/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/355731724":"2024-09-22T03:34:22.549156+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:43:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.942+0000 7f1bcb000640 1 -- 172.21.15.16:0/1511574572 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1ba80b6510 msgr2=0x7f1ba80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.942+0000 7f1bcb000640 1 --2- 172.21.15.16:0/1511574572 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1ba80b6510 0x7f1ba80b8a20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1bac02aa00 tx=0x7f1bac005a20 comp rx=0 tx=0).stop 2024-09-21T03:43:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.942+0000 7f1bcb000640 1 -- 172.21.15.16:0/1511574572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bc4103b80 msgr2=0x7f1bc419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.942+0000 7f1bcb000640 1 --2- 172.21.15.16:0/1511574572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bc4103b80 0x7f1bc419c6f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400e9f0 tx=0x7f1bb400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.942+0000 7f1bcb000640 1 -- 172.21.15.16:0/1511574572 shutdown_connections 2024-09-21T03:43:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.942+0000 7f1bcb000640 1 --2- 172.21.15.16:0/1511574572 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1ba80b6510 0x7f1ba80b8a20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.942+0000 7f1bcb000640 1 --2- 172.21.15.16:0/1511574572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bc4109e10 0x7f1bc41a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.942+0000 7f1bcb000640 1 --2- 172.21.15.16:0/1511574572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bc41094c0 0x7f1bc419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.942+0000 7f1bcb000640 1 --2- 172.21.15.16:0/1511574572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bc4103b80 0x7f1bc419c6f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.942+0000 7f1bcb000640 1 -- 172.21.15.16:0/1511574572 >> 172.21.15.16:0/1511574572 conn(0x7f1bc40ff5d0 msgr2=0x7f1bc4108b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.943+0000 7f1bcb000640 1 -- 172.21.15.16:0/1511574572 shutdown_connections 2024-09-21T03:43:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:33.943+0000 7f1bcb000640 1 -- 172.21.15.16:0/1511574572 wait complete. 2024-09-21T03:43:34.627 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:34 smithi016 ceph-mon[30375]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:34.627 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:34 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1511574572' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:34.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:34 smithi123 ceph-mon[34673]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:34.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:34 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1511574572' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:34.665 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:43:09.332305+0000', 'modified': '2024-09-21T03:43:22.247004+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24731}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24731': {'gid': 24731, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.123:6839/459522816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 459522816}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 459522816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24731, 'qdb_cluster': [24731]} 2024-09-21T03:43:34.665 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:43:34.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:43:34.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:34 smithi134 ceph-mon[35031]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:34.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:34 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1511574572' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.650+0000 7fb87fe00640 1 -- 172.21.15.16:0/3707499479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87810b6c0 msgr2=0x7fb87810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.650+0000 7fb87fe00640 1 --2- 172.21.15.16:0/3707499479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87810b6c0 0x7fb87810bad0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb864009900 tx=0x7fb86402a410 comp rx=0 tx=0).stop 2024-09-21T03:43:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.650+0000 7fb87fe00640 1 -- 172.21.15.16:0/3707499479 shutdown_connections 2024-09-21T03:43:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.650+0000 7fb87fe00640 1 --2- 172.21.15.16:0/3707499479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87810c010 0x7fb878112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.650+0000 7fb87fe00640 1 --2- 172.21.15.16:0/3707499479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87810b6c0 0x7fb87810bad0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.650+0000 7fb87fe00640 1 --2- 172.21.15.16:0/3707499479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb878105d80 0x7fb878106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.650+0000 7fb87fe00640 1 -- 172.21.15.16:0/3707499479 >> 172.21.15.16:0/3707499479 conn(0x7fb878101810 msgr2=0x7fb878103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.650+0000 7fb87fe00640 1 -- 172.21.15.16:0/3707499479 shutdown_connections 2024-09-21T03:43:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.650+0000 7fb87fe00640 1 -- 172.21.15.16:0/3707499479 wait complete. 2024-09-21T03:43:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.651+0000 7fb87fe00640 1 Processor -- start 2024-09-21T03:43:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.651+0000 7fb87fe00640 1 -- start start 2024-09-21T03:43:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.651+0000 7fb87fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb878105d80 0x7fb87819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.651+0000 7fb87fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87810b6c0 0x7fb87819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.651+0000 7fb87fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87810c010 0x7fb8781a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.652+0000 7fb87fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb878115d20 con 0x7fb87810b6c0 2024-09-21T03:43:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.652+0000 7fb87fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb878115ba0 con 0x7fb878105d80 2024-09-21T03:43:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.652+0000 7fb87fe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb878115ea0 con 0x7fb87810c010 2024-09-21T03:43:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.652+0000 7fb87ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87810b6c0 0x7fb87819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.652+0000 7fb87ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87810b6c0 0x7fb87819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35248/0 (socket says 172.21.15.16:35248) 2024-09-21T03:43:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.652+0000 7fb87ca00640 1 -- 172.21.15.16:0/2267859307 learned_addr learned my addr 172.21.15.16:0/2267859307 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.652+0000 7fb87de00640 1 --2- 172.21.15.16:0/2267859307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87810c010 0x7fb8781a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.652+0000 7fb87d400640 1 --2- 172.21.15.16:0/2267859307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb878105d80 0x7fb87819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.652+0000 7fb87ca00640 1 -- 172.21.15.16:0/2267859307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87810c010 msgr2=0x7fb8781a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.652+0000 7fb87ca00640 1 --2- 172.21.15.16:0/2267859307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87810c010 0x7fb8781a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.652+0000 7fb87ca00640 1 -- 172.21.15.16:0/2267859307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb878105d80 msgr2=0x7fb87819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.652+0000 7fb87ca00640 1 --2- 172.21.15.16:0/2267859307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb878105d80 0x7fb87819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.652+0000 7fb87ca00640 1 -- 172.21.15.16:0/2267859307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8781a2620 con 0x7fb87810b6c0 2024-09-21T03:43:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.653+0000 7fb87ca00640 1 --2- 172.21.15.16:0/2267859307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87810b6c0 0x7fb87819cc50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb864005ab0 tx=0x7fb864003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.653+0000 7fb876000640 1 -- 172.21.15.16:0/2267859307 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86403f070 con 0x7fb87810b6c0 2024-09-21T03:43:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.653+0000 7fb876000640 1 -- 172.21.15.16:0/2267859307 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb864031d60 con 0x7fb87810b6c0 2024-09-21T03:43:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.653+0000 7fb87fe00640 1 -- 172.21.15.16:0/2267859307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8781a28b0 con 0x7fb87810b6c0 2024-09-21T03:43:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.653+0000 7fb876000640 1 -- 172.21.15.16:0/2267859307 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864035030 con 0x7fb87810b6c0 2024-09-21T03:43:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.653+0000 7fb87fe00640 1 -- 172.21.15.16:0/2267859307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8781a9070 con 0x7fb87810b6c0 2024-09-21T03:43:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.656+0000 7fb886bf0740 1 Processor -- start 2024-09-21T03:43:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.657+0000 7fb886bf0740 1 -- start start 2024-09-21T03:43:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.657+0000 7fb876000640 1 -- 172.21.15.16:0/2267859307 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fb864042070 con 0x7fb87810b6c0 2024-09-21T03:43:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.657+0000 7fb886bf0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699beeaf60 0x55699c291490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.657+0000 7fb886bf0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699bec20a0 0x55699c2919d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.657+0000 7fb886bf0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699beca280 0x55699c291f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.657+0000 7fb886bf0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55699c208d50 con 0x55699bec20a0 2024-09-21T03:43:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.657+0000 7fb886bf0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55699c1f8180 con 0x55699beca280 2024-09-21T03:43:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.657+0000 7fb886bf0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55699c1fbf90 con 0x55699beeaf60 2024-09-21T03:43:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.657+0000 7fb87d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699bec20a0 0x55699c2919d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.657+0000 7fb87de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699beeaf60 0x55699c291490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.658+0000 7fb87d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699bec20a0 0x55699c2919d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35250/0 (socket says 172.21.15.16:35250) 2024-09-21T03:43:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.657+0000 7fb87ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699beca280 0x55699c291f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.658+0000 7fb87d400640 1 -- 172.21.15.16:0/903569970 learned_addr learned my addr 172.21.15.16:0/903569970 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.658+0000 7fb87de00640 1 -- 172.21.15.16:0/903569970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699beca280 msgr2=0x55699c291f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.658+0000 7fb87de00640 1 --2- 172.21.15.16:0/903569970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699beca280 0x55699c291f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.658+0000 7fb87de00640 1 -- 172.21.15.16:0/903569970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699bec20a0 msgr2=0x55699c2919d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.658+0000 7fb87de00640 1 --2- 172.21.15.16:0/903569970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699bec20a0 0x55699c2919d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.658+0000 7fb87de00640 1 -- 172.21.15.16:0/903569970 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55699c1c2a70 con 0x55699beeaf60 2024-09-21T03:43:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.658+0000 7fb87de00640 1 --2- 172.21.15.16:0/903569970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699beeaf60 0x55699c291490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb870009eb0 tx=0x7fb870010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dad54d5e684329c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.658+0000 7fb86a600640 1 -- 172.21.15.16:0/903569970 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870016070 con 0x55699beeaf60 2024-09-21T03:43:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.659+0000 7fb86a600640 1 -- 172.21.15.16:0/903569970 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb870004d20 con 0x55699beeaf60 2024-09-21T03:43:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.659+0000 7fb86a600640 1 -- 172.21.15.16:0/903569970 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870012690 con 0x55699beeaf60 2024-09-21T03:43:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.659+0000 7fb876000640 1 --2- 172.21.15.16:0/2267859307 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8600b6510 0x7fb8600b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.659+0000 7fb876000640 1 -- 172.21.15.16:0/2267859307 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8802+0+0 (secure 0 0 0) 0x7fb864107cf0 con 0x7fb87810b6c0 2024-09-21T03:43:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.659+0000 7fb87d400640 1 --2- 172.21.15.16:0/2267859307 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8600b6510 0x7fb8600b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.659+0000 7fb886bf0740 1 -- 172.21.15.16:0/903569970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699beeaf60 msgr2=0x55699c291490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.659+0000 7fb886bf0740 1 --2- 172.21.15.16:0/903569970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699beeaf60 0x55699c291490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb870009eb0 tx=0x7fb870010740 comp rx=0 tx=0).stop 2024-09-21T03:43:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.660+0000 7fb886bf0740 1 -- 172.21.15.16:0/903569970 shutdown_connections 2024-09-21T03:43:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.660+0000 7fb886bf0740 1 --2- 172.21.15.16:0/903569970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699beca280 0x55699c291f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.660+0000 7fb886bf0740 1 --2- 172.21.15.16:0/903569970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699bec20a0 0x55699c2919d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.660+0000 7fb886bf0740 1 --2- 172.21.15.16:0/903569970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699beeaf60 0x55699c291490 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.660+0000 7fb886bf0740 1 -- 172.21.15.16:0/903569970 >> 172.21.15.16:0/903569970 conn(0x55699bedf350 msgr2=0x55699c28f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.660+0000 7fb886bf0740 1 -- 172.21.15.16:0/903569970 shutdown_connections 2024-09-21T03:43:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.660+0000 7fb87d400640 1 --2- 172.21.15.16:0/2267859307 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8600b6510 0x7fb8600b8a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55699c290110 tx=0x7fb86c00b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.660+0000 7fb886bf0740 1 -- 172.21.15.16:0/903569970 wait complete. 2024-09-21T03:43:35.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.660+0000 7fb886bf0740 1 -- start start 2024-09-21T03:43:35.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.661+0000 7fb886bf0740 1 Processor -- start 2024-09-21T03:43:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.661+0000 7fb886bf0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699bec20a0 0x55699c29d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.661+0000 7fb886bf0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699beca280 0x55699c29d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.661+0000 7fb886bf0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beeaf60 0x55699c2a8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.661+0000 7fb886bf0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55699c0595f0 con 0x55699beeaf60 2024-09-21T03:43:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.661+0000 7fb886bf0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55699c193460 con 0x55699beca280 2024-09-21T03:43:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.661+0000 7fb886bf0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55699c29fc70 con 0x55699bec20a0 2024-09-21T03:43:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.661+0000 7fb886bf0740 10 client.0 _ll_register_callbacks cb 0x55699c1a3210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.661+0000 7fb87de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beeaf60 0x55699c2a8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.662+0000 7fb87de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beeaf60 0x55699c2a8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35266/0 (socket says 172.21.15.16:35266) 2024-09-21T03:43:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.662+0000 7fb87de00640 1 -- 172.21.15.16:0/2235080668 learned_addr learned my addr 172.21.15.16:0/2235080668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.662+0000 7fb87d400640 1 --2- 172.21.15.16:0/2235080668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699bec20a0 0x55699c29d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.662+0000 7fb87ca00640 1 --2- 172.21.15.16:0/2235080668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699beca280 0x55699c29d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.662+0000 7fb87de00640 1 -- 172.21.15.16:0/2235080668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699bec20a0 msgr2=0x55699c29d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.662+0000 7fb87de00640 1 --2- 172.21.15.16:0/2235080668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699bec20a0 0x55699c29d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.662+0000 7fb87de00640 1 -- 172.21.15.16:0/2235080668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699beca280 msgr2=0x55699c29d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.662+0000 7fb87de00640 1 --2- 172.21.15.16:0/2235080668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699beca280 0x55699c29d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.662+0000 7fb87de00640 1 -- 172.21.15.16:0/2235080668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55699bf56540 con 0x55699beeaf60 2024-09-21T03:43:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.662+0000 7fb87d400640 1 --2- 172.21.15.16:0/2235080668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699bec20a0 0x55699c29d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.662+0000 7fb87de00640 1 --2- 172.21.15.16:0/2235080668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beeaf60 0x55699c2a8fc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb870006010 tx=0x7fb870012680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cca91af3b1cb37b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.662+0000 7fb85ea00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:43:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.663+0000 7fb85ea00640 1 -- 172.21.15.16:0/2235080668 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870029060 con 0x55699beeaf60 2024-09-21T03:43:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.663+0000 7fb85ea00640 1 -- 172.21.15.16:0/2235080668 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8802+0+0 (secure 0 0 0) 0x7fb870012a00 con 0x55699beeaf60 2024-09-21T03:43:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.663+0000 7fb886bf0740 1 -- 172.21.15.16:0/2235080668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55699c195a70 con 0x55699beeaf60 2024-09-21T03:43:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.663+0000 7fb85ea00640 1 -- 172.21.15.16:0/2235080668 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb870030050 con 0x55699beeaf60 2024-09-21T03:43:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.663+0000 7fb886bf0740 10 client.14817 fetch_fsmap learned FSMap version 17 2024-09-21T03:43:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.663+0000 7fb886bf0740 1 -- 172.21.15.16:0/2235080668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55699c1b06f0 con 0x55699beeaf60 2024-09-21T03:43:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.663+0000 7fb85ea00640 1 -- 172.21.15.16:0/2235080668 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55699c195a70 con 0x55699beeaf60 2024-09-21T03:43:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.664+0000 7fb85ea00640 1 -- 172.21.15.16:0/2235080668 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fb87000f040 con 0x55699beeaf60 2024-09-21T03:43:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.664+0000 7fb886bf0740 10 client.14817 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T03:43:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.664+0000 7fb886bf0740 10 client.14817 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:43:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.664+0000 7fb886bf0740 20 client.14817 populate_metadata read hostname 'smithi016' 2024-09-21T03:43:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.664+0000 7fb886bf0740 1 --2- 172.21.15.16:0/2235080668 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55699bea98e0 0x55699c2af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.664+0000 7fb886bf0740 4 client.14817 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.664+0000 7fb886bf0740 1 -- 172.21.15.16:0/2235080668 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55699c20aca0 con 0x55699bea98e0 2024-09-21T03:43:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.664+0000 7fb87d400640 1 --2- 172.21.15.16:0/2235080668 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55699bea98e0 0x55699c2af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.665+0000 7fb87d400640 1 --2- 172.21.15.16:0/2235080668 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55699bea98e0 0x55699c2af1c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8881ba15d7e1999 server_cookie=828d467207e50469 in_seq=0 out_seq=0 2024-09-21T03:43:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.665+0000 7fb85ea00640 10 client.14817 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:43:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.666+0000 7fb85ea00640 1 -- 172.21.15.16:0/2235080668 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55699c20aca0 con 0x55699bea98e0 2024-09-21T03:43:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.666+0000 7fb85ea00640 10 client.14817 handle_command_reply: tid=0 2024-09-21T03:43:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.666+0000 7fb886bf0740 1 client.14817 shutdown 2024-09-21T03:43:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.666+0000 7fb886bf0740 1 -- 172.21.15.16:0/2235080668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beeaf60 msgr2=0x55699c2a8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.666+0000 7fb886bf0740 1 --2- 172.21.15.16:0/2235080668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beeaf60 0x55699c2a8fc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb870006010 tx=0x7fb870012680 comp rx=0 tx=0).stop 2024-09-21T03:43:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.666+0000 7fb886bf0740 1 -- 172.21.15.16:0/2235080668 shutdown_connections 2024-09-21T03:43:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.666+0000 7fb886bf0740 1 --2- 172.21.15.16:0/2235080668 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55699bea98e0 0x55699c2af1c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.666+0000 7fb886bf0740 1 --2- 172.21.15.16:0/2235080668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beeaf60 0x55699c2a8fc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.666+0000 7fb886bf0740 1 --2- 172.21.15.16:0/2235080668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699beca280 0x55699c29d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.666+0000 7fb886bf0740 1 --2- 172.21.15.16:0/2235080668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699bec20a0 0x55699c29d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.666+0000 7fb886bf0740 1 -- 172.21.15.16:0/2235080668 >> 172.21.15.16:0/2235080668 conn(0x55699bedf350 msgr2=0x55699c294620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.666+0000 7fb85ea00640 0 client.14817 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:43:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.666+0000 7fb886bf0740 1 -- 172.21.15.16:0/2235080668 shutdown_connections 2024-09-21T03:43:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.666+0000 7fb886bf0740 1 -- 172.21.15.16:0/2235080668 wait complete. 2024-09-21T03:43:35.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.666+0000 7fb886bf0740 20 client.14817 trim_cache size 0 max 16384 2024-09-21T03:43:35.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.685+0000 7fb886bf0740 1 Processor -- start 2024-09-21T03:43:35.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.685+0000 7fb886bf0740 1 -- start start 2024-09-21T03:43:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.686+0000 7fb886bf0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699bec20a0 0x55699c2af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.686+0000 7fb886bf0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beca280 0x55699c2a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.686+0000 7fb886bf0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699beeaf60 0x55699c298090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.686+0000 7fb886bf0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55699c2ad500 con 0x55699beca280 2024-09-21T03:43:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.686+0000 7fb886bf0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55699c2adbd0 con 0x55699beeaf60 2024-09-21T03:43:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.686+0000 7fb886bf0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55699c2ae2a0 con 0x55699bec20a0 2024-09-21T03:43:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.686+0000 7fb87ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beca280 0x55699c2a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.686+0000 7fb87ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beca280 0x55699c2a8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35274/0 (socket says 172.21.15.16:35274) 2024-09-21T03:43:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.686+0000 7fb87ca00640 1 -- 172.21.15.16:0/3904493145 learned_addr learned my addr 172.21.15.16:0/3904493145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.686+0000 7fb87d400640 1 --2- 172.21.15.16:0/3904493145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699bec20a0 0x55699c2af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.686+0000 7fb87de00640 1 --2- 172.21.15.16:0/3904493145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699beeaf60 0x55699c298090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.686+0000 7fb87ca00640 1 -- 172.21.15.16:0/3904493145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699bec20a0 msgr2=0x55699c2af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.686+0000 7fb87ca00640 1 --2- 172.21.15.16:0/3904493145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699bec20a0 0x55699c2af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.686+0000 7fb87ca00640 1 -- 172.21.15.16:0/3904493145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699beeaf60 msgr2=0x55699c298090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.686+0000 7fb87ca00640 1 --2- 172.21.15.16:0/3904493145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699beeaf60 0x55699c298090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.686+0000 7fb87ca00640 1 -- 172.21.15.16:0/3904493145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55699c29e1a0 con 0x55699beca280 2024-09-21T03:43:35.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.687+0000 7fb87ca00640 1 --2- 172.21.15.16:0/3904493145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beca280 0x55699c2a8f30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb86410b4d0 tx=0x7fb86410b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76329ac60fd58f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:35.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.687+0000 7fb869c00640 1 -- 172.21.15.16:0/3904493145 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864049040 con 0x55699beca280 2024-09-21T03:43:35.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.687+0000 7fb869c00640 1 -- 172.21.15.16:0/3904493145 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8640cd7a0 con 0x55699beca280 2024-09-21T03:43:35.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.687+0000 7fb869c00640 1 -- 172.21.15.16:0/3904493145 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8640cd400 con 0x55699beca280 2024-09-21T03:43:35.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.688+0000 7fb886bf0740 1 -- 172.21.15.16:0/3904493145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beca280 msgr2=0x55699c2a8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.688+0000 7fb886bf0740 1 --2- 172.21.15.16:0/3904493145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beca280 0x55699c2a8f30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb86410b4d0 tx=0x7fb86410b730 comp rx=0 tx=0).stop 2024-09-21T03:43:35.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.688+0000 7fb886bf0740 1 -- 172.21.15.16:0/3904493145 shutdown_connections 2024-09-21T03:43:35.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.688+0000 7fb886bf0740 1 --2- 172.21.15.16:0/3904493145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699beeaf60 0x55699c298090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.688+0000 7fb886bf0740 1 --2- 172.21.15.16:0/3904493145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beca280 0x55699c2a8f30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.688+0000 7fb886bf0740 1 --2- 172.21.15.16:0/3904493145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699bec20a0 0x55699c2af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.688+0000 7fb886bf0740 1 -- 172.21.15.16:0/3904493145 >> 172.21.15.16:0/3904493145 conn(0x55699bedf350 msgr2=0x55699c29d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:35.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.688+0000 7fb886bf0740 1 -- 172.21.15.16:0/3904493145 shutdown_connections 2024-09-21T03:43:35.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.688+0000 7fb886bf0740 1 -- 172.21.15.16:0/3904493145 wait complete. 2024-09-21T03:43:35.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.689+0000 7fb886bf0740 1 -- start start 2024-09-21T03:43:35.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.689+0000 7fb886bf0740 1 Processor -- start 2024-09-21T03:43:35.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.689+0000 7fb886bf0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699bec20a0 0x55699c2966d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.689+0000 7fb886bf0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699beca280 0x55699c296c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.689+0000 7fb886bf0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beeaf60 0x55699c318680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.689+0000 7fb886bf0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55699c2af040 con 0x55699beeaf60 2024-09-21T03:43:35.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.689+0000 7fb886bf0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55699c2ae970 con 0x55699bec20a0 2024-09-21T03:43:35.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.689+0000 7fb886bf0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55699c2af710 con 0x55699beca280 2024-09-21T03:43:35.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.689+0000 7fb886bf0740 10 client.0 _ll_register_callbacks cb 0x7fb870009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:35.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.690+0000 7fb87de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beeaf60 0x55699c318680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.690+0000 7fb87de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beeaf60 0x55699c318680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35284/0 (socket says 172.21.15.16:35284) 2024-09-21T03:43:35.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.690+0000 7fb87de00640 1 -- 172.21.15.16:0/3225399669 learned_addr learned my addr 172.21.15.16:0/3225399669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:35.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.690+0000 7fb87d400640 1 --2- 172.21.15.16:0/3225399669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699bec20a0 0x55699c2966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.690+0000 7fb87ca00640 1 --2- 172.21.15.16:0/3225399669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699beca280 0x55699c296c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.690+0000 7fb87d400640 1 -- 172.21.15.16:0/3225399669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699beca280 msgr2=0x55699c296c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:35.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.690+0000 7fb87d400640 1 --2- 172.21.15.16:0/3225399669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699beca280 0x55699c296c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.690+0000 7fb87d400640 1 -- 172.21.15.16:0/3225399669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beeaf60 msgr2=0x55699c318680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:35.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.690+0000 7fb87d400640 1 --2- 172.21.15.16:0/3225399669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beeaf60 0x55699c318680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.690+0000 7fb87d400640 1 -- 172.21.15.16:0/3225399669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55699c318cc0 con 0x55699bec20a0 2024-09-21T03:43:35.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.690+0000 7fb87de00640 1 --2- 172.21.15.16:0/3225399669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beeaf60 0x55699c318680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:35.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.690+0000 7fb87d400640 1 --2- 172.21.15.16:0/3225399669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699bec20a0 0x55699c2966d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55699c2af8c0 tx=0x7fb86c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6cd2a5bd3fe1b4b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:35.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.690+0000 7fb85ea00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:43:35.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.691+0000 7fb85ea00640 1 -- 172.21.15.16:0/3225399669 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c004020 con 0x55699bec20a0 2024-09-21T03:43:35.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.691+0000 7fb886bf0740 1 -- 172.21.15.16:0/3225399669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8640cd7a0 con 0x55699bec20a0 2024-09-21T03:43:35.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.691+0000 7fb85ea00640 1 -- 172.21.15.16:0/3225399669 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8802+0+0 (secure 0 0 0) 0x7fb86c0041c0 con 0x55699bec20a0 2024-09-21T03:43:35.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.691+0000 7fb85ea00640 1 -- 172.21.15.16:0/3225399669 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb86c0043c0 con 0x55699bec20a0 2024-09-21T03:43:35.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.691+0000 7fb886bf0740 10 client.24839 fetch_fsmap learned FSMap version 17 2024-09-21T03:43:35.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.691+0000 7fb886bf0740 1 -- 172.21.15.16:0/3225399669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55699c319fd0 con 0x55699bec20a0 2024-09-21T03:43:35.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.691+0000 7fb85ea00640 1 -- 172.21.15.16:0/3225399669 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8640cd7a0 con 0x55699bec20a0 2024-09-21T03:43:35.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.692+0000 7fb85ea00640 1 -- 172.21.15.16:0/3225399669 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fb86c030710 con 0x55699bec20a0 2024-09-21T03:43:35.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.692+0000 7fb886bf0740 10 client.24839 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T03:43:35.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.692+0000 7fb886bf0740 10 client.24839 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:43:35.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.692+0000 7fb886bf0740 20 client.24839 populate_metadata read hostname 'smithi016' 2024-09-21T03:43:35.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.692+0000 7fb886bf0740 1 --2- 172.21.15.16:0/3225399669 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55699be7ec90 0x55699c31d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:35.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.692+0000 7fb886bf0740 4 client.24839 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:43:35.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.692+0000 7fb886bf0740 1 -- 172.21.15.16:0/3225399669 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "status"}) -- 0x55699c206750 con 0x55699be7ec90 2024-09-21T03:43:35.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.692+0000 7fb87ca00640 1 --2- 172.21.15.16:0/3225399669 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55699be7ec90 0x55699c31d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:35.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.692+0000 7fb87ca00640 1 --2- 172.21.15.16:0/3225399669 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55699be7ec90 0x55699c31d8f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d09e8a216e8678aa server_cookie=ebb86aa001e2762d in_seq=0 out_seq=0 2024-09-21T03:43:35.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.692+0000 7fb85ea00640 10 client.24839 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:43:35.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.693+0000 7fb85ea00640 1 -- 172.21.15.16:0/3225399669 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55699c206750 con 0x55699be7ec90 2024-09-21T03:43:35.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.693+0000 7fb85ea00640 10 client.24839 handle_command_reply: tid=0 2024-09-21T03:43:35.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.693+0000 7fb886bf0740 1 client.24839 shutdown 2024-09-21T03:43:35.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.693+0000 7fb886bf0740 1 -- 172.21.15.16:0/3225399669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699bec20a0 msgr2=0x55699c2966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.693+0000 7fb886bf0740 1 --2- 172.21.15.16:0/3225399669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699bec20a0 0x55699c2966d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55699c2af8c0 tx=0x7fb86c0046c0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.693+0000 7fb886bf0740 1 -- 172.21.15.16:0/3225399669 shutdown_connections 2024-09-21T03:43:35.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.693+0000 7fb886bf0740 1 --2- 172.21.15.16:0/3225399669 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55699be7ec90 0x55699c31d8f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.693+0000 7fb886bf0740 1 --2- 172.21.15.16:0/3225399669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55699beeaf60 0x55699c318680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.693+0000 7fb886bf0740 1 --2- 172.21.15.16:0/3225399669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699beca280 0x55699c296c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.693+0000 7fb85ea00640 0 client.24839 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:43:35.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.693+0000 7fb886bf0740 1 --2- 172.21.15.16:0/3225399669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55699bec20a0 0x55699c2966d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.693+0000 7fb886bf0740 1 -- 172.21.15.16:0/3225399669 >> 172.21.15.16:0/3225399669 conn(0x55699bedf350 msgr2=0x55699c29b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:35.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.693+0000 7fb886bf0740 1 -- 172.21.15.16:0/3225399669 shutdown_connections 2024-09-21T03:43:35.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.693+0000 7fb886bf0740 1 -- 172.21.15.16:0/3225399669 wait complete. 2024-09-21T03:43:35.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.693+0000 7fb886bf0740 20 client.24839 trim_cache size 0 max 16384 2024-09-21T03:43:35.699 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:43:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:43:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-21T03:43:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24731, 2024-09-21T03:43:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T03:43:35.699 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T03:43:35.700 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T03:43:35.700 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 25.687747492, 2024-09-21T03:43:35.700 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 17, 2024-09-21T03:43:35.700 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 82, 2024-09-21T03:43:35.700 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 82, 2024-09-21T03:43:35.700 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 42.090314448999997 2024-09-21T03:43:35.700 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:43:35.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.694+0000 7fb87fe00640 1 -- 172.21.15.16:0/2267859307 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8600b6510 msgr2=0x7fb8600b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.694+0000 7fb87fe00640 1 --2- 172.21.15.16:0/2267859307 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8600b6510 0x7fb8600b8a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55699c290110 tx=0x7fb86c00b410 comp rx=0 tx=0).stop 2024-09-21T03:43:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.694+0000 7fb87fe00640 1 -- 172.21.15.16:0/2267859307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87810b6c0 msgr2=0x7fb87819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.694+0000 7fb87fe00640 1 --2- 172.21.15.16:0/2267859307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87810b6c0 0x7fb87819cc50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb864005ab0 tx=0x7fb864003cd0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.694+0000 7fb87fe00640 1 -- 172.21.15.16:0/2267859307 shutdown_connections 2024-09-21T03:43:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.694+0000 7fb87fe00640 1 --2- 172.21.15.16:0/2267859307 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8600b6510 0x7fb8600b8a20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.694+0000 7fb87fe00640 1 --2- 172.21.15.16:0/2267859307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87810c010 0x7fb8781a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.694+0000 7fb87fe00640 1 --2- 172.21.15.16:0/2267859307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87810b6c0 0x7fb87819cc50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.694+0000 7fb87fe00640 1 --2- 172.21.15.16:0/2267859307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb878105d80 0x7fb87819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.694+0000 7fb87fe00640 1 -- 172.21.15.16:0/2267859307 >> 172.21.15.16:0/2267859307 conn(0x7fb878101810 msgr2=0x7fb87810ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.694+0000 7fb87fe00640 1 -- 172.21.15.16:0/2267859307 shutdown_connections 2024-09-21T03:43:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:35.694+0000 7fb87fe00640 1 -- 172.21.15.16:0/2267859307 wait complete. 2024-09-21T03:43:36.570 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:36 smithi016 ceph-mon[30375]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:36.611 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T03:43:36.611 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:43:36.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T03:43:36.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:36 smithi123 ceph-mon[34673]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:36.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:36 smithi134 ceph-mon[35031]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:36.993 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:37.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.614+0000 7faef1800640 1 -- 172.21.15.16:0/355752202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec1092f0 msgr2=0x7faeec109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:37.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.614+0000 7faef1800640 1 --2- 172.21.15.16:0/355752202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec1092f0 0x7faeec109700 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faed8009900 tx=0x7faed802a410 comp rx=0 tx=0).stop 2024-09-21T03:43:37.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.614+0000 7faef1800640 1 -- 172.21.15.16:0/355752202 shutdown_connections 2024-09-21T03:43:37.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.614+0000 7faef1800640 1 --2- 172.21.15.16:0/355752202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeec109c40 0x7faeec110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.614+0000 7faef1800640 1 --2- 172.21.15.16:0/355752202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec1092f0 0x7faeec109700 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.614+0000 7faef1800640 1 --2- 172.21.15.16:0/355752202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeec1039b0 0x7faeec103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.614+0000 7faef1800640 1 -- 172.21.15.16:0/355752202 >> 172.21.15.16:0/355752202 conn(0x7faeec0ff400 msgr2=0x7faeec101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:37.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.615+0000 7faef1800640 1 -- 172.21.15.16:0/355752202 shutdown_connections 2024-09-21T03:43:37.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.615+0000 7faef1800640 1 -- 172.21.15.16:0/355752202 wait complete. 2024-09-21T03:43:37.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.616+0000 7faef1800640 1 Processor -- start 2024-09-21T03:43:37.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.616+0000 7faef1800640 1 -- start start 2024-09-21T03:43:37.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.616+0000 7faef1800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeec1039b0 0x7faeec19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:37.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.616+0000 7faef1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec1092f0 0x7faeec19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:37.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.616+0000 7faef1800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeec109c40 0x7faeec1a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:37.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.616+0000 7faef1800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faeec113900 con 0x7faeec1092f0 2024-09-21T03:43:37.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.616+0000 7faef1800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faeec113780 con 0x7faeec109c40 2024-09-21T03:43:37.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.616+0000 7faef1800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faeec113a80 con 0x7faeec1039b0 2024-09-21T03:43:37.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.617+0000 7faeeb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec1092f0 0x7faeec19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:37.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.617+0000 7faef0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeec109c40 0x7faeec1a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:37.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.617+0000 7faeebe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeec1039b0 0x7faeec19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:37.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.617+0000 7faef0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeec109c40 0x7faeec1a1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:57248/0 (socket says 172.21.15.16:57248) 2024-09-21T03:43:37.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.617+0000 7faeeb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec1092f0 0x7faeec19cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35300/0 (socket says 172.21.15.16:35300) 2024-09-21T03:43:37.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.617+0000 7faef0e00640 1 -- 172.21.15.16:0/983971056 learned_addr learned my addr 172.21.15.16:0/983971056 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:37.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.617+0000 7faef0e00640 1 -- 172.21.15.16:0/983971056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeec1039b0 msgr2=0x7faeec19c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:37.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.617+0000 7faef0e00640 1 --2- 172.21.15.16:0/983971056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeec1039b0 0x7faeec19c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.617+0000 7faef0e00640 1 -- 172.21.15.16:0/983971056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec1092f0 msgr2=0x7faeec19cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:37.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.617+0000 7faef0e00640 1 --2- 172.21.15.16:0/983971056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec1092f0 0x7faeec19cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.617+0000 7faef0e00640 1 -- 172.21.15.16:0/983971056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeec1a24a0 con 0x7faeec109c40 2024-09-21T03:43:37.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.617+0000 7faef0e00640 1 --2- 172.21.15.16:0/983971056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeec109c40 0x7faeec1a1d70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faee400eb10 tx=0x7faee400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:37.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.618+0000 7faee8c00640 1 -- 172.21.15.16:0/983971056 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee4004b20 con 0x7faeec109c40 2024-09-21T03:43:37.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.618+0000 7faee8c00640 1 -- 172.21.15.16:0/983971056 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faee4004cc0 con 0x7faeec109c40 2024-09-21T03:43:37.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.618+0000 7faee8c00640 1 -- 172.21.15.16:0/983971056 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee4010670 con 0x7faeec109c40 2024-09-21T03:43:37.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.618+0000 7faef1800640 1 -- 172.21.15.16:0/983971056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeec1a2790 con 0x7faeec109c40 2024-09-21T03:43:37.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.618+0000 7faef1800640 1 -- 172.21.15.16:0/983971056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeec1a8ef0 con 0x7faeec109c40 2024-09-21T03:43:37.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.619+0000 7faef1800640 1 -- 172.21.15.16:0/983971056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeec103dc0 con 0x7faeec109c40 2024-09-21T03:43:37.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.621+0000 7faee8c00640 1 -- 172.21.15.16:0/983971056 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7faee4005000 con 0x7faeec109c40 2024-09-21T03:43:37.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.623+0000 7faee8c00640 1 --2- 172.21.15.16:0/983971056 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faed00b6510 0x7faed00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:37.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.623+0000 7faee8c00640 1 -- 172.21.15.16:0/983971056 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8802+0+0 (secure 0 0 0) 0x7faee40e7ce0 con 0x7faeec109c40 2024-09-21T03:43:37.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.623+0000 7faeebe00640 1 --2- 172.21.15.16:0/983971056 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faed00b6510 0x7faed00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:37.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.624+0000 7faeebe00640 1 --2- 172.21.15.16:0/983971056 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faed00b6510 0x7faed00b8a20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faedc009bd0 tx=0x7faedc005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:37.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.625+0000 7faee8c00640 1 -- 172.21.15.16:0/983971056 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faee4056710 con 0x7faeec109c40 2024-09-21T03:43:37.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.863+0000 7faef1800640 1 -- 172.21.15.16:0/983971056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faeec19d660 con 0x7faeec109c40 2024-09-21T03:43:37.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.864+0000 7faee8c00640 1 -- 172.21.15.16:0/983971056 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7faeec19d660 con 0x7faeec109c40 2024-09-21T03:43:37.867 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T03:43:37.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.867+0000 7faef1800640 1 -- 172.21.15.16:0/983971056 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faed00b6510 msgr2=0x7faed00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:37.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.867+0000 7faef1800640 1 --2- 172.21.15.16:0/983971056 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faed00b6510 0x7faed00b8a20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faedc009bd0 tx=0x7faedc005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:37.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.867+0000 7faef1800640 1 -- 172.21.15.16:0/983971056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeec109c40 msgr2=0x7faeec1a1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:37.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.867+0000 7faef1800640 1 --2- 172.21.15.16:0/983971056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeec109c40 0x7faeec1a1d70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faee400eb10 tx=0x7faee400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.867+0000 7faef1800640 1 -- 172.21.15.16:0/983971056 shutdown_connections 2024-09-21T03:43:37.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.867+0000 7faef1800640 1 --2- 172.21.15.16:0/983971056 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faed00b6510 0x7faed00b8a20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.867+0000 7faef1800640 1 --2- 172.21.15.16:0/983971056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeec109c40 0x7faeec1a1d70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.867+0000 7faef1800640 1 --2- 172.21.15.16:0/983971056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec1092f0 0x7faeec19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.867+0000 7faef1800640 1 --2- 172.21.15.16:0/983971056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeec1039b0 0x7faeec19c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:37.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.867+0000 7faef1800640 1 -- 172.21.15.16:0/983971056 >> 172.21.15.16:0/983971056 conn(0x7faeec0ff400 msgr2=0x7faeec1089f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:37.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.867+0000 7faef1800640 1 -- 172.21.15.16:0/983971056 shutdown_connections 2024-09-21T03:43:37.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:37.867+0000 7faef1800640 1 -- 172.21.15.16:0/983971056 wait complete. 2024-09-21T03:43:38.621 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:38 smithi016 ceph-mon[30375]: pgmap v334: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:38.621 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:38 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/983971056' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:43:38.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:38 smithi123 ceph-mon[34673]: pgmap v334: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:38.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:38 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/983971056' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:43:38.658 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:43:38.658 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:43:38.658 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-21T03:43:38.658 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-21T03:43:38.658 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:43:38.658 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:43:38.658 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T03:43:38.658 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:43:38.658 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T03:43:38.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:38.658 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-21T03:43:38.695 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:43:38.695 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:43:38.695 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:43:38.695 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:38.695 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-21T03:43:38.695 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:38.695 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42125sec preferred_lft 42125sec 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:43:38.696 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-21T03:43:38.697 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T03:43:38.697 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:43:38.697 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-21T03:43:38.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-09-21T03:43:38.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-09-21T03:43:38.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-09-21T03:43:38.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T03:43:38.697 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-21T03:43:38.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:43:38.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:38 smithi134 ceph-mon[35031]: pgmap v334: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:38.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:38 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/983971056' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:43:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:43:39.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:39.132 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:43:39.192 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-21T03:43:39.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:39.195 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-21T03:43:39.256 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-21T03:43:39.256 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:43:39.256 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:43:39.256 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:43:39.256 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:43:39.256 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:43:39.257 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-21T03:43:39.257 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:43:39.257 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:43:39.257 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:43:39.257 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-21T03:43:39.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:43:39.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:43:39.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:39.870 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-21T03:43:39.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:39.906 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-21T03:43:39.983 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:43:39.983 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-21T03:43:39.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:43:39.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-21T03:43:39.983 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-21T03:43:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:43:40.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:43:40.164 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-21T03:43:40.164 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:43:40.164 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-21T03:43:40.164 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-21T03:43:40.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T03:43:40.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-21T03:43:40.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-21T03:43:40.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-21T03:43:40.165 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-21T03:43:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:43:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:43:40.421 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:43:40.421 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-21T03:43:40.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-21T03:43:40.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T03:43:40.421 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-21T03:43:40.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:43:40.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:43:40.589 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-21T03:43:40.590 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:43:40.590 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:43:40.627 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T03:43:40.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:43:40.628 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:43:40.633 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:40 smithi016 ceph-mon[30375]: pgmap v335: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:40.667 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-21T03:43:40.765 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T03:43:40.802 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-21T03:43:40.802 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-21T03:43:40.802 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-21T03:43:40.802 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-21T03:43:40.802 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-21T03:43:40.802 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-21T03:43:40.802 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-21T03:43:40.802 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-21T03:43:40.802 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-21T03:43:40.803 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-21T03:43:40.803 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-21T03:43:40.803 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-21T03:43:40.803 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:43:40.803 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-21T03:43:40.803 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-21T03:43:40.803 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-21T03:43:40.803 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-21T03:43:40.803 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-21T03:43:40.803 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-21T03:43:40.803 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T03:43:40.803 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T03:43:40.803 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T03:43:40.804 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T03:43:40.804 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-21T03:43:40.804 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-21T03:43:40.804 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay 2024-09-21T03:43:40.804 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/197b6b25ec9119083039650fb8e05e3115b096f8acf6f89cc839455b3bf0d2e8/merged 2024-09-21T03:43:40.804 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e0c180f0765563e35866af2f6bf2a6cf9bca24856153f41a7fae569c8419e0d4/merged 2024-09-21T03:43:40.804 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/0 2024-09-21T03:43:40.804 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:43:40.804 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/8d47583fcb9874bbccbdd0be253f019a7ffcf5e2adbd441f8d9362bc8c64644a/merged 2024-09-21T03:43:40.804 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/58b6e9496a21f02695dd10fc7682704ad91ebd0d0c47ead3b528e29f6c6061f1/merged 2024-09-21T03:43:40.804 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/7623dede57dbec52b518d2161e66d8c8962fd2c47e7cb77f9fcbb3f5724c5b9e/merged 2024-09-21T03:43:40.804 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/93f07b3c24b95bd4a17077fc65857240f525a1e6f9ac9375b1bd98d0b81f02fb/merged 2024-09-21T03:43:40.804 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/9af4270ab5ce400544b75e62870bcf9f4ea4a0aff02ae5280038c08bc1a53d83/merged 2024-09-21T03:43:40.804 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/ed0c68c96f1052185599f37212236ea19915bf44dc7ae7b827516349ef64f67e/merged 2024-09-21T03:43:40.805 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/20fe7322123f5cab9c3753a1258c0c78b13084afe9abf29a456514f9097880e5/merged 2024-09-21T03:43:40.805 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/194775f0c3181c2717c75400ddde72783e7172d300ef29114d64775d77ba5cea/merged 2024-09-21T03:43:40.805 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-21T03:43:40.805 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:43:40.805 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:43:40.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:40.806 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-21T03:43:40.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:40 smithi134 ceph-mon[35031]: pgmap v335: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:40.881 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T03:43:40.882 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-21T03:43:40.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:40 smithi123 ceph-mon[34673]: pgmap v335: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:40.926 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-21T03:43:40.982 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T03:43:41.018 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-21T03:43:41.019 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-21T03:43:41.019 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-21T03:43:41.019 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-21T03:43:41.019 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-21T03:43:41.019 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-21T03:43:41.019 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-21T03:43:41.019 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-21T03:43:41.019 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-21T03:43:41.019 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-21T03:43:41.019 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-21T03:43:41.019 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-21T03:43:41.020 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:43:41.020 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-21T03:43:41.020 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-21T03:43:41.020 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-21T03:43:41.020 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-21T03:43:41.020 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-21T03:43:41.020 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-21T03:43:41.020 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T03:43:41.020 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T03:43:41.020 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T03:43:41.020 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T03:43:41.020 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-21T03:43:41.020 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-21T03:43:41.021 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay 2024-09-21T03:43:41.021 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/197b6b25ec9119083039650fb8e05e3115b096f8acf6f89cc839455b3bf0d2e8/merged 2024-09-21T03:43:41.021 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e0c180f0765563e35866af2f6bf2a6cf9bca24856153f41a7fae569c8419e0d4/merged 2024-09-21T03:43:41.021 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/0 2024-09-21T03:43:41.021 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:43:41.021 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/8d47583fcb9874bbccbdd0be253f019a7ffcf5e2adbd441f8d9362bc8c64644a/merged 2024-09-21T03:43:41.021 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/58b6e9496a21f02695dd10fc7682704ad91ebd0d0c47ead3b528e29f6c6061f1/merged 2024-09-21T03:43:41.021 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/7623dede57dbec52b518d2161e66d8c8962fd2c47e7cb77f9fcbb3f5724c5b9e/merged 2024-09-21T03:43:41.021 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/93f07b3c24b95bd4a17077fc65857240f525a1e6f9ac9375b1bd98d0b81f02fb/merged 2024-09-21T03:43:41.021 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/9af4270ab5ce400544b75e62870bcf9f4ea4a0aff02ae5280038c08bc1a53d83/merged 2024-09-21T03:43:41.021 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/ed0c68c96f1052185599f37212236ea19915bf44dc7ae7b827516349ef64f67e/merged 2024-09-21T03:43:41.021 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/20fe7322123f5cab9c3753a1258c0c78b13084afe9abf29a456514f9097880e5/merged 2024-09-21T03:43:41.021 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/194775f0c3181c2717c75400ddde72783e7172d300ef29114d64775d77ba5cea/merged 2024-09-21T03:43:41.021 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-21T03:43:41.021 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:43:41.022 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:43:41.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:41.022 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-21T03:43:41.202 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:2024-09-21T03:43:41.200+0000 7fb5a5e3b480 -1 init, newargv = 0x7fb58c004c60 newargc=15 2024-09-21T03:43:41.202 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[75087]: starting ceph client 2024-09-21T03:43:41.212 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[75087]: starting fuse 2024-09-21T03:43:42.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-21T03:43:42.265 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T03:43:42.303 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-21T03:43:42.303 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-21T03:43:42.304 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-21T03:43:42.304 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-21T03:43:42.304 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-21T03:43:42.304 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-21T03:43:42.304 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-21T03:43:42.304 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-21T03:43:42.304 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-21T03:43:42.304 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-21T03:43:42.304 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-21T03:43:42.304 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-21T03:43:42.304 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:43:42.304 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-21T03:43:42.304 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-21T03:43:42.304 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-21T03:43:42.304 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-21T03:43:42.305 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-21T03:43:42.305 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-21T03:43:42.305 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T03:43:42.305 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T03:43:42.305 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T03:43:42.305 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T03:43:42.305 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-21T03:43:42.305 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-21T03:43:42.305 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay 2024-09-21T03:43:42.305 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/197b6b25ec9119083039650fb8e05e3115b096f8acf6f89cc839455b3bf0d2e8/merged 2024-09-21T03:43:42.305 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e0c180f0765563e35866af2f6bf2a6cf9bca24856153f41a7fae569c8419e0d4/merged 2024-09-21T03:43:42.305 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/0 2024-09-21T03:43:42.305 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:43:42.305 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/8d47583fcb9874bbccbdd0be253f019a7ffcf5e2adbd441f8d9362bc8c64644a/merged 2024-09-21T03:43:42.305 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/58b6e9496a21f02695dd10fc7682704ad91ebd0d0c47ead3b528e29f6c6061f1/merged 2024-09-21T03:43:42.306 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/7623dede57dbec52b518d2161e66d8c8962fd2c47e7cb77f9fcbb3f5724c5b9e/merged 2024-09-21T03:43:42.306 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/93f07b3c24b95bd4a17077fc65857240f525a1e6f9ac9375b1bd98d0b81f02fb/merged 2024-09-21T03:43:42.306 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/9af4270ab5ce400544b75e62870bcf9f4ea4a0aff02ae5280038c08bc1a53d83/merged 2024-09-21T03:43:42.306 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/ed0c68c96f1052185599f37212236ea19915bf44dc7ae7b827516349ef64f67e/merged 2024-09-21T03:43:42.306 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/20fe7322123f5cab9c3753a1258c0c78b13084afe9abf29a456514f9097880e5/merged 2024-09-21T03:43:42.306 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/194775f0c3181c2717c75400ddde72783e7172d300ef29114d64775d77ba5cea/merged 2024-09-21T03:43:42.306 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-21T03:43:42.306 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:43:42.306 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:43:42.306 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-21T03:43:42.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:42.307 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-21T03:43:42.385 INFO:teuthology.orchestra.run.smithi016.stdout:96 2024-09-21T03:43:42.386 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [96] 2024-09-21T03:43:42.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-21T03:43:42.386 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-21T03:43:42.386 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-21T03:43:42.386 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-21T03:43:42.386 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-21T03:43:42.386 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-21T03:43:42.386 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-21T03:43:42.386 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-21T03:43:42.386 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-21T03:43:42.386 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-21T03:43:42.386 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-21T03:43:42.386 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-21T03:43:42.386 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-21T03:43:42.387 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-21T03:43:42.387 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-21T03:43:42.387 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-21T03:43:42.387 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-21T03:43:42.387 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T03:43:42.387 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T03:43:42.387 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T03:43:42.387 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-21T03:43:42.387 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-21T03:43:42.387 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-21T03:43:42.387 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T03:43:42.387 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-21T03:43:42.387 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.admin")) 2024-09-21T03:43:42.387 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-21T03:43:42.552 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.admin.75087.asok 2024-09-21T03:43:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:43:42.564 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:42 smithi016 ceph-mon[30375]: pgmap v336: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:42.566 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75087.asok 2024-09-21T03:43:42.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:42.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75087.asok status 2024-09-21T03:43:42.818 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:43:42.818 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-21T03:43:42.818 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T03:43:42.818 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:43:42.818 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "admin", 2024-09-21T03:43:42.819 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-21T03:43:42.819 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-21T03:43:42.819 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "75087", 2024-09-21T03:43:42.819 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-21T03:43:42.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:43:42.819 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-21T03:43:42.819 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-21T03:43:42.819 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 14835, 2024-09-21T03:43:42.819 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-21T03:43:42.819 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-21T03:43:42.819 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-21T03:43:42.819 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 14835 2024-09-21T03:43:42.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:43:42.819 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-21T03:43:42.819 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-21T03:43:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-21T03:43:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2243176127 2024-09-21T03:43:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:43:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:43:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-21T03:43:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-21T03:43:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-21T03:43:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2243176127 2024-09-21T03:43:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:43:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.14835 192.168.0.1:0/2243176127", 2024-09-21T03:43:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/2243176127", 2024-09-21T03:43:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-21T03:43:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 17, 2024-09-21T03:43:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 82, 2024-09-21T03:43:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-21T03:43:42.821 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-21T03:43:42.821 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-21T03:43:42.821 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:43:42.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:42.837 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-21T03:43:42.837 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:43:42.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:42 smithi134 ceph-mon[35031]: pgmap v336: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:42.873 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-21T03:43:42.873 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:43:42.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:42.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:43:42.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:42 smithi123 ceph-mon[34673]: pgmap v336: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:42.971 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-21T03:43:43.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:43:43.079 INFO:teuthology.orchestra.run.smithi016.stdout:# file: . 2024-09-21T03:43:43.079 INFO:teuthology.orchestra.run.smithi016.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-21T03:43:43.080 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:43:43.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:43:43.088 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:43:43.088 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:43:43.124 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T03:43:43.546 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.197+0000 7f6fde000640 1 -- 172.21.15.16:0/3918974169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd81039b0 msgr2=0x7f6fd8103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.197+0000 7f6fde000640 1 --2- 172.21.15.16:0/3918974169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd81039b0 0x7f6fd8103dc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009900 tx=0x7f6fcc02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.197+0000 7f6fde000640 1 -- 172.21.15.16:0/3918974169 shutdown_connections 2024-09-21T03:43:44.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.197+0000 7f6fde000640 1 --2- 172.21.15.16:0/3918974169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd8109c40 0x7f6fd8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.197+0000 7f6fde000640 1 --2- 172.21.15.16:0/3918974169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd81092f0 0x7f6fd8109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.197+0000 7f6fde000640 1 --2- 172.21.15.16:0/3918974169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd81039b0 0x7f6fd8103dc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.197+0000 7f6fde000640 1 -- 172.21.15.16:0/3918974169 >> 172.21.15.16:0/3918974169 conn(0x7f6fd80ff400 msgr2=0x7f6fd8101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:44.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.197+0000 7f6fde000640 1 -- 172.21.15.16:0/3918974169 shutdown_connections 2024-09-21T03:43:44.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.197+0000 7f6fde000640 1 -- 172.21.15.16:0/3918974169 wait complete. 2024-09-21T03:43:44.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.198+0000 7f6fde000640 1 Processor -- start 2024-09-21T03:43:44.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.198+0000 7f6fde000640 1 -- start start 2024-09-21T03:43:44.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.198+0000 7f6fde000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd81039b0 0x7f6fd819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:44.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.198+0000 7f6fde000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd81092f0 0x7f6fd819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.198+0000 7f6fde000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd8109c40 0x7f6fd81a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.198+0000 7f6fde000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd8113950 con 0x7f6fd8109c40 2024-09-21T03:43:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.198+0000 7f6fde000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd81137d0 con 0x7f6fd81092f0 2024-09-21T03:43:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.198+0000 7f6fde000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd8113ad0 con 0x7f6fd81039b0 2024-09-21T03:43:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.199+0000 7f6fdcc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd81039b0 0x7f6fd819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.199+0000 7f6fdcc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd81039b0 0x7f6fd819c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:44596/0 (socket says 172.21.15.16:44596) 2024-09-21T03:43:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.199+0000 7f6fdcc00640 1 -- 172.21.15.16:0/2903441585 learned_addr learned my addr 172.21.15.16:0/2903441585 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.199+0000 7f6fdd600640 1 --2- 172.21.15.16:0/2903441585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd8109c40 0x7f6fd81a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.199+0000 7f6fd7e00640 1 --2- 172.21.15.16:0/2903441585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd81092f0 0x7f6fd819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.199+0000 7f6fdcc00640 1 -- 172.21.15.16:0/2903441585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd81092f0 msgr2=0x7f6fd819cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.199+0000 7f6fdcc00640 1 --2- 172.21.15.16:0/2903441585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd81092f0 0x7f6fd819cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.199+0000 7f6fdcc00640 1 -- 172.21.15.16:0/2903441585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd8109c40 msgr2=0x7f6fd81a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.199+0000 7f6fdcc00640 1 --2- 172.21.15.16:0/2903441585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd8109c40 0x7f6fd81a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.199+0000 7f6fdcc00640 1 -- 172.21.15.16:0/2903441585 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd81a24b0 con 0x7f6fd81039b0 2024-09-21T03:43:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.200+0000 7f6fdcc00640 1 --2- 172.21.15.16:0/2903441585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd81039b0 0x7f6fd819c540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc005ab0 tx=0x7f6fcc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.200+0000 7f6fd5600640 1 -- 172.21.15.16:0/2903441585 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc03f070 con 0x7f6fd81039b0 2024-09-21T03:43:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.200+0000 7f6fde000640 1 -- 172.21.15.16:0/2903441585 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd81a2740 con 0x7f6fd81039b0 2024-09-21T03:43:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.200+0000 7f6fd5600640 1 -- 172.21.15.16:0/2903441585 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fcc031b60 con 0x7f6fd81039b0 2024-09-21T03:43:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.200+0000 7f6fd5600640 1 -- 172.21.15.16:0/2903441585 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc035030 con 0x7f6fd81039b0 2024-09-21T03:43:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.200+0000 7f6fde000640 1 -- 172.21.15.16:0/2903441585 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd81a8f00 con 0x7f6fd81039b0 2024-09-21T03:43:44.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.201+0000 7f6fde000640 1 -- 172.21.15.16:0/2903441585 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd8104c30 con 0x7f6fd81039b0 2024-09-21T03:43:44.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.204+0000 7f6fd5600640 1 -- 172.21.15.16:0/2903441585 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f6fcc042070 con 0x7f6fd81039b0 2024-09-21T03:43:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.205+0000 7f6fd5600640 1 --2- 172.21.15.16:0/2903441585 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6fc00b6510 0x7f6fc00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.206+0000 7f6fd5600640 1 -- 172.21.15.16:0/2903441585 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8802+0+0 (secure 0 0 0) 0x7f6fcc1084c0 con 0x7f6fd81039b0 2024-09-21T03:43:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.206+0000 7f6fd7e00640 1 --2- 172.21.15.16:0/2903441585 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6fc00b6510 0x7f6fc00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:44.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.206+0000 7f6fd7e00640 1 --2- 172.21.15.16:0/2903441585 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6fc00b6510 0x7f6fc00b8a20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8105a60 tx=0x7f6fc4005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:44.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.207+0000 7f6fd5600640 1 -- 172.21.15.16:0/2903441585 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6fcc0ceab0 con 0x7f6fd81039b0 2024-09-21T03:43:44.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.423+0000 7f6fde000640 1 -- 172.21.15.16:0/2903441585 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6fd81a90d0 con 0x7f6fd81039b0 2024-09-21T03:43:44.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.424+0000 7f6fd5600640 1 -- 172.21.15.16:0/2903441585 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v82) ==== 81+0+586 (secure 0 0 0) 0x7f6fcc0d2e90 con 0x7f6fd81039b0 2024-09-21T03:43:44.426 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/557742783 2024-09-22T03:34:53.217208+0000 2024-09-21T03:43:44.426 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3238676548 2024-09-22T03:34:53.217208+0000 2024-09-21T03:43:44.426 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/500178737 2024-09-22T03:34:53.217208+0000 2024-09-21T03:43:44.426 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2357336233 2024-09-22T03:34:22.549156+0000 2024-09-21T03:43:44.426 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/428542610 2024-09-22T03:34:22.549156+0000 2024-09-21T03:43:44.426 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/428542610 2024-09-22T03:34:22.549156+0000 2024-09-21T03:43:44.426 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3806844703 2024-09-22T03:34:22.549156+0000 2024-09-21T03:43:44.426 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2351581621 2024-09-22T03:34:53.217208+0000 2024-09-21T03:43:44.427 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/500178737 2024-09-22T03:34:53.217208+0000 2024-09-21T03:43:44.427 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/355731724 2024-09-22T03:34:22.549156+0000 2024-09-21T03:43:44.427 INFO:teuthology.orchestra.run.smithi016.stderr:listed 10 entries 2024-09-21T03:43:44.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.426+0000 7f6fde000640 1 -- 172.21.15.16:0/2903441585 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6fc00b6510 msgr2=0x7f6fc00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:44.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.426+0000 7f6fde000640 1 --2- 172.21.15.16:0/2903441585 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6fc00b6510 0x7f6fc00b8a20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8105a60 tx=0x7f6fc4005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:44.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.426+0000 7f6fde000640 1 -- 172.21.15.16:0/2903441585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd81039b0 msgr2=0x7f6fd819c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:44.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.426+0000 7f6fde000640 1 --2- 172.21.15.16:0/2903441585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd81039b0 0x7f6fd819c540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc005ab0 tx=0x7f6fcc003c10 comp rx=0 tx=0).stop 2024-09-21T03:43:44.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.427+0000 7f6fde000640 1 -- 172.21.15.16:0/2903441585 shutdown_connections 2024-09-21T03:43:44.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.427+0000 7f6fde000640 1 --2- 172.21.15.16:0/2903441585 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6fc00b6510 0x7f6fc00b8a20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.427+0000 7f6fde000640 1 --2- 172.21.15.16:0/2903441585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd8109c40 0x7f6fd81a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.427+0000 7f6fde000640 1 --2- 172.21.15.16:0/2903441585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd81092f0 0x7f6fd819cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.427+0000 7f6fde000640 1 --2- 172.21.15.16:0/2903441585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd81039b0 0x7f6fd819c540 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:44.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.427+0000 7f6fde000640 1 -- 172.21.15.16:0/2903441585 >> 172.21.15.16:0/2903441585 conn(0x7f6fd80ff400 msgr2=0x7f6fd81089b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:44.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.427+0000 7f6fde000640 1 -- 172.21.15.16:0/2903441585 shutdown_connections 2024-09-21T03:43:44.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:44.427+0000 7f6fde000640 1 -- 172.21.15.16:0/2903441585 wait complete. 2024-09-21T03:43:44.708 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:44 smithi123 ceph-mon[34673]: pgmap v337: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T03:43:44.832 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:44 smithi016 ceph-mon[30375]: pgmap v337: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T03:43:44.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:44 smithi134 ceph-mon[35031]: pgmap v337: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T03:43:45.232 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:43:45.233 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:43:45.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T03:43:45.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:45 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2903441585' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:43:45.694 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:45.728 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:45 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2903441585' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:43:45.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:45 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2903441585' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:43:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.299+0000 7f1421a00640 1 -- 172.21.15.16:0/1550574164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f141c1039b0 msgr2=0x7f141c103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.299+0000 7f1421a00640 1 --2- 172.21.15.16:0/1550574164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f141c1039b0 0x7f141c103dc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f140c009900 tx=0x7f140c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.299+0000 7f1421a00640 1 -- 172.21.15.16:0/1550574164 shutdown_connections 2024-09-21T03:43:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.299+0000 7f1421a00640 1 --2- 172.21.15.16:0/1550574164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141c109c40 0x7f141c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.299+0000 7f1421a00640 1 --2- 172.21.15.16:0/1550574164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f141c1092f0 0x7f141c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.299+0000 7f1421a00640 1 --2- 172.21.15.16:0/1550574164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f141c1039b0 0x7f141c103dc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.299+0000 7f1421a00640 1 -- 172.21.15.16:0/1550574164 >> 172.21.15.16:0/1550574164 conn(0x7f141c0ff400 msgr2=0x7f141c101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.299+0000 7f1421a00640 1 -- 172.21.15.16:0/1550574164 shutdown_connections 2024-09-21T03:43:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.299+0000 7f1421a00640 1 -- 172.21.15.16:0/1550574164 wait complete. 2024-09-21T03:43:46.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.300+0000 7f1421a00640 1 Processor -- start 2024-09-21T03:43:46.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.300+0000 7f1421a00640 1 -- start start 2024-09-21T03:43:46.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.300+0000 7f1421a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141c1039b0 0x7f141c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.301+0000 7f1421a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f141c1092f0 0x7f141c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.301+0000 7f1421a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f141c109c40 0x7f141c1a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.301+0000 7f1421a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f141c113950 con 0x7f141c1039b0 2024-09-21T03:43:46.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.301+0000 7f1421a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f141c1137d0 con 0x7f141c109c40 2024-09-21T03:43:46.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.301+0000 7f1421a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f141c113ad0 con 0x7f141c1092f0 2024-09-21T03:43:46.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.301+0000 7f141be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141c1039b0 0x7f141c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.301+0000 7f1421000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f141c109c40 0x7f141c1a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.301+0000 7f141be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141c1039b0 0x7f141c19c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35698/0 (socket says 172.21.15.16:35698) 2024-09-21T03:43:46.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.301+0000 7f141be00640 1 -- 172.21.15.16:0/2717218896 learned_addr learned my addr 172.21.15.16:0/2717218896 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:46.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.301+0000 7f141b400640 1 --2- 172.21.15.16:0/2717218896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f141c1092f0 0x7f141c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.301+0000 7f1421000640 1 -- 172.21.15.16:0/2717218896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f141c1092f0 msgr2=0x7f141c19caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.301+0000 7f1421000640 1 --2- 172.21.15.16:0/2717218896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f141c1092f0 0x7f141c19caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.301+0000 7f1421000640 1 -- 172.21.15.16:0/2717218896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141c1039b0 msgr2=0x7f141c19c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.301+0000 7f1421000640 1 --2- 172.21.15.16:0/2717218896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141c1039b0 0x7f141c19c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.301+0000 7f1421000640 1 -- 172.21.15.16:0/2717218896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141c1a2470 con 0x7f141c109c40 2024-09-21T03:43:46.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.302+0000 7f1421000640 1 --2- 172.21.15.16:0/2717218896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f141c109c40 0x7f141c1a1d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f141400ec50 tx=0x7f141400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:46.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.302+0000 7f1418c00640 1 -- 172.21.15.16:0/2717218896 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414003b70 con 0x7f141c109c40 2024-09-21T03:43:46.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.302+0000 7f1418c00640 1 -- 172.21.15.16:0/2717218896 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1414003d10 con 0x7f141c109c40 2024-09-21T03:43:46.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.302+0000 7f1421a00640 1 -- 172.21.15.16:0/2717218896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141c1a2760 con 0x7f141c109c40 2024-09-21T03:43:46.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.302+0000 7f1418c00640 1 -- 172.21.15.16:0/2717218896 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414010830 con 0x7f141c109c40 2024-09-21T03:43:46.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.302+0000 7f1421a00640 1 -- 172.21.15.16:0/2717218896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141c1a8ec0 con 0x7f141c109c40 2024-09-21T03:43:46.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.303+0000 7f1421a00640 1 -- 172.21.15.16:0/2717218896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f141c103dc0 con 0x7f141c109c40 2024-09-21T03:43:46.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.305+0000 7f1418c00640 1 -- 172.21.15.16:0/2717218896 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f1414004b40 con 0x7f141c109c40 2024-09-21T03:43:46.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.307+0000 7f1418c00640 1 --2- 172.21.15.16:0/2717218896 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f14000b6590 0x7f14000b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:46.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.307+0000 7f1418c00640 1 -- 172.21.15.16:0/2717218896 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8802+0+0 (secure 0 0 0) 0x7f14140e7e20 con 0x7f141c109c40 2024-09-21T03:43:46.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.307+0000 7f141be00640 1 --2- 172.21.15.16:0/2717218896 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f14000b6590 0x7f14000b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:46.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.308+0000 7f141be00640 1 --2- 172.21.15.16:0/2717218896 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f14000b6590 0x7f14000b8aa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f140c0096c0 tx=0x7f140c005a20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:46.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.309+0000 7f1418c00640 1 -- 172.21.15.16:0/2717218896 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f14140ae410 con 0x7f141c109c40 2024-09-21T03:43:46.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.520+0000 7f1421a00640 1 -- 172.21.15.16:0/2717218896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f141c19d630 con 0x7f141c109c40 2024-09-21T03:43:46.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.520+0000 7f1418c00640 1 -- 172.21.15.16:0/2717218896 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v82) ==== 81+0+586 (secure 0 0 0) 0x7f14140b27f0 con 0x7f141c109c40 2024-09-21T03:43:46.522 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/557742783 2024-09-22T03:34:53.217208+0000 2024-09-21T03:43:46.522 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3238676548 2024-09-22T03:34:53.217208+0000 2024-09-21T03:43:46.522 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/500178737 2024-09-22T03:34:53.217208+0000 2024-09-21T03:43:46.522 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2357336233 2024-09-22T03:34:22.549156+0000 2024-09-21T03:43:46.523 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/428542610 2024-09-22T03:34:22.549156+0000 2024-09-21T03:43:46.523 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/428542610 2024-09-22T03:34:22.549156+0000 2024-09-21T03:43:46.523 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3806844703 2024-09-22T03:34:22.549156+0000 2024-09-21T03:43:46.523 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2351581621 2024-09-22T03:34:53.217208+0000 2024-09-21T03:43:46.523 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/500178737 2024-09-22T03:34:53.217208+0000 2024-09-21T03:43:46.523 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/355731724 2024-09-22T03:34:22.549156+0000 2024-09-21T03:43:46.523 INFO:teuthology.orchestra.run.smithi016.stderr:listed 10 entries 2024-09-21T03:43:46.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.523+0000 7f1421a00640 1 -- 172.21.15.16:0/2717218896 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f14000b6590 msgr2=0x7f14000b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.523+0000 7f1421a00640 1 --2- 172.21.15.16:0/2717218896 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f14000b6590 0x7f14000b8aa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f140c0096c0 tx=0x7f140c005a20 comp rx=0 tx=0).stop 2024-09-21T03:43:46.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.523+0000 7f1421a00640 1 -- 172.21.15.16:0/2717218896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f141c109c40 msgr2=0x7f141c1a1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:46.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.523+0000 7f1421a00640 1 --2- 172.21.15.16:0/2717218896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f141c109c40 0x7f141c1a1d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f141400ec50 tx=0x7f141400c6a0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.523+0000 7f1421a00640 1 -- 172.21.15.16:0/2717218896 shutdown_connections 2024-09-21T03:43:46.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.523+0000 7f1421a00640 1 --2- 172.21.15.16:0/2717218896 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f14000b6590 0x7f14000b8aa0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.523+0000 7f1421a00640 1 --2- 172.21.15.16:0/2717218896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f141c109c40 0x7f141c1a1d40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.523+0000 7f1421a00640 1 --2- 172.21.15.16:0/2717218896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f141c1092f0 0x7f141c19caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.523+0000 7f1421a00640 1 --2- 172.21.15.16:0/2717218896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141c1039b0 0x7f141c19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:46.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.523+0000 7f1421a00640 1 -- 172.21.15.16:0/2717218896 >> 172.21.15.16:0/2717218896 conn(0x7f141c0ff400 msgr2=0x7f141c108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:46.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.523+0000 7f1421a00640 1 -- 172.21.15.16:0/2717218896 shutdown_connections 2024-09-21T03:43:46.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:46.523+0000 7f1421a00640 1 -- 172.21.15.16:0/2717218896 wait complete. 2024-09-21T03:43:46.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:46 smithi134 ceph-mon[35031]: pgmap v338: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T03:43:46.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:46 smithi123 ceph-mon[34673]: pgmap v338: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T03:43:46.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:46 smithi016 ceph-mon[30375]: pgmap v338: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T03:43:47.348 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-21T03:43:47.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:47.349 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:43:47.405 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[75087]: fuse finished with error 0 and tester_r 0 2024-09-21T03:43:47.408 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:43:47.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:47 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2717218896' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:43:47.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:47 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2717218896' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:43:47.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:47 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2717218896' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:43:48.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:48 smithi134 ceph-mon[35031]: pgmap v339: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T03:43:48.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:48 smithi123 ceph-mon[34673]: pgmap v339: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T03:43:48.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:48 smithi016 ceph-mon[30375]: pgmap v339: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T03:43:50.713 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:50 smithi016 ceph-mon[30375]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T03:43:50.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:50 smithi134 ceph-mon[35031]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T03:43:50.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:50 smithi123 ceph-mon[34673]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T03:43:52.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:52 smithi134 ceph-mon[35031]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T03:43:52.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:52 smithi123 ceph-mon[34673]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T03:43:52.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:52 smithi016 ceph-mon[30375]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T03:43:53.511 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-21T03:43:53.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:43:53.511 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-21T03:43:53.587 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:43:53.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-09-21T03:43:53.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T03:43:53.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]} : dispatch 2024-09-21T03:43:53.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]} : dispatch 2024-09-21T03:43:53.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]} : dispatch 2024-09-21T03:43:53.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-09-21T03:43:53.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]} : dispatch 2024-09-21T03:43:53.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]} : dispatch 2024-09-21T03:43:53.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]} : dispatch 2024-09-21T03:43:53.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]} : dispatch 2024-09-21T03:43:53.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-09-21T03:43:53.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T03:43:53.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]} : dispatch 2024-09-21T03:43:53.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]} : dispatch 2024-09-21T03:43:53.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]} : dispatch 2024-09-21T03:43:53.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-09-21T03:43:53.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]} : dispatch 2024-09-21T03:43:53.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]} : dispatch 2024-09-21T03:43:53.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]} : dispatch 2024-09-21T03:43:53.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]} : dispatch 2024-09-21T03:43:53.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-09-21T03:43:53.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T03:43:53.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]} : dispatch 2024-09-21T03:43:53.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]} : dispatch 2024-09-21T03:43:53.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]} : dispatch 2024-09-21T03:43:53.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-09-21T03:43:53.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]} : dispatch 2024-09-21T03:43:53.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]} : dispatch 2024-09-21T03:43:53.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]} : dispatch 2024-09-21T03:43:53.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]} : dispatch 2024-09-21T03:43:53.989 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.596+0000 7fe446c00640 1 -- 172.21.15.16:0/2304557145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe440105110 msgr2=0x7fe44010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.596+0000 7fe446c00640 1 --2- 172.21.15.16:0/2304557145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe440105110 0x7fe44010d660 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009900 tx=0x7fe42c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:43:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.596+0000 7fe446c00640 1 -- 172.21.15.16:0/2304557145 shutdown_connections 2024-09-21T03:43:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.596+0000 7fe446c00640 1 --2- 172.21.15.16:0/2304557145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe44010dc20 0x7fe440110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.596+0000 7fe446c00640 1 --2- 172.21.15.16:0/2304557145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe440105110 0x7fe44010d660 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.596+0000 7fe446c00640 1 --2- 172.21.15.16:0/2304557145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4401047c0 0x7fe440104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.596+0000 7fe446c00640 1 -- 172.21.15.16:0/2304557145 >> 172.21.15.16:0/2304557145 conn(0x7fe4400fe2c0 msgr2=0x7fe440100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.596+0000 7fe446c00640 1 -- 172.21.15.16:0/2304557145 shutdown_connections 2024-09-21T03:43:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.596+0000 7fe446c00640 1 -- 172.21.15.16:0/2304557145 wait complete. 2024-09-21T03:43:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.597+0000 7fe446c00640 1 Processor -- start 2024-09-21T03:43:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.598+0000 7fe446c00640 1 -- start start 2024-09-21T03:43:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.598+0000 7fe446c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4401047c0 0x7fe44019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.598+0000 7fe446c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe440105110 0x7fe44019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.598+0000 7fe446c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe44010dc20 0x7fe44019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.598+0000 7fe446c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe440113840 con 0x7fe44010dc20 2024-09-21T03:43:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.598+0000 7fe446c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401136c0 con 0x7fe4401047c0 2024-09-21T03:43:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.598+0000 7fe446c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401139c0 con 0x7fe440105110 2024-09-21T03:43:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.598+0000 7fe446200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe44010dc20 0x7fe44019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.598+0000 7fe446200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe44010dc20 0x7fe44019fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37190/0 (socket says 172.21.15.16:37190) 2024-09-21T03:43:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.599+0000 7fe446200640 1 -- 172.21.15.16:0/1188375067 learned_addr learned my addr 172.21.15.16:0/1188375067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.599+0000 7fe444e00640 1 --2- 172.21.15.16:0/1188375067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe440105110 0x7fe44019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.599+0000 7fe445800640 1 --2- 172.21.15.16:0/1188375067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4401047c0 0x7fe44019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:54.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.599+0000 7fe446200640 1 -- 172.21.15.16:0/1188375067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe440105110 msgr2=0x7fe44019a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:54.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.599+0000 7fe446200640 1 --2- 172.21.15.16:0/1188375067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe440105110 0x7fe44019a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:54.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.599+0000 7fe446200640 1 -- 172.21.15.16:0/1188375067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4401047c0 msgr2=0x7fe44019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:54.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.599+0000 7fe446200640 1 --2- 172.21.15.16:0/1188375067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4401047c0 0x7fe44019a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:54.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.599+0000 7fe446200640 1 -- 172.21.15.16:0/1188375067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4401a02e0 con 0x7fe44010dc20 2024-09-21T03:43:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.599+0000 7fe446200640 1 --2- 172.21.15.16:0/1188375067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe44010dc20 0x7fe44019fbb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe43800eaa0 tx=0x7fe43800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.600+0000 7fe43e000640 1 -- 172.21.15.16:0/1188375067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43800cdb0 con 0x7fe44010dc20 2024-09-21T03:43:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.600+0000 7fe43e000640 1 -- 172.21.15.16:0/1188375067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe438005020 con 0x7fe44010dc20 2024-09-21T03:43:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.600+0000 7fe446c00640 1 -- 172.21.15.16:0/1188375067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4401a05d0 con 0x7fe44010dc20 2024-09-21T03:43:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.600+0000 7fe43e000640 1 -- 172.21.15.16:0/1188375067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4380116d0 con 0x7fe44010dc20 2024-09-21T03:43:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.600+0000 7fe446c00640 1 -- 172.21.15.16:0/1188375067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4401a6d30 con 0x7fe44010dc20 2024-09-21T03:43:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.601+0000 7fe446c00640 1 -- 172.21.15.16:0/1188375067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe440104bd0 con 0x7fe44010dc20 2024-09-21T03:43:54.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.604+0000 7fe43e000640 1 -- 172.21.15.16:0/1188375067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fe4380051d0 con 0x7fe44010dc20 2024-09-21T03:43:54.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.605+0000 7fe43e000640 1 --2- 172.21.15.16:0/1188375067 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe4280b6510 0x7fe4280b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:54.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.605+0000 7fe43e000640 1 -- 172.21.15.16:0/1188375067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7fe4380e7af0 con 0x7fe44010dc20 2024-09-21T03:43:54.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.605+0000 7fe445800640 1 --2- 172.21.15.16:0/1188375067 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe4280b6510 0x7fe4280b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:54.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.606+0000 7fe445800640 1 --2- 172.21.15.16:0/1188375067 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe4280b6510 0x7fe4280b8a20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe43400bbe0 tx=0x7fe43400b390 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:54.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.607+0000 7fe43e000640 1 -- 172.21.15.16:0/1188375067 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe438016030 con 0x7fe44010dc20 2024-09-21T03:43:54.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:54 smithi016 ceph-mon[30375]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T03:43:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-21T03:43:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T03:43:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-09-21T03:43:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-21T03:43:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-09-21T03:43:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-21T03:43:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-21T03:43:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-09-21T03:43:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-21T03:43:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-21T03:43:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:54 smithi016 ceph-mon[30375]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:43:54.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.812+0000 7fe446c00640 1 -- 172.21.15.16:0/1188375067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe44019b4a0 con 0x7fe44010dc20 2024-09-21T03:43:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.814+0000 7fe43e000640 1 -- 172.21.15.16:0/1188375067 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+24704 (secure 0 0 0) 0x7fe44019b4a0 con 0x7fe44010dc20 2024-09-21T03:43:54.816 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:43:54.816 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:33:53.220358+0000","modified":"2024-09-21T03:43:54.450112+0000","last_up_change":"2024-09-21T03:42:06.614247+0000","last_in_change":"2024-09-21T03:41:43.379404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:37:55.708915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:43:03.567522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:43:06.048083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:43:16.061356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6a7e4b9e-078f-4ae9-ab9e-37be995e4deb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6803","nonce":311974551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6805","nonce":311974551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6809","nonce":311974551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6807","nonce":311974551}]},"public_addr":"172.21.15.16:6803/311974551","cluster_addr":"172.21.15.16:6805/311974551","heartbeat_back_addr":"172.21.15.16:6809/311974551","heartbeat_front_addr":"172.21.15.16:6807/311974551","state":["exists","up"]},{"osd":1,"uuid":"3a497a64-e1a8-43ef-8509-0a79515da0aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6801","nonce":2152486571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6803","nonce":2152486571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6807","nonce":2152486571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6805","nonce":2152486571}]},"public_addr":"172.21.15.123:6801/2152486571","cluster_addr":"172.21.15.123:6803/2152486571","heartbeat_back_addr":"172.21.15.123:6807/2152486571","heartbeat_front_addr":"172.21.15.123:6805/2152486571","state":["exists","up"]},{"osd":2,"uuid":"1c39518e-6cce-4a0f-9ee8-55c60ad325f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6801","nonce":2812351279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6803","nonce":2812351279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6807","nonce":2812351279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6805","nonce":2812351279}]},"public_addr":"172.21.15.134:6801/2812351279","cluster_addr":"172.21.15.134:6803/2812351279","heartbeat_back_addr":"172.21.15.134:6807/2812351279","heartbeat_front_addr":"172.21.15.134:6805/2812351279","state":["exists","up"]},{"osd":3,"uuid":"0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6811","nonce":2366216026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6813","nonce":2366216026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6817","nonce":2366216026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6815","nonce":2366216026}]},"public_addr":"172.21.15.16:6811/2366216026","cluster_addr":"172.21.15.16:6813/2366216026","heartbeat_back_addr":"172.21.15.16:6817/2366216026","heartbeat_front_addr":"172.21.15.16:6815/2366216026","state":["exists","up"]},{"osd":4,"uuid":"ad5e6d0e-31e8-4779-9e74-f215032777ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6809","nonce":2354279116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6811","nonce":2354279116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6815","nonce":2354279116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6813","nonce":2354279116}]},"public_addr":"172.21.15.123:6809/2354279116","cluster_addr":"172.21.15.123:6811/2354279116","heartbeat_back_addr":"172.21.15.123:6815/2354279116","heartbeat_front_addr":"172.21.15.123:6813/2354279116","state":["exists","up"]},{"osd":5,"uuid":"ca09bf88-1772-4dc0-8711-cbf9a5b31977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6809","nonce":2678416182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6811","nonce":2678416182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6815","nonce":2678416182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6813","nonce":2678416182}]},"public_addr":"172.21.15.134:6809/2678416182","cluster_addr":"172.21.15.134:6811/2678416182","heartbeat_back_addr":"172.21.15.134:6815/2678416182","heartbeat_front_addr":"172.21.15.134:6813/2678416182","state":["exists","up"]},{"osd":6,"uuid":"0d41ed4e-8699-4d88-9254-12d3b84941e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6819","nonce":3377807744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6821","nonce":3377807744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6825","nonce":3377807744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6823","nonce":3377807744}]},"public_addr":"172.21.15.16:6819/3377807744","cluster_addr":"172.21.15.16:6821/3377807744","heartbeat_back_addr":"172.21.15.16:6825/3377807744","heartbeat_front_addr":"172.21.15.16:6823/3377807744","state":["exists","up"]},{"osd":7,"uuid":"1541a272-c524-4f12-9b94-763f69bb1c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6817","nonce":1463412739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6819","nonce":1463412739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6823","nonce":1463412739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6821","nonce":1463412739}]},"public_addr":"172.21.15.123:6817/1463412739","cluster_addr":"172.21.15.123:6819/1463412739","heartbeat_back_addr":"172.21.15.123:6823/1463412739","heartbeat_front_addr":"172.21.15.123:6821/1463412739","state":["exists","up"]},{"osd":8,"uuid":"cc00014c-eb99-49ab-984c-0bc26ef85130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6817","nonce":2644021450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6819","nonce":2644021450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6823","nonce":2644021450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6821","nonce":2644021450}]},"public_addr":"172.21.15.134:6817/2644021450","cluster_addr":"172.21.15.134:6819/2644021450","heartbeat_back_addr":"172.21.15.134:6823/2644021450","heartbeat_front_addr":"172.21.15.134:6821/2644021450","state":["exists","up"]},{"osd":9,"uuid":"3cbfff74-31ff-4ffd-844b-54ec8563ba32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6827","nonce":1653164371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6829","nonce":1653164371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6833","nonce":1653164371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6831","nonce":1653164371}]},"public_addr":"172.21.15.16:6827/1653164371","cluster_addr":"172.21.15.16:6829/1653164371","heartbeat_back_addr":"172.21.15.16:6833/1653164371","heartbeat_front_addr":"172.21.15.16:6831/1653164371","state":["exists","up"]},{"osd":10,"uuid":"0a1caaad-70cd-4c2c-88d3-e9a5cc885655","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6825","nonce":3682234496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6827","nonce":3682234496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6831","nonce":3682234496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6829","nonce":3682234496}]},"public_addr":"172.21.15.123:6825/3682234496","cluster_addr":"172.21.15.123:6827/3682234496","heartbeat_back_addr":"172.21.15.123:6831/3682234496","heartbeat_front_addr":"172.21.15.123:6829/3682234496","state":["exists","up"]},{"osd":11,"uuid":"3aff3258-5a38-41ff-aac3-4a082feffdae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6825","nonce":584758685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6827","nonce":584758685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6831","nonce":584758685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6829","nonce":584758685}]},"public_addr":"172.21.15.134:6825/584758685","cluster_addr":"172.21.15.134:6827/584758685","heartbeat_back_addr":"172.21.15.134:6831/584758685","heartbeat_front_addr":"172.21.15.134:6829/584758685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:02.177731+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:28.847562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:54.272384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:24.338596+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:52.430371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:17.832737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:50.397890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:16.842245+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:42.102933+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:13.074732+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:40.057905+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:04.898190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/557742783":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/3238676548":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6801/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/2357336233":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6801/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6800/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/3806844703":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/2351581621":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6800/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/355731724":"2024-09-22T03:34:22.549156+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:43:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.817+0000 7fe446c00640 1 -- 172.21.15.16:0/1188375067 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe4280b6510 msgr2=0x7fe4280b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.817+0000 7fe446c00640 1 --2- 172.21.15.16:0/1188375067 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe4280b6510 0x7fe4280b8a20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe43400bbe0 tx=0x7fe43400b390 comp rx=0 tx=0).stop 2024-09-21T03:43:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.817+0000 7fe446c00640 1 -- 172.21.15.16:0/1188375067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe44010dc20 msgr2=0x7fe44019fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.817+0000 7fe446c00640 1 --2- 172.21.15.16:0/1188375067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe44010dc20 0x7fe44019fbb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe43800eaa0 tx=0x7fe43800ef60 comp rx=0 tx=0).stop 2024-09-21T03:43:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.817+0000 7fe446c00640 1 -- 172.21.15.16:0/1188375067 shutdown_connections 2024-09-21T03:43:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.817+0000 7fe446c00640 1 --2- 172.21.15.16:0/1188375067 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe4280b6510 0x7fe4280b8a20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.817+0000 7fe446c00640 1 --2- 172.21.15.16:0/1188375067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe44010dc20 0x7fe44019fbb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.817+0000 7fe446c00640 1 --2- 172.21.15.16:0/1188375067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe440105110 0x7fe44019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.817+0000 7fe446c00640 1 --2- 172.21.15.16:0/1188375067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4401047c0 0x7fe44019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.817+0000 7fe446c00640 1 -- 172.21.15.16:0/1188375067 >> 172.21.15.16:0/1188375067 conn(0x7fe4400fe2c0 msgr2=0x7fe4400fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.817+0000 7fe446c00640 1 -- 172.21.15.16:0/1188375067 shutdown_connections 2024-09-21T03:43:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:54.817+0000 7fe446c00640 1 -- 172.21.15.16:0/1188375067 wait complete. 2024-09-21T03:43:54.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:54 smithi134 ceph-mon[35031]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T03:43:54.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-21T03:43:54.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T03:43:54.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-09-21T03:43:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-21T03:43:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-09-21T03:43:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-21T03:43:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-21T03:43:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-09-21T03:43:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-21T03:43:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-21T03:43:54.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:54 smithi134 ceph-mon[35031]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:43:54.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:54 smithi123 ceph-mon[34673]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T03:43:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-21T03:43:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T03:43:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-09-21T03:43:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-21T03:43:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-09-21T03:43:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-21T03:43:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-21T03:43:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-09-21T03:43:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-21T03:43:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-21T03:43:54.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:54 smithi123 ceph-mon[34673]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:43:55.643 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:43:55.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:55 smithi134 ceph-mon[35031]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:43:55.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:55 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1188375067' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:55.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:55 smithi123 ceph-mon[34673]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:43:55.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:55 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1188375067' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:55.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:55 smithi016 ceph-mon[30375]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:43:55.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:55 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1188375067' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:56.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:56 smithi016 ceph-mon[30375]: pgmap v345: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-09-21T03:43:56.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.728+0000 7f1679a00640 1 -- 172.21.15.16:0/2937170566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1674075730 msgr2=0x7f1674075b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:56.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.728+0000 7f1679a00640 1 --2- 172.21.15.16:0/2937170566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1674075730 0x7f1674075b40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1664009900 tx=0x7f166402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:43:56.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.729+0000 7f1679a00640 1 -- 172.21.15.16:0/2937170566 shutdown_connections 2024-09-21T03:43:56.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.729+0000 7f1679a00640 1 --2- 172.21.15.16:0/2937170566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1674110140 0x7f1674112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:56.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.729+0000 7f1679a00640 1 --2- 172.21.15.16:0/2937170566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1674076080 0x7f167410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:56.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.729+0000 7f1679a00640 1 --2- 172.21.15.16:0/2937170566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1674075730 0x7f1674075b40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:56.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.729+0000 7f1679a00640 1 -- 172.21.15.16:0/2937170566 >> 172.21.15.16:0/2937170566 conn(0x7f167407b850 msgr2=0x7f167407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:56.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.729+0000 7f1679a00640 1 -- 172.21.15.16:0/2937170566 shutdown_connections 2024-09-21T03:43:56.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.729+0000 7f1679a00640 1 -- 172.21.15.16:0/2937170566 wait complete. 2024-09-21T03:43:56.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.730+0000 7f1679a00640 1 Processor -- start 2024-09-21T03:43:56.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.730+0000 7f1679a00640 1 -- start start 2024-09-21T03:43:56.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.731+0000 7f1679a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1674075730 0x7f167419c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.731+0000 7f1679a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1674076080 0x7f167419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.731+0000 7f1679a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1674110140 0x7f16741a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.731+0000 7f1679a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1674115cf0 con 0x7f1674076080 2024-09-21T03:43:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.731+0000 7f1679a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1674115b70 con 0x7f1674075730 2024-09-21T03:43:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.731+0000 7f1679a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1674115e70 con 0x7f1674110140 2024-09-21T03:43:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.731+0000 7f166be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1674076080 0x7f167419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.731+0000 7f1673400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1674110140 0x7f16741a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.731+0000 7f1673400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1674110140 0x7f16741a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:33704/0 (socket says 172.21.15.16:33704) 2024-09-21T03:43:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.731+0000 7f1673400640 1 -- 172.21.15.16:0/3256582107 learned_addr learned my addr 172.21.15.16:0/3256582107 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.731+0000 7f1672a00640 1 --2- 172.21.15.16:0/3256582107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1674075730 0x7f167419c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.732+0000 7f1672a00640 1 -- 172.21.15.16:0/3256582107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1674110140 msgr2=0x7f16741a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.732+0000 7f1672a00640 1 --2- 172.21.15.16:0/3256582107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1674110140 0x7f16741a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.732+0000 7f1672a00640 1 -- 172.21.15.16:0/3256582107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1674076080 msgr2=0x7f167419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.732+0000 7f1672a00640 1 --2- 172.21.15.16:0/3256582107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1674076080 0x7f167419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.732+0000 7f1672a00640 1 -- 172.21.15.16:0/3256582107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16741a26b0 con 0x7f1674075730 2024-09-21T03:43:56.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.732+0000 7f1673400640 1 --2- 172.21.15.16:0/3256582107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1674110140 0x7f16741a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.732+0000 7f166be00640 1 --2- 172.21.15.16:0/3256582107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1674076080 0x7f167419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.732+0000 7f1672a00640 1 --2- 172.21.15.16:0/3256582107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1674075730 0x7f167419c740 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f166402a9e0 tx=0x7f1664003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.733+0000 7f166b400640 1 -- 172.21.15.16:0/3256582107 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166403f070 con 0x7f1674075730 2024-09-21T03:43:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.733+0000 7f1679a00640 1 -- 172.21.15.16:0/3256582107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16741a2940 con 0x7f1674075730 2024-09-21T03:43:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.733+0000 7f1679a00640 1 -- 172.21.15.16:0/3256582107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16741a9100 con 0x7f1674075730 2024-09-21T03:43:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.733+0000 7f166b400640 1 -- 172.21.15.16:0/3256582107 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1664031b50 con 0x7f1674075730 2024-09-21T03:43:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.733+0000 7f166b400640 1 -- 172.21.15.16:0/3256582107 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664049060 con 0x7f1674075730 2024-09-21T03:43:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.734+0000 7f1679a00640 1 -- 172.21.15.16:0/3256582107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1674075b40 con 0x7f1674075730 2024-09-21T03:43:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.736+0000 7f166b400640 1 -- 172.21.15.16:0/3256582107 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f1664042070 con 0x7f1674075730 2024-09-21T03:43:56.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.738+0000 7f166b400640 1 --2- 172.21.15.16:0/3256582107 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f16580b6510 0x7f16580b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:56.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.738+0000 7f166b400640 1 -- 172.21.15.16:0/3256582107 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f1664107ae0 con 0x7f1674075730 2024-09-21T03:43:56.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.738+0000 7f166be00640 1 --2- 172.21.15.16:0/3256582107 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f16580b6510 0x7f16580b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:56.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.739+0000 7f166be00640 1 --2- 172.21.15.16:0/3256582107 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f16580b6510 0x7f16580b8a20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f165c0095f0 tx=0x7f165c005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:56.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.739+0000 7f166b400640 1 -- 172.21.15.16:0/3256582107 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f16640d2390 con 0x7f1674075730 2024-09-21T03:43:56.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:56 smithi134 ceph-mon[35031]: pgmap v345: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-09-21T03:43:56.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:56 smithi123 ceph-mon[34673]: pgmap v345: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-09-21T03:43:56.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.940+0000 7f1679a00640 1 -- 172.21.15.16:0/3256582107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16741a9470 con 0x7f1674075730 2024-09-21T03:43:56.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.942+0000 7f166b400640 1 -- 172.21.15.16:0/3256582107 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+24704 (secure 0 0 0) 0x7f16640d72d0 con 0x7f1674075730 2024-09-21T03:43:56.944 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:43:56.944 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:33:53.220358+0000","modified":"2024-09-21T03:43:54.450112+0000","last_up_change":"2024-09-21T03:42:06.614247+0000","last_in_change":"2024-09-21T03:41:43.379404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:37:55.708915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:43:03.567522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:43:06.048083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:43:16.061356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6a7e4b9e-078f-4ae9-ab9e-37be995e4deb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6803","nonce":311974551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6805","nonce":311974551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6809","nonce":311974551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6807","nonce":311974551}]},"public_addr":"172.21.15.16:6803/311974551","cluster_addr":"172.21.15.16:6805/311974551","heartbeat_back_addr":"172.21.15.16:6809/311974551","heartbeat_front_addr":"172.21.15.16:6807/311974551","state":["exists","up"]},{"osd":1,"uuid":"3a497a64-e1a8-43ef-8509-0a79515da0aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6801","nonce":2152486571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6803","nonce":2152486571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6807","nonce":2152486571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6805","nonce":2152486571}]},"public_addr":"172.21.15.123:6801/2152486571","cluster_addr":"172.21.15.123:6803/2152486571","heartbeat_back_addr":"172.21.15.123:6807/2152486571","heartbeat_front_addr":"172.21.15.123:6805/2152486571","state":["exists","up"]},{"osd":2,"uuid":"1c39518e-6cce-4a0f-9ee8-55c60ad325f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6801","nonce":2812351279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6803","nonce":2812351279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6807","nonce":2812351279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6805","nonce":2812351279}]},"public_addr":"172.21.15.134:6801/2812351279","cluster_addr":"172.21.15.134:6803/2812351279","heartbeat_back_addr":"172.21.15.134:6807/2812351279","heartbeat_front_addr":"172.21.15.134:6805/2812351279","state":["exists","up"]},{"osd":3,"uuid":"0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6811","nonce":2366216026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6813","nonce":2366216026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6817","nonce":2366216026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6815","nonce":2366216026}]},"public_addr":"172.21.15.16:6811/2366216026","cluster_addr":"172.21.15.16:6813/2366216026","heartbeat_back_addr":"172.21.15.16:6817/2366216026","heartbeat_front_addr":"172.21.15.16:6815/2366216026","state":["exists","up"]},{"osd":4,"uuid":"ad5e6d0e-31e8-4779-9e74-f215032777ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6809","nonce":2354279116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6811","nonce":2354279116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6815","nonce":2354279116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6813","nonce":2354279116}]},"public_addr":"172.21.15.123:6809/2354279116","cluster_addr":"172.21.15.123:6811/2354279116","heartbeat_back_addr":"172.21.15.123:6815/2354279116","heartbeat_front_addr":"172.21.15.123:6813/2354279116","state":["exists","up"]},{"osd":5,"uuid":"ca09bf88-1772-4dc0-8711-cbf9a5b31977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6809","nonce":2678416182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6811","nonce":2678416182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6815","nonce":2678416182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6813","nonce":2678416182}]},"public_addr":"172.21.15.134:6809/2678416182","cluster_addr":"172.21.15.134:6811/2678416182","heartbeat_back_addr":"172.21.15.134:6815/2678416182","heartbeat_front_addr":"172.21.15.134:6813/2678416182","state":["exists","up"]},{"osd":6,"uuid":"0d41ed4e-8699-4d88-9254-12d3b84941e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6819","nonce":3377807744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6821","nonce":3377807744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6825","nonce":3377807744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6823","nonce":3377807744}]},"public_addr":"172.21.15.16:6819/3377807744","cluster_addr":"172.21.15.16:6821/3377807744","heartbeat_back_addr":"172.21.15.16:6825/3377807744","heartbeat_front_addr":"172.21.15.16:6823/3377807744","state":["exists","up"]},{"osd":7,"uuid":"1541a272-c524-4f12-9b94-763f69bb1c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6817","nonce":1463412739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6819","nonce":1463412739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6823","nonce":1463412739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6821","nonce":1463412739}]},"public_addr":"172.21.15.123:6817/1463412739","cluster_addr":"172.21.15.123:6819/1463412739","heartbeat_back_addr":"172.21.15.123:6823/1463412739","heartbeat_front_addr":"172.21.15.123:6821/1463412739","state":["exists","up"]},{"osd":8,"uuid":"cc00014c-eb99-49ab-984c-0bc26ef85130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6817","nonce":2644021450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6819","nonce":2644021450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6823","nonce":2644021450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6821","nonce":2644021450}]},"public_addr":"172.21.15.134:6817/2644021450","cluster_addr":"172.21.15.134:6819/2644021450","heartbeat_back_addr":"172.21.15.134:6823/2644021450","heartbeat_front_addr":"172.21.15.134:6821/2644021450","state":["exists","up"]},{"osd":9,"uuid":"3cbfff74-31ff-4ffd-844b-54ec8563ba32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6827","nonce":1653164371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6829","nonce":1653164371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6833","nonce":1653164371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6831","nonce":1653164371}]},"public_addr":"172.21.15.16:6827/1653164371","cluster_addr":"172.21.15.16:6829/1653164371","heartbeat_back_addr":"172.21.15.16:6833/1653164371","heartbeat_front_addr":"172.21.15.16:6831/1653164371","state":["exists","up"]},{"osd":10,"uuid":"0a1caaad-70cd-4c2c-88d3-e9a5cc885655","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6825","nonce":3682234496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6827","nonce":3682234496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6831","nonce":3682234496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6829","nonce":3682234496}]},"public_addr":"172.21.15.123:6825/3682234496","cluster_addr":"172.21.15.123:6827/3682234496","heartbeat_back_addr":"172.21.15.123:6831/3682234496","heartbeat_front_addr":"172.21.15.123:6829/3682234496","state":["exists","up"]},{"osd":11,"uuid":"3aff3258-5a38-41ff-aac3-4a082feffdae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6825","nonce":584758685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6827","nonce":584758685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6831","nonce":584758685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6829","nonce":584758685}]},"public_addr":"172.21.15.134:6825/584758685","cluster_addr":"172.21.15.134:6827/584758685","heartbeat_back_addr":"172.21.15.134:6831/584758685","heartbeat_front_addr":"172.21.15.134:6829/584758685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:02.177731+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:28.847562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:54.272384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:24.338596+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:52.430371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:17.832737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:50.397890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:16.842245+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:42.102933+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:13.074732+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:40.057905+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:04.898190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/557742783":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/3238676548":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6801/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/2357336233":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6801/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6800/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/3806844703":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/2351581621":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6800/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/355731724":"2024-09-22T03:34:22.549156+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:43:56.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.944+0000 7f1679a00640 1 -- 172.21.15.16:0/3256582107 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f16580b6510 msgr2=0x7f16580b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:56.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.944+0000 7f1679a00640 1 --2- 172.21.15.16:0/3256582107 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f16580b6510 0x7f16580b8a20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f165c0095f0 tx=0x7f165c005f90 comp rx=0 tx=0).stop 2024-09-21T03:43:56.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.944+0000 7f1679a00640 1 -- 172.21.15.16:0/3256582107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1674075730 msgr2=0x7f167419c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:56.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.944+0000 7f1679a00640 1 --2- 172.21.15.16:0/3256582107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1674075730 0x7f167419c740 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f166402a9e0 tx=0x7f1664003b90 comp rx=0 tx=0).stop 2024-09-21T03:43:56.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.944+0000 7f1679a00640 1 -- 172.21.15.16:0/3256582107 shutdown_connections 2024-09-21T03:43:56.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.944+0000 7f1679a00640 1 --2- 172.21.15.16:0/3256582107 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f16580b6510 0x7f16580b8a20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:56.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.944+0000 7f1679a00640 1 --2- 172.21.15.16:0/3256582107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1674110140 0x7f16741a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:56.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.944+0000 7f1679a00640 1 --2- 172.21.15.16:0/3256582107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1674076080 0x7f167419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:56.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.944+0000 7f1679a00640 1 --2- 172.21.15.16:0/3256582107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1674075730 0x7f167419c740 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:56.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.944+0000 7f1679a00640 1 -- 172.21.15.16:0/3256582107 >> 172.21.15.16:0/3256582107 conn(0x7f167407b850 msgr2=0x7f1674109c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:56.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.944+0000 7f1679a00640 1 -- 172.21.15.16:0/3256582107 shutdown_connections 2024-09-21T03:43:56.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:56.945+0000 7f1679a00640 1 -- 172.21.15.16:0/3256582107 wait complete. 2024-09-21T03:43:57.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:57 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3256582107' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:57.778 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T03:43:57.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:57 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3256582107' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:57.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:57 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3256582107' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:43:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:58.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:58 smithi016 ceph-mon[30375]: pgmap v346: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.825+0000 7f7f0aa00640 1 -- 172.21.15.16:0/2754971138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f04102d40 msgr2=0x7f7f0410fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.825+0000 7f7f0aa00640 1 --2- 172.21.15.16:0/2754971138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f04102d40 0x7f7f0410fa60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009900 tx=0x7f7ef002a480 comp rx=0 tx=0).stop 2024-09-21T03:43:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.825+0000 7f7f0aa00640 1 -- 172.21.15.16:0/2754971138 shutdown_connections 2024-09-21T03:43:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.825+0000 7f7f0aa00640 1 --2- 172.21.15.16:0/2754971138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f0410ffa0 0x7f7f041123c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.825+0000 7f7f0aa00640 1 --2- 172.21.15.16:0/2754971138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f04102d40 0x7f7f0410fa60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.825+0000 7f7f0aa00640 1 --2- 172.21.15.16:0/2754971138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f041023f0 0x7f7f04102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.825+0000 7f7f0aa00640 1 -- 172.21.15.16:0/2754971138 >> 172.21.15.16:0/2754971138 conn(0x7f7f040fe2d0 msgr2=0x7f7f04100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.825+0000 7f7f0aa00640 1 -- 172.21.15.16:0/2754971138 shutdown_connections 2024-09-21T03:43:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.825+0000 7f7f0aa00640 1 -- 172.21.15.16:0/2754971138 wait complete. 2024-09-21T03:43:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.827+0000 7f7f0aa00640 1 Processor -- start 2024-09-21T03:43:58.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.827+0000 7f7f0aa00640 1 -- start start 2024-09-21T03:43:58.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.828+0000 7f7f0aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f041023f0 0x7f7f0419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.828+0000 7f7f0aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f04102d40 0x7f7f0419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.828+0000 7f7f0aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f0410ffa0 0x7f7f041a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.828+0000 7f7f0aa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7f04115ae0 con 0x7f7f041023f0 2024-09-21T03:43:58.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.828+0000 7f7f0aa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f04115960 con 0x7f7f0410ffa0 2024-09-21T03:43:58.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.828+0000 7f7f0aa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7f04115c60 con 0x7f7f04102d40 2024-09-21T03:43:58.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.828+0000 7f7f09600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f041023f0 0x7f7f0419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.828+0000 7f7f09600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f041023f0 0x7f7f0419c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37218/0 (socket says 172.21.15.16:37218) 2024-09-21T03:43:58.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.828+0000 7f7f09600640 1 -- 172.21.15.16:0/3839771286 learned_addr learned my addr 172.21.15.16:0/3839771286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:43:58.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.828+0000 7f7f08c00640 1 --2- 172.21.15.16:0/3839771286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f04102d40 0x7f7f0419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.828+0000 7f7f0a000640 1 --2- 172.21.15.16:0/3839771286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f0410ffa0 0x7f7f041a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.829+0000 7f7f09600640 1 -- 172.21.15.16:0/3839771286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f04102d40 msgr2=0x7f7f0419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.829+0000 7f7f09600640 1 --2- 172.21.15.16:0/3839771286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f04102d40 0x7f7f0419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.829+0000 7f7f09600640 1 -- 172.21.15.16:0/3839771286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f0410ffa0 msgr2=0x7f7f041a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:58.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.829+0000 7f7f09600640 1 --2- 172.21.15.16:0/3839771286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f0410ffa0 0x7f7f041a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.829+0000 7f7f09600640 1 -- 172.21.15.16:0/3839771286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f041a2400 con 0x7f7f041023f0 2024-09-21T03:43:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.829+0000 7f7f08c00640 1 --2- 172.21.15.16:0/3839771286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f04102d40 0x7f7f0419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.829+0000 7f7f09600640 1 --2- 172.21.15.16:0/3839771286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f041023f0 0x7f7f0419c490 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800e9f0 tx=0x7f7ef800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.829+0000 7f7f02000640 1 -- 172.21.15.16:0/3839771286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef800ccc0 con 0x7f7f041023f0 2024-09-21T03:43:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.829+0000 7f7f02000640 1 -- 172.21.15.16:0/3839771286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ef80108e0 con 0x7f7f041023f0 2024-09-21T03:43:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.829+0000 7f7f0aa00640 1 -- 172.21.15.16:0/3839771286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f041a26f0 con 0x7f7f041023f0 2024-09-21T03:43:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.829+0000 7f7f02000640 1 -- 172.21.15.16:0/3839771286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8018880 con 0x7f7f041023f0 2024-09-21T03:43:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.830+0000 7f7f0aa00640 1 -- 172.21.15.16:0/3839771286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f041a8e50 con 0x7f7f041023f0 2024-09-21T03:43:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.831+0000 7f7f0aa00640 1 -- 172.21.15.16:0/3839771286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f04102800 con 0x7f7f041023f0 2024-09-21T03:43:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.833+0000 7f7f02000640 1 -- 172.21.15.16:0/3839771286 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f7ef8010430 con 0x7f7f041023f0 2024-09-21T03:43:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.834+0000 7f7f02000640 1 --2- 172.21.15.16:0/3839771286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7eec0b6590 0x7f7eec0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.835+0000 7f7f02000640 1 -- 172.21.15.16:0/3839771286 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f7ef80e79e0 con 0x7f7f041023f0 2024-09-21T03:43:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.835+0000 7f7f08c00640 1 --2- 172.21.15.16:0/3839771286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7eec0b6590 0x7f7eec0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.835+0000 7f7f08c00640 1 --2- 172.21.15.16:0/3839771286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7eec0b6590 0x7f7eec0b8aa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0005ab0 tx=0x7f7ef0005a20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:58.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:58.838+0000 7f7f02000640 1 -- 172.21.15.16:0/3839771286 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ef80adeb0 con 0x7f7f041023f0 2024-09-21T03:43:58.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:58 smithi134 ceph-mon[35031]: pgmap v346: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:58.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:58 smithi123 ceph-mon[34673]: pgmap v346: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:59.080+0000 7f7f0aa00640 1 -- 172.21.15.16:0/3839771286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7f0402e640 con 0x7f7f041023f0 2024-09-21T03:43:59.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:59.484+0000 7f7f02000640 1 -- 172.21.15.16:0/3839771286 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f7ef80b2290 con 0x7f7f041023f0 2024-09-21T03:43:59.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:59.487+0000 7f7f0aa00640 1 -- 172.21.15.16:0/3839771286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7eec0b6590 msgr2=0x7f7eec0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:59.488+0000 7f7f0aa00640 1 --2- 172.21.15.16:0/3839771286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7eec0b6590 0x7f7eec0b8aa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0005ab0 tx=0x7f7ef0005a20 comp rx=0 tx=0).stop 2024-09-21T03:43:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:59.488+0000 7f7f0aa00640 1 -- 172.21.15.16:0/3839771286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f041023f0 msgr2=0x7f7f0419c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:59.488+0000 7f7f0aa00640 1 --2- 172.21.15.16:0/3839771286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f041023f0 0x7f7f0419c490 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800e9f0 tx=0x7f7ef800edf0 comp rx=0 tx=0).stop 2024-09-21T03:43:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:59.488+0000 7f7f0aa00640 1 -- 172.21.15.16:0/3839771286 shutdown_connections 2024-09-21T03:43:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:59.488+0000 7f7f0aa00640 1 --2- 172.21.15.16:0/3839771286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7eec0b6590 0x7f7eec0b8aa0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:59.488+0000 7f7f0aa00640 1 --2- 172.21.15.16:0/3839771286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f0410ffa0 0x7f7f041a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:59.488+0000 7f7f0aa00640 1 --2- 172.21.15.16:0/3839771286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f04102d40 0x7f7f0419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:59.488+0000 7f7f0aa00640 1 --2- 172.21.15.16:0/3839771286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f041023f0 0x7f7f0419c490 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:59.488+0000 7f7f0aa00640 1 -- 172.21.15.16:0/3839771286 >> 172.21.15.16:0/3839771286 conn(0x7f7f040fe2d0 msgr2=0x7f7f041120f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:59.488+0000 7f7f0aa00640 1 -- 172.21.15.16:0/3839771286 shutdown_connections 2024-09-21T03:43:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:43:59.488+0000 7f7f0aa00640 1 -- 172.21.15.16:0/3839771286 wait complete. 2024-09-21T03:43:59.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:43:59 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3839771286' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:43:59.891 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:43:59 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3839771286' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:43:59.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:43:59 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3839771286' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:44:00.264 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-21T03:44:00.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs set cephfs max_mds 5 2024-09-21T03:44:00.669 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:00.816 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:00 smithi016 ceph-mon[30375]: pgmap v347: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:00.817 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:00 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3839771286' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:44:00.817 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:00 smithi016 ceph-mon[30375]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:00.817 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:00 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:44:00.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:00 smithi134 ceph-mon[35031]: pgmap v347: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:00.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:00 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3839771286' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:44:00.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:00 smithi134 ceph-mon[35031]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:00.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:00 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:44:00.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:00 smithi123 ceph-mon[34673]: pgmap v347: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:00.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:00 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3839771286' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:44:00.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:00 smithi123 ceph-mon[34673]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:00.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:00 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:44:01.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.344+0000 7fd211000640 1 -- 172.21.15.16:0/2861684231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c1100c0 msgr2=0x7fd20c112500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.344+0000 7fd211000640 1 --2- 172.21.15.16:0/2861684231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c1100c0 0x7fd20c112500 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd204009ca0 tx=0x7fd20402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.344+0000 7fd211000640 1 -- 172.21.15.16:0/2861684231 shutdown_connections 2024-09-21T03:44:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.344+0000 7fd211000640 1 --2- 172.21.15.16:0/2861684231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c1100c0 0x7fd20c112500 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.344+0000 7fd211000640 1 --2- 172.21.15.16:0/2861684231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd20c076100 0x7fd20c10fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.344+0000 7fd211000640 1 --2- 172.21.15.16:0/2861684231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd20c0757b0 0x7fd20c075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.344+0000 7fd211000640 1 -- 172.21.15.16:0/2861684231 >> 172.21.15.16:0/2861684231 conn(0x7fd20c07b850 msgr2=0x7fd20c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.344+0000 7fd211000640 1 -- 172.21.15.16:0/2861684231 shutdown_connections 2024-09-21T03:44:01.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.344+0000 7fd211000640 1 -- 172.21.15.16:0/2861684231 wait complete. 2024-09-21T03:44:01.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.345+0000 7fd211000640 1 Processor -- start 2024-09-21T03:44:01.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.345+0000 7fd211000640 1 -- start start 2024-09-21T03:44:01.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.346+0000 7fd211000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd20c0757b0 0x7fd20c19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.346+0000 7fd211000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c076100 0x7fd20c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.346+0000 7fd211000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd20c1100c0 0x7fd20c1a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.346+0000 7fd211000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c115c70 con 0x7fd20c076100 2024-09-21T03:44:01.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.346+0000 7fd211000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c115af0 con 0x7fd20c1100c0 2024-09-21T03:44:01.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.346+0000 7fd211000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c115df0 con 0x7fd20c0757b0 2024-09-21T03:44:01.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.346+0000 7fd209e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd20c0757b0 0x7fd20c19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.346+0000 7fd203e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c076100 0x7fd20c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.346+0000 7fd209e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd20c0757b0 0x7fd20c19c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:33732/0 (socket says 172.21.15.16:33732) 2024-09-21T03:44:01.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.346+0000 7fd209e00640 1 -- 172.21.15.16:0/206080356 learned_addr learned my addr 172.21.15.16:0/206080356 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:01.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.346+0000 7fd20a800640 1 --2- 172.21.15.16:0/206080356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd20c1100c0 0x7fd20c1a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.347+0000 7fd203e00640 1 -- 172.21.15.16:0/206080356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd20c0757b0 msgr2=0x7fd20c19c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.347+0000 7fd203e00640 1 --2- 172.21.15.16:0/206080356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd20c0757b0 0x7fd20c19c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.347+0000 7fd203e00640 1 -- 172.21.15.16:0/206080356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd20c1100c0 msgr2=0x7fd20c1a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:01.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.347+0000 7fd203e00640 1 --2- 172.21.15.16:0/206080356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd20c1100c0 0x7fd20c1a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:01.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.347+0000 7fd203e00640 1 -- 172.21.15.16:0/206080356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20c1a25f0 con 0x7fd20c076100 2024-09-21T03:44:01.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.347+0000 7fd209e00640 1 --2- 172.21.15.16:0/206080356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd20c0757b0 0x7fd20c19c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:01.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.347+0000 7fd203e00640 1 --2- 172.21.15.16:0/206080356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c076100 0x7fd20c19cc20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400e9c0 tx=0x7fd1f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:01.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.347+0000 7fd202a00640 1 -- 172.21.15.16:0/206080356 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f400cd80 con 0x7fd20c076100 2024-09-21T03:44:01.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.347+0000 7fd211000640 1 -- 172.21.15.16:0/206080356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd20c1a28e0 con 0x7fd20c076100 2024-09-21T03:44:01.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.347+0000 7fd202a00640 1 -- 172.21.15.16:0/206080356 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1f40108e0 con 0x7fd20c076100 2024-09-21T03:44:01.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.347+0000 7fd202a00640 1 -- 172.21.15.16:0/206080356 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4018900 con 0x7fd20c076100 2024-09-21T03:44:01.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.348+0000 7fd211000640 1 -- 172.21.15.16:0/206080356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd20c1a9040 con 0x7fd20c076100 2024-09-21T03:44:01.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.349+0000 7fd211000640 1 -- 172.21.15.16:0/206080356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd20c075bc0 con 0x7fd20c076100 2024-09-21T03:44:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.352+0000 7fd202a00640 1 -- 172.21.15.16:0/206080356 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fd1f401f020 con 0x7fd20c076100 2024-09-21T03:44:01.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.354+0000 7fd202a00640 1 --2- 172.21.15.16:0/206080356 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd1f00b6510 0x7fd1f00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:01.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.354+0000 7fd202a00640 1 -- 172.21.15.16:0/206080356 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7fd1f40e7670 con 0x7fd20c076100 2024-09-21T03:44:01.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.354+0000 7fd209e00640 1 --2- 172.21.15.16:0/206080356 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd1f00b6510 0x7fd1f00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.355+0000 7fd209e00640 1 --2- 172.21.15.16:0/206080356 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd1f00b6510 0x7fd1f00b8a20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc011500 tx=0x7fd1fc005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.358+0000 7fd202a00640 1 -- 172.21.15.16:0/206080356 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd1f40adb40 con 0x7fd20c076100 2024-09-21T03:44:01.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:01.611+0000 7fd211000640 1 -- 172.21.15.16:0/206080356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fd20c1a9330 con 0x7fd20c076100 2024-09-21T03:44:02.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:02.494+0000 7fd202a00640 1 -- 172.21.15.16:0/206080356 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fd1f40b1f20 con 0x7fd20c076100 2024-09-21T03:44:02.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:02.498+0000 7fd211000640 1 -- 172.21.15.16:0/206080356 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd1f00b6510 msgr2=0x7fd1f00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:02.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:02.498+0000 7fd211000640 1 --2- 172.21.15.16:0/206080356 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd1f00b6510 0x7fd1f00b8a20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc011500 tx=0x7fd1fc005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:02.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:02.498+0000 7fd211000640 1 -- 172.21.15.16:0/206080356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c076100 msgr2=0x7fd20c19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:02.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:02.498+0000 7fd211000640 1 --2- 172.21.15.16:0/206080356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c076100 0x7fd20c19cc20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400e9c0 tx=0x7fd1f400ee80 comp rx=0 tx=0).stop 2024-09-21T03:44:02.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:02.498+0000 7fd211000640 1 -- 172.21.15.16:0/206080356 shutdown_connections 2024-09-21T03:44:02.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:02.499+0000 7fd211000640 1 --2- 172.21.15.16:0/206080356 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd1f00b6510 0x7fd1f00b8a20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:02.499+0000 7fd211000640 1 --2- 172.21.15.16:0/206080356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd20c1100c0 0x7fd20c1a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:02.499+0000 7fd211000640 1 --2- 172.21.15.16:0/206080356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c076100 0x7fd20c19cc20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:02.499+0000 7fd211000640 1 --2- 172.21.15.16:0/206080356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd20c0757b0 0x7fd20c19c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:02.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:02.499+0000 7fd211000640 1 -- 172.21.15.16:0/206080356 >> 172.21.15.16:0/206080356 conn(0x7fd20c07b850 msgr2=0x7fd20c100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:02.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:02.499+0000 7fd211000640 1 -- 172.21.15.16:0/206080356 shutdown_connections 2024-09-21T03:44:02.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:02.499+0000 7fd211000640 1 -- 172.21.15.16:0/206080356 wait complete. 2024-09-21T03:44:02.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:02 smithi134 ceph-mon[35031]: pgmap v348: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:02.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:02 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/206080356' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T03:44:02.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:02 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:02.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:02 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:02.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:02 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:02.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:02 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:02.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:02 smithi123 ceph-mon[34673]: pgmap v348: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:02.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:02 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/206080356' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T03:44:02.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:02 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:02.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:02 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:02.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:02 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:02.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:02 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:02.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:02 smithi016 ceph-mon[30375]: pgmap v348: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:44:02.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:02 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/206080356' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T03:44:02.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:02 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:02.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:02 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:02.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:02 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:02.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:02 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:03.347 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T03:44:03.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:03 smithi016 ceph-mon[30375]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T03:44:03.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:03 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/206080356' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T03:44:03.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:03 smithi016 ceph-mon[30375]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T03:44:03.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:03 smithi016 ceph-mon[30375]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:03.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:03 smithi016 ceph-mon[30375]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-21T03:44:03.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:03 smithi016 ceph-mon[30375]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-21T03:44:03.762 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:03.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:03 smithi134 ceph-mon[35031]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T03:44:03.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:03 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/206080356' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T03:44:03.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:03 smithi134 ceph-mon[35031]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T03:44:03.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:03 smithi134 ceph-mon[35031]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:03.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:03 smithi134 ceph-mon[35031]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-21T03:44:03.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:03 smithi134 ceph-mon[35031]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-21T03:44:03.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:03 smithi123 ceph-mon[34673]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T03:44:03.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:03 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/206080356' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T03:44:03.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:03 smithi123 ceph-mon[34673]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T03:44:03.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:03 smithi123 ceph-mon[34673]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:44:03.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:03 smithi123 ceph-mon[34673]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-21T03:44:03.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:03 smithi123 ceph-mon[34673]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-21T03:44:04.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.571+0000 7faae3c00640 1 -- 172.21.15.16:0/1117419099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc075750 msgr2=0x7faadc075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.571+0000 7faae3c00640 1 --2- 172.21.15.16:0/1117419099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc075750 0x7faadc075b60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faad0009900 tx=0x7faad002a410 comp rx=0 tx=0).stop 2024-09-21T03:44:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.571+0000 7faae3c00640 1 -- 172.21.15.16:0/1117419099 shutdown_connections 2024-09-21T03:44:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.571+0000 7faae3c00640 1 --2- 172.21.15.16:0/1117419099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faadc110120 0x7faadc112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.571+0000 7faae3c00640 1 --2- 172.21.15.16:0/1117419099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faadc0760a0 0x7faadc10fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.571+0000 7faae3c00640 1 --2- 172.21.15.16:0/1117419099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc075750 0x7faadc075b60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.571+0000 7faae3c00640 1 -- 172.21.15.16:0/1117419099 >> 172.21.15.16:0/1117419099 conn(0x7faadc07b850 msgr2=0x7faadc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.571+0000 7faae3c00640 1 -- 172.21.15.16:0/1117419099 shutdown_connections 2024-09-21T03:44:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.572+0000 7faae3c00640 1 -- 172.21.15.16:0/1117419099 wait complete. 2024-09-21T03:44:04.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.572+0000 7faae3c00640 1 Processor -- start 2024-09-21T03:44:04.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.573+0000 7faae3c00640 1 -- start start 2024-09-21T03:44:04.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.573+0000 7faae3c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faadc075750 0x7faadc19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:04.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.573+0000 7faae3c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faadc0760a0 0x7faadc19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:04.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.573+0000 7faae3c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc110120 0x7faadc1a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.573+0000 7faae3c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faadc115c90 con 0x7faadc110120 2024-09-21T03:44:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.573+0000 7faae3c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faadc115b10 con 0x7faadc0760a0 2024-09-21T03:44:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.573+0000 7faae3c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faadc115e10 con 0x7faadc075750 2024-09-21T03:44:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.573+0000 7faadbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faadc0760a0 0x7faadc19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.573+0000 7faae1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc110120 0x7faadc1a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.574+0000 7faae1200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faadc075750 0x7faadc19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.574+0000 7faadbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faadc0760a0 0x7faadc19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:41386/0 (socket says 172.21.15.16:41386) 2024-09-21T03:44:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.574+0000 7faadbe00640 1 -- 172.21.15.16:0/3277770516 learned_addr learned my addr 172.21.15.16:0/3277770516 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.574+0000 7faadbe00640 1 -- 172.21.15.16:0/3277770516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faadc075750 msgr2=0x7faadc19c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.574+0000 7faadbe00640 1 --2- 172.21.15.16:0/3277770516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faadc075750 0x7faadc19c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.574+0000 7faadbe00640 1 -- 172.21.15.16:0/3277770516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc110120 msgr2=0x7faadc1a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.574+0000 7faadbe00640 1 --2- 172.21.15.16:0/3277770516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc110120 0x7faadc1a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.574+0000 7faadbe00640 1 -- 172.21.15.16:0/3277770516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faadc1a25f0 con 0x7faadc0760a0 2024-09-21T03:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.574+0000 7faadbe00640 1 --2- 172.21.15.16:0/3277770516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faadc0760a0 0x7faadc19cc20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faac800d950 tx=0x7faac800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.575+0000 7faad9600640 1 -- 172.21.15.16:0/3277770516 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8017070 con 0x7faadc0760a0 2024-09-21T03:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.575+0000 7faae3c00640 1 -- 172.21.15.16:0/3277770516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faadc1a28e0 con 0x7faadc0760a0 2024-09-21T03:44:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.575+0000 7faad9600640 1 -- 172.21.15.16:0/3277770516 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faac8009c30 con 0x7faadc0760a0 2024-09-21T03:44:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.575+0000 7faad9600640 1 -- 172.21.15.16:0/3277770516 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac80047a0 con 0x7faadc0760a0 2024-09-21T03:44:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.575+0000 7faae3c00640 1 -- 172.21.15.16:0/3277770516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faadc1a9040 con 0x7faadc0760a0 2024-09-21T03:44:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.576+0000 7faae3c00640 1 -- 172.21.15.16:0/3277770516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faadc075b60 con 0x7faadc0760a0 2024-09-21T03:44:04.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.578+0000 7faad9600640 1 -- 172.21.15.16:0/3277770516 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7faac8004020 con 0x7faadc0760a0 2024-09-21T03:44:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.581+0000 7faad9600640 1 --2- 172.21.15.16:0/3277770516 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faac40b6510 0x7faac40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.581+0000 7faad9600640 1 -- 172.21.15.16:0/3277770516 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7faac80e7740 con 0x7faadc0760a0 2024-09-21T03:44:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.581+0000 7faae1200640 1 --2- 172.21.15.16:0/3277770516 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faac40b6510 0x7faac40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.581+0000 7faad9600640 1 -- 172.21.15.16:0/3277770516 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faac8020020 con 0x7faadc0760a0 2024-09-21T03:44:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.582+0000 7faae1200640 1 --2- 172.21.15.16:0/3277770516 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faac40b6510 0x7faac40b8a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faad0031440 tx=0x7faad0009560 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:04.656 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 03:44:04 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-b[62470]: 2024-09-21T03:44:04.390+0000 7f4e89757640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T03:44:04.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:04 smithi123 ceph-mon[34673]: pgmap v349: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 0 op/s 2024-09-21T03:44:04.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:04 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] up:active 2024-09-21T03:44:04.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:04 smithi123 ceph-mon[34673]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T03:44:04.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:04 smithi123 ceph-mon[34673]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-21T03:44:04.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:04 smithi123 ceph-mon[34673]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-21T03:44:04.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:04 smithi123 ceph-mon[34673]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T03:44:04.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:04 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:04.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:04 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:04.826+0000 7faae3c00640 1 -- 172.21.15.16:0/3277770516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7faadc1a92e0 con 0x7faadc0760a0 2024-09-21T03:44:04.828 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:04 smithi016 ceph-mon[30375]: pgmap v349: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 0 op/s 2024-09-21T03:44:04.828 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:04 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] up:active 2024-09-21T03:44:04.828 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:04 smithi016 ceph-mon[30375]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T03:44:04.828 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:04 smithi016 ceph-mon[30375]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-21T03:44:04.828 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:04 smithi016 ceph-mon[30375]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-21T03:44:04.828 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:04 smithi016 ceph-mon[30375]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T03:44:04.828 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:04 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:04.828 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:04 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:04.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:04 smithi134 ceph-mon[35031]: pgmap v349: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 0 op/s 2024-09-21T03:44:04.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:04 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] up:active 2024-09-21T03:44:04.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:04 smithi134 ceph-mon[35031]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T03:44:04.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:04 smithi134 ceph-mon[35031]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-21T03:44:04.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:04 smithi134 ceph-mon[35031]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-21T03:44:04.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:04 smithi134 ceph-mon[35031]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T03:44:04.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:04 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:04.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:04 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:05.530+0000 7faad9600640 1 -- 172.21.15.16:0/3277770516 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7faac80adc10 con 0x7faadc0760a0 2024-09-21T03:44:05.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:05.533+0000 7faae3c00640 1 -- 172.21.15.16:0/3277770516 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faac40b6510 msgr2=0x7faac40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:05.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:05.533+0000 7faae3c00640 1 --2- 172.21.15.16:0/3277770516 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faac40b6510 0x7faac40b8a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faad0031440 tx=0x7faad0009560 comp rx=0 tx=0).stop 2024-09-21T03:44:05.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:05.533+0000 7faae3c00640 1 -- 172.21.15.16:0/3277770516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faadc0760a0 msgr2=0x7faadc19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:05.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:05.533+0000 7faae3c00640 1 --2- 172.21.15.16:0/3277770516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faadc0760a0 0x7faadc19cc20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faac800d950 tx=0x7faac800de10 comp rx=0 tx=0).stop 2024-09-21T03:44:05.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:05.533+0000 7faae3c00640 1 -- 172.21.15.16:0/3277770516 shutdown_connections 2024-09-21T03:44:05.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:05.533+0000 7faae3c00640 1 --2- 172.21.15.16:0/3277770516 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faac40b6510 0x7faac40b8a20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:05.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:05.533+0000 7faae3c00640 1 --2- 172.21.15.16:0/3277770516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faadc110120 0x7faadc1a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:05.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:05.533+0000 7faae3c00640 1 --2- 172.21.15.16:0/3277770516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faadc0760a0 0x7faadc19cc20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:05.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:05.533+0000 7faae3c00640 1 --2- 172.21.15.16:0/3277770516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faadc075750 0x7faadc19c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:05.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:05.533+0000 7faae3c00640 1 -- 172.21.15.16:0/3277770516 >> 172.21.15.16:0/3277770516 conn(0x7faadc07b850 msgr2=0x7faadc101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:05.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:05.534+0000 7faae3c00640 1 -- 172.21.15.16:0/3277770516 shutdown_connections 2024-09-21T03:44:05.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:05.534+0000 7faae3c00640 1 -- 172.21.15.16:0/3277770516 wait complete. 2024-09-21T03:44:05.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:05 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] up:active 2024-09-21T03:44:05.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:05 smithi134 ceph-mon[35031]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T03:44:05.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:05 smithi134 ceph-mon[35031]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-21T03:44:05.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:05 smithi134 ceph-mon[35031]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-21T03:44:05.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:05 smithi134 ceph-mon[35031]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T03:44:05.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:05.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:44:05.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:05.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:05 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3277770516' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:44:05.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:05 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:44:05.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:05 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] up:active 2024-09-21T03:44:05.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:05 smithi123 ceph-mon[34673]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T03:44:05.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:05 smithi123 ceph-mon[34673]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-21T03:44:05.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:05 smithi123 ceph-mon[34673]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-21T03:44:05.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:05 smithi123 ceph-mon[34673]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T03:44:05.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:05.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:44:05.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:05.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:05 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3277770516' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:44:05.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:05 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:44:05.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:05 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] up:active 2024-09-21T03:44:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:05 smithi016 ceph-mon[30375]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T03:44:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:05 smithi016 ceph-mon[30375]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-21T03:44:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:05 smithi016 ceph-mon[30375]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-21T03:44:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:05 smithi016 ceph-mon[30375]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T03:44:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:44:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:44:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:05 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3277770516' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:44:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:05 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:44:06.362 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T03:44:06.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:06 smithi016 ceph-mon[30375]: pgmap v350: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-21T03:44:06.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:06 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:44:06.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:06 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] up:active 2024-09-21T03:44:06.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:06 smithi016 ceph-mon[30375]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T03:44:06.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:06 smithi016 ceph-mon[30375]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T03:44:06.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:06 smithi016 ceph-mon[30375]: Cluster is now healthy 2024-09-21T03:44:06.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:06 smithi016 ceph-mon[30375]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-21T03:44:06.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:06 smithi016 ceph-mon[30375]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T03:44:06.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:06 smithi016 ceph-mon[30375]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-21T03:44:06.751 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:06.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:06 smithi134 ceph-mon[35031]: pgmap v350: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-21T03:44:06.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:06 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:44:06.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:06 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] up:active 2024-09-21T03:44:06.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:06 smithi134 ceph-mon[35031]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T03:44:06.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:06 smithi134 ceph-mon[35031]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T03:44:06.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:06 smithi134 ceph-mon[35031]: Cluster is now healthy 2024-09-21T03:44:06.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:06 smithi134 ceph-mon[35031]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-21T03:44:06.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:06 smithi134 ceph-mon[35031]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T03:44:06.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:06 smithi134 ceph-mon[35031]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-21T03:44:06.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:06 smithi123 ceph-mon[34673]: pgmap v350: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-21T03:44:06.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:06 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:44:06.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:06 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] up:active 2024-09-21T03:44:06.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:06 smithi123 ceph-mon[34673]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T03:44:06.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:06 smithi123 ceph-mon[34673]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T03:44:06.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:06 smithi123 ceph-mon[34673]: Cluster is now healthy 2024-09-21T03:44:06.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:06 smithi123 ceph-mon[34673]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-21T03:44:06.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:06 smithi123 ceph-mon[34673]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T03:44:06.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:06 smithi123 ceph-mon[34673]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-21T03:44:07.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.387+0000 7fbab4400640 1 -- 172.21.15.16:0/3115957922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaac105d80 msgr2=0x7fbaac106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:07.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.387+0000 7fbab4400640 1 --2- 172.21.15.16:0/3115957922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaac105d80 0x7fbaac106190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009900 tx=0x7fbaa002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:44:07.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.388+0000 7fbab4400640 1 -- 172.21.15.16:0/3115957922 shutdown_connections 2024-09-21T03:44:07.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.388+0000 7fbab4400640 1 --2- 172.21.15.16:0/3115957922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac10c010 0x7fbaac112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:07.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.388+0000 7fbab4400640 1 --2- 172.21.15.16:0/3115957922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaac10b6c0 0x7fbaac10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:07.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.388+0000 7fbab4400640 1 --2- 172.21.15.16:0/3115957922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaac105d80 0x7fbaac106190 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:07.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.388+0000 7fbab4400640 1 -- 172.21.15.16:0/3115957922 >> 172.21.15.16:0/3115957922 conn(0x7fbaac101810 msgr2=0x7fbaac103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:07.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.388+0000 7fbab4400640 1 -- 172.21.15.16:0/3115957922 shutdown_connections 2024-09-21T03:44:07.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.388+0000 7fbab4400640 1 -- 172.21.15.16:0/3115957922 wait complete. 2024-09-21T03:44:07.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.389+0000 7fbab4400640 1 Processor -- start 2024-09-21T03:44:07.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.389+0000 7fbab4400640 1 -- start start 2024-09-21T03:44:07.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.390+0000 7fbab4400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaac105d80 0x7fbaac19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:07.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.390+0000 7fbab4400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac10b6c0 0x7fbaac19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:07.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.390+0000 7fbab4400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaac10c010 0x7fbaac1a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:07.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.390+0000 7fbab4400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac115cd0 con 0x7fbaac105d80 2024-09-21T03:44:07.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.390+0000 7fbab4400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac115b50 con 0x7fbaac10b6c0 2024-09-21T03:44:07.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.390+0000 7fbab4400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac115e50 con 0x7fbaac10c010 2024-09-21T03:44:07.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.390+0000 7fbab1a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaac105d80 0x7fbaac19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:07.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.390+0000 7fbab1a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaac105d80 0x7fbaac19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46508/0 (socket says 172.21.15.16:46508) 2024-09-21T03:44:07.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.390+0000 7fbab1000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac10b6c0 0x7fbaac19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:07.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.390+0000 7fbab1a00640 1 -- 172.21.15.16:0/2191510725 learned_addr learned my addr 172.21.15.16:0/2191510725 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:07.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.390+0000 7fbab2400640 1 --2- 172.21.15.16:0/2191510725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaac10c010 0x7fbaac1a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:07.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.390+0000 7fbab1000640 1 -- 172.21.15.16:0/2191510725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaac10c010 msgr2=0x7fbaac1a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:07.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.390+0000 7fbab1000640 1 --2- 172.21.15.16:0/2191510725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaac10c010 0x7fbaac1a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:07.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.390+0000 7fbab1000640 1 -- 172.21.15.16:0/2191510725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaac105d80 msgr2=0x7fbaac19c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:07.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.390+0000 7fbab1000640 1 --2- 172.21.15.16:0/2191510725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaac105d80 0x7fbaac19c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:07.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.390+0000 7fbab1000640 1 -- 172.21.15.16:0/2191510725 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaac1a2630 con 0x7fbaac10b6c0 2024-09-21T03:44:07.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.391+0000 7fbab1000640 1 --2- 172.21.15.16:0/2191510725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac10b6c0 0x7fbaac19cc60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbaac06caa0 tx=0x7fba9800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:07.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.391+0000 7fbaaa000640 1 -- 172.21.15.16:0/2191510725 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9800cc40 con 0x7fbaac10b6c0 2024-09-21T03:44:07.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.391+0000 7fbaaa000640 1 -- 172.21.15.16:0/2191510725 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba9800cde0 con 0x7fbaac10b6c0 2024-09-21T03:44:07.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.391+0000 7fbab4400640 1 -- 172.21.15.16:0/2191510725 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaac1a2920 con 0x7fbaac10b6c0 2024-09-21T03:44:07.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.391+0000 7fbaaa000640 1 -- 172.21.15.16:0/2191510725 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98018780 con 0x7fbaac10b6c0 2024-09-21T03:44:07.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.391+0000 7fbab4400640 1 -- 172.21.15.16:0/2191510725 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaac1a9080 con 0x7fbaac10b6c0 2024-09-21T03:44:07.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.393+0000 7fbab4400640 1 -- 172.21.15.16:0/2191510725 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaac106190 con 0x7fbaac10b6c0 2024-09-21T03:44:07.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.395+0000 7fbaaa000640 1 -- 172.21.15.16:0/2191510725 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fba98010a90 con 0x7fbaac10b6c0 2024-09-21T03:44:07.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.397+0000 7fbaaa000640 1 --2- 172.21.15.16:0/2191510725 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fba940b6510 0x7fba940b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:07.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.398+0000 7fbaaa000640 1 -- 172.21.15.16:0/2191510725 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7fba980e7920 con 0x7fbaac10b6c0 2024-09-21T03:44:07.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.398+0000 7fbab1a00640 1 --2- 172.21.15.16:0/2191510725 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fba940b6510 0x7fba940b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.399+0000 7fbab1a00640 1 --2- 172.21.15.16:0/2191510725 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fba940b6510 0x7fba940b8a20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0031440 tx=0x7fbaa00058d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.401+0000 7fbaaa000640 1 -- 172.21.15.16:0/2191510725 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fba980b21d0 con 0x7fbaac10b6c0 2024-09-21T03:44:07.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:07.643+0000 7fbab4400640 1 -- 172.21.15.16:0/2191510725 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fbaac10de40 con 0x7fbaac10b6c0 2024-09-21T03:44:07.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:07 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] up:active 2024-09-21T03:44:07.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:07 smithi134 ceph-mon[35031]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T03:44:07.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:07 smithi134 ceph-mon[35031]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:44:07.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:07 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] up:active 2024-09-21T03:44:07.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:07 smithi123 ceph-mon[34673]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T03:44:07.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:07 smithi123 ceph-mon[34673]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:44:07.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:07 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] up:active 2024-09-21T03:44:07.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:07 smithi016 ceph-mon[30375]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T03:44:07.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:07 smithi016 ceph-mon[30375]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:44:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:08.557+0000 7fbaaa000640 1 -- 172.21.15.16:0/2191510725 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7fba980b7110 con 0x7fbaac10b6c0 2024-09-21T03:44:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:08.559+0000 7fbab4400640 1 -- 172.21.15.16:0/2191510725 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fba940b6510 msgr2=0x7fba940b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:08.559+0000 7fbab4400640 1 --2- 172.21.15.16:0/2191510725 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fba940b6510 0x7fba940b8a20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0031440 tx=0x7fbaa00058d0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:08.559+0000 7fbab4400640 1 -- 172.21.15.16:0/2191510725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac10b6c0 msgr2=0x7fbaac19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:08.559+0000 7fbab4400640 1 --2- 172.21.15.16:0/2191510725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac10b6c0 0x7fbaac19cc60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbaac06caa0 tx=0x7fba9800ee50 comp rx=0 tx=0).stop 2024-09-21T03:44:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:08.559+0000 7fbab4400640 1 -- 172.21.15.16:0/2191510725 shutdown_connections 2024-09-21T03:44:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:08.559+0000 7fbab4400640 1 --2- 172.21.15.16:0/2191510725 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fba940b6510 0x7fba940b8a20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:08.559+0000 7fbab4400640 1 --2- 172.21.15.16:0/2191510725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaac10c010 0x7fbaac1a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:08.559+0000 7fbab4400640 1 --2- 172.21.15.16:0/2191510725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac10b6c0 0x7fbaac19cc60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:08.559+0000 7fbab4400640 1 --2- 172.21.15.16:0/2191510725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaac105d80 0x7fbaac19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:08.559+0000 7fbab4400640 1 -- 172.21.15.16:0/2191510725 >> 172.21.15.16:0/2191510725 conn(0x7fbaac101810 msgr2=0x7fbaac103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:08.560+0000 7fbab4400640 1 -- 172.21.15.16:0/2191510725 shutdown_connections 2024-09-21T03:44:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:08.560+0000 7fbab4400640 1 -- 172.21.15.16:0/2191510725 wait complete. 2024-09-21T03:44:08.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:08 smithi134 ceph-mon[35031]: pgmap v351: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-21T03:44:08.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:08 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2191510725' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:44:08.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:08 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:44:08.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:08 smithi123 ceph-mon[34673]: pgmap v351: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-21T03:44:08.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:08 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2191510725' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:44:08.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:08 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:44:08.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:08 smithi016 ceph-mon[30375]: pgmap v351: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-21T03:44:08.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:08 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2191510725' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:44:08.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:08 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:44:09.525 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-21T03:44:09.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T03:44:09.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:09 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:44:09.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:09 smithi123 ceph-mon[34673]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:44:09.657 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 03:44:09 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-b[62470]: 2024-09-21T03:44:09.390+0000 7f4e89757640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-21T03:44:09.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:09 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:44:09.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:09 smithi134 ceph-mon[35031]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:44:09.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:09 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:44:09.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:09 smithi016 ceph-mon[30375]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:44:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.601+0000 7f386b200640 1 -- 172.21.15.16:0/3579825126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c0757b0 msgr2=0x7f386c075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.601+0000 7f386b200640 1 --2- 172.21.15.16:0/3579825126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c0757b0 0x7f386c075bc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f385c009900 tx=0x7f385c02a400 comp rx=0 tx=0).stop 2024-09-21T03:44:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.601+0000 7f386b200640 1 -- 172.21.15.16:0/3579825126 shutdown_connections 2024-09-21T03:44:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.601+0000 7f386b200640 1 --2- 172.21.15.16:0/3579825126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f386c10ff10 0x7f386c112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.601+0000 7f386b200640 1 --2- 172.21.15.16:0/3579825126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f386c076100 0x7f386c10f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.602+0000 7f386b200640 1 --2- 172.21.15.16:0/3579825126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c0757b0 0x7f386c075bc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.602+0000 7f386b200640 1 -- 172.21.15.16:0/3579825126 >> 172.21.15.16:0/3579825126 conn(0x7f386c0ff440 msgr2=0x7f386c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.602+0000 7f386b200640 1 -- 172.21.15.16:0/3579825126 shutdown_connections 2024-09-21T03:44:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.602+0000 7f386b200640 1 -- 172.21.15.16:0/3579825126 wait complete. 2024-09-21T03:44:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.603+0000 7f386b200640 1 Processor -- start 2024-09-21T03:44:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.603+0000 7f386b200640 1 -- start start 2024-09-21T03:44:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.603+0000 7f386b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f386c0757b0 0x7f386c19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.603+0000 7f386b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c076100 0x7f386c19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.603+0000 7f386b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f386c10ff10 0x7f386c1a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.603+0000 7f386b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f386c115ac0 con 0x7f386c076100 2024-09-21T03:44:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.603+0000 7f386b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f386c115940 con 0x7f386c0757b0 2024-09-21T03:44:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.603+0000 7f386b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f386c115c40 con 0x7f386c10ff10 2024-09-21T03:44:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.604+0000 7f3869400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c076100 0x7f386c19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.604+0000 7f3869e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f386c0757b0 0x7f386c19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.604+0000 7f3869400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c076100 0x7f386c19ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46526/0 (socket says 172.21.15.16:46526) 2024-09-21T03:44:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.604+0000 7f3869400640 1 -- 172.21.15.16:0/856900930 learned_addr learned my addr 172.21.15.16:0/856900930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.604+0000 7f386a800640 1 --2- 172.21.15.16:0/856900930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f386c10ff10 0x7f386c1a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.604+0000 7f3869e00640 1 -- 172.21.15.16:0/856900930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f386c10ff10 msgr2=0x7f386c1a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.604+0000 7f3869e00640 1 --2- 172.21.15.16:0/856900930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f386c10ff10 0x7f386c1a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.604+0000 7f3869e00640 1 -- 172.21.15.16:0/856900930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c076100 msgr2=0x7f386c19ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.604+0000 7f3869e00640 1 --2- 172.21.15.16:0/856900930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c076100 0x7f386c19ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.604+0000 7f3869e00640 1 -- 172.21.15.16:0/856900930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386c1a2460 con 0x7f386c0757b0 2024-09-21T03:44:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.604+0000 7f3869400640 1 --2- 172.21.15.16:0/856900930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c076100 0x7f386c19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:10.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.605+0000 7f3869e00640 1 --2- 172.21.15.16:0/856900930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f386c0757b0 0x7f386c19c4f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f385c02a920 tx=0x7f385c031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:10.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.605+0000 7f3862a00640 1 -- 172.21.15.16:0/856900930 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c03f030 con 0x7f386c0757b0 2024-09-21T03:44:10.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.605+0000 7f386b200640 1 -- 172.21.15.16:0/856900930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f386c1a26f0 con 0x7f386c0757b0 2024-09-21T03:44:10.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.605+0000 7f3862a00640 1 -- 172.21.15.16:0/856900930 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f385c0043d0 con 0x7f386c0757b0 2024-09-21T03:44:10.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.605+0000 7f386b200640 1 -- 172.21.15.16:0/856900930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f386c1a8eb0 con 0x7f386c0757b0 2024-09-21T03:44:10.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.605+0000 7f3862a00640 1 -- 172.21.15.16:0/856900930 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c035070 con 0x7f386c0757b0 2024-09-21T03:44:10.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.607+0000 7f386b200640 1 -- 172.21.15.16:0/856900930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f386c075bc0 con 0x7f386c0757b0 2024-09-21T03:44:10.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.609+0000 7f3862a00640 1 -- 172.21.15.16:0/856900930 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f385c042030 con 0x7f386c0757b0 2024-09-21T03:44:10.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.611+0000 7f3862a00640 1 --2- 172.21.15.16:0/856900930 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38500b6590 0x7f38500b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:10.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.612+0000 7f3862a00640 1 -- 172.21.15.16:0/856900930 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f385c107820 con 0x7f386c0757b0 2024-09-21T03:44:10.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.612+0000 7f3869400640 1 --2- 172.21.15.16:0/856900930 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38500b6590 0x7f38500b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:10.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.612+0000 7f3869400640 1 --2- 172.21.15.16:0/856900930 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38500b6590 0x7f38500b8aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f386c108680 tx=0x7f3858005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:10.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.614+0000 7f3862a00640 1 -- 172.21.15.16:0/856900930 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f385c0cdcf0 con 0x7f386c0757b0 2024-09-21T03:44:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.839+0000 7f386b200640 1 -- 172.21.15.16:0/856900930 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f386c10a1e0 con 0x7f38500b6590 2024-09-21T03:44:10.841 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:10 smithi016 ceph-mon[30375]: pgmap v352: 129 pgs: 129 active+clean; 585 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-21T03:44:10.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:10 smithi134 ceph-mon[35031]: pgmap v352: 129 pgs: 129 active+clean; 585 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-21T03:44:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.873+0000 7f3862a00640 1 -- 172.21.15.16:0/856900930 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f386c10a1e0 con 0x7f38500b6590 2024-09-21T03:44:10.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.875+0000 7f386b200640 1 -- 172.21.15.16:0/856900930 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38500b6590 msgr2=0x7f38500b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:10.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.875+0000 7f386b200640 1 --2- 172.21.15.16:0/856900930 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38500b6590 0x7f38500b8aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f386c108680 tx=0x7f3858005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.875+0000 7f386b200640 1 -- 172.21.15.16:0/856900930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f386c0757b0 msgr2=0x7f386c19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.875+0000 7f386b200640 1 --2- 172.21.15.16:0/856900930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f386c0757b0 0x7f386c19c4f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f385c02a920 tx=0x7f385c031a20 comp rx=0 tx=0).stop 2024-09-21T03:44:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.875+0000 7f386b200640 1 -- 172.21.15.16:0/856900930 shutdown_connections 2024-09-21T03:44:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.875+0000 7f386b200640 1 --2- 172.21.15.16:0/856900930 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38500b6590 0x7f38500b8aa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.875+0000 7f386b200640 1 --2- 172.21.15.16:0/856900930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f386c10ff10 0x7f386c1a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.876+0000 7f386b200640 1 --2- 172.21.15.16:0/856900930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c076100 0x7f386c19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.876+0000 7f386b200640 1 --2- 172.21.15.16:0/856900930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f386c0757b0 0x7f386c19c4f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.876+0000 7f386b200640 1 -- 172.21.15.16:0/856900930 >> 172.21.15.16:0/856900930 conn(0x7f386c0ff440 msgr2=0x7f386c109ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.876+0000 7f386b200640 1 -- 172.21.15.16:0/856900930 shutdown_connections 2024-09-21T03:44:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:10.876+0000 7f386b200640 1 -- 172.21.15.16:0/856900930 wait complete. 2024-09-21T03:44:10.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:10 smithi123 ceph-mon[34673]: pgmap v352: 129 pgs: 129 active+clean; 585 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-21T03:44:11.162 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:44:10 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: 2024-09-21T03:44:10.848+0000 7efca8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:44:11.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:44:10 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: 2024-09-21T03:44:10.848+0000 7efca8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:44:11.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:44:10 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: 2024-09-21T03:44:10.848+0000 7efca8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:44:11.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:44:10 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: 2024-09-21T03:44:10.848+0000 7efca8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:44:11.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:44:10 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: 2024-09-21T03:44:10.848+0000 7efca8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:44:11.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T03:44:11.831 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:11 smithi016 ceph-mon[30375]: from='client.24905 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:11.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:11 smithi134 ceph-mon[35031]: from='client.24905 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:11.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:11 smithi123 ceph-mon[34673]: from='client.24905 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:12.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.770+0000 7f79fa400640 1 -- 172.21.15.16:0/2857449008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79f41051d0 msgr2=0x7f79f410d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:12.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.770+0000 7f79fa400640 1 --2- 172.21.15.16:0/2857449008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79f41051d0 0x7f79f410d720 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009900 tx=0x7f79e002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.771+0000 7f79fa400640 1 -- 172.21.15.16:0/2857449008 shutdown_connections 2024-09-21T03:44:12.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.771+0000 7f79fa400640 1 --2- 172.21.15.16:0/2857449008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79f410dce0 0x7f79f41101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.771+0000 7f79fa400640 1 --2- 172.21.15.16:0/2857449008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79f41051d0 0x7f79f410d720 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.771+0000 7f79fa400640 1 --2- 172.21.15.16:0/2857449008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79f4104880 0x7f79f4104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.771+0000 7f79fa400640 1 -- 172.21.15.16:0/2857449008 >> 172.21.15.16:0/2857449008 conn(0x7f79f40fe360 msgr2=0x7f79f41007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:12.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.771+0000 7f79fa400640 1 -- 172.21.15.16:0/2857449008 shutdown_connections 2024-09-21T03:44:12.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.771+0000 7f79fa400640 1 -- 172.21.15.16:0/2857449008 wait complete. 2024-09-21T03:44:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.772+0000 7f79fa400640 1 Processor -- start 2024-09-21T03:44:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.772+0000 7f79fa400640 1 -- start start 2024-09-21T03:44:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.772+0000 7f79fa400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79f4104880 0x7f79f419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.772+0000 7f79fa400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79f41051d0 0x7f79f419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.772+0000 7f79fa400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79f410dce0 0x7f79f419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.772+0000 7f79fa400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f79f4113940 con 0x7f79f4104880 2024-09-21T03:44:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.772+0000 7f79fa400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79f41137c0 con 0x7f79f41051d0 2024-09-21T03:44:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.772+0000 7f79fa400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f79f4113ac0 con 0x7f79f410dce0 2024-09-21T03:44:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.773+0000 7f79f3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79f41051d0 0x7f79f419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.773+0000 7f79f9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79f4104880 0x7f79f419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.773+0000 7f79f3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79f41051d0 0x7f79f419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:40736/0 (socket says 172.21.15.16:40736) 2024-09-21T03:44:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.773+0000 7f79f3e00640 1 -- 172.21.15.16:0/1695792371 learned_addr learned my addr 172.21.15.16:0/1695792371 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:12.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.773+0000 7f79f9a00640 1 --2- 172.21.15.16:0/1695792371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79f410dce0 0x7f79f419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:12.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.773+0000 7f79f3e00640 1 -- 172.21.15.16:0/1695792371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79f410dce0 msgr2=0x7f79f419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:12.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.773+0000 7f79f3e00640 1 --2- 172.21.15.16:0/1695792371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79f410dce0 0x7f79f419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.773+0000 7f79f3e00640 1 -- 172.21.15.16:0/1695792371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79f4104880 msgr2=0x7f79f419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.773+0000 7f79f3e00640 1 --2- 172.21.15.16:0/1695792371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79f4104880 0x7f79f419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.773+0000 7f79f3e00640 1 -- 172.21.15.16:0/1695792371 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f41a0360 con 0x7f79f41051d0 2024-09-21T03:44:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.774+0000 7f79f3e00640 1 --2- 172.21.15.16:0/1695792371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79f41051d0 0x7f79f419a990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f79e002a910 tx=0x7f79e0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.774+0000 7f79f1600640 1 -- 172.21.15.16:0/1695792371 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e003f030 con 0x7f79f41051d0 2024-09-21T03:44:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.774+0000 7f79fa400640 1 -- 172.21.15.16:0/1695792371 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f41a05f0 con 0x7f79f41051d0 2024-09-21T03:44:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.774+0000 7f79f1600640 1 -- 172.21.15.16:0/1695792371 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79e0003aa0 con 0x7f79f41051d0 2024-09-21T03:44:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.774+0000 7f79f1600640 1 -- 172.21.15.16:0/1695792371 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0035070 con 0x7f79f41051d0 2024-09-21T03:44:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.774+0000 7f79fa400640 1 -- 172.21.15.16:0/1695792371 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f41a6db0 con 0x7f79f41051d0 2024-09-21T03:44:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.775+0000 7f79fa400640 1 -- 172.21.15.16:0/1695792371 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f4104c90 con 0x7f79f41051d0 2024-09-21T03:44:12.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.778+0000 7f79f1600640 1 -- 172.21.15.16:0/1695792371 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f79e0042030 con 0x7f79f41051d0 2024-09-21T03:44:12.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.779+0000 7f79f1600640 1 --2- 172.21.15.16:0/1695792371 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f79dc0b6620 0x7f79dc0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:12.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.779+0000 7f79f1600640 1 -- 172.21.15.16:0/1695792371 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f79e0107e00 con 0x7f79f41051d0 2024-09-21T03:44:12.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.779+0000 7f79f9000640 1 --2- 172.21.15.16:0/1695792371 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f79dc0b6620 0x7f79dc0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:12.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.780+0000 7f79f9000640 1 --2- 172.21.15.16:0/1695792371 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f79dc0b6620 0x7f79dc0b8b30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f79e800bbe0 tx=0x7f79e800b390 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:12.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:12.781+0000 7f79f1600640 1 -- 172.21.15.16:0/1695792371 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f79e00d2550 con 0x7f79f41051d0 2024-09-21T03:44:12.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:12 smithi134 ceph-mon[35031]: pgmap v353: 129 pgs: 129 active+clean; 587 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-09-21T03:44:12.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:12 smithi134 ceph-mon[35031]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-09-21T03:44:12.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:12 smithi123 ceph-mon[34673]: pgmap v353: 129 pgs: 129 active+clean; 587 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-09-21T03:44:12.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:12 smithi123 ceph-mon[34673]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-09-21T03:44:12.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:12 smithi016 ceph-mon[30375]: pgmap v353: 129 pgs: 129 active+clean; 587 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-09-21T03:44:12.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:12 smithi016 ceph-mon[30375]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-09-21T03:44:13.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:13.011+0000 7f79fa400640 1 -- 172.21.15.16:0/1695792371 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f79f4107f70 con 0x7f79dc0b6620 2024-09-21T03:44:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:13.043+0000 7f79f1600640 1 -- 172.21.15.16:0/1695792371 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f79f4107f70 con 0x7f79dc0b6620 2024-09-21T03:44:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:13.045+0000 7f79fa400640 1 -- 172.21.15.16:0/1695792371 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f79dc0b6620 msgr2=0x7f79dc0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:13.045+0000 7f79fa400640 1 --2- 172.21.15.16:0/1695792371 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f79dc0b6620 0x7f79dc0b8b30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f79e800bbe0 tx=0x7f79e800b390 comp rx=0 tx=0).stop 2024-09-21T03:44:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:13.045+0000 7f79fa400640 1 -- 172.21.15.16:0/1695792371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79f41051d0 msgr2=0x7f79f419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:13.045+0000 7f79fa400640 1 --2- 172.21.15.16:0/1695792371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79f41051d0 0x7f79f419a990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f79e002a910 tx=0x7f79e0004270 comp rx=0 tx=0).stop 2024-09-21T03:44:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:13.045+0000 7f79fa400640 1 -- 172.21.15.16:0/1695792371 shutdown_connections 2024-09-21T03:44:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:13.045+0000 7f79fa400640 1 --2- 172.21.15.16:0/1695792371 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f79dc0b6620 0x7f79dc0b8b30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:13.045+0000 7f79fa400640 1 --2- 172.21.15.16:0/1695792371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79f410dce0 0x7f79f419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:13.045+0000 7f79fa400640 1 --2- 172.21.15.16:0/1695792371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79f41051d0 0x7f79f419a990 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:13.045+0000 7f79fa400640 1 --2- 172.21.15.16:0/1695792371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79f4104880 0x7f79f419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:13.045+0000 7f79fa400640 1 -- 172.21.15.16:0/1695792371 >> 172.21.15.16:0/1695792371 conn(0x7f79f40fe360 msgr2=0x7f79f4107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:13.046+0000 7f79fa400640 1 -- 172.21.15.16:0/1695792371 shutdown_connections 2024-09-21T03:44:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:13.046+0000 7f79fa400640 1 -- 172.21.15.16:0/1695792371 wait complete. 2024-09-21T03:44:13.811 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:13 smithi016 ceph-mon[30375]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:13.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T03:44:13.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:13 smithi134 ceph-mon[35031]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:13.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:13 smithi123 ceph-mon[34673]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:14.233 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:14.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:14 smithi016 ceph-mon[30375]: pgmap v354: 129 pgs: 129 active+clean; 593 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.8 KiB/s wr, 21 op/s 2024-09-21T03:44:14.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:14 smithi134 ceph-mon[35031]: pgmap v354: 129 pgs: 129 active+clean; 593 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.8 KiB/s wr, 21 op/s 2024-09-21T03:44:14.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.897+0000 7fee53e00640 1 -- 172.21.15.16:0/1547947183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c109e10 msgr2=0x7fee4c110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:14.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.897+0000 7fee53e00640 1 --2- 172.21.15.16:0/1547947183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c109e10 0x7fee4c110640 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fee44009ca0 tx=0x7fee4402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.897+0000 7fee53e00640 1 -- 172.21.15.16:0/1547947183 shutdown_connections 2024-09-21T03:44:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.897+0000 7fee53e00640 1 --2- 172.21.15.16:0/1547947183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c109e10 0x7fee4c110640 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.897+0000 7fee53e00640 1 --2- 172.21.15.16:0/1547947183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee4c1094c0 0x7fee4c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.897+0000 7fee53e00640 1 --2- 172.21.15.16:0/1547947183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c103b80 0x7fee4c103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.897+0000 7fee53e00640 1 -- 172.21.15.16:0/1547947183 >> 172.21.15.16:0/1547947183 conn(0x7fee4c0ff5d0 msgr2=0x7fee4c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.897+0000 7fee53e00640 1 -- 172.21.15.16:0/1547947183 shutdown_connections 2024-09-21T03:44:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.897+0000 7fee53e00640 1 -- 172.21.15.16:0/1547947183 wait complete. 2024-09-21T03:44:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.899+0000 7fee53e00640 1 Processor -- start 2024-09-21T03:44:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.899+0000 7fee53e00640 1 -- start start 2024-09-21T03:44:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.899+0000 7fee53e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee4c103b80 0x7fee4c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.899+0000 7fee53e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c1094c0 0x7fee4c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.899+0000 7fee53e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c109e10 0x7fee4c1a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.899+0000 7fee53e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c113ad0 con 0x7fee4c1094c0 2024-09-21T03:44:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.899+0000 7fee53e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c113950 con 0x7fee4c109e10 2024-09-21T03:44:14.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.899+0000 7fee53e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c113c50 con 0x7fee4c103b80 2024-09-21T03:44:14.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.899+0000 7fee51400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee4c103b80 0x7fee4c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.899+0000 7fee50a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c1094c0 0x7fee4c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.899+0000 7fee51400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee4c103b80 0x7fee4c19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:48236/0 (socket says 172.21.15.16:48236) 2024-09-21T03:44:14.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.899+0000 7fee51e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c109e10 0x7fee4c1a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.899+0000 7fee51400640 1 -- 172.21.15.16:0/1422432630 learned_addr learned my addr 172.21.15.16:0/1422432630 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:14.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.900+0000 7fee50a00640 1 -- 172.21.15.16:0/1422432630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee4c103b80 msgr2=0x7fee4c19c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:14.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.900+0000 7fee50a00640 1 --2- 172.21.15.16:0/1422432630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee4c103b80 0x7fee4c19c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.900+0000 7fee50a00640 1 -- 172.21.15.16:0/1422432630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c109e10 msgr2=0x7fee4c1a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:14.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.900+0000 7fee50a00640 1 --2- 172.21.15.16:0/1422432630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c109e10 0x7fee4c1a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.900+0000 7fee50a00640 1 -- 172.21.15.16:0/1422432630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4c1a2630 con 0x7fee4c1094c0 2024-09-21T03:44:14.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.900+0000 7fee51e00640 1 --2- 172.21.15.16:0/1422432630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c109e10 0x7fee4c1a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:14.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.900+0000 7fee50a00640 1 --2- 172.21.15.16:0/1422432630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c1094c0 0x7fee4c19cc60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fee4c06ca10 tx=0x7fee3800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:14.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.901+0000 7fee4a000640 1 -- 172.21.15.16:0/1422432630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3800cc40 con 0x7fee4c1094c0 2024-09-21T03:44:14.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.901+0000 7fee4a000640 1 -- 172.21.15.16:0/1422432630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee3800cde0 con 0x7fee4c1094c0 2024-09-21T03:44:14.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.901+0000 7fee53e00640 1 -- 172.21.15.16:0/1422432630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4c1a2920 con 0x7fee4c1094c0 2024-09-21T03:44:14.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.901+0000 7fee4a000640 1 -- 172.21.15.16:0/1422432630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee38004a00 con 0x7fee4c1094c0 2024-09-21T03:44:14.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.901+0000 7fee53e00640 1 -- 172.21.15.16:0/1422432630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee4c1a9080 con 0x7fee4c1094c0 2024-09-21T03:44:14.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.902+0000 7fee53e00640 1 -- 172.21.15.16:0/1422432630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee4c103f90 con 0x7fee4c1094c0 2024-09-21T03:44:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.904+0000 7fee4a000640 1 -- 172.21.15.16:0/1422432630 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fee38004ba0 con 0x7fee4c1094c0 2024-09-21T03:44:14.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:14 smithi123 ceph-mon[34673]: pgmap v354: 129 pgs: 129 active+clean; 593 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.8 KiB/s wr, 21 op/s 2024-09-21T03:44:14.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.906+0000 7fee4a000640 1 --2- 172.21.15.16:0/1422432630 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fee340b6620 0x7fee340b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.906+0000 7fee4a000640 1 -- 172.21.15.16:0/1422432630 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7fee380e7720 con 0x7fee4c1094c0 2024-09-21T03:44:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.906+0000 7fee51400640 1 --2- 172.21.15.16:0/1422432630 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fee340b6620 0x7fee340b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.907+0000 7fee51400640 1 --2- 172.21.15.16:0/1422432630 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fee340b6620 0x7fee340b8b30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fee40009c90 tx=0x7fee40005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:14.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:14.910+0000 7fee4a000640 1 -- 172.21.15.16:0/1422432630 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fee380ada90 con 0x7fee4c1094c0 2024-09-21T03:44:15.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:15.138+0000 7fee53e00640 1 -- 172.21.15.16:0/1422432630 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fee4c10f910 con 0x7fee340b6620 2024-09-21T03:44:15.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:15.144+0000 7fee4a000640 1 -- 172.21.15.16:0/1422432630 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fee4c10f910 con 0x7fee340b6620 2024-09-21T03:44:15.145 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_0/f9ebd23e-130b-4649-b081-1d8708a49444 2024-09-21T03:44:15.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:15.146+0000 7fee53e00640 1 -- 172.21.15.16:0/1422432630 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fee340b6620 msgr2=0x7fee340b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:15.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:15.146+0000 7fee53e00640 1 --2- 172.21.15.16:0/1422432630 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fee340b6620 0x7fee340b8b30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fee40009c90 tx=0x7fee40005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:15.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:15.146+0000 7fee53e00640 1 -- 172.21.15.16:0/1422432630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c1094c0 msgr2=0x7fee4c19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:15.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:15.146+0000 7fee53e00640 1 --2- 172.21.15.16:0/1422432630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c1094c0 0x7fee4c19cc60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fee4c06ca10 tx=0x7fee3800ee50 comp rx=0 tx=0).stop 2024-09-21T03:44:15.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:15.146+0000 7fee53e00640 1 -- 172.21.15.16:0/1422432630 shutdown_connections 2024-09-21T03:44:15.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:15.146+0000 7fee53e00640 1 --2- 172.21.15.16:0/1422432630 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fee340b6620 0x7fee340b8b30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:15.146+0000 7fee53e00640 1 --2- 172.21.15.16:0/1422432630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c109e10 0x7fee4c1a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:15.146+0000 7fee53e00640 1 --2- 172.21.15.16:0/1422432630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c1094c0 0x7fee4c19cc60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:15.146+0000 7fee53e00640 1 --2- 172.21.15.16:0/1422432630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee4c103b80 0x7fee4c19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:15.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:15.146+0000 7fee53e00640 1 -- 172.21.15.16:0/1422432630 >> 172.21.15.16:0/1422432630 conn(0x7fee4c0ff5d0 msgr2=0x7fee4c108b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:15.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:15.147+0000 7fee53e00640 1 -- 172.21.15.16:0/1422432630 shutdown_connections 2024-09-21T03:44:15.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:15.147+0000 7fee53e00640 1 -- 172.21.15.16:0/1422432630 wait complete. 2024-09-21T03:44:15.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T03:44:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:16.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:16 smithi016 ceph-mon[30375]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:16.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:16 smithi016 ceph-mon[30375]: pgmap v355: 129 pgs: 129 active+clean; 593 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.8 KiB/s wr, 21 op/s 2024-09-21T03:44:16.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:16 smithi134 ceph-mon[35031]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:16.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:16 smithi134 ceph-mon[35031]: pgmap v355: 129 pgs: 129 active+clean; 593 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.8 KiB/s wr, 21 op/s 2024-09-21T03:44:16.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:16 smithi123 ceph-mon[34673]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:16.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:16 smithi123 ceph-mon[34673]: pgmap v355: 129 pgs: 129 active+clean; 593 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.8 KiB/s wr, 21 op/s 2024-09-21T03:44:16.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.948+0000 7ff4eb600640 1 -- 172.21.15.16:0/413128955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e4105db0 msgr2=0x7ff4e41061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:16.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.948+0000 7ff4eb600640 1 --2- 172.21.15.16:0/413128955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e4105db0 0x7ff4e41061c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8009900 tx=0x7ff4d802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.948+0000 7ff4eb600640 1 -- 172.21.15.16:0/413128955 shutdown_connections 2024-09-21T03:44:16.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.948+0000 7ff4eb600640 1 --2- 172.21.15.16:0/413128955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e410c040 0x7ff4e4112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.948+0000 7ff4eb600640 1 --2- 172.21.15.16:0/413128955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4e410b6f0 0x7ff4e410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.948+0000 7ff4eb600640 1 --2- 172.21.15.16:0/413128955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e4105db0 0x7ff4e41061c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.948+0000 7ff4eb600640 1 -- 172.21.15.16:0/413128955 >> 172.21.15.16:0/413128955 conn(0x7ff4e41017e0 msgr2=0x7ff4e4103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:16.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.948+0000 7ff4eb600640 1 -- 172.21.15.16:0/413128955 shutdown_connections 2024-09-21T03:44:16.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.948+0000 7ff4eb600640 1 -- 172.21.15.16:0/413128955 wait complete. 2024-09-21T03:44:16.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.949+0000 7ff4eb600640 1 Processor -- start 2024-09-21T03:44:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.950+0000 7ff4eb600640 1 -- start start 2024-09-21T03:44:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.950+0000 7ff4eb600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4e4105db0 0x7ff4e419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.950+0000 7ff4eb600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e410b6f0 0x7ff4e419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.950+0000 7ff4eb600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e410c040 0x7ff4e41a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.950+0000 7ff4eb600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4115d00 con 0x7ff4e410c040 2024-09-21T03:44:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.950+0000 7ff4eb600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4115b80 con 0x7ff4e410b6f0 2024-09-21T03:44:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.950+0000 7ff4eb600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4115e80 con 0x7ff4e4105db0 2024-09-21T03:44:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.950+0000 7ff4e8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4e4105db0 0x7ff4e419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.950+0000 7ff4e9600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e410c040 0x7ff4e41a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.950+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e410b6f0 0x7ff4e419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.950+0000 7ff4e9600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e410c040 0x7ff4e41a1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52162/0 (socket says 172.21.15.16:52162) 2024-09-21T03:44:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.950+0000 7ff4e8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4e4105db0 0x7ff4e419c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:48264/0 (socket says 172.21.15.16:48264) 2024-09-21T03:44:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.950+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e410b6f0 0x7ff4e419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:40744/0 (socket says 172.21.15.16:40744) 2024-09-21T03:44:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.950+0000 7ff4e9600640 1 -- 172.21.15.16:0/3376698733 learned_addr learned my addr 172.21.15.16:0/3376698733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.951+0000 7ff4e8c00640 1 -- 172.21.15.16:0/3376698733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e410b6f0 msgr2=0x7ff4e419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.951+0000 7ff4e8c00640 1 --2- 172.21.15.16:0/3376698733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e410b6f0 0x7ff4e419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.951+0000 7ff4e8c00640 1 -- 172.21.15.16:0/3376698733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e410c040 msgr2=0x7ff4e41a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.951+0000 7ff4e8c00640 1 --2- 172.21.15.16:0/3376698733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e410c040 0x7ff4e41a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.951+0000 7ff4e8c00640 1 -- 172.21.15.16:0/3376698733 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e41a2680 con 0x7ff4e4105db0 2024-09-21T03:44:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.951+0000 7ff4e3e00640 1 --2- 172.21.15.16:0/3376698733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e410b6f0 0x7ff4e419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.951+0000 7ff4e8c00640 1 --2- 172.21.15.16:0/3376698733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4e4105db0 0x7ff4e419c710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff4d802a910 tx=0x7ff4d8031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.952+0000 7ff4e1600640 1 -- 172.21.15.16:0/3376698733 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d803f030 con 0x7ff4e4105db0 2024-09-21T03:44:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.952+0000 7ff4eb600640 1 -- 172.21.15.16:0/3376698733 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e41a2910 con 0x7ff4e4105db0 2024-09-21T03:44:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.952+0000 7ff4e1600640 1 -- 172.21.15.16:0/3376698733 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d8004490 con 0x7ff4e4105db0 2024-09-21T03:44:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.952+0000 7ff4e1600640 1 -- 172.21.15.16:0/3376698733 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8035070 con 0x7ff4e4105db0 2024-09-21T03:44:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.952+0000 7ff4eb600640 1 -- 172.21.15.16:0/3376698733 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e41a90d0 con 0x7ff4e4105db0 2024-09-21T03:44:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.953+0000 7ff4eb600640 1 -- 172.21.15.16:0/3376698733 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e41061c0 con 0x7ff4e4105db0 2024-09-21T03:44:16.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.955+0000 7ff4e1600640 1 -- 172.21.15.16:0/3376698733 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7ff4d8042030 con 0x7ff4e4105db0 2024-09-21T03:44:16.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.957+0000 7ff4e1600640 1 --2- 172.21.15.16:0/3376698733 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff4cc0b65a0 0x7ff4cc0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:16.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.957+0000 7ff4e1600640 1 -- 172.21.15.16:0/3376698733 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7ff4d8107d10 con 0x7ff4e4105db0 2024-09-21T03:44:16.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.957+0000 7ff4e3e00640 1 --2- 172.21.15.16:0/3376698733 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff4cc0b65a0 0x7ff4cc0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:16.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.958+0000 7ff4e3e00640 1 --2- 172.21.15.16:0/3376698733 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff4cc0b65a0 0x7ff4cc0b8ab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4104ff0 tx=0x7ff4d0005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:16.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:16.959+0000 7ff4e1600640 1 -- 172.21.15.16:0/3376698733 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff4d80d2460 con 0x7ff4e4105db0 2024-09-21T03:44:17.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:17.190+0000 7ff4eb600640 1 -- 172.21.15.16:0/3376698733 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff4e4063800 con 0x7ff4cc0b65a0 2024-09-21T03:44:17.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:17.222+0000 7ff4e1600640 1 -- 172.21.15.16:0/3376698733 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff4e4063800 con 0x7ff4cc0b65a0 2024-09-21T03:44:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:17.225+0000 7ff4eb600640 1 -- 172.21.15.16:0/3376698733 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff4cc0b65a0 msgr2=0x7ff4cc0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:17.225+0000 7ff4eb600640 1 --2- 172.21.15.16:0/3376698733 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff4cc0b65a0 0x7ff4cc0b8ab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4104ff0 tx=0x7ff4d0005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:17.225+0000 7ff4eb600640 1 -- 172.21.15.16:0/3376698733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4e4105db0 msgr2=0x7ff4e419c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:17.225+0000 7ff4eb600640 1 --2- 172.21.15.16:0/3376698733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4e4105db0 0x7ff4e419c710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff4d802a910 tx=0x7ff4d8031a20 comp rx=0 tx=0).stop 2024-09-21T03:44:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:17.225+0000 7ff4eb600640 1 -- 172.21.15.16:0/3376698733 shutdown_connections 2024-09-21T03:44:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:17.225+0000 7ff4eb600640 1 --2- 172.21.15.16:0/3376698733 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff4cc0b65a0 0x7ff4cc0b8ab0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:17.225+0000 7ff4eb600640 1 --2- 172.21.15.16:0/3376698733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e410c040 0x7ff4e41a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:17.225+0000 7ff4eb600640 1 --2- 172.21.15.16:0/3376698733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e410b6f0 0x7ff4e419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:17.225+0000 7ff4eb600640 1 --2- 172.21.15.16:0/3376698733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4e4105db0 0x7ff4e419c710 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:17.225+0000 7ff4eb600640 1 -- 172.21.15.16:0/3376698733 >> 172.21.15.16:0/3376698733 conn(0x7ff4e41017e0 msgr2=0x7ff4e410adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:17.225+0000 7ff4eb600640 1 -- 172.21.15.16:0/3376698733 shutdown_connections 2024-09-21T03:44:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:17.225+0000 7ff4eb600640 1 -- 172.21.15.16:0/3376698733 wait complete. 2024-09-21T03:44:18.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T03:44:18.500 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:18.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:18 smithi134 ceph-mon[35031]: from='client.24907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:18.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:18 smithi134 ceph-mon[35031]: pgmap v356: 129 pgs: 129 active+clean; 593 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.3 KiB/s wr, 19 op/s 2024-09-21T03:44:18.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:18 smithi123 ceph-mon[34673]: from='client.24907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:18.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:18 smithi123 ceph-mon[34673]: pgmap v356: 129 pgs: 129 active+clean; 593 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.3 KiB/s wr, 19 op/s 2024-09-21T03:44:18.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:18 smithi016 ceph-mon[30375]: from='client.24907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:18.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:18 smithi016 ceph-mon[30375]: pgmap v356: 129 pgs: 129 active+clean; 593 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.3 KiB/s wr, 19 op/s 2024-09-21T03:44:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.142+0000 7fd41d600640 1 -- 172.21.15.16:0/4264857799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41810c020 msgr2=0x7fd418112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.142+0000 7fd41d600640 1 --2- 172.21.15.16:0/4264857799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41810c020 0x7fd418112850 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd410009ca0 tx=0x7fd41002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.142+0000 7fd41d600640 1 -- 172.21.15.16:0/4264857799 shutdown_connections 2024-09-21T03:44:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.142+0000 7fd41d600640 1 --2- 172.21.15.16:0/4264857799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41810c020 0x7fd418112850 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.142+0000 7fd41d600640 1 --2- 172.21.15.16:0/4264857799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd41810b6d0 0x7fd41810bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.142+0000 7fd41d600640 1 --2- 172.21.15.16:0/4264857799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd418105d90 0x7fd4181061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.142+0000 7fd41d600640 1 -- 172.21.15.16:0/4264857799 >> 172.21.15.16:0/4264857799 conn(0x7fd418101860 msgr2=0x7fd418103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.142+0000 7fd41d600640 1 -- 172.21.15.16:0/4264857799 shutdown_connections 2024-09-21T03:44:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.142+0000 7fd41d600640 1 -- 172.21.15.16:0/4264857799 wait complete. 2024-09-21T03:44:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.144+0000 7fd41d600640 1 Processor -- start 2024-09-21T03:44:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.144+0000 7fd41d600640 1 -- start start 2024-09-21T03:44:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.144+0000 7fd41d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd418105d90 0x7fd41819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.144+0000 7fd41d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41810b6d0 0x7fd41819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.144+0000 7fd41d600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd41810c020 0x7fd4181a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.144+0000 7fd41d600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd418115d30 con 0x7fd41810b6d0 2024-09-21T03:44:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.144+0000 7fd41d600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd418115bb0 con 0x7fd418105d90 2024-09-21T03:44:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.144+0000 7fd41d600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd418115eb0 con 0x7fd41810c020 2024-09-21T03:44:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.144+0000 7fd416000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41810b6d0 0x7fd41819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.145+0000 7fd416000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41810b6d0 0x7fd41819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52184/0 (socket says 172.21.15.16:52184) 2024-09-21T03:44:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.145+0000 7fd416000640 1 -- 172.21.15.16:0/2233722315 learned_addr learned my addr 172.21.15.16:0/2233722315 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.145+0000 7fd416a00640 1 --2- 172.21.15.16:0/2233722315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd418105d90 0x7fd41819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.145+0000 7fd417400640 1 --2- 172.21.15.16:0/2233722315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd41810c020 0x7fd4181a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.145+0000 7fd416000640 1 -- 172.21.15.16:0/2233722315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd41810c020 msgr2=0x7fd4181a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.145+0000 7fd416000640 1 --2- 172.21.15.16:0/2233722315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd41810c020 0x7fd4181a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.145+0000 7fd416000640 1 -- 172.21.15.16:0/2233722315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd418105d90 msgr2=0x7fd41819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.145+0000 7fd416000640 1 --2- 172.21.15.16:0/2233722315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd418105d90 0x7fd41819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.145+0000 7fd416000640 1 -- 172.21.15.16:0/2233722315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4181a2610 con 0x7fd41810b6d0 2024-09-21T03:44:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.145+0000 7fd416a00640 1 --2- 172.21.15.16:0/2233722315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd418105d90 0x7fd41819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.145+0000 7fd416000640 1 --2- 172.21.15.16:0/2233722315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41810b6d0 0x7fd41819cc40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd40400e9f0 tx=0x7fd40400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.146+0000 7fd40f400640 1 -- 172.21.15.16:0/2233722315 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40400ccc0 con 0x7fd41810b6d0 2024-09-21T03:44:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.146+0000 7fd40f400640 1 -- 172.21.15.16:0/2233722315 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4040108e0 con 0x7fd41810b6d0 2024-09-21T03:44:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.146+0000 7fd40f400640 1 -- 172.21.15.16:0/2233722315 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404018860 con 0x7fd41810b6d0 2024-09-21T03:44:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.146+0000 7fd41d600640 1 -- 172.21.15.16:0/2233722315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4181a2900 con 0x7fd41810b6d0 2024-09-21T03:44:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.146+0000 7fd41d600640 1 -- 172.21.15.16:0/2233722315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4181a9060 con 0x7fd41810b6d0 2024-09-21T03:44:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.147+0000 7fd41d600640 1 -- 172.21.15.16:0/2233722315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4181061a0 con 0x7fd41810b6d0 2024-09-21T03:44:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.149+0000 7fd40f400640 1 -- 172.21.15.16:0/2233722315 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fd404010430 con 0x7fd41810b6d0 2024-09-21T03:44:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.152+0000 7fd40f400640 1 --2- 172.21.15.16:0/2233722315 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd3fc0b65a0 0x7fd3fc0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.152+0000 7fd40f400640 1 -- 172.21.15.16:0/2233722315 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7fd4040e7ae0 con 0x7fd41810b6d0 2024-09-21T03:44:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.152+0000 7fd416a00640 1 --2- 172.21.15.16:0/2233722315 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd3fc0b65a0 0x7fd3fc0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.153+0000 7fd416a00640 1 --2- 172.21.15.16:0/2233722315 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd3fc0b65a0 0x7fd3fc0b8ab0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd408009fc0 tx=0x7fd408005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.156+0000 7fd40f400640 1 -- 172.21.15.16:0/2233722315 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd4040b2230 con 0x7fd41810b6d0 2024-09-21T03:44:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.385+0000 7fd41d600640 1 -- 172.21.15.16:0/2233722315 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fd418063800 con 0x7fd3fc0b65a0 2024-09-21T03:44:19.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.389+0000 7fd40f400640 1 -- 172.21.15.16:0/2233722315 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd418063800 con 0x7fd3fc0b65a0 2024-09-21T03:44:19.392 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_1/bc0903f4-b2b4-4deb-8269-0a424bf4f740 2024-09-21T03:44:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.392+0000 7fd41d600640 1 -- 172.21.15.16:0/2233722315 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd3fc0b65a0 msgr2=0x7fd3fc0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.392+0000 7fd41d600640 1 --2- 172.21.15.16:0/2233722315 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd3fc0b65a0 0x7fd3fc0b8ab0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd408009fc0 tx=0x7fd408005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.392+0000 7fd41d600640 1 -- 172.21.15.16:0/2233722315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41810b6d0 msgr2=0x7fd41819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.392+0000 7fd41d600640 1 --2- 172.21.15.16:0/2233722315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41810b6d0 0x7fd41819cc40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd40400e9f0 tx=0x7fd40400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.392+0000 7fd41d600640 1 -- 172.21.15.16:0/2233722315 shutdown_connections 2024-09-21T03:44:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.392+0000 7fd41d600640 1 --2- 172.21.15.16:0/2233722315 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd3fc0b65a0 0x7fd3fc0b8ab0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.392+0000 7fd41d600640 1 --2- 172.21.15.16:0/2233722315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd41810c020 0x7fd4181a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.392+0000 7fd41d600640 1 --2- 172.21.15.16:0/2233722315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd41810b6d0 0x7fd41819cc40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.392+0000 7fd41d600640 1 --2- 172.21.15.16:0/2233722315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd418105d90 0x7fd41819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.392+0000 7fd41d600640 1 -- 172.21.15.16:0/2233722315 >> 172.21.15.16:0/2233722315 conn(0x7fd418101860 msgr2=0x7fd418108c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.392+0000 7fd41d600640 1 -- 172.21.15.16:0/2233722315 shutdown_connections 2024-09-21T03:44:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:19.392+0000 7fd41d600640 1 -- 172.21.15.16:0/2233722315 wait complete. 2024-09-21T03:44:20.292 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:44:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:20.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:20 smithi134 ceph-mon[35031]: pgmap v357: 129 pgs: 129 active+clean; 630 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 19 op/s 2024-09-21T03:44:20.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:20 smithi134 ceph-mon[35031]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:20.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:20 smithi016 ceph-mon[30375]: pgmap v357: 129 pgs: 129 active+clean; 630 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 19 op/s 2024-09-21T03:44:20.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:20 smithi016 ceph-mon[30375]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:20.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:20 smithi123 ceph-mon[34673]: pgmap v357: 129 pgs: 129 active+clean; 630 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 19 op/s 2024-09-21T03:44:20.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:20 smithi123 ceph-mon[34673]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.330+0000 7f8cdb200640 1 -- 172.21.15.16:0/1937401245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41092f0 msgr2=0x7f8cd4109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.330+0000 7f8cdb200640 1 --2- 172.21.15.16:0/1937401245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41092f0 0x7f8cd4109700 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009900 tx=0x7f8cc402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.331+0000 7f8cdb200640 1 -- 172.21.15.16:0/1937401245 shutdown_connections 2024-09-21T03:44:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.331+0000 7f8cdb200640 1 --2- 172.21.15.16:0/1937401245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd4109c40 0x7f8cd4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.331+0000 7f8cdb200640 1 --2- 172.21.15.16:0/1937401245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41092f0 0x7f8cd4109700 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.331+0000 7f8cdb200640 1 --2- 172.21.15.16:0/1937401245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cd41039b0 0x7f8cd4103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.331+0000 7f8cdb200640 1 -- 172.21.15.16:0/1937401245 >> 172.21.15.16:0/1937401245 conn(0x7f8cd40ff400 msgr2=0x7f8cd4101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.331+0000 7f8cdb200640 1 -- 172.21.15.16:0/1937401245 shutdown_connections 2024-09-21T03:44:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.331+0000 7f8cdb200640 1 -- 172.21.15.16:0/1937401245 wait complete. 2024-09-21T03:44:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.332+0000 7f8cdb200640 1 Processor -- start 2024-09-21T03:44:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.332+0000 7f8cdb200640 1 -- start start 2024-09-21T03:44:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.332+0000 7f8cdb200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cd41039b0 0x7f8cd419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.332+0000 7f8cdb200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd41092f0 0x7f8cd419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.332+0000 7f8cdb200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd4109c40 0x7f8cd419fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.332+0000 7f8cdb200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd4113950 con 0x7f8cd4109c40 2024-09-21T03:44:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.332+0000 7f8cdb200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41137d0 con 0x7f8cd41092f0 2024-09-21T03:44:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.332+0000 7f8cdb200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd4113ad0 con 0x7f8cd41039b0 2024-09-21T03:44:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.333+0000 7f8cd9e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cd41039b0 0x7f8cd419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.333+0000 7f8cd9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd41092f0 0x7f8cd419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.333+0000 7f8cda800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd4109c40 0x7f8cd419fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.333+0000 7f8cd9e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cd41039b0 0x7f8cd419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:48296/0 (socket says 172.21.15.16:48296) 2024-09-21T03:44:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.333+0000 7f8cd9e00640 1 -- 172.21.15.16:0/3714877971 learned_addr learned my addr 172.21.15.16:0/3714877971 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.333+0000 7f8cd9e00640 1 -- 172.21.15.16:0/3714877971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd41092f0 msgr2=0x7f8cd419a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.333+0000 7f8cd9e00640 1 --2- 172.21.15.16:0/3714877971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd41092f0 0x7f8cd419a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.333+0000 7f8cd9e00640 1 -- 172.21.15.16:0/3714877971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd4109c40 msgr2=0x7f8cd419fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.333+0000 7f8cd9e00640 1 --2- 172.21.15.16:0/3714877971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd4109c40 0x7f8cd419fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.333+0000 7f8cd9e00640 1 -- 172.21.15.16:0/3714877971 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd41a0270 con 0x7f8cd41039b0 2024-09-21T03:44:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.333+0000 7f8cda800640 1 --2- 172.21.15.16:0/3714877971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd4109c40 0x7f8cd419fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.334+0000 7f8cd9e00640 1 --2- 172.21.15.16:0/3714877971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cd41039b0 0x7f8cd419a300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800e9f0 tx=0x7f8cc800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.334+0000 7f8ccea00640 1 -- 172.21.15.16:0/3714877971 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc800cd80 con 0x7f8cd41039b0 2024-09-21T03:44:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.334+0000 7f8cdb200640 1 -- 172.21.15.16:0/3714877971 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd41a0560 con 0x7f8cd41039b0 2024-09-21T03:44:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.334+0000 7f8ccea00640 1 -- 172.21.15.16:0/3714877971 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc80054d0 con 0x7f8cd41039b0 2024-09-21T03:44:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.334+0000 7f8ccea00640 1 -- 172.21.15.16:0/3714877971 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8010690 con 0x7f8cd41039b0 2024-09-21T03:44:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.334+0000 7f8cdb200640 1 -- 172.21.15.16:0/3714877971 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd41a6cc0 con 0x7f8cd41039b0 2024-09-21T03:44:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.335+0000 7f8cdb200640 1 -- 172.21.15.16:0/3714877971 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd4103dc0 con 0x7f8cd41039b0 2024-09-21T03:44:21.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.338+0000 7f8ccea00640 1 -- 172.21.15.16:0/3714877971 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f8cc8004e30 con 0x7f8cd41039b0 2024-09-21T03:44:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.339+0000 7f8ccea00640 1 --2- 172.21.15.16:0/3714877971 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8cbc0b65a0 0x7f8cbc0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.340+0000 7f8ccea00640 1 -- 172.21.15.16:0/3714877971 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f8cc80e7e10 con 0x7f8cd41039b0 2024-09-21T03:44:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.340+0000 7f8cd9400640 1 --2- 172.21.15.16:0/3714877971 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8cbc0b65a0 0x7f8cbc0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.340+0000 7f8cd9400640 1 --2- 172.21.15.16:0/3714877971 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8cbc0b65a0 0x7f8cbc0b8ab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8cc402a9e0 tx=0x7f8cc4005a20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.341+0000 7f8ccea00640 1 -- 172.21.15.16:0/3714877971 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8cc80ae180 con 0x7f8cd41039b0 2024-09-21T03:44:21.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.579+0000 7f8cdb200640 1 -- 172.21.15.16:0/3714877971 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cd419b430 con 0x7f8cd41039b0 2024-09-21T03:44:21.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.581+0000 7f8ccea00640 1 -- 172.21.15.16:0/3714877971 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11971 (secure 0 0 0) 0x7f8cc80b2560 con 0x7f8cd41039b0 2024-09-21T03:44:21.585 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:44:21.586 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"btime":"2024-09-21T03:44:08:542131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:44:07.645054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24668,24670,24682,24689,24731]},"id":1}]} 2024-09-21T03:44:21.587 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-09-21T03:44:21.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.583+0000 7f8cdb200640 1 -- 172.21.15.16:0/3714877971 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8cbc0b65a0 msgr2=0x7f8cbc0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.583+0000 7f8cdb200640 1 --2- 172.21.15.16:0/3714877971 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8cbc0b65a0 0x7f8cbc0b8ab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8cc402a9e0 tx=0x7f8cc4005a20 comp rx=0 tx=0).stop 2024-09-21T03:44:21.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.583+0000 7f8cdb200640 1 -- 172.21.15.16:0/3714877971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cd41039b0 msgr2=0x7f8cd419a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.583+0000 7f8cdb200640 1 --2- 172.21.15.16:0/3714877971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cd41039b0 0x7f8cd419a300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800e9f0 tx=0x7f8cc800edf0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.583+0000 7f8cdb200640 1 -- 172.21.15.16:0/3714877971 shutdown_connections 2024-09-21T03:44:21.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.583+0000 7f8cdb200640 1 --2- 172.21.15.16:0/3714877971 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8cbc0b65a0 0x7f8cbc0b8ab0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.583+0000 7f8cdb200640 1 --2- 172.21.15.16:0/3714877971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd4109c40 0x7f8cd419fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.583+0000 7f8cdb200640 1 --2- 172.21.15.16:0/3714877971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd41092f0 0x7f8cd419a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.583+0000 7f8cdb200640 1 --2- 172.21.15.16:0/3714877971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cd41039b0 0x7f8cd419a300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.583+0000 7f8cdb200640 1 -- 172.21.15.16:0/3714877971 >> 172.21.15.16:0/3714877971 conn(0x7f8cd40ff400 msgr2=0x7f8cd4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:21.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.583+0000 7f8cdb200640 1 -- 172.21.15.16:0/3714877971 shutdown_connections 2024-09-21T03:44:21.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:21.583+0000 7f8cdb200640 1 -- 172.21.15.16:0/3714877971 wait complete. 2024-09-21T03:44:21.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:21 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3714877971' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:21.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:21 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3714877971' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:21.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:21 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3714877971' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:22.467 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:44:22.848 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:22.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:22 smithi134 ceph-mon[35031]: pgmap v358: 129 pgs: 129 active+clean; 630 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-09-21T03:44:22.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:22 smithi123 ceph-mon[34673]: pgmap v358: 129 pgs: 129 active+clean; 630 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-09-21T03:44:22.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:22 smithi016 ceph-mon[30375]: pgmap v358: 129 pgs: 129 active+clean; 630 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-09-21T03:44:23.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.476+0000 7fcb98a00640 1 -- 172.21.15.16:0/1853427453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb9410ff70 msgr2=0x7fcb94112390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.476+0000 7fcb98a00640 1 --2- 172.21.15.16:0/1853427453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb9410ff70 0x7fcb94112390 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009ca0 tx=0x7fcb8c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.476+0000 7fcb98a00640 1 -- 172.21.15.16:0/1853427453 shutdown_connections 2024-09-21T03:44:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.476+0000 7fcb98a00640 1 --2- 172.21.15.16:0/1853427453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb9410ff70 0x7fcb94112390 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.476+0000 7fcb98a00640 1 --2- 172.21.15.16:0/1853427453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb94076080 0x7fcb9410fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.476+0000 7fcb98a00640 1 --2- 172.21.15.16:0/1853427453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb94075730 0x7fcb94075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.476+0000 7fcb98a00640 1 -- 172.21.15.16:0/1853427453 >> 172.21.15.16:0/1853427453 conn(0x7fcb940ff3e0 msgr2=0x7fcb94101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.476+0000 7fcb98a00640 1 -- 172.21.15.16:0/1853427453 shutdown_connections 2024-09-21T03:44:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.476+0000 7fcb98a00640 1 -- 172.21.15.16:0/1853427453 wait complete. 2024-09-21T03:44:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.477+0000 7fcb98a00640 1 Processor -- start 2024-09-21T03:44:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.477+0000 7fcb98a00640 1 -- start start 2024-09-21T03:44:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.478+0000 7fcb98a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb94075730 0x7fcb9419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.478+0000 7fcb98a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb94076080 0x7fcb9419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.478+0000 7fcb98a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb9410ff70 0x7fcb941a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.478+0000 7fcb98a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb94115b20 con 0x7fcb94076080 2024-09-21T03:44:23.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.478+0000 7fcb98a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcb941159a0 con 0x7fcb9410ff70 2024-09-21T03:44:23.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.478+0000 7fcb98a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcb94115ca0 con 0x7fcb94075730 2024-09-21T03:44:23.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.478+0000 7fcb91e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb94076080 0x7fcb9419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.478+0000 7fcb92800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb94075730 0x7fcb9419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.478+0000 7fcb92800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb94075730 0x7fcb9419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:42400/0 (socket says 172.21.15.16:42400) 2024-09-21T03:44:23.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.478+0000 7fcb92800640 1 -- 172.21.15.16:0/548795613 learned_addr learned my addr 172.21.15.16:0/548795613 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:23.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.478+0000 7fcb93200640 1 --2- 172.21.15.16:0/548795613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb9410ff70 0x7fcb941a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.479+0000 7fcb92800640 1 -- 172.21.15.16:0/548795613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb9410ff70 msgr2=0x7fcb941a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.479+0000 7fcb92800640 1 --2- 172.21.15.16:0/548795613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb9410ff70 0x7fcb941a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.479+0000 7fcb92800640 1 -- 172.21.15.16:0/548795613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb94076080 msgr2=0x7fcb9419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.479+0000 7fcb92800640 1 --2- 172.21.15.16:0/548795613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb94076080 0x7fcb9419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.479+0000 7fcb92800640 1 -- 172.21.15.16:0/548795613 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb941a24c0 con 0x7fcb94075730 2024-09-21T03:44:23.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.479+0000 7fcb92800640 1 --2- 172.21.15.16:0/548795613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb94075730 0x7fcb9419c550 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400e9f0 tx=0x7fcb8400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:23.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.479+0000 7fcb8b400640 1 -- 172.21.15.16:0/548795613 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8400cd80 con 0x7fcb94075730 2024-09-21T03:44:23.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.479+0000 7fcb8b400640 1 -- 172.21.15.16:0/548795613 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb840054d0 con 0x7fcb94075730 2024-09-21T03:44:23.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.479+0000 7fcb98a00640 1 -- 172.21.15.16:0/548795613 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb941a27b0 con 0x7fcb94075730 2024-09-21T03:44:23.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.480+0000 7fcb8b400640 1 -- 172.21.15.16:0/548795613 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84010690 con 0x7fcb94075730 2024-09-21T03:44:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.480+0000 7fcb98a00640 1 -- 172.21.15.16:0/548795613 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb941a8f10 con 0x7fcb94075730 2024-09-21T03:44:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.481+0000 7fcb98a00640 1 -- 172.21.15.16:0/548795613 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb94075b40 con 0x7fcb94075730 2024-09-21T03:44:23.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.483+0000 7fcb8b400640 1 -- 172.21.15.16:0/548795613 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fcb84004e30 con 0x7fcb94075730 2024-09-21T03:44:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.485+0000 7fcb8b400640 1 --2- 172.21.15.16:0/548795613 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcb780b65a0 0x7fcb780b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.485+0000 7fcb8b400640 1 -- 172.21.15.16:0/548795613 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7fcb840e7e10 con 0x7fcb94075730 2024-09-21T03:44:23.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.485+0000 7fcb91e00640 1 --2- 172.21.15.16:0/548795613 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcb780b65a0 0x7fcb780b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.486+0000 7fcb91e00640 1 --2- 172.21.15.16:0/548795613 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcb780b65a0 0x7fcb780b8ab0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcb800095f0 tx=0x7fcb80005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:23.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.487+0000 7fcb8b400640 1 -- 172.21.15.16:0/548795613 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcb840ae180 con 0x7fcb94075730 2024-09-21T03:44:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.689+0000 7fcb98a00640 1 -- 172.21.15.16:0/548795613 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb9419d680 con 0x7fcb94075730 2024-09-21T03:44:23.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.691+0000 7fcb8b400640 1 -- 172.21.15.16:0/548795613 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+24704 (secure 0 0 0) 0x7fcb9419d680 con 0x7fcb94075730 2024-09-21T03:44:23.694 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:44:23.694 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:33:53.220358+0000","modified":"2024-09-21T03:43:54.450112+0000","last_up_change":"2024-09-21T03:42:06.614247+0000","last_in_change":"2024-09-21T03:41:43.379404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:37:55.708915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:43:03.567522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:43:06.048083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:43:16.061356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6a7e4b9e-078f-4ae9-ab9e-37be995e4deb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6803","nonce":311974551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6805","nonce":311974551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6809","nonce":311974551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6807","nonce":311974551}]},"public_addr":"172.21.15.16:6803/311974551","cluster_addr":"172.21.15.16:6805/311974551","heartbeat_back_addr":"172.21.15.16:6809/311974551","heartbeat_front_addr":"172.21.15.16:6807/311974551","state":["exists","up"]},{"osd":1,"uuid":"3a497a64-e1a8-43ef-8509-0a79515da0aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6801","nonce":2152486571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6803","nonce":2152486571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6807","nonce":2152486571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6805","nonce":2152486571}]},"public_addr":"172.21.15.123:6801/2152486571","cluster_addr":"172.21.15.123:6803/2152486571","heartbeat_back_addr":"172.21.15.123:6807/2152486571","heartbeat_front_addr":"172.21.15.123:6805/2152486571","state":["exists","up"]},{"osd":2,"uuid":"1c39518e-6cce-4a0f-9ee8-55c60ad325f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6801","nonce":2812351279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6803","nonce":2812351279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6807","nonce":2812351279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6805","nonce":2812351279}]},"public_addr":"172.21.15.134:6801/2812351279","cluster_addr":"172.21.15.134:6803/2812351279","heartbeat_back_addr":"172.21.15.134:6807/2812351279","heartbeat_front_addr":"172.21.15.134:6805/2812351279","state":["exists","up"]},{"osd":3,"uuid":"0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6811","nonce":2366216026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6813","nonce":2366216026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6817","nonce":2366216026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6815","nonce":2366216026}]},"public_addr":"172.21.15.16:6811/2366216026","cluster_addr":"172.21.15.16:6813/2366216026","heartbeat_back_addr":"172.21.15.16:6817/2366216026","heartbeat_front_addr":"172.21.15.16:6815/2366216026","state":["exists","up"]},{"osd":4,"uuid":"ad5e6d0e-31e8-4779-9e74-f215032777ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6809","nonce":2354279116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6811","nonce":2354279116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6815","nonce":2354279116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6813","nonce":2354279116}]},"public_addr":"172.21.15.123:6809/2354279116","cluster_addr":"172.21.15.123:6811/2354279116","heartbeat_back_addr":"172.21.15.123:6815/2354279116","heartbeat_front_addr":"172.21.15.123:6813/2354279116","state":["exists","up"]},{"osd":5,"uuid":"ca09bf88-1772-4dc0-8711-cbf9a5b31977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6809","nonce":2678416182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6811","nonce":2678416182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6815","nonce":2678416182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6813","nonce":2678416182}]},"public_addr":"172.21.15.134:6809/2678416182","cluster_addr":"172.21.15.134:6811/2678416182","heartbeat_back_addr":"172.21.15.134:6815/2678416182","heartbeat_front_addr":"172.21.15.134:6813/2678416182","state":["exists","up"]},{"osd":6,"uuid":"0d41ed4e-8699-4d88-9254-12d3b84941e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6819","nonce":3377807744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6821","nonce":3377807744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6825","nonce":3377807744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6823","nonce":3377807744}]},"public_addr":"172.21.15.16:6819/3377807744","cluster_addr":"172.21.15.16:6821/3377807744","heartbeat_back_addr":"172.21.15.16:6825/3377807744","heartbeat_front_addr":"172.21.15.16:6823/3377807744","state":["exists","up"]},{"osd":7,"uuid":"1541a272-c524-4f12-9b94-763f69bb1c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6817","nonce":1463412739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6819","nonce":1463412739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6823","nonce":1463412739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6821","nonce":1463412739}]},"public_addr":"172.21.15.123:6817/1463412739","cluster_addr":"172.21.15.123:6819/1463412739","heartbeat_back_addr":"172.21.15.123:6823/1463412739","heartbeat_front_addr":"172.21.15.123:6821/1463412739","state":["exists","up"]},{"osd":8,"uuid":"cc00014c-eb99-49ab-984c-0bc26ef85130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6817","nonce":2644021450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6819","nonce":2644021450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6823","nonce":2644021450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6821","nonce":2644021450}]},"public_addr":"172.21.15.134:6817/2644021450","cluster_addr":"172.21.15.134:6819/2644021450","heartbeat_back_addr":"172.21.15.134:6823/2644021450","heartbeat_front_addr":"172.21.15.134:6821/2644021450","state":["exists","up"]},{"osd":9,"uuid":"3cbfff74-31ff-4ffd-844b-54ec8563ba32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6827","nonce":1653164371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6829","nonce":1653164371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6833","nonce":1653164371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6831","nonce":1653164371}]},"public_addr":"172.21.15.16:6827/1653164371","cluster_addr":"172.21.15.16:6829/1653164371","heartbeat_back_addr":"172.21.15.16:6833/1653164371","heartbeat_front_addr":"172.21.15.16:6831/1653164371","state":["exists","up"]},{"osd":10,"uuid":"0a1caaad-70cd-4c2c-88d3-e9a5cc885655","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6825","nonce":3682234496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6827","nonce":3682234496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6831","nonce":3682234496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6829","nonce":3682234496}]},"public_addr":"172.21.15.123:6825/3682234496","cluster_addr":"172.21.15.123:6827/3682234496","heartbeat_back_addr":"172.21.15.123:6831/3682234496","heartbeat_front_addr":"172.21.15.123:6829/3682234496","state":["exists","up"]},{"osd":11,"uuid":"3aff3258-5a38-41ff-aac3-4a082feffdae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6825","nonce":584758685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6827","nonce":584758685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6831","nonce":584758685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6829","nonce":584758685}]},"public_addr":"172.21.15.134:6825/584758685","cluster_addr":"172.21.15.134:6827/584758685","heartbeat_back_addr":"172.21.15.134:6831/584758685","heartbeat_front_addr":"172.21.15.134:6829/584758685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:02.177731+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:28.847562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:54.272384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:24.338596+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:52.430371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:17.832737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:50.397890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:16.842245+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:42.102933+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:13.074732+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:40.057905+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:04.898190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/557742783":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/3238676548":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6801/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/2357336233":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6801/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6800/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/3806844703":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/2351581621":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6800/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/355731724":"2024-09-22T03:34:22.549156+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:44:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.693+0000 7fcb98a00640 1 -- 172.21.15.16:0/548795613 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcb780b65a0 msgr2=0x7fcb780b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.693+0000 7fcb98a00640 1 --2- 172.21.15.16:0/548795613 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcb780b65a0 0x7fcb780b8ab0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcb800095f0 tx=0x7fcb80005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.693+0000 7fcb98a00640 1 -- 172.21.15.16:0/548795613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb94075730 msgr2=0x7fcb9419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.693+0000 7fcb98a00640 1 --2- 172.21.15.16:0/548795613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb94075730 0x7fcb9419c550 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400e9f0 tx=0x7fcb8400edf0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.694+0000 7fcb98a00640 1 -- 172.21.15.16:0/548795613 shutdown_connections 2024-09-21T03:44:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.694+0000 7fcb98a00640 1 --2- 172.21.15.16:0/548795613 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcb780b65a0 0x7fcb780b8ab0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.694+0000 7fcb98a00640 1 --2- 172.21.15.16:0/548795613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb9410ff70 0x7fcb941a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.694+0000 7fcb98a00640 1 --2- 172.21.15.16:0/548795613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb94076080 0x7fcb9419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.694+0000 7fcb98a00640 1 --2- 172.21.15.16:0/548795613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb94075730 0x7fcb9419c550 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.694+0000 7fcb98a00640 1 -- 172.21.15.16:0/548795613 >> 172.21.15.16:0/548795613 conn(0x7fcb940ff3e0 msgr2=0x7fcb94109a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.694+0000 7fcb98a00640 1 -- 172.21.15.16:0/548795613 shutdown_connections 2024-09-21T03:44:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:23.694+0000 7fcb98a00640 1 -- 172.21.15.16:0/548795613 wait complete. 2024-09-21T03:44:24.428 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T03:44:24.429 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:44:24.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:24 smithi016 ceph-mon[30375]: pgmap v359: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 14 op/s 2024-09-21T03:44:24.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:24 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/548795613' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:24.809 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:24.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:24 smithi134 ceph-mon[35031]: pgmap v359: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 14 op/s 2024-09-21T03:44:24.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:24 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/548795613' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:24.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:24 smithi123 ceph-mon[34673]: pgmap v359: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 14 op/s 2024-09-21T03:44:24.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:24 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/548795613' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:25.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.453+0000 7f445b200640 1 -- 172.21.15.16:0/3973289991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f445c114280 msgr2=0x7f445c116750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.453+0000 7f445b200640 1 --2- 172.21.15.16:0/3973289991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f445c114280 0x7f445c116750 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4454009ca0 tx=0x7f445402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.453+0000 7f445b200640 1 -- 172.21.15.16:0/3973289991 shutdown_connections 2024-09-21T03:44:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.453+0000 7f445b200640 1 --2- 172.21.15.16:0/3973289991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f445c114280 0x7f445c116750 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.453+0000 7f445b200640 1 --2- 172.21.15.16:0/3973289991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f445c075dd0 0x7f445c0761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.453+0000 7f445b200640 1 --2- 172.21.15.16:0/3973289991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f445c077380 0x7f445c0757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.453+0000 7f445b200640 1 -- 172.21.15.16:0/3973289991 >> 172.21.15.16:0/3973289991 conn(0x7f445c0ff440 msgr2=0x7f445c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.453+0000 7f445b200640 1 -- 172.21.15.16:0/3973289991 shutdown_connections 2024-09-21T03:44:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.453+0000 7f445b200640 1 -- 172.21.15.16:0/3973289991 wait complete. 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.454+0000 7f445b200640 1 Processor -- start 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.455+0000 7f445b200640 1 -- start start 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.455+0000 7f445b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f445c075dd0 0x7f445c1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.455+0000 7f445b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f445c077380 0x7f445c1a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.455+0000 7f445b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f445c114280 0x7f445c1a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.455+0000 7f445b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f445c119f10 con 0x7f445c077380 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.455+0000 7f445b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f445c119d90 con 0x7f445c075dd0 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.455+0000 7f445b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f445c11a090 con 0x7f445c114280 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.455+0000 7f4459400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f445c077380 0x7f445c1a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.456+0000 7f4459e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f445c075dd0 0x7f445c1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.456+0000 7f4459400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f445c077380 0x7f445c1a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38234/0 (socket says 172.21.15.16:38234) 2024-09-21T03:44:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.456+0000 7f445a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f445c114280 0x7f445c1a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.456+0000 7f4459400640 1 -- 172.21.15.16:0/1051785618 learned_addr learned my addr 172.21.15.16:0/1051785618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.456+0000 7f4459e00640 1 -- 172.21.15.16:0/1051785618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f445c114280 msgr2=0x7f445c1a60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.456+0000 7f4459e00640 1 --2- 172.21.15.16:0/1051785618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f445c114280 0x7f445c1a60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:25.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.456+0000 7f4459e00640 1 -- 172.21.15.16:0/1051785618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f445c077380 msgr2=0x7f445c1a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:25.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.456+0000 7f4459e00640 1 --2- 172.21.15.16:0/1051785618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f445c077380 0x7f445c1a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:25.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.456+0000 7f4459e00640 1 -- 172.21.15.16:0/1051785618 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445c1a6810 con 0x7f445c075dd0 2024-09-21T03:44:25.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.456+0000 7f4459e00640 1 --2- 172.21.15.16:0/1051785618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f445c075dd0 0x7f445c1a08a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f444c00d6d0 tx=0x7f444c00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:25.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.457+0000 7f4452a00640 1 -- 172.21.15.16:0/1051785618 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c017070 con 0x7f445c075dd0 2024-09-21T03:44:25.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.457+0000 7f445b200640 1 -- 172.21.15.16:0/1051785618 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445c1a6b00 con 0x7f445c075dd0 2024-09-21T03:44:25.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.457+0000 7f4452a00640 1 -- 172.21.15.16:0/1051785618 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f444c009820 con 0x7f445c075dd0 2024-09-21T03:44:25.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.457+0000 7f4452a00640 1 -- 172.21.15.16:0/1051785618 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c004c40 con 0x7f445c075dd0 2024-09-21T03:44:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.457+0000 7f445b200640 1 -- 172.21.15.16:0/1051785618 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f445c1ad260 con 0x7f445c075dd0 2024-09-21T03:44:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.458+0000 7f445b200640 1 -- 172.21.15.16:0/1051785618 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f445c0761e0 con 0x7f445c075dd0 2024-09-21T03:44:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.460+0000 7f4452a00640 1 -- 172.21.15.16:0/1051785618 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f444c005220 con 0x7f445c075dd0 2024-09-21T03:44:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.462+0000 7f4452a00640 1 --2- 172.21.15.16:0/1051785618 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f44400b65a0 0x7f44400b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.462+0000 7f4452a00640 1 -- 172.21.15.16:0/1051785618 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f444c0e76a0 con 0x7f445c075dd0 2024-09-21T03:44:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.462+0000 7f4459400640 1 --2- 172.21.15.16:0/1051785618 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f44400b65a0 0x7f44400b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.463+0000 7f4459400640 1 --2- 172.21.15.16:0/1051785618 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f44400b65a0 0x7f44400b8ab0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f445c1003a0 tx=0x7f4448005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.464+0000 7f4452a00640 1 -- 172.21.15.16:0/1051785618 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f444c0b1df0 con 0x7f445c075dd0 2024-09-21T03:44:25.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.659+0000 7f445b200640 1 -- 172.21.15.16:0/1051785618 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f445c063800 con 0x7f44400b65a0 2024-09-21T03:44:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.667+0000 7f4452a00640 1 -- 172.21.15.16:0/1051785618 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340377 (secure 0 0 0) 0x7f445c063800 con 0x7f44400b65a0 2024-09-21T03:44:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:44:25.678 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.672+0000 7f445b200640 1 -- 172.21.15.16:0/1051785618 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f44400b65a0 msgr2=0x7f44400b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.672+0000 7f445b200640 1 --2- 172.21.15.16:0/1051785618 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f44400b65a0 0x7f44400b8ab0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f445c1003a0 tx=0x7f4448005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.672+0000 7f445b200640 1 -- 172.21.15.16:0/1051785618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f445c075dd0 msgr2=0x7f445c1a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.672+0000 7f445b200640 1 --2- 172.21.15.16:0/1051785618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f445c075dd0 0x7f445c1a08a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f444c00d6d0 tx=0x7f444c00db90 comp rx=0 tx=0).stop 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.672+0000 7f445b200640 1 -- 172.21.15.16:0/1051785618 shutdown_connections 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.672+0000 7f445b200640 1 --2- 172.21.15.16:0/1051785618 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f44400b65a0 0x7f44400b8ab0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.672+0000 7f445b200640 1 --2- 172.21.15.16:0/1051785618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f445c114280 0x7f445c1a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.672+0000 7f445b200640 1 --2- 172.21.15.16:0/1051785618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f445c077380 0x7f445c1a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.672+0000 7f445b200640 1 --2- 172.21.15.16:0/1051785618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f445c075dd0 0x7f445c1a08a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.672+0000 7f445b200640 1 -- 172.21.15.16:0/1051785618 >> 172.21.15.16:0/1051785618 conn(0x7f445c0ff440 msgr2=0x7f445c110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.672+0000 7f445b200640 1 -- 172.21.15.16:0/1051785618 shutdown_connections 2024-09-21T03:44:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:25.673+0000 7f445b200640 1 -- 172.21.15.16:0/1051785618 wait complete. 2024-09-21T03:44:26.530 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-21T03:44:25.349036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653964,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":285,"num_read_kb":236,"num_write":364,"num_write_kb":2284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379336,"kb_used_data":13448,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562488,"statfs":{"total":1151940427776,"available":1151551987712,"internally_reserved":0,"allocated":13770752,"data_stored":7711718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46341,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":97,"num_write":27,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004058"},"pg_stats":[{"pgid":"2.1f","version":"84'13","reported_seq":70,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:19.569055+0000","last_change":"2024-09-21T03:43:04.980132+0000","last_active":"2024-09-21T03:44:19.569055+0000","last_peered":"2024-09-21T03:44:19.569055+0000","last_clean":"2024-09-21T03:44:19.569055+0000","last_became_active":"2024-09-21T03:43:04.979274+0000","last_became_peered":"2024-09-21T03:43:04.979274+0000","last_unstale":"2024-09-21T03:44:19.569055+0000","last_undegraded":"2024-09-21T03:44:19.569055+0000","last_fullsized":"2024-09-21T03:44:19.569055+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:35.047129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462231+0000","last_change":"2024-09-21T03:43:08.011576+0000","last_active":"2024-09-21T03:43:54.462231+0000","last_peered":"2024-09-21T03:43:54.462231+0000","last_clean":"2024-09-21T03:43:54.462231+0000","last_became_active":"2024-09-21T03:43:08.011188+0000","last_became_peered":"2024-09-21T03:43:08.011188+0000","last_unstale":"2024-09-21T03:43:54.462231+0000","last_undegraded":"2024-09-21T03:43:54.462231+0000","last_fullsized":"2024-09-21T03:43:54.462231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:34.477639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.493297+0000","last_change":"2024-09-21T03:43:54.493183+0000","last_active":"2024-09-21T03:43:54.493297+0000","last_peered":"2024-09-21T03:43:54.493297+0000","last_clean":"2024-09-21T03:43:54.493297+0000","last_became_active":"2024-09-21T03:43:54.492605+0000","last_became_peered":"2024-09-21T03:43:54.492605+0000","last_unstale":"2024-09-21T03:43:54.493297+0000","last_undegraded":"2024-09-21T03:43:54.493297+0000","last_fullsized":"2024-09-21T03:43:54.493297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:26.634042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462613+0000","last_change":"2024-09-21T03:43:05.141790+0000","last_active":"2024-09-21T03:43:54.462613+0000","last_peered":"2024-09-21T03:43:54.462613+0000","last_clean":"2024-09-21T03:43:54.462613+0000","last_became_active":"2024-09-21T03:43:05.141299+0000","last_became_peered":"2024-09-21T03:43:05.141299+0000","last_unstale":"2024-09-21T03:43:54.462613+0000","last_undegraded":"2024-09-21T03:43:54.462613+0000","last_fullsized":"2024-09-21T03:43:54.462613+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:48.844615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489722+0000","last_change":"2024-09-21T03:43:08.018245+0000","last_active":"2024-09-21T03:43:54.489722+0000","last_peered":"2024-09-21T03:43:54.489722+0000","last_clean":"2024-09-21T03:43:54.489722+0000","last_became_active":"2024-09-21T03:43:08.015719+0000","last_became_peered":"2024-09-21T03:43:08.015719+0000","last_unstale":"2024-09-21T03:43:54.489722+0000","last_undegraded":"2024-09-21T03:43:54.489722+0000","last_fullsized":"2024-09-21T03:43:54.489722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:39.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.493401+0000","last_change":"2024-09-21T03:43:54.493279+0000","last_active":"2024-09-21T03:43:54.493401+0000","last_peered":"2024-09-21T03:43:54.493401+0000","last_clean":"2024-09-21T03:43:54.493401+0000","last_became_active":"2024-09-21T03:43:54.492592+0000","last_became_peered":"2024-09-21T03:43:54.492592+0000","last_unstale":"2024-09-21T03:43:54.493401+0000","last_undegraded":"2024-09-21T03:43:54.493401+0000","last_fullsized":"2024-09-21T03:43:54.493401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:28.520990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":54,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:06.537311+0000","last_change":"2024-09-21T03:43:04.982919+0000","last_active":"2024-09-21T03:44:06.537311+0000","last_peered":"2024-09-21T03:44:06.537311+0000","last_clean":"2024-09-21T03:44:06.537311+0000","last_became_active":"2024-09-21T03:43:04.982427+0000","last_became_peered":"2024-09-21T03:43:04.982427+0000","last_unstale":"2024-09-21T03:44:06.537311+0000","last_undegraded":"2024-09-21T03:44:06.537311+0000","last_fullsized":"2024-09-21T03:44:06.537311+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:27.042078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489984+0000","last_change":"2024-09-21T03:43:08.009018+0000","last_active":"2024-09-21T03:43:54.489984+0000","last_peered":"2024-09-21T03:43:54.489984+0000","last_clean":"2024-09-21T03:43:54.489984+0000","last_became_active":"2024-09-21T03:43:08.008614+0000","last_became_peered":"2024-09-21T03:43:08.008614+0000","last_unstale":"2024-09-21T03:43:54.489984+0000","last_undegraded":"2024-09-21T03:43:54.489984+0000","last_fullsized":"2024-09-21T03:43:54.489984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:31.000616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.457441+0000","last_change":"2024-09-21T03:43:18.092750+0000","last_active":"2024-09-21T03:43:54.457441+0000","last_peered":"2024-09-21T03:43:54.457441+0000","last_clean":"2024-09-21T03:43:54.457441+0000","last_became_active":"2024-09-21T03:43:18.092122+0000","last_became_peered":"2024-09-21T03:43:18.092122+0000","last_unstale":"2024-09-21T03:43:54.457441+0000","last_undegraded":"2024-09-21T03:43:54.457441+0000","last_fullsized":"2024-09-21T03:43:54.457441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:15.032388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.543975+0000","last_change":"2024-09-21T03:43:04.978045+0000","last_active":"2024-09-21T03:44:05.543975+0000","last_peered":"2024-09-21T03:44:05.543975+0000","last_clean":"2024-09-21T03:44:05.543975+0000","last_became_active":"2024-09-21T03:43:04.977565+0000","last_became_peered":"2024-09-21T03:43:04.977565+0000","last_unstale":"2024-09-21T03:44:05.543975+0000","last_undegraded":"2024-09-21T03:44:05.543975+0000","last_fullsized":"2024-09-21T03:44:05.543975+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:42.537892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.459614+0000","last_change":"2024-09-21T03:43:08.007094+0000","last_active":"2024-09-21T03:43:54.459614+0000","last_peered":"2024-09-21T03:43:54.459614+0000","last_clean":"2024-09-21T03:43:54.459614+0000","last_became_active":"2024-09-21T03:43:08.006666+0000","last_became_peered":"2024-09-21T03:43:08.006666+0000","last_unstale":"2024-09-21T03:43:54.459614+0000","last_undegraded":"2024-09-21T03:43:54.459614+0000","last_fullsized":"2024-09-21T03:43:54.459614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:59.247459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456502+0000","last_change":"2024-09-21T03:43:18.081141+0000","last_active":"2024-09-21T03:43:54.456502+0000","last_peered":"2024-09-21T03:43:54.456502+0000","last_clean":"2024-09-21T03:43:54.456502+0000","last_became_active":"2024-09-21T03:43:18.080623+0000","last_became_peered":"2024-09-21T03:43:18.080623+0000","last_unstale":"2024-09-21T03:43:54.456502+0000","last_undegraded":"2024-09-21T03:43:54.456502+0000","last_fullsized":"2024-09-21T03:43:54.456502+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:44.932673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462692+0000","last_change":"2024-09-21T03:43:08.008602+0000","last_active":"2024-09-21T03:43:54.462692+0000","last_peered":"2024-09-21T03:43:54.462692+0000","last_clean":"2024-09-21T03:43:54.462692+0000","last_became_active":"2024-09-21T03:43:08.008220+0000","last_became_peered":"2024-09-21T03:43:08.008220+0000","last_unstale":"2024-09-21T03:43:54.462692+0000","last_undegraded":"2024-09-21T03:43:54.462692+0000","last_fullsized":"2024-09-21T03:43:54.462692+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:28.286016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":46,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:02.510319+0000","last_change":"2024-09-21T03:43:04.979357+0000","last_active":"2024-09-21T03:44:02.510319+0000","last_peered":"2024-09-21T03:44:02.510319+0000","last_clean":"2024-09-21T03:44:02.510319+0000","last_became_active":"2024-09-21T03:43:04.978934+0000","last_became_peered":"2024-09-21T03:43:04.978934+0000","last_unstale":"2024-09-21T03:44:02.510319+0000","last_undegraded":"2024-09-21T03:44:02.510319+0000","last_fullsized":"2024-09-21T03:44:02.510319+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:00.858158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456966+0000","last_change":"2024-09-21T03:43:18.483523+0000","last_active":"2024-09-21T03:43:54.456966+0000","last_peered":"2024-09-21T03:43:54.456966+0000","last_clean":"2024-09-21T03:43:54.456966+0000","last_became_active":"2024-09-21T03:43:18.482833+0000","last_became_peered":"2024-09-21T03:43:18.482833+0000","last_unstale":"2024-09-21T03:43:54.456966+0000","last_undegraded":"2024-09-21T03:43:54.456966+0000","last_fullsized":"2024-09-21T03:43:54.456966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:48.236569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.461722+0000","last_change":"2024-09-21T03:43:08.022592+0000","last_active":"2024-09-21T03:43:54.461722+0000","last_peered":"2024-09-21T03:43:54.461722+0000","last_clean":"2024-09-21T03:43:54.461722+0000","last_became_active":"2024-09-21T03:43:08.018065+0000","last_became_peered":"2024-09-21T03:43:08.018065+0000","last_unstale":"2024-09-21T03:43:54.461722+0000","last_undegraded":"2024-09-21T03:43:54.461722+0000","last_fullsized":"2024-09-21T03:43:54.461722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:41.524402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:04.536611+0000","last_change":"2024-09-21T03:43:04.978788+0000","last_active":"2024-09-21T03:44:04.536611+0000","last_peered":"2024-09-21T03:44:04.536611+0000","last_clean":"2024-09-21T03:44:04.536611+0000","last_became_active":"2024-09-21T03:43:04.978382+0000","last_became_peered":"2024-09-21T03:43:04.978382+0000","last_unstale":"2024-09-21T03:44:04.536611+0000","last_undegraded":"2024-09-21T03:44:04.536611+0000","last_fullsized":"2024-09-21T03:44:04.536611+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:12.571217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.494138+0000","last_change":"2024-09-21T03:43:54.493347+0000","last_active":"2024-09-21T03:43:54.494138+0000","last_peered":"2024-09-21T03:43:54.494138+0000","last_clean":"2024-09-21T03:43:54.494138+0000","last_became_active":"2024-09-21T03:43:54.492598+0000","last_became_peered":"2024-09-21T03:43:54.492598+0000","last_unstale":"2024-09-21T03:43:54.494138+0000","last_undegraded":"2024-09-21T03:43:54.494138+0000","last_fullsized":"2024-09-21T03:43:54.494138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:34.788296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.491320+0000","last_change":"2024-09-21T03:43:08.014368+0000","last_active":"2024-09-21T03:43:54.491320+0000","last_peered":"2024-09-21T03:43:54.491320+0000","last_clean":"2024-09-21T03:43:54.491320+0000","last_became_active":"2024-09-21T03:43:08.013952+0000","last_became_peered":"2024-09-21T03:43:08.013952+0000","last_unstale":"2024-09-21T03:43:54.491320+0000","last_undegraded":"2024-09-21T03:43:54.491320+0000","last_fullsized":"2024-09-21T03:43:54.491320+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:16.893588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.543519+0000","last_change":"2024-09-21T03:43:04.976162+0000","last_active":"2024-09-21T03:44:05.543519+0000","last_peered":"2024-09-21T03:44:05.543519+0000","last_clean":"2024-09-21T03:44:05.543519+0000","last_became_active":"2024-09-21T03:43:04.973790+0000","last_became_peered":"2024-09-21T03:43:04.973790+0000","last_unstale":"2024-09-21T03:44:05.543519+0000","last_undegraded":"2024-09-21T03:44:05.543519+0000","last_fullsized":"2024-09-21T03:44:05.543519+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:47.838229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.454608+0000","last_change":"2024-09-21T03:43:18.084344+0000","last_active":"2024-09-21T03:43:54.454608+0000","last_peered":"2024-09-21T03:43:54.454608+0000","last_clean":"2024-09-21T03:43:54.454608+0000","last_became_active":"2024-09-21T03:43:18.083232+0000","last_became_peered":"2024-09-21T03:43:18.083232+0000","last_unstale":"2024-09-21T03:43:54.454608+0000","last_undegraded":"2024-09-21T03:43:54.454608+0000","last_fullsized":"2024-09-21T03:43:54.454608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:46.254405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.491379+0000","last_change":"2024-09-21T03:43:08.009932+0000","last_active":"2024-09-21T03:43:54.491379+0000","last_peered":"2024-09-21T03:43:54.491379+0000","last_clean":"2024-09-21T03:43:54.491379+0000","last_became_active":"2024-09-21T03:43:08.009492+0000","last_became_peered":"2024-09-21T03:43:08.009492+0000","last_unstale":"2024-09-21T03:43:54.491379+0000","last_undegraded":"2024-09-21T03:43:54.491379+0000","last_fullsized":"2024-09-21T03:43:54.491379+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:58.679348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:02.512989+0000","last_change":"2024-09-21T03:43:05.141927+0000","last_active":"2024-09-21T03:44:02.512989+0000","last_peered":"2024-09-21T03:44:02.512989+0000","last_clean":"2024-09-21T03:44:02.512989+0000","last_became_active":"2024-09-21T03:43:05.141516+0000","last_became_peered":"2024-09-21T03:43:05.141516+0000","last_unstale":"2024-09-21T03:44:02.512989+0000","last_undegraded":"2024-09-21T03:44:02.512989+0000","last_fullsized":"2024-09-21T03:44:02.512989+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:35.666981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456824+0000","last_change":"2024-09-21T03:43:18.484478+0000","last_active":"2024-09-21T03:43:54.456824+0000","last_peered":"2024-09-21T03:43:54.456824+0000","last_clean":"2024-09-21T03:43:54.456824+0000","last_became_active":"2024-09-21T03:43:18.483890+0000","last_became_peered":"2024-09-21T03:43:18.483890+0000","last_unstale":"2024-09-21T03:43:54.456824+0000","last_undegraded":"2024-09-21T03:43:54.456824+0000","last_fullsized":"2024-09-21T03:43:54.456824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:03.542783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489942+0000","last_change":"2024-09-21T03:43:08.017785+0000","last_active":"2024-09-21T03:43:54.489942+0000","last_peered":"2024-09-21T03:43:54.489942+0000","last_clean":"2024-09-21T03:43:54.489942+0000","last_became_active":"2024-09-21T03:43:08.017389+0000","last_became_peered":"2024-09-21T03:43:08.017389+0000","last_unstale":"2024-09-21T03:43:54.489942+0000","last_undegraded":"2024-09-21T03:43:54.489942+0000","last_fullsized":"2024-09-21T03:43:54.489942+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:37.557061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.545136+0000","last_change":"2024-09-21T03:43:05.567829+0000","last_active":"2024-09-21T03:44:05.545136+0000","last_peered":"2024-09-21T03:44:05.545136+0000","last_clean":"2024-09-21T03:44:05.545136+0000","last_became_active":"2024-09-21T03:43:05.567401+0000","last_became_peered":"2024-09-21T03:43:05.567401+0000","last_unstale":"2024-09-21T03:44:05.545136+0000","last_undegraded":"2024-09-21T03:44:05.545136+0000","last_fullsized":"2024-09-21T03:44:05.545136+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:03.954280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.457270+0000","last_change":"2024-09-21T03:43:18.080064+0000","last_active":"2024-09-21T03:43:54.457270+0000","last_peered":"2024-09-21T03:43:54.457270+0000","last_clean":"2024-09-21T03:43:54.457270+0000","last_became_active":"2024-09-21T03:43:18.079569+0000","last_became_peered":"2024-09-21T03:43:18.079569+0000","last_unstale":"2024-09-21T03:43:54.457270+0000","last_undegraded":"2024-09-21T03:43:54.457270+0000","last_fullsized":"2024-09-21T03:43:54.457270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:29.568476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462852+0000","last_change":"2024-09-21T03:43:08.011603+0000","last_active":"2024-09-21T03:43:54.462852+0000","last_peered":"2024-09-21T03:43:54.462852+0000","last_clean":"2024-09-21T03:43:54.462852+0000","last_became_active":"2024-09-21T03:43:08.011078+0000","last_became_peered":"2024-09-21T03:43:08.011078+0000","last_unstale":"2024-09-21T03:43:54.462852+0000","last_undegraded":"2024-09-21T03:43:54.462852+0000","last_fullsized":"2024-09-21T03:43:54.462852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:44.764713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'12","reported_seq":53,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:18.022235+0000","last_change":"2024-09-21T03:43:04.979223+0000","last_active":"2024-09-21T03:44:18.022235+0000","last_peered":"2024-09-21T03:44:18.022235+0000","last_clean":"2024-09-21T03:44:18.022235+0000","last_became_active":"2024-09-21T03:43:04.978675+0000","last_became_peered":"2024-09-21T03:43:04.978675+0000","last_unstale":"2024-09-21T03:44:18.022235+0000","last_undegraded":"2024-09-21T03:44:18.022235+0000","last_fullsized":"2024-09-21T03:44:18.022235+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:09.717921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.460027+0000","last_change":"2024-09-21T03:43:18.482334+0000","last_active":"2024-09-21T03:43:54.460027+0000","last_peered":"2024-09-21T03:43:54.460027+0000","last_clean":"2024-09-21T03:43:54.460027+0000","last_became_active":"2024-09-21T03:43:18.481881+0000","last_became_peered":"2024-09-21T03:43:18.481881+0000","last_unstale":"2024-09-21T03:43:54.460027+0000","last_undegraded":"2024-09-21T03:43:54.460027+0000","last_fullsized":"2024-09-21T03:43:54.460027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:02:32.497348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.458435+0000","last_change":"2024-09-21T03:43:07.993771+0000","last_active":"2024-09-21T03:43:54.458435+0000","last_peered":"2024-09-21T03:43:54.458435+0000","last_clean":"2024-09-21T03:43:54.458435+0000","last_became_active":"2024-09-21T03:43:07.993268+0000","last_became_peered":"2024-09-21T03:43:07.993268+0000","last_unstale":"2024-09-21T03:43:54.458435+0000","last_undegraded":"2024-09-21T03:43:54.458435+0000","last_fullsized":"2024-09-21T03:43:54.458435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:42.348561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":46,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.544796+0000","last_change":"2024-09-21T03:43:05.141786+0000","last_active":"2024-09-21T03:44:05.544796+0000","last_peered":"2024-09-21T03:44:05.544796+0000","last_clean":"2024-09-21T03:44:05.544796+0000","last_became_active":"2024-09-21T03:43:05.141307+0000","last_became_peered":"2024-09-21T03:43:05.141307+0000","last_unstale":"2024-09-21T03:44:05.544796+0000","last_undegraded":"2024-09-21T03:44:05.544796+0000","last_fullsized":"2024-09-21T03:44:05.544796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:12.139398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489872+0000","last_change":"2024-09-21T03:43:18.087848+0000","last_active":"2024-09-21T03:43:54.489872+0000","last_peered":"2024-09-21T03:43:54.489872+0000","last_clean":"2024-09-21T03:43:54.489872+0000","last_became_active":"2024-09-21T03:43:18.087400+0000","last_became_peered":"2024-09-21T03:43:18.087400+0000","last_unstale":"2024-09-21T03:43:54.489872+0000","last_undegraded":"2024-09-21T03:43:54.489872+0000","last_fullsized":"2024-09-21T03:43:54.489872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:46.902979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.454367+0000","last_change":"2024-09-21T03:43:08.010051+0000","last_active":"2024-09-21T03:43:54.454367+0000","last_peered":"2024-09-21T03:43:54.454367+0000","last_clean":"2024-09-21T03:43:54.454367+0000","last_became_active":"2024-09-21T03:43:08.007700+0000","last_became_peered":"2024-09-21T03:43:08.007700+0000","last_unstale":"2024-09-21T03:43:54.454367+0000","last_undegraded":"2024-09-21T03:43:54.454367+0000","last_fullsized":"2024-09-21T03:43:54.454367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:31.828193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.454472+0000","last_change":"2024-09-21T03:43:05.140490+0000","last_active":"2024-09-21T03:43:54.454472+0000","last_peered":"2024-09-21T03:43:54.454472+0000","last_clean":"2024-09-21T03:43:54.454472+0000","last_became_active":"2024-09-21T03:43:05.139979+0000","last_became_peered":"2024-09-21T03:43:05.139979+0000","last_unstale":"2024-09-21T03:43:54.454472+0000","last_undegraded":"2024-09-21T03:43:54.454472+0000","last_fullsized":"2024-09-21T03:43:54.454472+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:20.957596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.491072+0000","last_change":"2024-09-21T03:43:18.482777+0000","last_active":"2024-09-21T03:43:54.491072+0000","last_peered":"2024-09-21T03:43:54.491072+0000","last_clean":"2024-09-21T03:43:54.491072+0000","last_became_active":"2024-09-21T03:43:18.482244+0000","last_became_peered":"2024-09-21T03:43:18.482244+0000","last_unstale":"2024-09-21T03:43:54.491072+0000","last_undegraded":"2024-09-21T03:43:54.491072+0000","last_fullsized":"2024-09-21T03:43:54.491072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:41.997309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.454399+0000","last_change":"2024-09-21T03:43:08.019087+0000","last_active":"2024-09-21T03:43:54.454399+0000","last_peered":"2024-09-21T03:43:54.454399+0000","last_clean":"2024-09-21T03:43:54.454399+0000","last_became_active":"2024-09-21T03:43:08.018467+0000","last_became_peered":"2024-09-21T03:43:08.018467+0000","last_unstale":"2024-09-21T03:43:54.454399+0000","last_undegraded":"2024-09-21T03:43:54.454399+0000","last_fullsized":"2024-09-21T03:43:54.454399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:25.824955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"84'5","reported_seq":66,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:23.575159+0000","last_change":"2024-09-21T03:43:04.973433+0000","last_active":"2024-09-21T03:44:23.575159+0000","last_peered":"2024-09-21T03:44:23.575159+0000","last_clean":"2024-09-21T03:44:23.575159+0000","last_became_active":"2024-09-21T03:43:04.973033+0000","last_became_peered":"2024-09-21T03:43:04.973033+0000","last_unstale":"2024-09-21T03:44:23.575159+0000","last_undegraded":"2024-09-21T03:44:23.575159+0000","last_fullsized":"2024-09-21T03:44:23.575159+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:25.462635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.458312+0000","last_change":"2024-09-21T03:43:18.082385+0000","last_active":"2024-09-21T03:43:54.458312+0000","last_peered":"2024-09-21T03:43:54.458312+0000","last_clean":"2024-09-21T03:43:54.458312+0000","last_became_active":"2024-09-21T03:43:18.081958+0000","last_became_peered":"2024-09-21T03:43:18.081958+0000","last_unstale":"2024-09-21T03:43:54.458312+0000","last_undegraded":"2024-09-21T03:43:54.458312+0000","last_fullsized":"2024-09-21T03:43:54.458312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:36.554838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.459553+0000","last_change":"2024-09-21T03:43:08.008307+0000","last_active":"2024-09-21T03:43:54.459553+0000","last_peered":"2024-09-21T03:43:54.459553+0000","last_clean":"2024-09-21T03:43:54.459553+0000","last_became_active":"2024-09-21T03:43:08.006736+0000","last_became_peered":"2024-09-21T03:43:08.006736+0000","last_unstale":"2024-09-21T03:43:54.459553+0000","last_undegraded":"2024-09-21T03:43:54.459553+0000","last_fullsized":"2024-09-21T03:43:54.459553+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:31.694655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:21.571759+0000","last_change":"2024-09-21T03:43:05.140263+0000","last_active":"2024-09-21T03:44:21.571759+0000","last_peered":"2024-09-21T03:44:21.571759+0000","last_clean":"2024-09-21T03:44:21.571759+0000","last_became_active":"2024-09-21T03:43:05.139869+0000","last_became_peered":"2024-09-21T03:43:05.139869+0000","last_unstale":"2024-09-21T03:44:21.571759+0000","last_undegraded":"2024-09-21T03:44:21.571759+0000","last_fullsized":"2024-09-21T03:44:21.571759+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:19.313677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.458932+0000","last_change":"2024-09-21T03:43:18.483795+0000","last_active":"2024-09-21T03:43:54.458932+0000","last_peered":"2024-09-21T03:43:54.458932+0000","last_clean":"2024-09-21T03:43:54.458932+0000","last_became_active":"2024-09-21T03:43:18.483159+0000","last_became_peered":"2024-09-21T03:43:18.483159+0000","last_unstale":"2024-09-21T03:43:54.458932+0000","last_undegraded":"2024-09-21T03:43:54.458932+0000","last_fullsized":"2024-09-21T03:43:54.458932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:56.277956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456884+0000","last_change":"2024-09-21T03:43:08.001661+0000","last_active":"2024-09-21T03:43:54.456884+0000","last_peered":"2024-09-21T03:43:54.456884+0000","last_clean":"2024-09-21T03:43:54.456884+0000","last_became_active":"2024-09-21T03:43:08.001254+0000","last_became_peered":"2024-09-21T03:43:08.001254+0000","last_unstale":"2024-09-21T03:43:54.456884+0000","last_undegraded":"2024-09-21T03:43:54.456884+0000","last_fullsized":"2024-09-21T03:43:54.456884+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:19.020542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:03.526981+0000","last_change":"2024-09-21T03:43:04.979127+0000","last_active":"2024-09-21T03:44:03.526981+0000","last_peered":"2024-09-21T03:44:03.526981+0000","last_clean":"2024-09-21T03:44:03.526981+0000","last_became_active":"2024-09-21T03:43:04.978705+0000","last_became_peered":"2024-09-21T03:43:04.978705+0000","last_unstale":"2024-09-21T03:44:03.526981+0000","last_undegraded":"2024-09-21T03:44:03.526981+0000","last_fullsized":"2024-09-21T03:44:03.526981+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:48.548002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.459944+0000","last_change":"2024-09-21T03:43:18.483682+0000","last_active":"2024-09-21T03:43:54.459944+0000","last_peered":"2024-09-21T03:43:54.459944+0000","last_clean":"2024-09-21T03:43:54.459944+0000","last_became_active":"2024-09-21T03:43:18.483124+0000","last_became_peered":"2024-09-21T03:43:18.483124+0000","last_unstale":"2024-09-21T03:43:54.459944+0000","last_undegraded":"2024-09-21T03:43:54.459944+0000","last_fullsized":"2024-09-21T03:43:54.459944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:12.513795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.461321+0000","last_change":"2024-09-21T03:43:08.013193+0000","last_active":"2024-09-21T03:43:54.461321+0000","last_peered":"2024-09-21T03:43:54.461321+0000","last_clean":"2024-09-21T03:43:54.461321+0000","last_became_active":"2024-09-21T03:43:08.010360+0000","last_became_peered":"2024-09-21T03:43:08.010360+0000","last_unstale":"2024-09-21T03:43:54.461321+0000","last_undegraded":"2024-09-21T03:43:54.461321+0000","last_fullsized":"2024-09-21T03:43:54.461321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:28.101646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:06.544985+0000","last_change":"2024-09-21T03:43:04.971782+0000","last_active":"2024-09-21T03:44:06.544985+0000","last_peered":"2024-09-21T03:44:06.544985+0000","last_clean":"2024-09-21T03:44:06.544985+0000","last_became_active":"2024-09-21T03:43:04.971375+0000","last_became_peered":"2024-09-21T03:43:04.971375+0000","last_unstale":"2024-09-21T03:44:06.544985+0000","last_undegraded":"2024-09-21T03:44:06.544985+0000","last_fullsized":"2024-09-21T03:44:06.544985+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:49.573490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'92","reported_seq":111,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462612+0000","last_change":"2024-09-21T03:41:42.914435+0000","last_active":"2024-09-21T03:43:54.462612+0000","last_peered":"2024-09-21T03:43:54.462612+0000","last_clean":"2024-09-21T03:43:54.462612+0000","last_became_active":"2024-09-21T03:41:42.909490+0000","last_became_peered":"2024-09-21T03:41:42.909490+0000","last_unstale":"2024-09-21T03:43:54.462612+0000","last_undegraded":"2024-09-21T03:43:54.462612+0000","last_fullsized":"2024-09-21T03:43:54.462612+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:56.507027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:56.507027+0000","last_clean_scrub_stamp":"2024-09-21T03:37:56.507027+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:34.564062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.461799+0000","last_change":"2024-09-21T03:43:18.084701+0000","last_active":"2024-09-21T03:43:54.461799+0000","last_peered":"2024-09-21T03:43:54.461799+0000","last_clean":"2024-09-21T03:43:54.461799+0000","last_became_active":"2024-09-21T03:43:18.084251+0000","last_became_peered":"2024-09-21T03:43:18.084251+0000","last_unstale":"2024-09-21T03:43:54.461799+0000","last_undegraded":"2024-09-21T03:43:54.461799+0000","last_fullsized":"2024-09-21T03:43:54.461799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:15.660276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462254+0000","last_change":"2024-09-21T03:43:08.012195+0000","last_active":"2024-09-21T03:43:54.462254+0000","last_peered":"2024-09-21T03:43:54.462254+0000","last_clean":"2024-09-21T03:43:54.462254+0000","last_became_active":"2024-09-21T03:43:08.011670+0000","last_became_peered":"2024-09-21T03:43:08.011670+0000","last_unstale":"2024-09-21T03:43:54.462254+0000","last_undegraded":"2024-09-21T03:43:54.462254+0000","last_fullsized":"2024-09-21T03:43:54.462254+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:14.290074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":65,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:19.569654+0000","last_change":"2024-09-21T03:43:05.567898+0000","last_active":"2024-09-21T03:44:19.569654+0000","last_peered":"2024-09-21T03:44:19.569654+0000","last_clean":"2024-09-21T03:44:19.569654+0000","last_became_active":"2024-09-21T03:43:05.567367+0000","last_became_peered":"2024-09-21T03:43:05.567367+0000","last_unstale":"2024-09-21T03:44:19.569654+0000","last_undegraded":"2024-09-21T03:44:19.569654+0000","last_fullsized":"2024-09-21T03:44:19.569654+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:55.120838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489820+0000","last_change":"2024-09-21T03:43:18.073942+0000","last_active":"2024-09-21T03:43:54.489820+0000","last_peered":"2024-09-21T03:43:54.489820+0000","last_clean":"2024-09-21T03:43:54.489820+0000","last_became_active":"2024-09-21T03:43:18.073323+0000","last_became_peered":"2024-09-21T03:43:18.073323+0000","last_unstale":"2024-09-21T03:43:54.489820+0000","last_undegraded":"2024-09-21T03:43:54.489820+0000","last_fullsized":"2024-09-21T03:43:54.489820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:41.677143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.461712+0000","last_change":"2024-09-21T03:43:08.016636+0000","last_active":"2024-09-21T03:43:54.461712+0000","last_peered":"2024-09-21T03:43:54.461712+0000","last_clean":"2024-09-21T03:43:54.461712+0000","last_became_active":"2024-09-21T03:43:08.016235+0000","last_became_peered":"2024-09-21T03:43:08.016235+0000","last_unstale":"2024-09-21T03:43:54.461712+0000","last_undegraded":"2024-09-21T03:43:54.461712+0000","last_fullsized":"2024-09-21T03:43:54.461712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:35.365830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.543336+0000","last_change":"2024-09-21T03:43:04.980911+0000","last_active":"2024-09-21T03:44:05.543336+0000","last_peered":"2024-09-21T03:44:05.543336+0000","last_clean":"2024-09-21T03:44:05.543336+0000","last_became_active":"2024-09-21T03:43:04.980512+0000","last_became_peered":"2024-09-21T03:43:04.980512+0000","last_unstale":"2024-09-21T03:44:05.543336+0000","last_undegraded":"2024-09-21T03:44:05.543336+0000","last_fullsized":"2024-09-21T03:44:05.543336+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:04.136345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489524+0000","last_change":"2024-09-21T03:43:18.085874+0000","last_active":"2024-09-21T03:43:54.489524+0000","last_peered":"2024-09-21T03:43:54.489524+0000","last_clean":"2024-09-21T03:43:54.489524+0000","last_became_active":"2024-09-21T03:43:18.085381+0000","last_became_peered":"2024-09-21T03:43:18.085381+0000","last_unstale":"2024-09-21T03:43:54.489524+0000","last_undegraded":"2024-09-21T03:43:54.489524+0000","last_fullsized":"2024-09-21T03:43:54.489524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:02.829202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462064+0000","last_change":"2024-09-21T03:43:08.014776+0000","last_active":"2024-09-21T03:43:54.462064+0000","last_peered":"2024-09-21T03:43:54.462064+0000","last_clean":"2024-09-21T03:43:54.462064+0000","last_became_active":"2024-09-21T03:43:08.014191+0000","last_became_peered":"2024-09-21T03:43:08.014191+0000","last_unstale":"2024-09-21T03:43:54.462064+0000","last_undegraded":"2024-09-21T03:43:54.462064+0000","last_fullsized":"2024-09-21T03:43:54.462064+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:19.114838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":46,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.544909+0000","last_change":"2024-09-21T03:43:04.973714+0000","last_active":"2024-09-21T03:44:05.544909+0000","last_peered":"2024-09-21T03:44:05.544909+0000","last_clean":"2024-09-21T03:44:05.544909+0000","last_became_active":"2024-09-21T03:43:04.973348+0000","last_became_peered":"2024-09-21T03:43:04.973348+0000","last_unstale":"2024-09-21T03:44:05.544909+0000","last_undegraded":"2024-09-21T03:44:05.544909+0000","last_fullsized":"2024-09-21T03:44:05.544909+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:00.872589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.458790+0000","last_change":"2024-09-21T03:43:18.082643+0000","last_active":"2024-09-21T03:43:54.458790+0000","last_peered":"2024-09-21T03:43:54.458790+0000","last_clean":"2024-09-21T03:43:54.458790+0000","last_became_active":"2024-09-21T03:43:18.082157+0000","last_became_peered":"2024-09-21T03:43:18.082157+0000","last_unstale":"2024-09-21T03:43:54.458790+0000","last_undegraded":"2024-09-21T03:43:54.458790+0000","last_fullsized":"2024-09-21T03:43:54.458790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:52.652014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490309+0000","last_change":"2024-09-21T03:43:08.010913+0000","last_active":"2024-09-21T03:43:54.490309+0000","last_peered":"2024-09-21T03:43:54.490309+0000","last_clean":"2024-09-21T03:43:54.490309+0000","last_became_active":"2024-09-21T03:43:08.010483+0000","last_became_peered":"2024-09-21T03:43:08.010483+0000","last_unstale":"2024-09-21T03:43:54.490309+0000","last_undegraded":"2024-09-21T03:43:54.490309+0000","last_fullsized":"2024-09-21T03:43:54.490309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:52.158346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:04.522790+0000","last_change":"2024-09-21T03:43:04.980260+0000","last_active":"2024-09-21T03:44:04.522790+0000","last_peered":"2024-09-21T03:44:04.522790+0000","last_clean":"2024-09-21T03:44:04.522790+0000","last_became_active":"2024-09-21T03:43:04.978999+0000","last_became_peered":"2024-09-21T03:43:04.978999+0000","last_unstale":"2024-09-21T03:44:04.522790+0000","last_undegraded":"2024-09-21T03:44:04.522790+0000","last_fullsized":"2024-09-21T03:44:04.522790+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:33.682591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.458981+0000","last_change":"2024-09-21T03:43:18.083665+0000","last_active":"2024-09-21T03:43:54.458981+0000","last_peered":"2024-09-21T03:43:54.458981+0000","last_clean":"2024-09-21T03:43:54.458981+0000","last_became_active":"2024-09-21T03:43:18.083215+0000","last_became_peered":"2024-09-21T03:43:18.083215+0000","last_unstale":"2024-09-21T03:43:54.458981+0000","last_undegraded":"2024-09-21T03:43:54.458981+0000","last_fullsized":"2024-09-21T03:43:54.458981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:55.968968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.459158+0000","last_change":"2024-09-21T03:43:08.010570+0000","last_active":"2024-09-21T03:43:54.459158+0000","last_peered":"2024-09-21T03:43:54.459158+0000","last_clean":"2024-09-21T03:43:54.459158+0000","last_became_active":"2024-09-21T03:43:08.010163+0000","last_became_peered":"2024-09-21T03:43:08.010163+0000","last_unstale":"2024-09-21T03:43:54.459158+0000","last_undegraded":"2024-09-21T03:43:54.459158+0000","last_fullsized":"2024-09-21T03:43:54.459158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:17.368534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":59,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.543461+0000","last_change":"2024-09-21T03:43:05.140650+0000","last_active":"2024-09-21T03:44:05.543461+0000","last_peered":"2024-09-21T03:44:05.543461+0000","last_clean":"2024-09-21T03:44:05.543461+0000","last_became_active":"2024-09-21T03:43:05.140252+0000","last_became_peered":"2024-09-21T03:43:05.140252+0000","last_unstale":"2024-09-21T03:44:05.543461+0000","last_undegraded":"2024-09-21T03:44:05.543461+0000","last_fullsized":"2024-09-21T03:44:05.543461+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:23.693838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.459001+0000","last_change":"2024-09-21T03:43:18.089516+0000","last_active":"2024-09-21T03:43:54.459001+0000","last_peered":"2024-09-21T03:43:54.459001+0000","last_clean":"2024-09-21T03:43:54.459001+0000","last_became_active":"2024-09-21T03:43:18.089097+0000","last_became_peered":"2024-09-21T03:43:18.089097+0000","last_unstale":"2024-09-21T03:43:54.459001+0000","last_undegraded":"2024-09-21T03:43:54.459001+0000","last_fullsized":"2024-09-21T03:43:54.459001+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:50.428401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.461348+0000","last_change":"2024-09-21T03:43:08.015762+0000","last_active":"2024-09-21T03:43:54.461348+0000","last_peered":"2024-09-21T03:43:54.461348+0000","last_clean":"2024-09-21T03:43:54.461348+0000","last_became_active":"2024-09-21T03:43:08.015389+0000","last_became_peered":"2024-09-21T03:43:08.015389+0000","last_unstale":"2024-09-21T03:43:54.461348+0000","last_undegraded":"2024-09-21T03:43:54.461348+0000","last_fullsized":"2024-09-21T03:43:54.461348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:00.341479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'5","reported_seq":67,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:23.575428+0000","last_change":"2024-09-21T03:43:04.973173+0000","last_active":"2024-09-21T03:44:23.575428+0000","last_peered":"2024-09-21T03:44:23.575428+0000","last_clean":"2024-09-21T03:44:23.575428+0000","last_became_active":"2024-09-21T03:43:04.972740+0000","last_became_peered":"2024-09-21T03:43:04.972740+0000","last_unstale":"2024-09-21T03:44:23.575428+0000","last_undegraded":"2024-09-21T03:44:23.575428+0000","last_fullsized":"2024-09-21T03:44:23.575428+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:26.329220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.454727+0000","last_change":"2024-09-21T03:43:18.093730+0000","last_active":"2024-09-21T03:43:54.454727+0000","last_peered":"2024-09-21T03:43:54.454727+0000","last_clean":"2024-09-21T03:43:54.454727+0000","last_became_active":"2024-09-21T03:43:18.089621+0000","last_became_peered":"2024-09-21T03:43:18.089621+0000","last_unstale":"2024-09-21T03:43:54.454727+0000","last_undegraded":"2024-09-21T03:43:54.454727+0000","last_fullsized":"2024-09-21T03:43:54.454727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:05.227038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.465867+0000","last_change":"2024-09-21T03:43:54.465751+0000","last_active":"2024-09-21T03:43:54.465867+0000","last_peered":"2024-09-21T03:43:54.465867+0000","last_clean":"2024-09-21T03:43:54.465867+0000","last_became_active":"2024-09-21T03:43:54.465333+0000","last_became_peered":"2024-09-21T03:43:54.465333+0000","last_unstale":"2024-09-21T03:43:54.465867+0000","last_undegraded":"2024-09-21T03:43:54.465867+0000","last_fullsized":"2024-09-21T03:43:54.465867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:34.307699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.543641+0000","last_change":"2024-09-21T03:43:04.979385+0000","last_active":"2024-09-21T03:44:05.543641+0000","last_peered":"2024-09-21T03:44:05.543641+0000","last_clean":"2024-09-21T03:44:05.543641+0000","last_became_active":"2024-09-21T03:43:04.978764+0000","last_became_peered":"2024-09-21T03:43:04.978764+0000","last_unstale":"2024-09-21T03:44:05.543641+0000","last_undegraded":"2024-09-21T03:44:05.543641+0000","last_fullsized":"2024-09-21T03:44:05.543641+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:34.256966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490077+0000","last_change":"2024-09-21T03:43:18.095937+0000","last_active":"2024-09-21T03:43:54.490077+0000","last_peered":"2024-09-21T03:43:54.490077+0000","last_clean":"2024-09-21T03:43:54.490077+0000","last_became_active":"2024-09-21T03:43:18.095531+0000","last_became_peered":"2024-09-21T03:43:18.095531+0000","last_unstale":"2024-09-21T03:43:54.490077+0000","last_undegraded":"2024-09-21T03:43:54.490077+0000","last_fullsized":"2024-09-21T03:43:54.490077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:18.089661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.457193+0000","last_change":"2024-09-21T03:43:08.011342+0000","last_active":"2024-09-21T03:43:54.457193+0000","last_peered":"2024-09-21T03:43:54.457193+0000","last_clean":"2024-09-21T03:43:54.457193+0000","last_became_active":"2024-09-21T03:43:08.010963+0000","last_became_peered":"2024-09-21T03:43:08.010963+0000","last_unstale":"2024-09-21T03:43:54.457193+0000","last_undegraded":"2024-09-21T03:43:54.457193+0000","last_fullsized":"2024-09-21T03:43:54.457193+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:05.970863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":52,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:06.545562+0000","last_change":"2024-09-21T03:43:04.972896+0000","last_active":"2024-09-21T03:44:06.545562+0000","last_peered":"2024-09-21T03:44:06.545562+0000","last_clean":"2024-09-21T03:44:06.545562+0000","last_became_active":"2024-09-21T03:43:04.972535+0000","last_became_peered":"2024-09-21T03:43:04.972535+0000","last_unstale":"2024-09-21T03:44:06.545562+0000","last_undegraded":"2024-09-21T03:44:06.545562+0000","last_fullsized":"2024-09-21T03:44:06.545562+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:36.362019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.459478+0000","last_change":"2024-09-21T03:43:18.081300+0000","last_active":"2024-09-21T03:43:54.459478+0000","last_peered":"2024-09-21T03:43:54.459478+0000","last_clean":"2024-09-21T03:43:54.459478+0000","last_became_active":"2024-09-21T03:43:18.080874+0000","last_became_peered":"2024-09-21T03:43:18.080874+0000","last_unstale":"2024-09-21T03:43:54.459478+0000","last_undegraded":"2024-09-21T03:43:54.459478+0000","last_fullsized":"2024-09-21T03:43:54.459478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:13.246348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.491419+0000","last_change":"2024-09-21T03:43:08.014954+0000","last_active":"2024-09-21T03:43:54.491419+0000","last_peered":"2024-09-21T03:43:54.491419+0000","last_clean":"2024-09-21T03:43:54.491419+0000","last_became_active":"2024-09-21T03:43:08.014576+0000","last_became_peered":"2024-09-21T03:43:08.014576+0000","last_unstale":"2024-09-21T03:43:54.491419+0000","last_undegraded":"2024-09-21T03:43:54.491419+0000","last_fullsized":"2024-09-21T03:43:54.491419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:13.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"84'31","reported_seq":113,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:22.576813+0000","last_change":"2024-09-21T03:43:04.973305+0000","last_active":"2024-09-21T03:44:22.576813+0000","last_peered":"2024-09-21T03:44:22.576813+0000","last_clean":"2024-09-21T03:44:22.576813+0000","last_became_active":"2024-09-21T03:43:04.972891+0000","last_became_peered":"2024-09-21T03:43:04.972891+0000","last_unstale":"2024-09-21T03:44:22.576813+0000","last_undegraded":"2024-09-21T03:44:22.576813+0000","last_fullsized":"2024-09-21T03:44:22.576813+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:15.653769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56549,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":75,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.453933+0000","last_change":"2024-09-21T03:43:18.093613+0000","last_active":"2024-09-21T03:43:54.453933+0000","last_peered":"2024-09-21T03:43:54.453933+0000","last_clean":"2024-09-21T03:43:54.453933+0000","last_became_active":"2024-09-21T03:43:18.093195+0000","last_became_peered":"2024-09-21T03:43:18.093195+0000","last_unstale":"2024-09-21T03:43:54.453933+0000","last_undegraded":"2024-09-21T03:43:54.453933+0000","last_fullsized":"2024-09-21T03:43:54.453933+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:17.825071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490098+0000","last_change":"2024-09-21T03:43:08.014314+0000","last_active":"2024-09-21T03:43:54.490098+0000","last_peered":"2024-09-21T03:43:54.490098+0000","last_clean":"2024-09-21T03:43:54.490098+0000","last_became_active":"2024-09-21T03:43:08.012925+0000","last_became_peered":"2024-09-21T03:43:08.012925+0000","last_unstale":"2024-09-21T03:43:54.490098+0000","last_undegraded":"2024-09-21T03:43:54.490098+0000","last_fullsized":"2024-09-21T03:43:54.490098+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:47:28.599444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:04.536006+0000","last_change":"2024-09-21T03:43:05.141584+0000","last_active":"2024-09-21T03:44:04.536006+0000","last_peered":"2024-09-21T03:44:04.536006+0000","last_clean":"2024-09-21T03:44:04.536006+0000","last_became_active":"2024-09-21T03:43:05.141200+0000","last_became_peered":"2024-09-21T03:43:05.141200+0000","last_unstale":"2024-09-21T03:44:04.536006+0000","last_undegraded":"2024-09-21T03:44:04.536006+0000","last_fullsized":"2024-09-21T03:44:04.536006+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:04.357970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.457242+0000","last_change":"2024-09-21T03:43:18.092184+0000","last_active":"2024-09-21T03:43:54.457242+0000","last_peered":"2024-09-21T03:43:54.457242+0000","last_clean":"2024-09-21T03:43:54.457242+0000","last_became_active":"2024-09-21T03:43:18.091629+0000","last_became_peered":"2024-09-21T03:43:18.091629+0000","last_unstale":"2024-09-21T03:43:54.457242+0000","last_undegraded":"2024-09-21T03:43:54.457242+0000","last_fullsized":"2024-09-21T03:43:54.457242+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:39.448009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489762+0000","last_change":"2024-09-21T03:43:08.018335+0000","last_active":"2024-09-21T03:43:54.489762+0000","last_peered":"2024-09-21T03:43:54.489762+0000","last_clean":"2024-09-21T03:43:54.489762+0000","last_became_active":"2024-09-21T03:43:08.014005+0000","last_became_peered":"2024-09-21T03:43:08.014005+0000","last_unstale":"2024-09-21T03:43:54.489762+0000","last_undegraded":"2024-09-21T03:43:54.489762+0000","last_fullsized":"2024-09-21T03:43:54.489762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:35.749381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.543134+0000","last_change":"2024-09-21T03:43:04.978609+0000","last_active":"2024-09-21T03:44:05.543134+0000","last_peered":"2024-09-21T03:44:05.543134+0000","last_clean":"2024-09-21T03:44:05.543134+0000","last_became_active":"2024-09-21T03:43:04.978146+0000","last_became_peered":"2024-09-21T03:43:04.978146+0000","last_unstale":"2024-09-21T03:44:05.543134+0000","last_undegraded":"2024-09-21T03:44:05.543134+0000","last_fullsized":"2024-09-21T03:44:05.543134+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:41.651745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.453520+0000","last_change":"2024-09-21T03:43:18.089399+0000","last_active":"2024-09-21T03:43:54.453520+0000","last_peered":"2024-09-21T03:43:54.453520+0000","last_clean":"2024-09-21T03:43:54.453520+0000","last_became_active":"2024-09-21T03:43:18.086156+0000","last_became_peered":"2024-09-21T03:43:18.086156+0000","last_unstale":"2024-09-21T03:43:54.453520+0000","last_undegraded":"2024-09-21T03:43:54.453520+0000","last_fullsized":"2024-09-21T03:43:54.453520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:41.955341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490427+0000","last_change":"2024-09-21T03:43:08.008773+0000","last_active":"2024-09-21T03:43:54.490427+0000","last_peered":"2024-09-21T03:43:54.490427+0000","last_clean":"2024-09-21T03:43:54.490427+0000","last_became_active":"2024-09-21T03:43:08.008270+0000","last_became_peered":"2024-09-21T03:43:08.008270+0000","last_unstale":"2024-09-21T03:43:54.490427+0000","last_undegraded":"2024-09-21T03:43:54.490427+0000","last_fullsized":"2024-09-21T03:43:54.490427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:57.901701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":79,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:22.572573+0000","last_change":"2024-09-21T03:43:04.978296+0000","last_active":"2024-09-21T03:44:22.572573+0000","last_peered":"2024-09-21T03:44:22.572573+0000","last_clean":"2024-09-21T03:44:22.572573+0000","last_became_active":"2024-09-21T03:43:04.977782+0000","last_became_peered":"2024-09-21T03:43:04.977782+0000","last_unstale":"2024-09-21T03:44:22.572573+0000","last_undegraded":"2024-09-21T03:44:22.572573+0000","last_fullsized":"2024-09-21T03:44:22.572573+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:57.748737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.495673+0000","last_change":"2024-09-21T03:43:54.495536+0000","last_active":"2024-09-21T03:43:54.495673+0000","last_peered":"2024-09-21T03:43:54.495673+0000","last_clean":"2024-09-21T03:43:54.495673+0000","last_became_active":"2024-09-21T03:43:54.493492+0000","last_became_peered":"2024-09-21T03:43:54.493492+0000","last_unstale":"2024-09-21T03:43:54.495673+0000","last_undegraded":"2024-09-21T03:43:54.495673+0000","last_fullsized":"2024-09-21T03:43:54.495673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:11.291911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489721+0000","last_change":"2024-09-21T03:43:18.077741+0000","last_active":"2024-09-21T03:43:54.489721+0000","last_peered":"2024-09-21T03:43:54.489721+0000","last_clean":"2024-09-21T03:43:54.489721+0000","last_became_active":"2024-09-21T03:43:18.076619+0000","last_became_peered":"2024-09-21T03:43:18.076619+0000","last_unstale":"2024-09-21T03:43:54.489721+0000","last_undegraded":"2024-09-21T03:43:54.489721+0000","last_fullsized":"2024-09-21T03:43:54.489721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:33.297020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.453694+0000","last_change":"2024-09-21T03:43:08.008838+0000","last_active":"2024-09-21T03:43:54.453694+0000","last_peered":"2024-09-21T03:43:54.453694+0000","last_clean":"2024-09-21T03:43:54.453694+0000","last_became_active":"2024-09-21T03:43:08.008422+0000","last_became_peered":"2024-09-21T03:43:08.008422+0000","last_unstale":"2024-09-21T03:43:54.453694+0000","last_undegraded":"2024-09-21T03:43:54.453694+0000","last_fullsized":"2024-09-21T03:43:54.453694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:00:17.881443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.543919+0000","last_change":"2024-09-21T03:43:04.979289+0000","last_active":"2024-09-21T03:44:05.543919+0000","last_peered":"2024-09-21T03:44:05.543919+0000","last_clean":"2024-09-21T03:44:05.543919+0000","last_became_active":"2024-09-21T03:43:04.978843+0000","last_became_peered":"2024-09-21T03:43:04.978843+0000","last_unstale":"2024-09-21T03:44:05.543919+0000","last_undegraded":"2024-09-21T03:44:05.543919+0000","last_fullsized":"2024-09-21T03:44:05.543919+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:50.560607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490335+0000","last_change":"2024-09-21T03:43:18.096870+0000","last_active":"2024-09-21T03:43:54.490335+0000","last_peered":"2024-09-21T03:43:54.490335+0000","last_clean":"2024-09-21T03:43:54.490335+0000","last_became_active":"2024-09-21T03:43:18.096424+0000","last_became_peered":"2024-09-21T03:43:18.096424+0000","last_unstale":"2024-09-21T03:43:54.490335+0000","last_undegraded":"2024-09-21T03:43:54.490335+0000","last_fullsized":"2024-09-21T03:43:54.490335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:39.532368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.494294+0000","last_change":"2024-09-21T03:43:54.492499+0000","last_active":"2024-09-21T03:43:54.494294+0000","last_peered":"2024-09-21T03:43:54.494294+0000","last_clean":"2024-09-21T03:43:54.494294+0000","last_became_active":"2024-09-21T03:43:54.492043+0000","last_became_peered":"2024-09-21T03:43:54.492043+0000","last_unstale":"2024-09-21T03:43:54.494294+0000","last_undegraded":"2024-09-21T03:43:54.494294+0000","last_fullsized":"2024-09-21T03:43:54.494294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:25.687090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:03.530472+0000","last_change":"2024-09-21T03:43:04.974265+0000","last_active":"2024-09-21T03:44:03.530472+0000","last_peered":"2024-09-21T03:44:03.530472+0000","last_clean":"2024-09-21T03:44:03.530472+0000","last_became_active":"2024-09-21T03:43:04.973383+0000","last_became_peered":"2024-09-21T03:43:04.973383+0000","last_unstale":"2024-09-21T03:44:03.530472+0000","last_undegraded":"2024-09-21T03:44:03.530472+0000","last_fullsized":"2024-09-21T03:44:03.530472+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:02.312378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":48,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:04.536278+0000","last_change":"2024-09-21T03:43:04.980353+0000","last_active":"2024-09-21T03:44:04.536278+0000","last_peered":"2024-09-21T03:44:04.536278+0000","last_clean":"2024-09-21T03:44:04.536278+0000","last_became_active":"2024-09-21T03:43:04.979922+0000","last_became_peered":"2024-09-21T03:43:04.979922+0000","last_unstale":"2024-09-21T03:44:04.536278+0000","last_undegraded":"2024-09-21T03:44:04.536278+0000","last_fullsized":"2024-09-21T03:44:04.536278+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:17.636536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.459397+0000","last_change":"2024-09-21T03:43:08.009225+0000","last_active":"2024-09-21T03:43:54.459397+0000","last_peered":"2024-09-21T03:43:54.459397+0000","last_clean":"2024-09-21T03:43:54.459397+0000","last_became_active":"2024-09-21T03:43:08.008830+0000","last_became_peered":"2024-09-21T03:43:08.008830+0000","last_unstale":"2024-09-21T03:43:54.459397+0000","last_undegraded":"2024-09-21T03:43:54.459397+0000","last_fullsized":"2024-09-21T03:43:54.459397+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:38.047763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456518+0000","last_change":"2024-09-21T03:43:18.089302+0000","last_active":"2024-09-21T03:43:54.456518+0000","last_peered":"2024-09-21T03:43:54.456518+0000","last_clean":"2024-09-21T03:43:54.456518+0000","last_became_active":"2024-09-21T03:43:18.088867+0000","last_became_peered":"2024-09-21T03:43:18.088867+0000","last_unstale":"2024-09-21T03:43:54.456518+0000","last_undegraded":"2024-09-21T03:43:54.456518+0000","last_fullsized":"2024-09-21T03:43:54.456518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:08.960044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":73,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:22.573148+0000","last_change":"2024-09-21T03:43:54.464317+0000","last_active":"2024-09-21T03:44:22.573148+0000","last_peered":"2024-09-21T03:44:22.573148+0000","last_clean":"2024-09-21T03:44:22.573148+0000","last_became_active":"2024-09-21T03:43:54.463870+0000","last_became_peered":"2024-09-21T03:43:54.463870+0000","last_unstale":"2024-09-21T03:44:22.573148+0000","last_undegraded":"2024-09-21T03:44:22.573148+0000","last_fullsized":"2024-09-21T03:44:22.573148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:14.581726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.491212+0000","last_change":"2024-09-21T03:43:08.009398+0000","last_active":"2024-09-21T03:43:54.491212+0000","last_peered":"2024-09-21T03:43:54.491212+0000","last_clean":"2024-09-21T03:43:54.491212+0000","last_became_active":"2024-09-21T03:43:08.009000+0000","last_became_peered":"2024-09-21T03:43:08.009000+0000","last_unstale":"2024-09-21T03:43:54.491212+0000","last_undegraded":"2024-09-21T03:43:54.491212+0000","last_fullsized":"2024-09-21T03:43:54.491212+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:07.278613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456459+0000","last_change":"2024-09-21T03:43:18.081834+0000","last_active":"2024-09-21T03:43:54.456459+0000","last_peered":"2024-09-21T03:43:54.456459+0000","last_clean":"2024-09-21T03:43:54.456459+0000","last_became_active":"2024-09-21T03:43:18.081328+0000","last_became_peered":"2024-09-21T03:43:18.081328+0000","last_unstale":"2024-09-21T03:43:54.456459+0000","last_undegraded":"2024-09-21T03:43:54.456459+0000","last_fullsized":"2024-09-21T03:43:54.456459+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:37.638826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.491208+0000","last_change":"2024-09-21T03:43:08.005144+0000","last_active":"2024-09-21T03:43:54.491208+0000","last_peered":"2024-09-21T03:43:54.491208+0000","last_clean":"2024-09-21T03:43:54.491208+0000","last_became_active":"2024-09-21T03:43:08.003895+0000","last_became_peered":"2024-09-21T03:43:08.003895+0000","last_unstale":"2024-09-21T03:43:54.491208+0000","last_undegraded":"2024-09-21T03:43:54.491208+0000","last_fullsized":"2024-09-21T03:43:54.491208+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:25.822315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.457502+0000","last_change":"2024-09-21T03:43:08.000218+0000","last_active":"2024-09-21T03:43:54.457502+0000","last_peered":"2024-09-21T03:43:54.457502+0000","last_clean":"2024-09-21T03:43:54.457502+0000","last_became_active":"2024-09-21T03:43:07.999840+0000","last_became_peered":"2024-09-21T03:43:07.999840+0000","last_unstale":"2024-09-21T03:43:54.457502+0000","last_undegraded":"2024-09-21T03:43:54.457502+0000","last_fullsized":"2024-09-21T03:43:54.457502+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:55.391706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456190+0000","last_change":"2024-09-21T03:43:08.012075+0000","last_active":"2024-09-21T03:43:54.456190+0000","last_peered":"2024-09-21T03:43:54.456190+0000","last_clean":"2024-09-21T03:43:54.456190+0000","last_became_active":"2024-09-21T03:43:08.011666+0000","last_became_peered":"2024-09-21T03:43:08.011666+0000","last_unstale":"2024-09-21T03:43:54.456190+0000","last_undegraded":"2024-09-21T03:43:54.456190+0000","last_fullsized":"2024-09-21T03:43:54.456190+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:42.124948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489567+0000","last_change":"2024-09-21T03:43:08.021082+0000","last_active":"2024-09-21T03:43:54.489567+0000","last_peered":"2024-09-21T03:43:54.489567+0000","last_clean":"2024-09-21T03:43:54.489567+0000","last_became_active":"2024-09-21T03:43:08.020674+0000","last_became_peered":"2024-09-21T03:43:08.020674+0000","last_unstale":"2024-09-21T03:43:54.489567+0000","last_undegraded":"2024-09-21T03:43:54.489567+0000","last_fullsized":"2024-09-21T03:43:54.489567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:13.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.458400+0000","last_change":"2024-09-21T03:43:08.004199+0000","last_active":"2024-09-21T03:43:54.458400+0000","last_peered":"2024-09-21T03:43:54.458400+0000","last_clean":"2024-09-21T03:43:54.458400+0000","last_became_active":"2024-09-21T03:43:08.003806+0000","last_became_peered":"2024-09-21T03:43:08.003806+0000","last_unstale":"2024-09-21T03:43:54.458400+0000","last_undegraded":"2024-09-21T03:43:54.458400+0000","last_fullsized":"2024-09-21T03:43:54.458400+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:13.941130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490222+0000","last_change":"2024-09-21T03:43:08.007237+0000","last_active":"2024-09-21T03:43:54.490222+0000","last_peered":"2024-09-21T03:43:54.490222+0000","last_clean":"2024-09-21T03:43:54.490222+0000","last_became_active":"2024-09-21T03:43:08.006805+0000","last_became_peered":"2024-09-21T03:43:08.006805+0000","last_unstale":"2024-09-21T03:43:54.490222+0000","last_undegraded":"2024-09-21T03:43:54.490222+0000","last_fullsized":"2024-09-21T03:43:54.490222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:26.274494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456601+0000","last_change":"2024-09-21T03:43:08.007934+0000","last_active":"2024-09-21T03:43:54.456601+0000","last_peered":"2024-09-21T03:43:54.456601+0000","last_clean":"2024-09-21T03:43:54.456601+0000","last_became_active":"2024-09-21T03:43:08.007549+0000","last_became_peered":"2024-09-21T03:43:08.007549+0000","last_unstale":"2024-09-21T03:43:54.456601+0000","last_undegraded":"2024-09-21T03:43:54.456601+0000","last_fullsized":"2024-09-21T03:43:54.456601+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:58.158255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.493820+0000","last_change":"2024-09-21T03:43:54.493665+0000","last_active":"2024-09-21T03:43:54.493820+0000","last_peered":"2024-09-21T03:43:54.493820+0000","last_clean":"2024-09-21T03:43:54.493820+0000","last_became_active":"2024-09-21T03:43:54.493245+0000","last_became_peered":"2024-09-21T03:43:54.493245+0000","last_unstale":"2024-09-21T03:43:54.493820+0000","last_undegraded":"2024-09-21T03:43:54.493820+0000","last_fullsized":"2024-09-21T03:43:54.493820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:35.449852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.464246+0000","last_change":"2024-09-21T03:43:54.463419+0000","last_active":"2024-09-21T03:43:54.464246+0000","last_peered":"2024-09-21T03:43:54.464246+0000","last_clean":"2024-09-21T03:43:54.464246+0000","last_became_active":"2024-09-21T03:43:54.462951+0000","last_became_peered":"2024-09-21T03:43:54.462951+0000","last_unstale":"2024-09-21T03:43:54.464246+0000","last_undegraded":"2024-09-21T03:43:54.464246+0000","last_fullsized":"2024-09-21T03:43:54.464246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:23.620275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.460597+0000","last_change":"2024-09-21T03:43:08.011436+0000","last_active":"2024-09-21T03:43:54.460597+0000","last_peered":"2024-09-21T03:43:54.460597+0000","last_clean":"2024-09-21T03:43:54.460597+0000","last_became_active":"2024-09-21T03:43:08.011025+0000","last_became_peered":"2024-09-21T03:43:08.011025+0000","last_unstale":"2024-09-21T03:43:54.460597+0000","last_undegraded":"2024-09-21T03:43:54.460597+0000","last_fullsized":"2024-09-21T03:43:54.460597+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:31.411858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490007+0000","last_change":"2024-09-21T03:43:08.016378+0000","last_active":"2024-09-21T03:43:54.490007+0000","last_peered":"2024-09-21T03:43:54.490007+0000","last_clean":"2024-09-21T03:43:54.490007+0000","last_became_active":"2024-09-21T03:43:08.016013+0000","last_became_peered":"2024-09-21T03:43:08.016013+0000","last_unstale":"2024-09-21T03:43:54.490007+0000","last_undegraded":"2024-09-21T03:43:54.490007+0000","last_fullsized":"2024-09-21T03:43:54.490007+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:01.865669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.458144+0000","last_change":"2024-09-21T03:43:08.007862+0000","last_active":"2024-09-21T03:43:54.458144+0000","last_peered":"2024-09-21T03:43:54.458144+0000","last_clean":"2024-09-21T03:43:54.458144+0000","last_became_active":"2024-09-21T03:43:08.007433+0000","last_became_peered":"2024-09-21T03:43:08.007433+0000","last_unstale":"2024-09-21T03:43:54.458144+0000","last_undegraded":"2024-09-21T03:43:54.458144+0000","last_fullsized":"2024-09-21T03:43:54.458144+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:22:03.364767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.491033+0000","last_change":"2024-09-21T03:43:08.010542+0000","last_active":"2024-09-21T03:43:54.491033+0000","last_peered":"2024-09-21T03:43:54.491033+0000","last_clean":"2024-09-21T03:43:54.491033+0000","last_became_active":"2024-09-21T03:43:08.010195+0000","last_became_peered":"2024-09-21T03:43:08.010195+0000","last_unstale":"2024-09-21T03:43:54.491033+0000","last_undegraded":"2024-09-21T03:43:54.491033+0000","last_fullsized":"2024-09-21T03:43:54.491033+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:52.850657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456740+0000","last_change":"2024-09-21T03:43:08.006168+0000","last_active":"2024-09-21T03:43:54.456740+0000","last_peered":"2024-09-21T03:43:54.456740+0000","last_clean":"2024-09-21T03:43:54.456740+0000","last_became_active":"2024-09-21T03:43:08.005283+0000","last_became_peered":"2024-09-21T03:43:08.005283+0000","last_unstale":"2024-09-21T03:43:54.456740+0000","last_undegraded":"2024-09-21T03:43:54.456740+0000","last_fullsized":"2024-09-21T03:43:54.456740+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:59.852412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.454368+0000","last_change":"2024-09-21T03:43:08.005870+0000","last_active":"2024-09-21T03:43:54.454368+0000","last_peered":"2024-09-21T03:43:54.454368+0000","last_clean":"2024-09-21T03:43:54.454368+0000","last_became_active":"2024-09-21T03:43:08.005485+0000","last_became_peered":"2024-09-21T03:43:08.005485+0000","last_unstale":"2024-09-21T03:43:54.454368+0000","last_undegraded":"2024-09-21T03:43:54.454368+0000","last_fullsized":"2024-09-21T03:43:54.454368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:41.387865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490804+0000","last_change":"2024-09-21T03:43:08.015176+0000","last_active":"2024-09-21T03:43:54.490804+0000","last_peered":"2024-09-21T03:43:54.490804+0000","last_clean":"2024-09-21T03:43:54.490804+0000","last_became_active":"2024-09-21T03:43:08.014451+0000","last_became_peered":"2024-09-21T03:43:08.014451+0000","last_unstale":"2024-09-21T03:43:54.490804+0000","last_undegraded":"2024-09-21T03:43:54.490804+0000","last_fullsized":"2024-09-21T03:43:54.490804+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:49.524800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489769+0000","last_change":"2024-09-21T03:43:08.016798+0000","last_active":"2024-09-21T03:43:54.489769+0000","last_peered":"2024-09-21T03:43:54.489769+0000","last_clean":"2024-09-21T03:43:54.489769+0000","last_became_active":"2024-09-21T03:43:08.014842+0000","last_became_peered":"2024-09-21T03:43:08.014842+0000","last_unstale":"2024-09-21T03:43:54.489769+0000","last_undegraded":"2024-09-21T03:43:54.489769+0000","last_fullsized":"2024-09-21T03:43:54.489769+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:19.290618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489730+0000","last_change":"2024-09-21T03:43:08.021285+0000","last_active":"2024-09-21T03:43:54.489730+0000","last_peered":"2024-09-21T03:43:54.489730+0000","last_clean":"2024-09-21T03:43:54.489730+0000","last_became_active":"2024-09-21T03:43:08.020904+0000","last_became_peered":"2024-09-21T03:43:08.020904+0000","last_unstale":"2024-09-21T03:43:54.489730+0000","last_undegraded":"2024-09-21T03:43:54.489730+0000","last_fullsized":"2024-09-21T03:43:54.489730+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:56.589089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489290+0000","last_change":"2024-09-21T03:43:08.017599+0000","last_active":"2024-09-21T03:43:54.489290+0000","last_peered":"2024-09-21T03:43:54.489290+0000","last_clean":"2024-09-21T03:43:54.489290+0000","last_became_active":"2024-09-21T03:43:08.016138+0000","last_became_peered":"2024-09-21T03:43:08.016138+0000","last_unstale":"2024-09-21T03:43:54.489290+0000","last_undegraded":"2024-09-21T03:43:54.489290+0000","last_fullsized":"2024-09-21T03:43:54.489290+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:09.902246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.458119+0000","last_change":"2024-09-21T03:43:08.009598+0000","last_active":"2024-09-21T03:43:54.458119+0000","last_peered":"2024-09-21T03:43:54.458119+0000","last_clean":"2024-09-21T03:43:54.458119+0000","last_became_active":"2024-09-21T03:43:08.007488+0000","last_became_peered":"2024-09-21T03:43:08.007488+0000","last_unstale":"2024-09-21T03:43:54.458119+0000","last_undegraded":"2024-09-21T03:43:54.458119+0000","last_fullsized":"2024-09-21T03:43:54.458119+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:23.192761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.457719+0000","last_change":"2024-09-21T03:43:08.016450+0000","last_active":"2024-09-21T03:43:54.457719+0000","last_peered":"2024-09-21T03:43:54.457719+0000","last_clean":"2024-09-21T03:43:54.457719+0000","last_became_active":"2024-09-21T03:43:08.016037+0000","last_became_peered":"2024-09-21T03:43:08.016037+0000","last_unstale":"2024-09-21T03:43:54.457719+0000","last_undegraded":"2024-09-21T03:43:54.457719+0000","last_fullsized":"2024-09-21T03:43:54.457719+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:39.109408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462131+0000","last_change":"2024-09-21T03:43:08.010298+0000","last_active":"2024-09-21T03:43:54.462131+0000","last_peered":"2024-09-21T03:43:54.462131+0000","last_clean":"2024-09-21T03:43:54.462131+0000","last_became_active":"2024-09-21T03:43:08.009847+0000","last_became_peered":"2024-09-21T03:43:08.009847+0000","last_unstale":"2024-09-21T03:43:54.462131+0000","last_undegraded":"2024-09-21T03:43:54.462131+0000","last_fullsized":"2024-09-21T03:43:54.462131+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:37.463456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.454757+0000","last_change":"2024-09-21T03:43:08.017834+0000","last_active":"2024-09-21T03:43:54.454757+0000","last_peered":"2024-09-21T03:43:54.454757+0000","last_clean":"2024-09-21T03:43:54.454757+0000","last_became_active":"2024-09-21T03:43:08.016910+0000","last_became_peered":"2024-09-21T03:43:08.016910+0000","last_unstale":"2024-09-21T03:43:54.454757+0000","last_undegraded":"2024-09-21T03:43:54.454757+0000","last_fullsized":"2024-09-21T03:43:54.454757+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:50.528651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490598+0000","last_change":"2024-09-21T03:43:08.013945+0000","last_active":"2024-09-21T03:43:54.490598+0000","last_peered":"2024-09-21T03:43:54.490598+0000","last_clean":"2024-09-21T03:43:54.490598+0000","last_became_active":"2024-09-21T03:43:08.013598+0000","last_became_peered":"2024-09-21T03:43:08.013598+0000","last_unstale":"2024-09-21T03:43:54.490598+0000","last_undegraded":"2024-09-21T03:43:54.490598+0000","last_fullsized":"2024-09-21T03:43:54.490598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:58.182938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.460179+0000","last_change":"2024-09-21T03:43:08.018824+0000","last_active":"2024-09-21T03:43:54.460179+0000","last_peered":"2024-09-21T03:43:54.460179+0000","last_clean":"2024-09-21T03:43:54.460179+0000","last_became_active":"2024-09-21T03:43:08.017196+0000","last_became_peered":"2024-09-21T03:43:08.017196+0000","last_unstale":"2024-09-21T03:43:54.460179+0000","last_undegraded":"2024-09-21T03:43:54.460179+0000","last_fullsized":"2024-09-21T03:43:54.460179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:02.357523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462145+0000","last_change":"2024-09-21T03:43:08.003514+0000","last_active":"2024-09-21T03:43:54.462145+0000","last_peered":"2024-09-21T03:43:54.462145+0000","last_clean":"2024-09-21T03:43:54.462145+0000","last_became_active":"2024-09-21T03:43:08.003131+0000","last_became_peered":"2024-09-21T03:43:08.003131+0000","last_unstale":"2024-09-21T03:43:54.462145+0000","last_undegraded":"2024-09-21T03:43:54.462145+0000","last_fullsized":"2024-09-21T03:43:54.462145+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:23.170302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.454522+0000","last_change":"2024-09-21T03:43:08.015890+0000","last_active":"2024-09-21T03:43:54.454522+0000","last_peered":"2024-09-21T03:43:54.454522+0000","last_clean":"2024-09-21T03:43:54.454522+0000","last_became_active":"2024-09-21T03:43:08.015519+0000","last_became_peered":"2024-09-21T03:43:08.015519+0000","last_unstale":"2024-09-21T03:43:54.454522+0000","last_undegraded":"2024-09-21T03:43:54.454522+0000","last_fullsized":"2024-09-21T03:43:54.454522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:10.875923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.491133+0000","last_change":"2024-09-21T03:43:08.012261+0000","last_active":"2024-09-21T03:43:54.491133+0000","last_peered":"2024-09-21T03:43:54.491133+0000","last_clean":"2024-09-21T03:43:54.491133+0000","last_became_active":"2024-09-21T03:43:08.011783+0000","last_became_peered":"2024-09-21T03:43:08.011783+0000","last_unstale":"2024-09-21T03:43:54.491133+0000","last_undegraded":"2024-09-21T03:43:54.491133+0000","last_fullsized":"2024-09-21T03:43:54.491133+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:07.011340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.496630+0000","last_change":"2024-09-21T03:43:54.494826+0000","last_active":"2024-09-21T03:43:54.496630+0000","last_peered":"2024-09-21T03:43:54.496630+0000","last_clean":"2024-09-21T03:43:54.496630+0000","last_became_active":"2024-09-21T03:43:54.494404+0000","last_became_peered":"2024-09-21T03:43:54.494404+0000","last_unstale":"2024-09-21T03:43:54.496630+0000","last_undegraded":"2024-09-21T03:43:54.496630+0000","last_fullsized":"2024-09-21T03:43:54.496630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:42.109053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489653+0000","last_change":"2024-09-21T03:43:08.016214+0000","last_active":"2024-09-21T03:43:54.489653+0000","last_peered":"2024-09-21T03:43:54.489653+0000","last_clean":"2024-09-21T03:43:54.489653+0000","last_became_active":"2024-09-21T03:43:08.014815+0000","last_became_peered":"2024-09-21T03:43:08.014815+0000","last_unstale":"2024-09-21T03:43:54.489653+0000","last_undegraded":"2024-09-21T03:43:54.489653+0000","last_fullsized":"2024-09-21T03:43:54.489653+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:04.742829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489944+0000","last_change":"2024-09-21T03:43:08.019269+0000","last_active":"2024-09-21T03:43:54.489944+0000","last_peered":"2024-09-21T03:43:54.489944+0000","last_clean":"2024-09-21T03:43:54.489944+0000","last_became_active":"2024-09-21T03:43:08.017962+0000","last_became_peered":"2024-09-21T03:43:08.017962+0000","last_unstale":"2024-09-21T03:43:54.489944+0000","last_undegraded":"2024-09-21T03:43:54.489944+0000","last_fullsized":"2024-09-21T03:43:54.489944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:05.950697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490672+0000","last_change":"2024-09-21T03:43:08.014790+0000","last_active":"2024-09-21T03:43:54.490672+0000","last_peered":"2024-09-21T03:43:54.490672+0000","last_clean":"2024-09-21T03:43:54.490672+0000","last_became_active":"2024-09-21T03:43:08.013891+0000","last_became_peered":"2024-09-21T03:43:08.013891+0000","last_unstale":"2024-09-21T03:43:54.490672+0000","last_undegraded":"2024-09-21T03:43:54.490672+0000","last_fullsized":"2024-09-21T03:43:54.490672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:46.045675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63362,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":209,"num_read_kb":172,"num_write":239,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":190086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808862,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":482159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972387,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1129229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135911,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":481698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332142,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":482076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495668,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36892,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708260,"statfs":{"total":95995035648,"available":95957258240,"internally_reserved":0,"allocated":1585152,"data_stored":1078182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659193,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":477202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855423,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1593344,"data_stored":1073764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018949,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":481898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215177,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":476910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444112,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":534770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":534781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":479049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:44:26.536 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:44:26.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:26 smithi134 ceph-mon[35031]: pgmap v360: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-21T03:44:26.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:26 smithi134 ceph-mon[35031]: from='client.24953 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:44:26.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:26 smithi123 ceph-mon[34673]: pgmap v360: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-21T03:44:26.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:26 smithi123 ceph-mon[34673]: from='client.24953 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:44:26.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:26 smithi016 ceph-mon[30375]: pgmap v360: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-21T03:44:26.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:26 smithi016 ceph-mon[30375]: from='client.24953 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:44:26.913 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:27.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.565+0000 7fef66800640 1 -- 172.21.15.16:0/3283486201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef60077940 msgr2=0x7fef60077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:27.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.565+0000 7fef66800640 1 --2- 172.21.15.16:0/3283486201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef60077940 0x7fef60077d50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fef50009900 tx=0x7fef5002a410 comp rx=0 tx=0).stop 2024-09-21T03:44:27.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.566+0000 7fef66800640 1 -- 172.21.15.16:0/3283486201 shutdown_connections 2024-09-21T03:44:27.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.566+0000 7fef66800640 1 --2- 172.21.15.16:0/3283486201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef60114750 0x7fef60116b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:27.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.566+0000 7fef66800640 1 --2- 172.21.15.16:0/3283486201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef60078290 0x7fef60114210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:27.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.566+0000 7fef66800640 1 --2- 172.21.15.16:0/3283486201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef60077940 0x7fef60077d50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:27.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.566+0000 7fef66800640 1 -- 172.21.15.16:0/3283486201 >> 172.21.15.16:0/3283486201 conn(0x7fef60101860 msgr2=0x7fef60103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:27.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.566+0000 7fef66800640 1 -- 172.21.15.16:0/3283486201 shutdown_connections 2024-09-21T03:44:27.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.566+0000 7fef66800640 1 -- 172.21.15.16:0/3283486201 wait complete. 2024-09-21T03:44:27.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.567+0000 7fef66800640 1 Processor -- start 2024-09-21T03:44:27.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.567+0000 7fef66800640 1 -- start start 2024-09-21T03:44:27.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.567+0000 7fef66800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef60077940 0x7fef601a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:27.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.567+0000 7fef66800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef60078290 0x7fef601a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:27.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.567+0000 7fef66800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef60114750 0x7fef601a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:27.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.567+0000 7fef66800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef6011a130 con 0x7fef60077940 2024-09-21T03:44:27.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.567+0000 7fef66800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef60119fb0 con 0x7fef60078290 2024-09-21T03:44:27.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.567+0000 7fef66800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef6011a2b0 con 0x7fef60114750 2024-09-21T03:44:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.568+0000 7fef5f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef60077940 0x7fef601a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.568+0000 7fef5f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef60077940 0x7fef601a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38254/0 (socket says 172.21.15.16:38254) 2024-09-21T03:44:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.568+0000 7fef5f400640 1 -- 172.21.15.16:0/3355947151 learned_addr learned my addr 172.21.15.16:0/3355947151 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.568+0000 7fef5fe00640 1 --2- 172.21.15.16:0/3355947151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef60114750 0x7fef601a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.568+0000 7fef5ea00640 1 --2- 172.21.15.16:0/3355947151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef60078290 0x7fef601a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.568+0000 7fef5f400640 1 -- 172.21.15.16:0/3355947151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef60114750 msgr2=0x7fef601a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.568+0000 7fef5f400640 1 --2- 172.21.15.16:0/3355947151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef60114750 0x7fef601a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.568+0000 7fef5f400640 1 -- 172.21.15.16:0/3355947151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef60078290 msgr2=0x7fef601a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.568+0000 7fef5f400640 1 --2- 172.21.15.16:0/3355947151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef60078290 0x7fef601a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:27.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.568+0000 7fef5f400640 1 -- 172.21.15.16:0/3355947151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef601a6a30 con 0x7fef60077940 2024-09-21T03:44:27.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.568+0000 7fef5fe00640 1 --2- 172.21.15.16:0/3355947151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef60114750 0x7fef601a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:27.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.569+0000 7fef5f400640 1 --2- 172.21.15.16:0/3355947151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef60077940 0x7fef601a0ac0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fef50005ab0 tx=0x7fef50003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:27.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.569+0000 7fef57e00640 1 -- 172.21.15.16:0/3355947151 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5003f070 con 0x7fef60077940 2024-09-21T03:44:27.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.569+0000 7fef57e00640 1 -- 172.21.15.16:0/3355947151 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef50031b60 con 0x7fef60077940 2024-09-21T03:44:27.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.569+0000 7fef66800640 1 -- 172.21.15.16:0/3355947151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef601a6cc0 con 0x7fef60077940 2024-09-21T03:44:27.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.569+0000 7fef57e00640 1 -- 172.21.15.16:0/3355947151 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50035030 con 0x7fef60077940 2024-09-21T03:44:27.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.569+0000 7fef66800640 1 -- 172.21.15.16:0/3355947151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef601ad480 con 0x7fef60077940 2024-09-21T03:44:27.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.571+0000 7fef66800640 1 -- 172.21.15.16:0/3355947151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef60077d50 con 0x7fef60077940 2024-09-21T03:44:27.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.573+0000 7fef57e00640 1 -- 172.21.15.16:0/3355947151 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fef50042070 con 0x7fef60077940 2024-09-21T03:44:27.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.574+0000 7fef57e00640 1 --2- 172.21.15.16:0/3355947151 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fef440b6620 0x7fef440b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:27.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.575+0000 7fef57e00640 1 -- 172.21.15.16:0/3355947151 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7fef50108680 con 0x7fef60077940 2024-09-21T03:44:27.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.575+0000 7fef5ea00640 1 --2- 172.21.15.16:0/3355947151 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fef440b6620 0x7fef440b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:27.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.576+0000 7fef5ea00640 1 --2- 172.21.15.16:0/3355947151 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fef440b6620 0x7fef440b8b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fef60079da0 tx=0x7fef4c005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:27.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.582+0000 7fef57e00640 1 -- 172.21.15.16:0/3355947151 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fef500d2dd0 con 0x7fef60077940 2024-09-21T03:44:27.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.781+0000 7fef66800640 1 -- 172.21.15.16:0/3355947151 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef6010f940 con 0x7fef440b6620 2024-09-21T03:44:27.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.794+0000 7fef57e00640 1 -- 172.21.15.16:0/3355947151 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340378 (secure 0 0 0) 0x7fef6010f940 con 0x7fef440b6620 2024-09-21T03:44:27.801 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:44:27.802 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-21T03:44:27.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.800+0000 7fef66800640 1 -- 172.21.15.16:0/3355947151 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fef440b6620 msgr2=0x7fef440b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:27.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.800+0000 7fef66800640 1 --2- 172.21.15.16:0/3355947151 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fef440b6620 0x7fef440b8b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fef60079da0 tx=0x7fef4c005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:27.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.800+0000 7fef66800640 1 -- 172.21.15.16:0/3355947151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef60077940 msgr2=0x7fef601a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:27.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.800+0000 7fef66800640 1 --2- 172.21.15.16:0/3355947151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef60077940 0x7fef601a0ac0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fef50005ab0 tx=0x7fef50003c10 comp rx=0 tx=0).stop 2024-09-21T03:44:27.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.800+0000 7fef66800640 1 -- 172.21.15.16:0/3355947151 shutdown_connections 2024-09-21T03:44:27.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.800+0000 7fef66800640 1 --2- 172.21.15.16:0/3355947151 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fef440b6620 0x7fef440b8b30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:27.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.800+0000 7fef66800640 1 --2- 172.21.15.16:0/3355947151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef60114750 0x7fef601a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:27.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.800+0000 7fef66800640 1 --2- 172.21.15.16:0/3355947151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef60078290 0x7fef601a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:27.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.800+0000 7fef66800640 1 --2- 172.21.15.16:0/3355947151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef60077940 0x7fef601a0ac0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:27.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.800+0000 7fef66800640 1 -- 172.21.15.16:0/3355947151 >> 172.21.15.16:0/3355947151 conn(0x7fef60101860 msgr2=0x7fef60116750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:27.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.801+0000 7fef66800640 1 -- 172.21.15.16:0/3355947151 shutdown_connections 2024-09-21T03:44:27.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:27.801+0000 7fef66800640 1 -- 172.21.15.16:0/3355947151 wait complete. 2024-09-21T03:44:28.611 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-21T03:44:27.349611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653964,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":295,"num_read_kb":241,"num_write":370,"num_write_kb":2284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":303,"ondisk_log_size":303,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379344,"kb_used_data":13456,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562480,"statfs":{"total":1151940427776,"available":1151551979520,"internally_reserved":0,"allocated":13778944,"data_stored":7720248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46224,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":115,"num_read_kb":102,"num_write":32,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003935"},"pg_stats":[{"pgid":"2.1f","version":"84'13","reported_seq":75,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:24.575819+0000","last_change":"2024-09-21T03:43:04.980132+0000","last_active":"2024-09-21T03:44:24.575819+0000","last_peered":"2024-09-21T03:44:24.575819+0000","last_clean":"2024-09-21T03:44:24.575819+0000","last_became_active":"2024-09-21T03:43:04.979274+0000","last_became_peered":"2024-09-21T03:43:04.979274+0000","last_unstale":"2024-09-21T03:44:24.575819+0000","last_undegraded":"2024-09-21T03:44:24.575819+0000","last_fullsized":"2024-09-21T03:44:24.575819+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:35.047129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462231+0000","last_change":"2024-09-21T03:43:08.011576+0000","last_active":"2024-09-21T03:43:54.462231+0000","last_peered":"2024-09-21T03:43:54.462231+0000","last_clean":"2024-09-21T03:43:54.462231+0000","last_became_active":"2024-09-21T03:43:08.011188+0000","last_became_peered":"2024-09-21T03:43:08.011188+0000","last_unstale":"2024-09-21T03:43:54.462231+0000","last_undegraded":"2024-09-21T03:43:54.462231+0000","last_fullsized":"2024-09-21T03:43:54.462231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:34.477639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.493297+0000","last_change":"2024-09-21T03:43:54.493183+0000","last_active":"2024-09-21T03:43:54.493297+0000","last_peered":"2024-09-21T03:43:54.493297+0000","last_clean":"2024-09-21T03:43:54.493297+0000","last_became_active":"2024-09-21T03:43:54.492605+0000","last_became_peered":"2024-09-21T03:43:54.492605+0000","last_unstale":"2024-09-21T03:43:54.493297+0000","last_undegraded":"2024-09-21T03:43:54.493297+0000","last_fullsized":"2024-09-21T03:43:54.493297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:26.634042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462613+0000","last_change":"2024-09-21T03:43:05.141790+0000","last_active":"2024-09-21T03:43:54.462613+0000","last_peered":"2024-09-21T03:43:54.462613+0000","last_clean":"2024-09-21T03:43:54.462613+0000","last_became_active":"2024-09-21T03:43:05.141299+0000","last_became_peered":"2024-09-21T03:43:05.141299+0000","last_unstale":"2024-09-21T03:43:54.462613+0000","last_undegraded":"2024-09-21T03:43:54.462613+0000","last_fullsized":"2024-09-21T03:43:54.462613+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:48.844615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489722+0000","last_change":"2024-09-21T03:43:08.018245+0000","last_active":"2024-09-21T03:43:54.489722+0000","last_peered":"2024-09-21T03:43:54.489722+0000","last_clean":"2024-09-21T03:43:54.489722+0000","last_became_active":"2024-09-21T03:43:08.015719+0000","last_became_peered":"2024-09-21T03:43:08.015719+0000","last_unstale":"2024-09-21T03:43:54.489722+0000","last_undegraded":"2024-09-21T03:43:54.489722+0000","last_fullsized":"2024-09-21T03:43:54.489722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:39.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.493401+0000","last_change":"2024-09-21T03:43:54.493279+0000","last_active":"2024-09-21T03:43:54.493401+0000","last_peered":"2024-09-21T03:43:54.493401+0000","last_clean":"2024-09-21T03:43:54.493401+0000","last_became_active":"2024-09-21T03:43:54.492592+0000","last_became_peered":"2024-09-21T03:43:54.492592+0000","last_unstale":"2024-09-21T03:43:54.493401+0000","last_undegraded":"2024-09-21T03:43:54.493401+0000","last_fullsized":"2024-09-21T03:43:54.493401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:28.520990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":54,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:06.537311+0000","last_change":"2024-09-21T03:43:04.982919+0000","last_active":"2024-09-21T03:44:06.537311+0000","last_peered":"2024-09-21T03:44:06.537311+0000","last_clean":"2024-09-21T03:44:06.537311+0000","last_became_active":"2024-09-21T03:43:04.982427+0000","last_became_peered":"2024-09-21T03:43:04.982427+0000","last_unstale":"2024-09-21T03:44:06.537311+0000","last_undegraded":"2024-09-21T03:44:06.537311+0000","last_fullsized":"2024-09-21T03:44:06.537311+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:27.042078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489984+0000","last_change":"2024-09-21T03:43:08.009018+0000","last_active":"2024-09-21T03:43:54.489984+0000","last_peered":"2024-09-21T03:43:54.489984+0000","last_clean":"2024-09-21T03:43:54.489984+0000","last_became_active":"2024-09-21T03:43:08.008614+0000","last_became_peered":"2024-09-21T03:43:08.008614+0000","last_unstale":"2024-09-21T03:43:54.489984+0000","last_undegraded":"2024-09-21T03:43:54.489984+0000","last_fullsized":"2024-09-21T03:43:54.489984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:31.000616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.457441+0000","last_change":"2024-09-21T03:43:18.092750+0000","last_active":"2024-09-21T03:43:54.457441+0000","last_peered":"2024-09-21T03:43:54.457441+0000","last_clean":"2024-09-21T03:43:54.457441+0000","last_became_active":"2024-09-21T03:43:18.092122+0000","last_became_peered":"2024-09-21T03:43:18.092122+0000","last_unstale":"2024-09-21T03:43:54.457441+0000","last_undegraded":"2024-09-21T03:43:54.457441+0000","last_fullsized":"2024-09-21T03:43:54.457441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:15.032388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.543975+0000","last_change":"2024-09-21T03:43:04.978045+0000","last_active":"2024-09-21T03:44:05.543975+0000","last_peered":"2024-09-21T03:44:05.543975+0000","last_clean":"2024-09-21T03:44:05.543975+0000","last_became_active":"2024-09-21T03:43:04.977565+0000","last_became_peered":"2024-09-21T03:43:04.977565+0000","last_unstale":"2024-09-21T03:44:05.543975+0000","last_undegraded":"2024-09-21T03:44:05.543975+0000","last_fullsized":"2024-09-21T03:44:05.543975+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:42.537892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.459614+0000","last_change":"2024-09-21T03:43:08.007094+0000","last_active":"2024-09-21T03:43:54.459614+0000","last_peered":"2024-09-21T03:43:54.459614+0000","last_clean":"2024-09-21T03:43:54.459614+0000","last_became_active":"2024-09-21T03:43:08.006666+0000","last_became_peered":"2024-09-21T03:43:08.006666+0000","last_unstale":"2024-09-21T03:43:54.459614+0000","last_undegraded":"2024-09-21T03:43:54.459614+0000","last_fullsized":"2024-09-21T03:43:54.459614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:59.247459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456502+0000","last_change":"2024-09-21T03:43:18.081141+0000","last_active":"2024-09-21T03:43:54.456502+0000","last_peered":"2024-09-21T03:43:54.456502+0000","last_clean":"2024-09-21T03:43:54.456502+0000","last_became_active":"2024-09-21T03:43:18.080623+0000","last_became_peered":"2024-09-21T03:43:18.080623+0000","last_unstale":"2024-09-21T03:43:54.456502+0000","last_undegraded":"2024-09-21T03:43:54.456502+0000","last_fullsized":"2024-09-21T03:43:54.456502+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:44.932673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462692+0000","last_change":"2024-09-21T03:43:08.008602+0000","last_active":"2024-09-21T03:43:54.462692+0000","last_peered":"2024-09-21T03:43:54.462692+0000","last_clean":"2024-09-21T03:43:54.462692+0000","last_became_active":"2024-09-21T03:43:08.008220+0000","last_became_peered":"2024-09-21T03:43:08.008220+0000","last_unstale":"2024-09-21T03:43:54.462692+0000","last_undegraded":"2024-09-21T03:43:54.462692+0000","last_fullsized":"2024-09-21T03:43:54.462692+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:28.286016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":46,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:02.510319+0000","last_change":"2024-09-21T03:43:04.979357+0000","last_active":"2024-09-21T03:44:02.510319+0000","last_peered":"2024-09-21T03:44:02.510319+0000","last_clean":"2024-09-21T03:44:02.510319+0000","last_became_active":"2024-09-21T03:43:04.978934+0000","last_became_peered":"2024-09-21T03:43:04.978934+0000","last_unstale":"2024-09-21T03:44:02.510319+0000","last_undegraded":"2024-09-21T03:44:02.510319+0000","last_fullsized":"2024-09-21T03:44:02.510319+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:00.858158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456966+0000","last_change":"2024-09-21T03:43:18.483523+0000","last_active":"2024-09-21T03:43:54.456966+0000","last_peered":"2024-09-21T03:43:54.456966+0000","last_clean":"2024-09-21T03:43:54.456966+0000","last_became_active":"2024-09-21T03:43:18.482833+0000","last_became_peered":"2024-09-21T03:43:18.482833+0000","last_unstale":"2024-09-21T03:43:54.456966+0000","last_undegraded":"2024-09-21T03:43:54.456966+0000","last_fullsized":"2024-09-21T03:43:54.456966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:48.236569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.461722+0000","last_change":"2024-09-21T03:43:08.022592+0000","last_active":"2024-09-21T03:43:54.461722+0000","last_peered":"2024-09-21T03:43:54.461722+0000","last_clean":"2024-09-21T03:43:54.461722+0000","last_became_active":"2024-09-21T03:43:08.018065+0000","last_became_peered":"2024-09-21T03:43:08.018065+0000","last_unstale":"2024-09-21T03:43:54.461722+0000","last_undegraded":"2024-09-21T03:43:54.461722+0000","last_fullsized":"2024-09-21T03:43:54.461722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:41.524402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:04.536611+0000","last_change":"2024-09-21T03:43:04.978788+0000","last_active":"2024-09-21T03:44:04.536611+0000","last_peered":"2024-09-21T03:44:04.536611+0000","last_clean":"2024-09-21T03:44:04.536611+0000","last_became_active":"2024-09-21T03:43:04.978382+0000","last_became_peered":"2024-09-21T03:43:04.978382+0000","last_unstale":"2024-09-21T03:44:04.536611+0000","last_undegraded":"2024-09-21T03:44:04.536611+0000","last_fullsized":"2024-09-21T03:44:04.536611+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:12.571217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.494138+0000","last_change":"2024-09-21T03:43:54.493347+0000","last_active":"2024-09-21T03:43:54.494138+0000","last_peered":"2024-09-21T03:43:54.494138+0000","last_clean":"2024-09-21T03:43:54.494138+0000","last_became_active":"2024-09-21T03:43:54.492598+0000","last_became_peered":"2024-09-21T03:43:54.492598+0000","last_unstale":"2024-09-21T03:43:54.494138+0000","last_undegraded":"2024-09-21T03:43:54.494138+0000","last_fullsized":"2024-09-21T03:43:54.494138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:34.788296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.491320+0000","last_change":"2024-09-21T03:43:08.014368+0000","last_active":"2024-09-21T03:43:54.491320+0000","last_peered":"2024-09-21T03:43:54.491320+0000","last_clean":"2024-09-21T03:43:54.491320+0000","last_became_active":"2024-09-21T03:43:08.013952+0000","last_became_peered":"2024-09-21T03:43:08.013952+0000","last_unstale":"2024-09-21T03:43:54.491320+0000","last_undegraded":"2024-09-21T03:43:54.491320+0000","last_fullsized":"2024-09-21T03:43:54.491320+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:16.893588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.543519+0000","last_change":"2024-09-21T03:43:04.976162+0000","last_active":"2024-09-21T03:44:05.543519+0000","last_peered":"2024-09-21T03:44:05.543519+0000","last_clean":"2024-09-21T03:44:05.543519+0000","last_became_active":"2024-09-21T03:43:04.973790+0000","last_became_peered":"2024-09-21T03:43:04.973790+0000","last_unstale":"2024-09-21T03:44:05.543519+0000","last_undegraded":"2024-09-21T03:44:05.543519+0000","last_fullsized":"2024-09-21T03:44:05.543519+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:47.838229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.454608+0000","last_change":"2024-09-21T03:43:18.084344+0000","last_active":"2024-09-21T03:43:54.454608+0000","last_peered":"2024-09-21T03:43:54.454608+0000","last_clean":"2024-09-21T03:43:54.454608+0000","last_became_active":"2024-09-21T03:43:18.083232+0000","last_became_peered":"2024-09-21T03:43:18.083232+0000","last_unstale":"2024-09-21T03:43:54.454608+0000","last_undegraded":"2024-09-21T03:43:54.454608+0000","last_fullsized":"2024-09-21T03:43:54.454608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:46.254405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.491379+0000","last_change":"2024-09-21T03:43:08.009932+0000","last_active":"2024-09-21T03:43:54.491379+0000","last_peered":"2024-09-21T03:43:54.491379+0000","last_clean":"2024-09-21T03:43:54.491379+0000","last_became_active":"2024-09-21T03:43:08.009492+0000","last_became_peered":"2024-09-21T03:43:08.009492+0000","last_unstale":"2024-09-21T03:43:54.491379+0000","last_undegraded":"2024-09-21T03:43:54.491379+0000","last_fullsized":"2024-09-21T03:43:54.491379+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:58.679348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:02.512989+0000","last_change":"2024-09-21T03:43:05.141927+0000","last_active":"2024-09-21T03:44:02.512989+0000","last_peered":"2024-09-21T03:44:02.512989+0000","last_clean":"2024-09-21T03:44:02.512989+0000","last_became_active":"2024-09-21T03:43:05.141516+0000","last_became_peered":"2024-09-21T03:43:05.141516+0000","last_unstale":"2024-09-21T03:44:02.512989+0000","last_undegraded":"2024-09-21T03:44:02.512989+0000","last_fullsized":"2024-09-21T03:44:02.512989+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:35.666981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456824+0000","last_change":"2024-09-21T03:43:18.484478+0000","last_active":"2024-09-21T03:43:54.456824+0000","last_peered":"2024-09-21T03:43:54.456824+0000","last_clean":"2024-09-21T03:43:54.456824+0000","last_became_active":"2024-09-21T03:43:18.483890+0000","last_became_peered":"2024-09-21T03:43:18.483890+0000","last_unstale":"2024-09-21T03:43:54.456824+0000","last_undegraded":"2024-09-21T03:43:54.456824+0000","last_fullsized":"2024-09-21T03:43:54.456824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:03.542783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489942+0000","last_change":"2024-09-21T03:43:08.017785+0000","last_active":"2024-09-21T03:43:54.489942+0000","last_peered":"2024-09-21T03:43:54.489942+0000","last_clean":"2024-09-21T03:43:54.489942+0000","last_became_active":"2024-09-21T03:43:08.017389+0000","last_became_peered":"2024-09-21T03:43:08.017389+0000","last_unstale":"2024-09-21T03:43:54.489942+0000","last_undegraded":"2024-09-21T03:43:54.489942+0000","last_fullsized":"2024-09-21T03:43:54.489942+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:37.557061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.545136+0000","last_change":"2024-09-21T03:43:05.567829+0000","last_active":"2024-09-21T03:44:05.545136+0000","last_peered":"2024-09-21T03:44:05.545136+0000","last_clean":"2024-09-21T03:44:05.545136+0000","last_became_active":"2024-09-21T03:43:05.567401+0000","last_became_peered":"2024-09-21T03:43:05.567401+0000","last_unstale":"2024-09-21T03:44:05.545136+0000","last_undegraded":"2024-09-21T03:44:05.545136+0000","last_fullsized":"2024-09-21T03:44:05.545136+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:03.954280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.457270+0000","last_change":"2024-09-21T03:43:18.080064+0000","last_active":"2024-09-21T03:43:54.457270+0000","last_peered":"2024-09-21T03:43:54.457270+0000","last_clean":"2024-09-21T03:43:54.457270+0000","last_became_active":"2024-09-21T03:43:18.079569+0000","last_became_peered":"2024-09-21T03:43:18.079569+0000","last_unstale":"2024-09-21T03:43:54.457270+0000","last_undegraded":"2024-09-21T03:43:54.457270+0000","last_fullsized":"2024-09-21T03:43:54.457270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:29.568476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462852+0000","last_change":"2024-09-21T03:43:08.011603+0000","last_active":"2024-09-21T03:43:54.462852+0000","last_peered":"2024-09-21T03:43:54.462852+0000","last_clean":"2024-09-21T03:43:54.462852+0000","last_became_active":"2024-09-21T03:43:08.011078+0000","last_became_peered":"2024-09-21T03:43:08.011078+0000","last_unstale":"2024-09-21T03:43:54.462852+0000","last_undegraded":"2024-09-21T03:43:54.462852+0000","last_fullsized":"2024-09-21T03:43:54.462852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:44.764713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'15","reported_seq":56,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:23.022789+0000","last_change":"2024-09-21T03:43:04.979223+0000","last_active":"2024-09-21T03:44:23.022789+0000","last_peered":"2024-09-21T03:44:23.022789+0000","last_clean":"2024-09-21T03:44:23.022789+0000","last_became_active":"2024-09-21T03:43:04.978675+0000","last_became_peered":"2024-09-21T03:43:04.978675+0000","last_unstale":"2024-09-21T03:44:23.022789+0000","last_undegraded":"2024-09-21T03:44:23.022789+0000","last_fullsized":"2024-09-21T03:44:23.022789+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:09.717921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.460027+0000","last_change":"2024-09-21T03:43:18.482334+0000","last_active":"2024-09-21T03:43:54.460027+0000","last_peered":"2024-09-21T03:43:54.460027+0000","last_clean":"2024-09-21T03:43:54.460027+0000","last_became_active":"2024-09-21T03:43:18.481881+0000","last_became_peered":"2024-09-21T03:43:18.481881+0000","last_unstale":"2024-09-21T03:43:54.460027+0000","last_undegraded":"2024-09-21T03:43:54.460027+0000","last_fullsized":"2024-09-21T03:43:54.460027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:02:32.497348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.458435+0000","last_change":"2024-09-21T03:43:07.993771+0000","last_active":"2024-09-21T03:43:54.458435+0000","last_peered":"2024-09-21T03:43:54.458435+0000","last_clean":"2024-09-21T03:43:54.458435+0000","last_became_active":"2024-09-21T03:43:07.993268+0000","last_became_peered":"2024-09-21T03:43:07.993268+0000","last_unstale":"2024-09-21T03:43:54.458435+0000","last_undegraded":"2024-09-21T03:43:54.458435+0000","last_fullsized":"2024-09-21T03:43:54.458435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:42.348561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":46,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.544796+0000","last_change":"2024-09-21T03:43:05.141786+0000","last_active":"2024-09-21T03:44:05.544796+0000","last_peered":"2024-09-21T03:44:05.544796+0000","last_clean":"2024-09-21T03:44:05.544796+0000","last_became_active":"2024-09-21T03:43:05.141307+0000","last_became_peered":"2024-09-21T03:43:05.141307+0000","last_unstale":"2024-09-21T03:44:05.544796+0000","last_undegraded":"2024-09-21T03:44:05.544796+0000","last_fullsized":"2024-09-21T03:44:05.544796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:12.139398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489872+0000","last_change":"2024-09-21T03:43:18.087848+0000","last_active":"2024-09-21T03:43:54.489872+0000","last_peered":"2024-09-21T03:43:54.489872+0000","last_clean":"2024-09-21T03:43:54.489872+0000","last_became_active":"2024-09-21T03:43:18.087400+0000","last_became_peered":"2024-09-21T03:43:18.087400+0000","last_unstale":"2024-09-21T03:43:54.489872+0000","last_undegraded":"2024-09-21T03:43:54.489872+0000","last_fullsized":"2024-09-21T03:43:54.489872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:46.902979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.454367+0000","last_change":"2024-09-21T03:43:08.010051+0000","last_active":"2024-09-21T03:43:54.454367+0000","last_peered":"2024-09-21T03:43:54.454367+0000","last_clean":"2024-09-21T03:43:54.454367+0000","last_became_active":"2024-09-21T03:43:08.007700+0000","last_became_peered":"2024-09-21T03:43:08.007700+0000","last_unstale":"2024-09-21T03:43:54.454367+0000","last_undegraded":"2024-09-21T03:43:54.454367+0000","last_fullsized":"2024-09-21T03:43:54.454367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:31.828193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.454472+0000","last_change":"2024-09-21T03:43:05.140490+0000","last_active":"2024-09-21T03:43:54.454472+0000","last_peered":"2024-09-21T03:43:54.454472+0000","last_clean":"2024-09-21T03:43:54.454472+0000","last_became_active":"2024-09-21T03:43:05.139979+0000","last_became_peered":"2024-09-21T03:43:05.139979+0000","last_unstale":"2024-09-21T03:43:54.454472+0000","last_undegraded":"2024-09-21T03:43:54.454472+0000","last_fullsized":"2024-09-21T03:43:54.454472+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:20.957596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.491072+0000","last_change":"2024-09-21T03:43:18.482777+0000","last_active":"2024-09-21T03:43:54.491072+0000","last_peered":"2024-09-21T03:43:54.491072+0000","last_clean":"2024-09-21T03:43:54.491072+0000","last_became_active":"2024-09-21T03:43:18.482244+0000","last_became_peered":"2024-09-21T03:43:18.482244+0000","last_unstale":"2024-09-21T03:43:54.491072+0000","last_undegraded":"2024-09-21T03:43:54.491072+0000","last_fullsized":"2024-09-21T03:43:54.491072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:41.997309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.454399+0000","last_change":"2024-09-21T03:43:08.019087+0000","last_active":"2024-09-21T03:43:54.454399+0000","last_peered":"2024-09-21T03:43:54.454399+0000","last_clean":"2024-09-21T03:43:54.454399+0000","last_became_active":"2024-09-21T03:43:08.018467+0000","last_became_peered":"2024-09-21T03:43:08.018467+0000","last_unstale":"2024-09-21T03:43:54.454399+0000","last_undegraded":"2024-09-21T03:43:54.454399+0000","last_fullsized":"2024-09-21T03:43:54.454399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:25.824955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"84'5","reported_seq":66,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:23.575159+0000","last_change":"2024-09-21T03:43:04.973433+0000","last_active":"2024-09-21T03:44:23.575159+0000","last_peered":"2024-09-21T03:44:23.575159+0000","last_clean":"2024-09-21T03:44:23.575159+0000","last_became_active":"2024-09-21T03:43:04.973033+0000","last_became_peered":"2024-09-21T03:43:04.973033+0000","last_unstale":"2024-09-21T03:44:23.575159+0000","last_undegraded":"2024-09-21T03:44:23.575159+0000","last_fullsized":"2024-09-21T03:44:23.575159+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:25.462635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.458312+0000","last_change":"2024-09-21T03:43:18.082385+0000","last_active":"2024-09-21T03:43:54.458312+0000","last_peered":"2024-09-21T03:43:54.458312+0000","last_clean":"2024-09-21T03:43:54.458312+0000","last_became_active":"2024-09-21T03:43:18.081958+0000","last_became_peered":"2024-09-21T03:43:18.081958+0000","last_unstale":"2024-09-21T03:43:54.458312+0000","last_undegraded":"2024-09-21T03:43:54.458312+0000","last_fullsized":"2024-09-21T03:43:54.458312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:36.554838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.459553+0000","last_change":"2024-09-21T03:43:08.008307+0000","last_active":"2024-09-21T03:43:54.459553+0000","last_peered":"2024-09-21T03:43:54.459553+0000","last_clean":"2024-09-21T03:43:54.459553+0000","last_became_active":"2024-09-21T03:43:08.006736+0000","last_became_peered":"2024-09-21T03:43:08.006736+0000","last_unstale":"2024-09-21T03:43:54.459553+0000","last_undegraded":"2024-09-21T03:43:54.459553+0000","last_fullsized":"2024-09-21T03:43:54.459553+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:31.694655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:21.571759+0000","last_change":"2024-09-21T03:43:05.140263+0000","last_active":"2024-09-21T03:44:21.571759+0000","last_peered":"2024-09-21T03:44:21.571759+0000","last_clean":"2024-09-21T03:44:21.571759+0000","last_became_active":"2024-09-21T03:43:05.139869+0000","last_became_peered":"2024-09-21T03:43:05.139869+0000","last_unstale":"2024-09-21T03:44:21.571759+0000","last_undegraded":"2024-09-21T03:44:21.571759+0000","last_fullsized":"2024-09-21T03:44:21.571759+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:19.313677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.458932+0000","last_change":"2024-09-21T03:43:18.483795+0000","last_active":"2024-09-21T03:43:54.458932+0000","last_peered":"2024-09-21T03:43:54.458932+0000","last_clean":"2024-09-21T03:43:54.458932+0000","last_became_active":"2024-09-21T03:43:18.483159+0000","last_became_peered":"2024-09-21T03:43:18.483159+0000","last_unstale":"2024-09-21T03:43:54.458932+0000","last_undegraded":"2024-09-21T03:43:54.458932+0000","last_fullsized":"2024-09-21T03:43:54.458932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:56.277956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456884+0000","last_change":"2024-09-21T03:43:08.001661+0000","last_active":"2024-09-21T03:43:54.456884+0000","last_peered":"2024-09-21T03:43:54.456884+0000","last_clean":"2024-09-21T03:43:54.456884+0000","last_became_active":"2024-09-21T03:43:08.001254+0000","last_became_peered":"2024-09-21T03:43:08.001254+0000","last_unstale":"2024-09-21T03:43:54.456884+0000","last_undegraded":"2024-09-21T03:43:54.456884+0000","last_fullsized":"2024-09-21T03:43:54.456884+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:19.020542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:03.526981+0000","last_change":"2024-09-21T03:43:04.979127+0000","last_active":"2024-09-21T03:44:03.526981+0000","last_peered":"2024-09-21T03:44:03.526981+0000","last_clean":"2024-09-21T03:44:03.526981+0000","last_became_active":"2024-09-21T03:43:04.978705+0000","last_became_peered":"2024-09-21T03:43:04.978705+0000","last_unstale":"2024-09-21T03:44:03.526981+0000","last_undegraded":"2024-09-21T03:44:03.526981+0000","last_fullsized":"2024-09-21T03:44:03.526981+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:48.548002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.459944+0000","last_change":"2024-09-21T03:43:18.483682+0000","last_active":"2024-09-21T03:43:54.459944+0000","last_peered":"2024-09-21T03:43:54.459944+0000","last_clean":"2024-09-21T03:43:54.459944+0000","last_became_active":"2024-09-21T03:43:18.483124+0000","last_became_peered":"2024-09-21T03:43:18.483124+0000","last_unstale":"2024-09-21T03:43:54.459944+0000","last_undegraded":"2024-09-21T03:43:54.459944+0000","last_fullsized":"2024-09-21T03:43:54.459944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:12.513795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.461321+0000","last_change":"2024-09-21T03:43:08.013193+0000","last_active":"2024-09-21T03:43:54.461321+0000","last_peered":"2024-09-21T03:43:54.461321+0000","last_clean":"2024-09-21T03:43:54.461321+0000","last_became_active":"2024-09-21T03:43:08.010360+0000","last_became_peered":"2024-09-21T03:43:08.010360+0000","last_unstale":"2024-09-21T03:43:54.461321+0000","last_undegraded":"2024-09-21T03:43:54.461321+0000","last_fullsized":"2024-09-21T03:43:54.461321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:28.101646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:06.544985+0000","last_change":"2024-09-21T03:43:04.971782+0000","last_active":"2024-09-21T03:44:06.544985+0000","last_peered":"2024-09-21T03:44:06.544985+0000","last_clean":"2024-09-21T03:44:06.544985+0000","last_became_active":"2024-09-21T03:43:04.971375+0000","last_became_peered":"2024-09-21T03:43:04.971375+0000","last_unstale":"2024-09-21T03:44:06.544985+0000","last_undegraded":"2024-09-21T03:44:06.544985+0000","last_fullsized":"2024-09-21T03:44:06.544985+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:49.573490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'92","reported_seq":111,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462612+0000","last_change":"2024-09-21T03:41:42.914435+0000","last_active":"2024-09-21T03:43:54.462612+0000","last_peered":"2024-09-21T03:43:54.462612+0000","last_clean":"2024-09-21T03:43:54.462612+0000","last_became_active":"2024-09-21T03:41:42.909490+0000","last_became_peered":"2024-09-21T03:41:42.909490+0000","last_unstale":"2024-09-21T03:43:54.462612+0000","last_undegraded":"2024-09-21T03:43:54.462612+0000","last_fullsized":"2024-09-21T03:43:54.462612+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:56.507027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:56.507027+0000","last_clean_scrub_stamp":"2024-09-21T03:37:56.507027+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:34.564062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.461799+0000","last_change":"2024-09-21T03:43:18.084701+0000","last_active":"2024-09-21T03:43:54.461799+0000","last_peered":"2024-09-21T03:43:54.461799+0000","last_clean":"2024-09-21T03:43:54.461799+0000","last_became_active":"2024-09-21T03:43:18.084251+0000","last_became_peered":"2024-09-21T03:43:18.084251+0000","last_unstale":"2024-09-21T03:43:54.461799+0000","last_undegraded":"2024-09-21T03:43:54.461799+0000","last_fullsized":"2024-09-21T03:43:54.461799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:15.660276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462254+0000","last_change":"2024-09-21T03:43:08.012195+0000","last_active":"2024-09-21T03:43:54.462254+0000","last_peered":"2024-09-21T03:43:54.462254+0000","last_clean":"2024-09-21T03:43:54.462254+0000","last_became_active":"2024-09-21T03:43:08.011670+0000","last_became_peered":"2024-09-21T03:43:08.011670+0000","last_unstale":"2024-09-21T03:43:54.462254+0000","last_undegraded":"2024-09-21T03:43:54.462254+0000","last_fullsized":"2024-09-21T03:43:54.462254+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:14.290074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":70,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:24.576206+0000","last_change":"2024-09-21T03:43:05.567898+0000","last_active":"2024-09-21T03:44:24.576206+0000","last_peered":"2024-09-21T03:44:24.576206+0000","last_clean":"2024-09-21T03:44:24.576206+0000","last_became_active":"2024-09-21T03:43:05.567367+0000","last_became_peered":"2024-09-21T03:43:05.567367+0000","last_unstale":"2024-09-21T03:44:24.576206+0000","last_undegraded":"2024-09-21T03:44:24.576206+0000","last_fullsized":"2024-09-21T03:44:24.576206+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:55.120838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489820+0000","last_change":"2024-09-21T03:43:18.073942+0000","last_active":"2024-09-21T03:43:54.489820+0000","last_peered":"2024-09-21T03:43:54.489820+0000","last_clean":"2024-09-21T03:43:54.489820+0000","last_became_active":"2024-09-21T03:43:18.073323+0000","last_became_peered":"2024-09-21T03:43:18.073323+0000","last_unstale":"2024-09-21T03:43:54.489820+0000","last_undegraded":"2024-09-21T03:43:54.489820+0000","last_fullsized":"2024-09-21T03:43:54.489820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:41.677143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.461712+0000","last_change":"2024-09-21T03:43:08.016636+0000","last_active":"2024-09-21T03:43:54.461712+0000","last_peered":"2024-09-21T03:43:54.461712+0000","last_clean":"2024-09-21T03:43:54.461712+0000","last_became_active":"2024-09-21T03:43:08.016235+0000","last_became_peered":"2024-09-21T03:43:08.016235+0000","last_unstale":"2024-09-21T03:43:54.461712+0000","last_undegraded":"2024-09-21T03:43:54.461712+0000","last_fullsized":"2024-09-21T03:43:54.461712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:35.365830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.543336+0000","last_change":"2024-09-21T03:43:04.980911+0000","last_active":"2024-09-21T03:44:05.543336+0000","last_peered":"2024-09-21T03:44:05.543336+0000","last_clean":"2024-09-21T03:44:05.543336+0000","last_became_active":"2024-09-21T03:43:04.980512+0000","last_became_peered":"2024-09-21T03:43:04.980512+0000","last_unstale":"2024-09-21T03:44:05.543336+0000","last_undegraded":"2024-09-21T03:44:05.543336+0000","last_fullsized":"2024-09-21T03:44:05.543336+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:04.136345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489524+0000","last_change":"2024-09-21T03:43:18.085874+0000","last_active":"2024-09-21T03:43:54.489524+0000","last_peered":"2024-09-21T03:43:54.489524+0000","last_clean":"2024-09-21T03:43:54.489524+0000","last_became_active":"2024-09-21T03:43:18.085381+0000","last_became_peered":"2024-09-21T03:43:18.085381+0000","last_unstale":"2024-09-21T03:43:54.489524+0000","last_undegraded":"2024-09-21T03:43:54.489524+0000","last_fullsized":"2024-09-21T03:43:54.489524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:02.829202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462064+0000","last_change":"2024-09-21T03:43:08.014776+0000","last_active":"2024-09-21T03:43:54.462064+0000","last_peered":"2024-09-21T03:43:54.462064+0000","last_clean":"2024-09-21T03:43:54.462064+0000","last_became_active":"2024-09-21T03:43:08.014191+0000","last_became_peered":"2024-09-21T03:43:08.014191+0000","last_unstale":"2024-09-21T03:43:54.462064+0000","last_undegraded":"2024-09-21T03:43:54.462064+0000","last_fullsized":"2024-09-21T03:43:54.462064+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:19.114838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":46,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.544909+0000","last_change":"2024-09-21T03:43:04.973714+0000","last_active":"2024-09-21T03:44:05.544909+0000","last_peered":"2024-09-21T03:44:05.544909+0000","last_clean":"2024-09-21T03:44:05.544909+0000","last_became_active":"2024-09-21T03:43:04.973348+0000","last_became_peered":"2024-09-21T03:43:04.973348+0000","last_unstale":"2024-09-21T03:44:05.544909+0000","last_undegraded":"2024-09-21T03:44:05.544909+0000","last_fullsized":"2024-09-21T03:44:05.544909+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:00.872589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.458790+0000","last_change":"2024-09-21T03:43:18.082643+0000","last_active":"2024-09-21T03:43:54.458790+0000","last_peered":"2024-09-21T03:43:54.458790+0000","last_clean":"2024-09-21T03:43:54.458790+0000","last_became_active":"2024-09-21T03:43:18.082157+0000","last_became_peered":"2024-09-21T03:43:18.082157+0000","last_unstale":"2024-09-21T03:43:54.458790+0000","last_undegraded":"2024-09-21T03:43:54.458790+0000","last_fullsized":"2024-09-21T03:43:54.458790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:52.652014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490309+0000","last_change":"2024-09-21T03:43:08.010913+0000","last_active":"2024-09-21T03:43:54.490309+0000","last_peered":"2024-09-21T03:43:54.490309+0000","last_clean":"2024-09-21T03:43:54.490309+0000","last_became_active":"2024-09-21T03:43:08.010483+0000","last_became_peered":"2024-09-21T03:43:08.010483+0000","last_unstale":"2024-09-21T03:43:54.490309+0000","last_undegraded":"2024-09-21T03:43:54.490309+0000","last_fullsized":"2024-09-21T03:43:54.490309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:52.158346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:04.522790+0000","last_change":"2024-09-21T03:43:04.980260+0000","last_active":"2024-09-21T03:44:04.522790+0000","last_peered":"2024-09-21T03:44:04.522790+0000","last_clean":"2024-09-21T03:44:04.522790+0000","last_became_active":"2024-09-21T03:43:04.978999+0000","last_became_peered":"2024-09-21T03:43:04.978999+0000","last_unstale":"2024-09-21T03:44:04.522790+0000","last_undegraded":"2024-09-21T03:44:04.522790+0000","last_fullsized":"2024-09-21T03:44:04.522790+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:33.682591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.458981+0000","last_change":"2024-09-21T03:43:18.083665+0000","last_active":"2024-09-21T03:43:54.458981+0000","last_peered":"2024-09-21T03:43:54.458981+0000","last_clean":"2024-09-21T03:43:54.458981+0000","last_became_active":"2024-09-21T03:43:18.083215+0000","last_became_peered":"2024-09-21T03:43:18.083215+0000","last_unstale":"2024-09-21T03:43:54.458981+0000","last_undegraded":"2024-09-21T03:43:54.458981+0000","last_fullsized":"2024-09-21T03:43:54.458981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:55.968968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.459158+0000","last_change":"2024-09-21T03:43:08.010570+0000","last_active":"2024-09-21T03:43:54.459158+0000","last_peered":"2024-09-21T03:43:54.459158+0000","last_clean":"2024-09-21T03:43:54.459158+0000","last_became_active":"2024-09-21T03:43:08.010163+0000","last_became_peered":"2024-09-21T03:43:08.010163+0000","last_unstale":"2024-09-21T03:43:54.459158+0000","last_undegraded":"2024-09-21T03:43:54.459158+0000","last_fullsized":"2024-09-21T03:43:54.459158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:17.368534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":59,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.543461+0000","last_change":"2024-09-21T03:43:05.140650+0000","last_active":"2024-09-21T03:44:05.543461+0000","last_peered":"2024-09-21T03:44:05.543461+0000","last_clean":"2024-09-21T03:44:05.543461+0000","last_became_active":"2024-09-21T03:43:05.140252+0000","last_became_peered":"2024-09-21T03:43:05.140252+0000","last_unstale":"2024-09-21T03:44:05.543461+0000","last_undegraded":"2024-09-21T03:44:05.543461+0000","last_fullsized":"2024-09-21T03:44:05.543461+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:23.693838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.459001+0000","last_change":"2024-09-21T03:43:18.089516+0000","last_active":"2024-09-21T03:43:54.459001+0000","last_peered":"2024-09-21T03:43:54.459001+0000","last_clean":"2024-09-21T03:43:54.459001+0000","last_became_active":"2024-09-21T03:43:18.089097+0000","last_became_peered":"2024-09-21T03:43:18.089097+0000","last_unstale":"2024-09-21T03:43:54.459001+0000","last_undegraded":"2024-09-21T03:43:54.459001+0000","last_fullsized":"2024-09-21T03:43:54.459001+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:50.428401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.461348+0000","last_change":"2024-09-21T03:43:08.015762+0000","last_active":"2024-09-21T03:43:54.461348+0000","last_peered":"2024-09-21T03:43:54.461348+0000","last_clean":"2024-09-21T03:43:54.461348+0000","last_became_active":"2024-09-21T03:43:08.015389+0000","last_became_peered":"2024-09-21T03:43:08.015389+0000","last_unstale":"2024-09-21T03:43:54.461348+0000","last_undegraded":"2024-09-21T03:43:54.461348+0000","last_fullsized":"2024-09-21T03:43:54.461348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:00.341479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'5","reported_seq":67,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:23.575428+0000","last_change":"2024-09-21T03:43:04.973173+0000","last_active":"2024-09-21T03:44:23.575428+0000","last_peered":"2024-09-21T03:44:23.575428+0000","last_clean":"2024-09-21T03:44:23.575428+0000","last_became_active":"2024-09-21T03:43:04.972740+0000","last_became_peered":"2024-09-21T03:43:04.972740+0000","last_unstale":"2024-09-21T03:44:23.575428+0000","last_undegraded":"2024-09-21T03:44:23.575428+0000","last_fullsized":"2024-09-21T03:44:23.575428+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:26.329220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.454727+0000","last_change":"2024-09-21T03:43:18.093730+0000","last_active":"2024-09-21T03:43:54.454727+0000","last_peered":"2024-09-21T03:43:54.454727+0000","last_clean":"2024-09-21T03:43:54.454727+0000","last_became_active":"2024-09-21T03:43:18.089621+0000","last_became_peered":"2024-09-21T03:43:18.089621+0000","last_unstale":"2024-09-21T03:43:54.454727+0000","last_undegraded":"2024-09-21T03:43:54.454727+0000","last_fullsized":"2024-09-21T03:43:54.454727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:05.227038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.465867+0000","last_change":"2024-09-21T03:43:54.465751+0000","last_active":"2024-09-21T03:43:54.465867+0000","last_peered":"2024-09-21T03:43:54.465867+0000","last_clean":"2024-09-21T03:43:54.465867+0000","last_became_active":"2024-09-21T03:43:54.465333+0000","last_became_peered":"2024-09-21T03:43:54.465333+0000","last_unstale":"2024-09-21T03:43:54.465867+0000","last_undegraded":"2024-09-21T03:43:54.465867+0000","last_fullsized":"2024-09-21T03:43:54.465867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:34.307699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.543641+0000","last_change":"2024-09-21T03:43:04.979385+0000","last_active":"2024-09-21T03:44:05.543641+0000","last_peered":"2024-09-21T03:44:05.543641+0000","last_clean":"2024-09-21T03:44:05.543641+0000","last_became_active":"2024-09-21T03:43:04.978764+0000","last_became_peered":"2024-09-21T03:43:04.978764+0000","last_unstale":"2024-09-21T03:44:05.543641+0000","last_undegraded":"2024-09-21T03:44:05.543641+0000","last_fullsized":"2024-09-21T03:44:05.543641+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:34.256966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490077+0000","last_change":"2024-09-21T03:43:18.095937+0000","last_active":"2024-09-21T03:43:54.490077+0000","last_peered":"2024-09-21T03:43:54.490077+0000","last_clean":"2024-09-21T03:43:54.490077+0000","last_became_active":"2024-09-21T03:43:18.095531+0000","last_became_peered":"2024-09-21T03:43:18.095531+0000","last_unstale":"2024-09-21T03:43:54.490077+0000","last_undegraded":"2024-09-21T03:43:54.490077+0000","last_fullsized":"2024-09-21T03:43:54.490077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:18.089661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.457193+0000","last_change":"2024-09-21T03:43:08.011342+0000","last_active":"2024-09-21T03:43:54.457193+0000","last_peered":"2024-09-21T03:43:54.457193+0000","last_clean":"2024-09-21T03:43:54.457193+0000","last_became_active":"2024-09-21T03:43:08.010963+0000","last_became_peered":"2024-09-21T03:43:08.010963+0000","last_unstale":"2024-09-21T03:43:54.457193+0000","last_undegraded":"2024-09-21T03:43:54.457193+0000","last_fullsized":"2024-09-21T03:43:54.457193+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:05.970863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":52,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:06.545562+0000","last_change":"2024-09-21T03:43:04.972896+0000","last_active":"2024-09-21T03:44:06.545562+0000","last_peered":"2024-09-21T03:44:06.545562+0000","last_clean":"2024-09-21T03:44:06.545562+0000","last_became_active":"2024-09-21T03:43:04.972535+0000","last_became_peered":"2024-09-21T03:43:04.972535+0000","last_unstale":"2024-09-21T03:44:06.545562+0000","last_undegraded":"2024-09-21T03:44:06.545562+0000","last_fullsized":"2024-09-21T03:44:06.545562+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:36.362019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.459478+0000","last_change":"2024-09-21T03:43:18.081300+0000","last_active":"2024-09-21T03:43:54.459478+0000","last_peered":"2024-09-21T03:43:54.459478+0000","last_clean":"2024-09-21T03:43:54.459478+0000","last_became_active":"2024-09-21T03:43:18.080874+0000","last_became_peered":"2024-09-21T03:43:18.080874+0000","last_unstale":"2024-09-21T03:43:54.459478+0000","last_undegraded":"2024-09-21T03:43:54.459478+0000","last_fullsized":"2024-09-21T03:43:54.459478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:13.246348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.491419+0000","last_change":"2024-09-21T03:43:08.014954+0000","last_active":"2024-09-21T03:43:54.491419+0000","last_peered":"2024-09-21T03:43:54.491419+0000","last_clean":"2024-09-21T03:43:54.491419+0000","last_became_active":"2024-09-21T03:43:08.014576+0000","last_became_peered":"2024-09-21T03:43:08.014576+0000","last_unstale":"2024-09-21T03:43:54.491419+0000","last_undegraded":"2024-09-21T03:43:54.491419+0000","last_fullsized":"2024-09-21T03:43:54.491419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:13.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"84'31","reported_seq":113,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:22.576813+0000","last_change":"2024-09-21T03:43:04.973305+0000","last_active":"2024-09-21T03:44:22.576813+0000","last_peered":"2024-09-21T03:44:22.576813+0000","last_clean":"2024-09-21T03:44:22.576813+0000","last_became_active":"2024-09-21T03:43:04.972891+0000","last_became_peered":"2024-09-21T03:43:04.972891+0000","last_unstale":"2024-09-21T03:44:22.576813+0000","last_undegraded":"2024-09-21T03:44:22.576813+0000","last_fullsized":"2024-09-21T03:44:22.576813+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:15.653769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56549,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":75,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.453933+0000","last_change":"2024-09-21T03:43:18.093613+0000","last_active":"2024-09-21T03:43:54.453933+0000","last_peered":"2024-09-21T03:43:54.453933+0000","last_clean":"2024-09-21T03:43:54.453933+0000","last_became_active":"2024-09-21T03:43:18.093195+0000","last_became_peered":"2024-09-21T03:43:18.093195+0000","last_unstale":"2024-09-21T03:43:54.453933+0000","last_undegraded":"2024-09-21T03:43:54.453933+0000","last_fullsized":"2024-09-21T03:43:54.453933+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:17.825071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490098+0000","last_change":"2024-09-21T03:43:08.014314+0000","last_active":"2024-09-21T03:43:54.490098+0000","last_peered":"2024-09-21T03:43:54.490098+0000","last_clean":"2024-09-21T03:43:54.490098+0000","last_became_active":"2024-09-21T03:43:08.012925+0000","last_became_peered":"2024-09-21T03:43:08.012925+0000","last_unstale":"2024-09-21T03:43:54.490098+0000","last_undegraded":"2024-09-21T03:43:54.490098+0000","last_fullsized":"2024-09-21T03:43:54.490098+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:47:28.599444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:04.536006+0000","last_change":"2024-09-21T03:43:05.141584+0000","last_active":"2024-09-21T03:44:04.536006+0000","last_peered":"2024-09-21T03:44:04.536006+0000","last_clean":"2024-09-21T03:44:04.536006+0000","last_became_active":"2024-09-21T03:43:05.141200+0000","last_became_peered":"2024-09-21T03:43:05.141200+0000","last_unstale":"2024-09-21T03:44:04.536006+0000","last_undegraded":"2024-09-21T03:44:04.536006+0000","last_fullsized":"2024-09-21T03:44:04.536006+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:04.357970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.457242+0000","last_change":"2024-09-21T03:43:18.092184+0000","last_active":"2024-09-21T03:43:54.457242+0000","last_peered":"2024-09-21T03:43:54.457242+0000","last_clean":"2024-09-21T03:43:54.457242+0000","last_became_active":"2024-09-21T03:43:18.091629+0000","last_became_peered":"2024-09-21T03:43:18.091629+0000","last_unstale":"2024-09-21T03:43:54.457242+0000","last_undegraded":"2024-09-21T03:43:54.457242+0000","last_fullsized":"2024-09-21T03:43:54.457242+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:39.448009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489762+0000","last_change":"2024-09-21T03:43:08.018335+0000","last_active":"2024-09-21T03:43:54.489762+0000","last_peered":"2024-09-21T03:43:54.489762+0000","last_clean":"2024-09-21T03:43:54.489762+0000","last_became_active":"2024-09-21T03:43:08.014005+0000","last_became_peered":"2024-09-21T03:43:08.014005+0000","last_unstale":"2024-09-21T03:43:54.489762+0000","last_undegraded":"2024-09-21T03:43:54.489762+0000","last_fullsized":"2024-09-21T03:43:54.489762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:35.749381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.543134+0000","last_change":"2024-09-21T03:43:04.978609+0000","last_active":"2024-09-21T03:44:05.543134+0000","last_peered":"2024-09-21T03:44:05.543134+0000","last_clean":"2024-09-21T03:44:05.543134+0000","last_became_active":"2024-09-21T03:43:04.978146+0000","last_became_peered":"2024-09-21T03:43:04.978146+0000","last_unstale":"2024-09-21T03:44:05.543134+0000","last_undegraded":"2024-09-21T03:44:05.543134+0000","last_fullsized":"2024-09-21T03:44:05.543134+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:41.651745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.453520+0000","last_change":"2024-09-21T03:43:18.089399+0000","last_active":"2024-09-21T03:43:54.453520+0000","last_peered":"2024-09-21T03:43:54.453520+0000","last_clean":"2024-09-21T03:43:54.453520+0000","last_became_active":"2024-09-21T03:43:18.086156+0000","last_became_peered":"2024-09-21T03:43:18.086156+0000","last_unstale":"2024-09-21T03:43:54.453520+0000","last_undegraded":"2024-09-21T03:43:54.453520+0000","last_fullsized":"2024-09-21T03:43:54.453520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:41.955341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490427+0000","last_change":"2024-09-21T03:43:08.008773+0000","last_active":"2024-09-21T03:43:54.490427+0000","last_peered":"2024-09-21T03:43:54.490427+0000","last_clean":"2024-09-21T03:43:54.490427+0000","last_became_active":"2024-09-21T03:43:08.008270+0000","last_became_peered":"2024-09-21T03:43:08.008270+0000","last_unstale":"2024-09-21T03:43:54.490427+0000","last_undegraded":"2024-09-21T03:43:54.490427+0000","last_fullsized":"2024-09-21T03:43:54.490427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:57.901701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":79,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:22.572573+0000","last_change":"2024-09-21T03:43:04.978296+0000","last_active":"2024-09-21T03:44:22.572573+0000","last_peered":"2024-09-21T03:44:22.572573+0000","last_clean":"2024-09-21T03:44:22.572573+0000","last_became_active":"2024-09-21T03:43:04.977782+0000","last_became_peered":"2024-09-21T03:43:04.977782+0000","last_unstale":"2024-09-21T03:44:22.572573+0000","last_undegraded":"2024-09-21T03:44:22.572573+0000","last_fullsized":"2024-09-21T03:44:22.572573+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:57.748737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.495673+0000","last_change":"2024-09-21T03:43:54.495536+0000","last_active":"2024-09-21T03:43:54.495673+0000","last_peered":"2024-09-21T03:43:54.495673+0000","last_clean":"2024-09-21T03:43:54.495673+0000","last_became_active":"2024-09-21T03:43:54.493492+0000","last_became_peered":"2024-09-21T03:43:54.493492+0000","last_unstale":"2024-09-21T03:43:54.495673+0000","last_undegraded":"2024-09-21T03:43:54.495673+0000","last_fullsized":"2024-09-21T03:43:54.495673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:11.291911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489721+0000","last_change":"2024-09-21T03:43:18.077741+0000","last_active":"2024-09-21T03:43:54.489721+0000","last_peered":"2024-09-21T03:43:54.489721+0000","last_clean":"2024-09-21T03:43:54.489721+0000","last_became_active":"2024-09-21T03:43:18.076619+0000","last_became_peered":"2024-09-21T03:43:18.076619+0000","last_unstale":"2024-09-21T03:43:54.489721+0000","last_undegraded":"2024-09-21T03:43:54.489721+0000","last_fullsized":"2024-09-21T03:43:54.489721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:33.297020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.453694+0000","last_change":"2024-09-21T03:43:08.008838+0000","last_active":"2024-09-21T03:43:54.453694+0000","last_peered":"2024-09-21T03:43:54.453694+0000","last_clean":"2024-09-21T03:43:54.453694+0000","last_became_active":"2024-09-21T03:43:08.008422+0000","last_became_peered":"2024-09-21T03:43:08.008422+0000","last_unstale":"2024-09-21T03:43:54.453694+0000","last_undegraded":"2024-09-21T03:43:54.453694+0000","last_fullsized":"2024-09-21T03:43:54.453694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:00:17.881443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:05.543919+0000","last_change":"2024-09-21T03:43:04.979289+0000","last_active":"2024-09-21T03:44:05.543919+0000","last_peered":"2024-09-21T03:44:05.543919+0000","last_clean":"2024-09-21T03:44:05.543919+0000","last_became_active":"2024-09-21T03:43:04.978843+0000","last_became_peered":"2024-09-21T03:43:04.978843+0000","last_unstale":"2024-09-21T03:44:05.543919+0000","last_undegraded":"2024-09-21T03:44:05.543919+0000","last_fullsized":"2024-09-21T03:44:05.543919+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:50.560607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490335+0000","last_change":"2024-09-21T03:43:18.096870+0000","last_active":"2024-09-21T03:43:54.490335+0000","last_peered":"2024-09-21T03:43:54.490335+0000","last_clean":"2024-09-21T03:43:54.490335+0000","last_became_active":"2024-09-21T03:43:18.096424+0000","last_became_peered":"2024-09-21T03:43:18.096424+0000","last_unstale":"2024-09-21T03:43:54.490335+0000","last_undegraded":"2024-09-21T03:43:54.490335+0000","last_fullsized":"2024-09-21T03:43:54.490335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:39.532368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.494294+0000","last_change":"2024-09-21T03:43:54.492499+0000","last_active":"2024-09-21T03:43:54.494294+0000","last_peered":"2024-09-21T03:43:54.494294+0000","last_clean":"2024-09-21T03:43:54.494294+0000","last_became_active":"2024-09-21T03:43:54.492043+0000","last_became_peered":"2024-09-21T03:43:54.492043+0000","last_unstale":"2024-09-21T03:43:54.494294+0000","last_undegraded":"2024-09-21T03:43:54.494294+0000","last_fullsized":"2024-09-21T03:43:54.494294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:25.687090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:03.530472+0000","last_change":"2024-09-21T03:43:04.974265+0000","last_active":"2024-09-21T03:44:03.530472+0000","last_peered":"2024-09-21T03:44:03.530472+0000","last_clean":"2024-09-21T03:44:03.530472+0000","last_became_active":"2024-09-21T03:43:04.973383+0000","last_became_peered":"2024-09-21T03:43:04.973383+0000","last_unstale":"2024-09-21T03:44:03.530472+0000","last_undegraded":"2024-09-21T03:44:03.530472+0000","last_fullsized":"2024-09-21T03:44:03.530472+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:02.312378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":48,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:04.536278+0000","last_change":"2024-09-21T03:43:04.980353+0000","last_active":"2024-09-21T03:44:04.536278+0000","last_peered":"2024-09-21T03:44:04.536278+0000","last_clean":"2024-09-21T03:44:04.536278+0000","last_became_active":"2024-09-21T03:43:04.979922+0000","last_became_peered":"2024-09-21T03:43:04.979922+0000","last_unstale":"2024-09-21T03:44:04.536278+0000","last_undegraded":"2024-09-21T03:44:04.536278+0000","last_fullsized":"2024-09-21T03:44:04.536278+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:17.636536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.459397+0000","last_change":"2024-09-21T03:43:08.009225+0000","last_active":"2024-09-21T03:43:54.459397+0000","last_peered":"2024-09-21T03:43:54.459397+0000","last_clean":"2024-09-21T03:43:54.459397+0000","last_became_active":"2024-09-21T03:43:08.008830+0000","last_became_peered":"2024-09-21T03:43:08.008830+0000","last_unstale":"2024-09-21T03:43:54.459397+0000","last_undegraded":"2024-09-21T03:43:54.459397+0000","last_fullsized":"2024-09-21T03:43:54.459397+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:38.047763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456518+0000","last_change":"2024-09-21T03:43:18.089302+0000","last_active":"2024-09-21T03:43:54.456518+0000","last_peered":"2024-09-21T03:43:54.456518+0000","last_clean":"2024-09-21T03:43:54.456518+0000","last_became_active":"2024-09-21T03:43:18.088867+0000","last_became_peered":"2024-09-21T03:43:18.088867+0000","last_unstale":"2024-09-21T03:43:54.456518+0000","last_undegraded":"2024-09-21T03:43:54.456518+0000","last_fullsized":"2024-09-21T03:43:54.456518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:08.960044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":73,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:44:22.573148+0000","last_change":"2024-09-21T03:43:54.464317+0000","last_active":"2024-09-21T03:44:22.573148+0000","last_peered":"2024-09-21T03:44:22.573148+0000","last_clean":"2024-09-21T03:44:22.573148+0000","last_became_active":"2024-09-21T03:43:54.463870+0000","last_became_peered":"2024-09-21T03:43:54.463870+0000","last_unstale":"2024-09-21T03:44:22.573148+0000","last_undegraded":"2024-09-21T03:44:22.573148+0000","last_fullsized":"2024-09-21T03:44:22.573148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:14.581726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.491212+0000","last_change":"2024-09-21T03:43:08.009398+0000","last_active":"2024-09-21T03:43:54.491212+0000","last_peered":"2024-09-21T03:43:54.491212+0000","last_clean":"2024-09-21T03:43:54.491212+0000","last_became_active":"2024-09-21T03:43:08.009000+0000","last_became_peered":"2024-09-21T03:43:08.009000+0000","last_unstale":"2024-09-21T03:43:54.491212+0000","last_undegraded":"2024-09-21T03:43:54.491212+0000","last_fullsized":"2024-09-21T03:43:54.491212+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:07.278613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456459+0000","last_change":"2024-09-21T03:43:18.081834+0000","last_active":"2024-09-21T03:43:54.456459+0000","last_peered":"2024-09-21T03:43:54.456459+0000","last_clean":"2024-09-21T03:43:54.456459+0000","last_became_active":"2024-09-21T03:43:18.081328+0000","last_became_peered":"2024-09-21T03:43:18.081328+0000","last_unstale":"2024-09-21T03:43:54.456459+0000","last_undegraded":"2024-09-21T03:43:54.456459+0000","last_fullsized":"2024-09-21T03:43:54.456459+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:37.638826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.491208+0000","last_change":"2024-09-21T03:43:08.005144+0000","last_active":"2024-09-21T03:43:54.491208+0000","last_peered":"2024-09-21T03:43:54.491208+0000","last_clean":"2024-09-21T03:43:54.491208+0000","last_became_active":"2024-09-21T03:43:08.003895+0000","last_became_peered":"2024-09-21T03:43:08.003895+0000","last_unstale":"2024-09-21T03:43:54.491208+0000","last_undegraded":"2024-09-21T03:43:54.491208+0000","last_fullsized":"2024-09-21T03:43:54.491208+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:25.822315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.457502+0000","last_change":"2024-09-21T03:43:08.000218+0000","last_active":"2024-09-21T03:43:54.457502+0000","last_peered":"2024-09-21T03:43:54.457502+0000","last_clean":"2024-09-21T03:43:54.457502+0000","last_became_active":"2024-09-21T03:43:07.999840+0000","last_became_peered":"2024-09-21T03:43:07.999840+0000","last_unstale":"2024-09-21T03:43:54.457502+0000","last_undegraded":"2024-09-21T03:43:54.457502+0000","last_fullsized":"2024-09-21T03:43:54.457502+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:55.391706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456190+0000","last_change":"2024-09-21T03:43:08.012075+0000","last_active":"2024-09-21T03:43:54.456190+0000","last_peered":"2024-09-21T03:43:54.456190+0000","last_clean":"2024-09-21T03:43:54.456190+0000","last_became_active":"2024-09-21T03:43:08.011666+0000","last_became_peered":"2024-09-21T03:43:08.011666+0000","last_unstale":"2024-09-21T03:43:54.456190+0000","last_undegraded":"2024-09-21T03:43:54.456190+0000","last_fullsized":"2024-09-21T03:43:54.456190+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:42.124948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489567+0000","last_change":"2024-09-21T03:43:08.021082+0000","last_active":"2024-09-21T03:43:54.489567+0000","last_peered":"2024-09-21T03:43:54.489567+0000","last_clean":"2024-09-21T03:43:54.489567+0000","last_became_active":"2024-09-21T03:43:08.020674+0000","last_became_peered":"2024-09-21T03:43:08.020674+0000","last_unstale":"2024-09-21T03:43:54.489567+0000","last_undegraded":"2024-09-21T03:43:54.489567+0000","last_fullsized":"2024-09-21T03:43:54.489567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:13.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.458400+0000","last_change":"2024-09-21T03:43:08.004199+0000","last_active":"2024-09-21T03:43:54.458400+0000","last_peered":"2024-09-21T03:43:54.458400+0000","last_clean":"2024-09-21T03:43:54.458400+0000","last_became_active":"2024-09-21T03:43:08.003806+0000","last_became_peered":"2024-09-21T03:43:08.003806+0000","last_unstale":"2024-09-21T03:43:54.458400+0000","last_undegraded":"2024-09-21T03:43:54.458400+0000","last_fullsized":"2024-09-21T03:43:54.458400+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:13.941130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490222+0000","last_change":"2024-09-21T03:43:08.007237+0000","last_active":"2024-09-21T03:43:54.490222+0000","last_peered":"2024-09-21T03:43:54.490222+0000","last_clean":"2024-09-21T03:43:54.490222+0000","last_became_active":"2024-09-21T03:43:08.006805+0000","last_became_peered":"2024-09-21T03:43:08.006805+0000","last_unstale":"2024-09-21T03:43:54.490222+0000","last_undegraded":"2024-09-21T03:43:54.490222+0000","last_fullsized":"2024-09-21T03:43:54.490222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:26.274494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456601+0000","last_change":"2024-09-21T03:43:08.007934+0000","last_active":"2024-09-21T03:43:54.456601+0000","last_peered":"2024-09-21T03:43:54.456601+0000","last_clean":"2024-09-21T03:43:54.456601+0000","last_became_active":"2024-09-21T03:43:08.007549+0000","last_became_peered":"2024-09-21T03:43:08.007549+0000","last_unstale":"2024-09-21T03:43:54.456601+0000","last_undegraded":"2024-09-21T03:43:54.456601+0000","last_fullsized":"2024-09-21T03:43:54.456601+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:58.158255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.493820+0000","last_change":"2024-09-21T03:43:54.493665+0000","last_active":"2024-09-21T03:43:54.493820+0000","last_peered":"2024-09-21T03:43:54.493820+0000","last_clean":"2024-09-21T03:43:54.493820+0000","last_became_active":"2024-09-21T03:43:54.493245+0000","last_became_peered":"2024-09-21T03:43:54.493245+0000","last_unstale":"2024-09-21T03:43:54.493820+0000","last_undegraded":"2024-09-21T03:43:54.493820+0000","last_fullsized":"2024-09-21T03:43:54.493820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:35.449852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.464246+0000","last_change":"2024-09-21T03:43:54.463419+0000","last_active":"2024-09-21T03:43:54.464246+0000","last_peered":"2024-09-21T03:43:54.464246+0000","last_clean":"2024-09-21T03:43:54.464246+0000","last_became_active":"2024-09-21T03:43:54.462951+0000","last_became_peered":"2024-09-21T03:43:54.462951+0000","last_unstale":"2024-09-21T03:43:54.464246+0000","last_undegraded":"2024-09-21T03:43:54.464246+0000","last_fullsized":"2024-09-21T03:43:54.464246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:23.620275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.460597+0000","last_change":"2024-09-21T03:43:08.011436+0000","last_active":"2024-09-21T03:43:54.460597+0000","last_peered":"2024-09-21T03:43:54.460597+0000","last_clean":"2024-09-21T03:43:54.460597+0000","last_became_active":"2024-09-21T03:43:08.011025+0000","last_became_peered":"2024-09-21T03:43:08.011025+0000","last_unstale":"2024-09-21T03:43:54.460597+0000","last_undegraded":"2024-09-21T03:43:54.460597+0000","last_fullsized":"2024-09-21T03:43:54.460597+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:31.411858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490007+0000","last_change":"2024-09-21T03:43:08.016378+0000","last_active":"2024-09-21T03:43:54.490007+0000","last_peered":"2024-09-21T03:43:54.490007+0000","last_clean":"2024-09-21T03:43:54.490007+0000","last_became_active":"2024-09-21T03:43:08.016013+0000","last_became_peered":"2024-09-21T03:43:08.016013+0000","last_unstale":"2024-09-21T03:43:54.490007+0000","last_undegraded":"2024-09-21T03:43:54.490007+0000","last_fullsized":"2024-09-21T03:43:54.490007+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:01.865669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.458144+0000","last_change":"2024-09-21T03:43:08.007862+0000","last_active":"2024-09-21T03:43:54.458144+0000","last_peered":"2024-09-21T03:43:54.458144+0000","last_clean":"2024-09-21T03:43:54.458144+0000","last_became_active":"2024-09-21T03:43:08.007433+0000","last_became_peered":"2024-09-21T03:43:08.007433+0000","last_unstale":"2024-09-21T03:43:54.458144+0000","last_undegraded":"2024-09-21T03:43:54.458144+0000","last_fullsized":"2024-09-21T03:43:54.458144+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:22:03.364767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.491033+0000","last_change":"2024-09-21T03:43:08.010542+0000","last_active":"2024-09-21T03:43:54.491033+0000","last_peered":"2024-09-21T03:43:54.491033+0000","last_clean":"2024-09-21T03:43:54.491033+0000","last_became_active":"2024-09-21T03:43:08.010195+0000","last_became_peered":"2024-09-21T03:43:08.010195+0000","last_unstale":"2024-09-21T03:43:54.491033+0000","last_undegraded":"2024-09-21T03:43:54.491033+0000","last_fullsized":"2024-09-21T03:43:54.491033+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:52.850657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.456740+0000","last_change":"2024-09-21T03:43:08.006168+0000","last_active":"2024-09-21T03:43:54.456740+0000","last_peered":"2024-09-21T03:43:54.456740+0000","last_clean":"2024-09-21T03:43:54.456740+0000","last_became_active":"2024-09-21T03:43:08.005283+0000","last_became_peered":"2024-09-21T03:43:08.005283+0000","last_unstale":"2024-09-21T03:43:54.456740+0000","last_undegraded":"2024-09-21T03:43:54.456740+0000","last_fullsized":"2024-09-21T03:43:54.456740+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:59.852412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.454368+0000","last_change":"2024-09-21T03:43:08.005870+0000","last_active":"2024-09-21T03:43:54.454368+0000","last_peered":"2024-09-21T03:43:54.454368+0000","last_clean":"2024-09-21T03:43:54.454368+0000","last_became_active":"2024-09-21T03:43:08.005485+0000","last_became_peered":"2024-09-21T03:43:08.005485+0000","last_unstale":"2024-09-21T03:43:54.454368+0000","last_undegraded":"2024-09-21T03:43:54.454368+0000","last_fullsized":"2024-09-21T03:43:54.454368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:41.387865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490804+0000","last_change":"2024-09-21T03:43:08.015176+0000","last_active":"2024-09-21T03:43:54.490804+0000","last_peered":"2024-09-21T03:43:54.490804+0000","last_clean":"2024-09-21T03:43:54.490804+0000","last_became_active":"2024-09-21T03:43:08.014451+0000","last_became_peered":"2024-09-21T03:43:08.014451+0000","last_unstale":"2024-09-21T03:43:54.490804+0000","last_undegraded":"2024-09-21T03:43:54.490804+0000","last_fullsized":"2024-09-21T03:43:54.490804+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:49.524800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489769+0000","last_change":"2024-09-21T03:43:08.016798+0000","last_active":"2024-09-21T03:43:54.489769+0000","last_peered":"2024-09-21T03:43:54.489769+0000","last_clean":"2024-09-21T03:43:54.489769+0000","last_became_active":"2024-09-21T03:43:08.014842+0000","last_became_peered":"2024-09-21T03:43:08.014842+0000","last_unstale":"2024-09-21T03:43:54.489769+0000","last_undegraded":"2024-09-21T03:43:54.489769+0000","last_fullsized":"2024-09-21T03:43:54.489769+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:19.290618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489730+0000","last_change":"2024-09-21T03:43:08.021285+0000","last_active":"2024-09-21T03:43:54.489730+0000","last_peered":"2024-09-21T03:43:54.489730+0000","last_clean":"2024-09-21T03:43:54.489730+0000","last_became_active":"2024-09-21T03:43:08.020904+0000","last_became_peered":"2024-09-21T03:43:08.020904+0000","last_unstale":"2024-09-21T03:43:54.489730+0000","last_undegraded":"2024-09-21T03:43:54.489730+0000","last_fullsized":"2024-09-21T03:43:54.489730+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:56.589089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489290+0000","last_change":"2024-09-21T03:43:08.017599+0000","last_active":"2024-09-21T03:43:54.489290+0000","last_peered":"2024-09-21T03:43:54.489290+0000","last_clean":"2024-09-21T03:43:54.489290+0000","last_became_active":"2024-09-21T03:43:08.016138+0000","last_became_peered":"2024-09-21T03:43:08.016138+0000","last_unstale":"2024-09-21T03:43:54.489290+0000","last_undegraded":"2024-09-21T03:43:54.489290+0000","last_fullsized":"2024-09-21T03:43:54.489290+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:09.902246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.458119+0000","last_change":"2024-09-21T03:43:08.009598+0000","last_active":"2024-09-21T03:43:54.458119+0000","last_peered":"2024-09-21T03:43:54.458119+0000","last_clean":"2024-09-21T03:43:54.458119+0000","last_became_active":"2024-09-21T03:43:08.007488+0000","last_became_peered":"2024-09-21T03:43:08.007488+0000","last_unstale":"2024-09-21T03:43:54.458119+0000","last_undegraded":"2024-09-21T03:43:54.458119+0000","last_fullsized":"2024-09-21T03:43:54.458119+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:23.192761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.457719+0000","last_change":"2024-09-21T03:43:08.016450+0000","last_active":"2024-09-21T03:43:54.457719+0000","last_peered":"2024-09-21T03:43:54.457719+0000","last_clean":"2024-09-21T03:43:54.457719+0000","last_became_active":"2024-09-21T03:43:08.016037+0000","last_became_peered":"2024-09-21T03:43:08.016037+0000","last_unstale":"2024-09-21T03:43:54.457719+0000","last_undegraded":"2024-09-21T03:43:54.457719+0000","last_fullsized":"2024-09-21T03:43:54.457719+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:39.109408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462131+0000","last_change":"2024-09-21T03:43:08.010298+0000","last_active":"2024-09-21T03:43:54.462131+0000","last_peered":"2024-09-21T03:43:54.462131+0000","last_clean":"2024-09-21T03:43:54.462131+0000","last_became_active":"2024-09-21T03:43:08.009847+0000","last_became_peered":"2024-09-21T03:43:08.009847+0000","last_unstale":"2024-09-21T03:43:54.462131+0000","last_undegraded":"2024-09-21T03:43:54.462131+0000","last_fullsized":"2024-09-21T03:43:54.462131+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:37.463456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.454757+0000","last_change":"2024-09-21T03:43:08.017834+0000","last_active":"2024-09-21T03:43:54.454757+0000","last_peered":"2024-09-21T03:43:54.454757+0000","last_clean":"2024-09-21T03:43:54.454757+0000","last_became_active":"2024-09-21T03:43:08.016910+0000","last_became_peered":"2024-09-21T03:43:08.016910+0000","last_unstale":"2024-09-21T03:43:54.454757+0000","last_undegraded":"2024-09-21T03:43:54.454757+0000","last_fullsized":"2024-09-21T03:43:54.454757+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:50.528651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490598+0000","last_change":"2024-09-21T03:43:08.013945+0000","last_active":"2024-09-21T03:43:54.490598+0000","last_peered":"2024-09-21T03:43:54.490598+0000","last_clean":"2024-09-21T03:43:54.490598+0000","last_became_active":"2024-09-21T03:43:08.013598+0000","last_became_peered":"2024-09-21T03:43:08.013598+0000","last_unstale":"2024-09-21T03:43:54.490598+0000","last_undegraded":"2024-09-21T03:43:54.490598+0000","last_fullsized":"2024-09-21T03:43:54.490598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:58.182938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.460179+0000","last_change":"2024-09-21T03:43:08.018824+0000","last_active":"2024-09-21T03:43:54.460179+0000","last_peered":"2024-09-21T03:43:54.460179+0000","last_clean":"2024-09-21T03:43:54.460179+0000","last_became_active":"2024-09-21T03:43:08.017196+0000","last_became_peered":"2024-09-21T03:43:08.017196+0000","last_unstale":"2024-09-21T03:43:54.460179+0000","last_undegraded":"2024-09-21T03:43:54.460179+0000","last_fullsized":"2024-09-21T03:43:54.460179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:02.357523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.462145+0000","last_change":"2024-09-21T03:43:08.003514+0000","last_active":"2024-09-21T03:43:54.462145+0000","last_peered":"2024-09-21T03:43:54.462145+0000","last_clean":"2024-09-21T03:43:54.462145+0000","last_became_active":"2024-09-21T03:43:08.003131+0000","last_became_peered":"2024-09-21T03:43:08.003131+0000","last_unstale":"2024-09-21T03:43:54.462145+0000","last_undegraded":"2024-09-21T03:43:54.462145+0000","last_fullsized":"2024-09-21T03:43:54.462145+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:23.170302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.454522+0000","last_change":"2024-09-21T03:43:08.015890+0000","last_active":"2024-09-21T03:43:54.454522+0000","last_peered":"2024-09-21T03:43:54.454522+0000","last_clean":"2024-09-21T03:43:54.454522+0000","last_became_active":"2024-09-21T03:43:08.015519+0000","last_became_peered":"2024-09-21T03:43:08.015519+0000","last_unstale":"2024-09-21T03:43:54.454522+0000","last_undegraded":"2024-09-21T03:43:54.454522+0000","last_fullsized":"2024-09-21T03:43:54.454522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:10.875923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.491133+0000","last_change":"2024-09-21T03:43:08.012261+0000","last_active":"2024-09-21T03:43:54.491133+0000","last_peered":"2024-09-21T03:43:54.491133+0000","last_clean":"2024-09-21T03:43:54.491133+0000","last_became_active":"2024-09-21T03:43:08.011783+0000","last_became_peered":"2024-09-21T03:43:08.011783+0000","last_unstale":"2024-09-21T03:43:54.491133+0000","last_undegraded":"2024-09-21T03:43:54.491133+0000","last_fullsized":"2024-09-21T03:43:54.491133+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:07.011340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.496630+0000","last_change":"2024-09-21T03:43:54.494826+0000","last_active":"2024-09-21T03:43:54.496630+0000","last_peered":"2024-09-21T03:43:54.496630+0000","last_clean":"2024-09-21T03:43:54.496630+0000","last_became_active":"2024-09-21T03:43:54.494404+0000","last_became_peered":"2024-09-21T03:43:54.494404+0000","last_unstale":"2024-09-21T03:43:54.496630+0000","last_undegraded":"2024-09-21T03:43:54.496630+0000","last_fullsized":"2024-09-21T03:43:54.496630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:42.109053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489653+0000","last_change":"2024-09-21T03:43:08.016214+0000","last_active":"2024-09-21T03:43:54.489653+0000","last_peered":"2024-09-21T03:43:54.489653+0000","last_clean":"2024-09-21T03:43:54.489653+0000","last_became_active":"2024-09-21T03:43:08.014815+0000","last_became_peered":"2024-09-21T03:43:08.014815+0000","last_unstale":"2024-09-21T03:43:54.489653+0000","last_undegraded":"2024-09-21T03:43:54.489653+0000","last_fullsized":"2024-09-21T03:43:54.489653+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:04.742829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.489944+0000","last_change":"2024-09-21T03:43:08.019269+0000","last_active":"2024-09-21T03:43:54.489944+0000","last_peered":"2024-09-21T03:43:54.489944+0000","last_clean":"2024-09-21T03:43:54.489944+0000","last_became_active":"2024-09-21T03:43:08.017962+0000","last_became_peered":"2024-09-21T03:43:08.017962+0000","last_unstale":"2024-09-21T03:43:54.489944+0000","last_undegraded":"2024-09-21T03:43:54.489944+0000","last_fullsized":"2024-09-21T03:43:54.489944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:05.950697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:43:54.490672+0000","last_change":"2024-09-21T03:43:08.014790+0000","last_active":"2024-09-21T03:43:54.490672+0000","last_peered":"2024-09-21T03:43:54.490672+0000","last_clean":"2024-09-21T03:43:54.490672+0000","last_became_active":"2024-09-21T03:43:08.013891+0000","last_became_peered":"2024-09-21T03:43:08.013891+0000","last_unstale":"2024-09-21T03:43:54.490672+0000","last_undegraded":"2024-09-21T03:43:54.490672+0000","last_fullsized":"2024-09-21T03:43:54.490672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:46.045675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63362,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":219,"num_read_kb":177,"num_write":245,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":198616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808863,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":482159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972387,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1129229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135912,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":481698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332142,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":482076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495668,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36892,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708260,"statfs":{"total":95995035648,"available":95957258240,"internally_reserved":0,"allocated":1585152,"data_stored":1078182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659193,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":477202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855423,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1593344,"data_stored":1073764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018949,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":481898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215177,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":476910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444113,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":539035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":539046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":479049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:44:28.616 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T03:44:28.617 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T03:44:28.617 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T03:44:28.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:28 smithi123 ceph-mon[34673]: pgmap v361: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-21T03:44:28.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:28 smithi123 ceph-mon[34673]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:44:28.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:28 smithi016 ceph-mon[30375]: pgmap v361: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-21T03:44:28.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:28 smithi016 ceph-mon[30375]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:44:29.041 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:29.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:28 smithi134 ceph-mon[35031]: pgmap v361: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-21T03:44:29.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:28 smithi134 ceph-mon[35031]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:44:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.668+0000 7ff8eb200640 1 -- 172.21.15.16:0/1638644929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec1071c0 msgr2=0x7ff8ec10db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.668+0000 7ff8eb200640 1 --2- 172.21.15.16:0/1638644929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec1071c0 0x7ff8ec10db60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009ca0 tx=0x7ff8e402a950 comp rx=0 tx=0).stop 2024-09-21T03:44:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.668+0000 7ff8eb200640 1 -- 172.21.15.16:0/1638644929 shutdown_connections 2024-09-21T03:44:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.668+0000 7ff8eb200640 1 --2- 172.21.15.16:0/1638644929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec1071c0 0x7ff8ec10db60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.668+0000 7ff8eb200640 1 --2- 172.21.15.16:0/1638644929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ec106820 0x7ff8ec106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.668+0000 7ff8eb200640 1 --2- 172.21.15.16:0/1638644929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8ec105790 0x7ff8ec105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.668+0000 7ff8eb200640 1 -- 172.21.15.16:0/1638644929 >> 172.21.15.16:0/1638644929 conn(0x7ff8ec1010e0 msgr2=0x7ff8ec1035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.668+0000 7ff8eb200640 1 -- 172.21.15.16:0/1638644929 shutdown_connections 2024-09-21T03:44:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.668+0000 7ff8eb200640 1 -- 172.21.15.16:0/1638644929 wait complete. 2024-09-21T03:44:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.669+0000 7ff8eb200640 1 Processor -- start 2024-09-21T03:44:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.669+0000 7ff8eb200640 1 -- start start 2024-09-21T03:44:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.669+0000 7ff8eb200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ec105790 0x7ff8ec19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.670+0000 7ff8eb200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec106820 0x7ff8ec19c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.670+0000 7ff8eb200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8ec1071c0 0x7ff8ec19d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.670+0000 7ff8eb200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec111430 con 0x7ff8ec106820 2024-09-21T03:44:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.670+0000 7ff8eb200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1112b0 con 0x7ff8ec105790 2024-09-21T03:44:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.670+0000 7ff8eb200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1115b0 con 0x7ff8ec1071c0 2024-09-21T03:44:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.670+0000 7ff8e3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec106820 0x7ff8ec19c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.670+0000 7ff8e3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec106820 0x7ff8ec19c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38268/0 (socket says 172.21.15.16:38268) 2024-09-21T03:44:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.670+0000 7ff8e9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ec105790 0x7ff8ec19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.670+0000 7ff8e3e00640 1 -- 172.21.15.16:0/491008836 learned_addr learned my addr 172.21.15.16:0/491008836 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.670+0000 7ff8ea800640 1 --2- 172.21.15.16:0/491008836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8ec1071c0 0x7ff8ec19d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.670+0000 7ff8e9e00640 1 -- 172.21.15.16:0/491008836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8ec1071c0 msgr2=0x7ff8ec19d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.670+0000 7ff8e9e00640 1 --2- 172.21.15.16:0/491008836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8ec1071c0 0x7ff8ec19d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.670+0000 7ff8e9e00640 1 -- 172.21.15.16:0/491008836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec106820 msgr2=0x7ff8ec19c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.670+0000 7ff8e9e00640 1 --2- 172.21.15.16:0/491008836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec106820 0x7ff8ec19c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.670+0000 7ff8e9e00640 1 -- 172.21.15.16:0/491008836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec19d9c0 con 0x7ff8ec105790 2024-09-21T03:44:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.670+0000 7ff8ea800640 1 --2- 172.21.15.16:0/491008836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8ec1071c0 0x7ff8ec19d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.671+0000 7ff8e9e00640 1 --2- 172.21.15.16:0/491008836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ec105790 0x7ff8ec19bf90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00e9f0 tx=0x7ff8dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.671+0000 7ff8e2a00640 1 -- 172.21.15.16:0/491008836 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc00ccc0 con 0x7ff8ec105790 2024-09-21T03:44:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.671+0000 7ff8e2a00640 1 -- 172.21.15.16:0/491008836 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8dc005020 con 0x7ff8ec105790 2024-09-21T03:44:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.671+0000 7ff8eb200640 1 -- 172.21.15.16:0/491008836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ec19dcb0 con 0x7ff8ec105790 2024-09-21T03:44:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.671+0000 7ff8e2a00640 1 -- 172.21.15.16:0/491008836 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc004a00 con 0x7ff8ec105790 2024-09-21T03:44:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.671+0000 7ff8eb200640 1 -- 172.21.15.16:0/491008836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ec1a43b0 con 0x7ff8ec105790 2024-09-21T03:44:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.672+0000 7ff8eb200640 1 -- 172.21.15.16:0/491008836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ec10c970 con 0x7ff8ec105790 2024-09-21T03:44:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.675+0000 7ff8e2a00640 1 -- 172.21.15.16:0/491008836 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7ff8dc004ba0 con 0x7ff8ec105790 2024-09-21T03:44:29.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.676+0000 7ff8e2a00640 1 --2- 172.21.15.16:0/491008836 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff8d00b6590 0x7ff8d00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:29.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.676+0000 7ff8e2a00640 1 -- 172.21.15.16:0/491008836 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7ff8dc0e78b0 con 0x7ff8ec105790 2024-09-21T03:44:29.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.676+0000 7ff8e3e00640 1 --2- 172.21.15.16:0/491008836 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff8d00b6590 0x7ff8d00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:29.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.677+0000 7ff8e3e00640 1 --2- 172.21.15.16:0/491008836 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff8d00b6590 0x7ff8d00b8aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4009c90 tx=0x7ff8d4005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:29.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.680+0000 7ff8e2a00640 1 -- 172.21.15.16:0/491008836 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff8dc0adc20 con 0x7ff8ec105790 2024-09-21T03:44:29.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.954+0000 7ff8eb200640 1 -- 172.21.15.16:0/491008836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7ff8ec02e640 con 0x7ff8ec105790 2024-09-21T03:44:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.962+0000 7ff8e2a00640 1 -- 172.21.15.16:0/491008836 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7ff8dc0b2000 con 0x7ff8ec105790 2024-09-21T03:44:29.965 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-21T03:44:29.965 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQAdQe5mgy4SORAAHuZGCSdvm3vvaPO+l31Wxg== 2024-09-21T03:44:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.965+0000 7ff8eb200640 1 -- 172.21.15.16:0/491008836 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff8d00b6590 msgr2=0x7ff8d00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.965+0000 7ff8eb200640 1 --2- 172.21.15.16:0/491008836 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff8d00b6590 0x7ff8d00b8aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4009c90 tx=0x7ff8d4005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.965+0000 7ff8eb200640 1 -- 172.21.15.16:0/491008836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ec105790 msgr2=0x7ff8ec19bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.965+0000 7ff8eb200640 1 --2- 172.21.15.16:0/491008836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ec105790 0x7ff8ec19bf90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00e9f0 tx=0x7ff8dc00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.965+0000 7ff8eb200640 1 -- 172.21.15.16:0/491008836 shutdown_connections 2024-09-21T03:44:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.965+0000 7ff8eb200640 1 --2- 172.21.15.16:0/491008836 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff8d00b6590 0x7ff8d00b8aa0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.965+0000 7ff8eb200640 1 --2- 172.21.15.16:0/491008836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8ec1071c0 0x7ff8ec19d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.965+0000 7ff8eb200640 1 --2- 172.21.15.16:0/491008836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec106820 0x7ff8ec19c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.965+0000 7ff8eb200640 1 --2- 172.21.15.16:0/491008836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ec105790 0x7ff8ec19bf90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.965+0000 7ff8eb200640 1 -- 172.21.15.16:0/491008836 >> 172.21.15.16:0/491008836 conn(0x7ff8ec1010e0 msgr2=0x7ff8ec109b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.965+0000 7ff8eb200640 1 -- 172.21.15.16:0/491008836 shutdown_connections 2024-09-21T03:44:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:29.965+0000 7ff8eb200640 1 -- 172.21.15.16:0/491008836 wait complete. 2024-09-21T03:44:30.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:30 smithi016 ceph-mon[30375]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-09-21T03:44:30.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:30 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/491008836' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:44:30.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:30 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:44:30.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:30 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:44:30.869 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:44:30.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T03:44:30.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T03:44:30.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:30 smithi123 ceph-mon[34673]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-09-21T03:44:30.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:30 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/491008836' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:44:30.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:30 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:44:30.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:30 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:44:30.950 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T03:44:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:30 smithi134 ceph-mon[35031]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-09-21T03:44:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:30 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/491008836' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:44:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:30 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:44:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:30 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:44:31.368 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.027+0000 7f4003400640 1 -- 172.21.15.16:0/1830679160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc10de10 msgr2=0x7f3ffc110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.027+0000 7f4003400640 1 --2- 172.21.15.16:0/1830679160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc10de10 0x7f3ffc110300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009ca0 tx=0x7f3ff402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.027+0000 7f4003400640 1 -- 172.21.15.16:0/1830679160 shutdown_connections 2024-09-21T03:44:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.027+0000 7f4003400640 1 --2- 172.21.15.16:0/1830679160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc10de10 0x7f3ffc110300 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.027+0000 7f4003400640 1 --2- 172.21.15.16:0/1830679160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ffc105300 0x7f3ffc10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.027+0000 7f4003400640 1 --2- 172.21.15.16:0/1830679160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ffc1049b0 0x7f3ffc104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.027+0000 7f4003400640 1 -- 172.21.15.16:0/1830679160 >> 172.21.15.16:0/1830679160 conn(0x7f3ffc079760 msgr2=0x7f3ffc079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.027+0000 7f4003400640 1 -- 172.21.15.16:0/1830679160 shutdown_connections 2024-09-21T03:44:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.027+0000 7f4003400640 1 -- 172.21.15.16:0/1830679160 wait complete. 2024-09-21T03:44:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.029+0000 7f4003400640 1 Processor -- start 2024-09-21T03:44:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.029+0000 7f4003400640 1 -- start start 2024-09-21T03:44:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.029+0000 7f4003400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc1049b0 0x7f3ffc1026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.029+0000 7f4003400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ffc105300 0x7f3ffc102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.029+0000 7f4003400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ffc10de10 0x7f3ffc103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.029+0000 7f4003400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc113ae0 con 0x7f3ffc1049b0 2024-09-21T03:44:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.029+0000 7f4003400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc113960 con 0x7f3ffc105300 2024-09-21T03:44:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.029+0000 7f4003400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc113c60 con 0x7f3ffc10de10 2024-09-21T03:44:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.030+0000 7f4000a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc1049b0 0x7f3ffc1026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.030+0000 7f4000a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc1049b0 0x7f3ffc1026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48320/0 (socket says 172.21.15.16:48320) 2024-09-21T03:44:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.030+0000 7f4000a00640 1 -- 172.21.15.16:0/463586402 learned_addr learned my addr 172.21.15.16:0/463586402 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.030+0000 7f3ffbe00640 1 --2- 172.21.15.16:0/463586402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ffc105300 0x7f3ffc102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.030+0000 7f4001400640 1 --2- 172.21.15.16:0/463586402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ffc10de10 0x7f3ffc103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.030+0000 7f4000a00640 1 -- 172.21.15.16:0/463586402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ffc10de10 msgr2=0x7f3ffc103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.030+0000 7f4000a00640 1 --2- 172.21.15.16:0/463586402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ffc10de10 0x7f3ffc103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.030+0000 7f4000a00640 1 -- 172.21.15.16:0/463586402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ffc105300 msgr2=0x7f3ffc102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.030+0000 7f4000a00640 1 --2- 172.21.15.16:0/463586402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ffc105300 0x7f3ffc102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.030+0000 7f4000a00640 1 -- 172.21.15.16:0/463586402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc1a8710 con 0x7f3ffc1049b0 2024-09-21T03:44:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.031+0000 7f4000a00640 1 --2- 172.21.15.16:0/463586402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc1049b0 0x7f3ffc1026f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000e9f0 tx=0x7f3ff000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.031+0000 7f3ff9600640 1 -- 172.21.15.16:0/463586402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff000cd80 con 0x7f3ffc1049b0 2024-09-21T03:44:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.031+0000 7f3ff9600640 1 -- 172.21.15.16:0/463586402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff00108e0 con 0x7f3ffc1049b0 2024-09-21T03:44:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.031+0000 7f4003400640 1 -- 172.21.15.16:0/463586402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ffc1a8a00 con 0x7f3ffc1049b0 2024-09-21T03:44:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.031+0000 7f3ff9600640 1 -- 172.21.15.16:0/463586402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0018980 con 0x7f3ffc1049b0 2024-09-21T03:44:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.031+0000 7f4003400640 1 -- 172.21.15.16:0/463586402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ffc1a8f40 con 0x7f3ffc1049b0 2024-09-21T03:44:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.032+0000 7f4003400640 1 -- 172.21.15.16:0/463586402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ffc06c2c0 con 0x7f3ffc1049b0 2024-09-21T03:44:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.035+0000 7f3ff9600640 1 -- 172.21.15.16:0/463586402 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f3ff0010430 con 0x7f3ffc1049b0 2024-09-21T03:44:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.038+0000 7f3ff9600640 1 --2- 172.21.15.16:0/463586402 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fe40b65a0 0x7f3fe40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.038+0000 7f3ff9600640 1 -- 172.21.15.16:0/463586402 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f3ff00e7ff0 con 0x7f3ffc1049b0 2024-09-21T03:44:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.038+0000 7f3ffbe00640 1 --2- 172.21.15.16:0/463586402 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fe40b65a0 0x7f3fe40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.039+0000 7f3ffbe00640 1 --2- 172.21.15.16:0/463586402 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fe40b65a0 0x7f3fe40b8ab0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc10fba0 tx=0x7f3fe8005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.040+0000 7f3ff9600640 1 -- 172.21.15.16:0/463586402 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3ff00b2780 con 0x7f3ffc1049b0 2024-09-21T03:44:32.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.242+0000 7f4003400640 1 -- 172.21.15.16:0/463586402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f3ffc1039f0 con 0x7f3ffc1049b0 2024-09-21T03:44:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.243+0000 7f3ff9600640 1 -- 172.21.15.16:0/463586402 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v51) ==== 86+0+133 (secure 0 0 0) 0x7f3ff00b76c0 con 0x7f3ffc1049b0 2024-09-21T03:44:32.246 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:44:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.246+0000 7f4003400640 1 -- 172.21.15.16:0/463586402 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fe40b65a0 msgr2=0x7f3fe40b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.246+0000 7f4003400640 1 --2- 172.21.15.16:0/463586402 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fe40b65a0 0x7f3fe40b8ab0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc10fba0 tx=0x7f3fe8005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.246+0000 7f4003400640 1 -- 172.21.15.16:0/463586402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc1049b0 msgr2=0x7f3ffc1026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.246+0000 7f4003400640 1 --2- 172.21.15.16:0/463586402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc1049b0 0x7f3ffc1026f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000e9f0 tx=0x7f3ff000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.246+0000 7f4003400640 1 -- 172.21.15.16:0/463586402 shutdown_connections 2024-09-21T03:44:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.246+0000 7f4003400640 1 --2- 172.21.15.16:0/463586402 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fe40b65a0 0x7f3fe40b8ab0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.246+0000 7f4003400640 1 --2- 172.21.15.16:0/463586402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ffc10de10 0x7f3ffc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.246+0000 7f4003400640 1 --2- 172.21.15.16:0/463586402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ffc105300 0x7f3ffc102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.246+0000 7f4003400640 1 --2- 172.21.15.16:0/463586402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ffc1049b0 0x7f3ffc1026f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.246+0000 7f4003400640 1 -- 172.21.15.16:0/463586402 >> 172.21.15.16:0/463586402 conn(0x7f3ffc079760 msgr2=0x7f3ffc0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.246+0000 7f4003400640 1 -- 172.21.15.16:0/463586402 shutdown_connections 2024-09-21T03:44:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:32.246+0000 7f4003400640 1 -- 172.21.15.16:0/463586402 wait complete. 2024-09-21T03:44:32.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:32 smithi123 ceph-mon[34673]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-21T03:44:32.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:32 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/463586402' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:44:32.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:32 smithi016 ceph-mon[30375]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-21T03:44:32.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:32 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/463586402' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:44:33.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:32 smithi134 ceph-mon[35031]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-21T03:44:33.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:32 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/463586402' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:44:33.139 INFO:teuthology.orchestra.run.smithi016.stdout:[{"version":51,"timestamp":"2024-09-21T03:42:53.703456+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T03:44:33.139 INFO:tasks.ceph_manager:config epoch is 51 2024-09-21T03:44:33.140 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T03:44:33.140 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T03:44:33.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T03:44:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.212+0000 7f919b200640 1 -- 172.21.15.16:0/1640997008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f919c109c40 msgr2=0x7f919c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.212+0000 7f919b200640 1 --2- 172.21.15.16:0/1640997008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f919c109c40 0x7f919c110470 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9194009ca0 tx=0x7f919402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.212+0000 7f919b200640 1 -- 172.21.15.16:0/1640997008 shutdown_connections 2024-09-21T03:44:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.212+0000 7f919b200640 1 --2- 172.21.15.16:0/1640997008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f919c109c40 0x7f919c110470 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.212+0000 7f919b200640 1 --2- 172.21.15.16:0/1640997008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919c1092f0 0x7f919c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.212+0000 7f919b200640 1 --2- 172.21.15.16:0/1640997008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f919c1039b0 0x7f919c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.212+0000 7f919b200640 1 -- 172.21.15.16:0/1640997008 >> 172.21.15.16:0/1640997008 conn(0x7f919c0ff400 msgr2=0x7f919c101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.212+0000 7f919b200640 1 -- 172.21.15.16:0/1640997008 shutdown_connections 2024-09-21T03:44:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.213+0000 7f919b200640 1 -- 172.21.15.16:0/1640997008 wait complete. 2024-09-21T03:44:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.214+0000 7f919b200640 1 Processor -- start 2024-09-21T03:44:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.214+0000 7f919b200640 1 -- start start 2024-09-21T03:44:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.214+0000 7f919b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f919c1039b0 0x7f919c19c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.214+0000 7f919b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919c1092f0 0x7f919c19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.214+0000 7f919b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f919c109c40 0x7f919c1a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.214+0000 7f919b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f919c113900 con 0x7f919c109c40 2024-09-21T03:44:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.214+0000 7f919b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f919c113780 con 0x7f919c1092f0 2024-09-21T03:44:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.214+0000 7f919b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f919c113a80 con 0x7f919c1039b0 2024-09-21T03:44:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.214+0000 7f919a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f919c109c40 0x7f919c1a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.215+0000 7f919a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f919c109c40 0x7f919c1a1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48334/0 (socket says 172.21.15.16:48334) 2024-09-21T03:44:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.215+0000 7f919a800640 1 -- 172.21.15.16:0/944280516 learned_addr learned my addr 172.21.15.16:0/944280516 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.215+0000 7f9199400640 1 --2- 172.21.15.16:0/944280516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919c1092f0 0x7f919c19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.215+0000 7f9199e00640 1 --2- 172.21.15.16:0/944280516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f919c1039b0 0x7f919c19c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.215+0000 7f9199400640 1 -- 172.21.15.16:0/944280516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f919c1039b0 msgr2=0x7f919c19c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.215+0000 7f9199400640 1 --2- 172.21.15.16:0/944280516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f919c1039b0 0x7f919c19c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.215+0000 7f9199400640 1 -- 172.21.15.16:0/944280516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f919c109c40 msgr2=0x7f919c1a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.215+0000 7f9199400640 1 --2- 172.21.15.16:0/944280516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f919c109c40 0x7f919c1a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.215+0000 7f9199400640 1 -- 172.21.15.16:0/944280516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c1a2440 con 0x7f919c1092f0 2024-09-21T03:44:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.215+0000 7f9199400640 1 --2- 172.21.15.16:0/944280516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919c1092f0 0x7f919c19ca70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f919c06b930 tx=0x7f918800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.216+0000 7f9192a00640 1 -- 172.21.15.16:0/944280516 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918800cc40 con 0x7f919c1092f0 2024-09-21T03:44:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.216+0000 7f9192a00640 1 -- 172.21.15.16:0/944280516 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f918800cde0 con 0x7f919c1092f0 2024-09-21T03:44:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.216+0000 7f919b200640 1 -- 172.21.15.16:0/944280516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919c1a2730 con 0x7f919c1092f0 2024-09-21T03:44:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.216+0000 7f9192a00640 1 -- 172.21.15.16:0/944280516 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188018760 con 0x7f919c1092f0 2024-09-21T03:44:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.216+0000 7f919b200640 1 -- 172.21.15.16:0/944280516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f919c1a8e90 con 0x7f919c1092f0 2024-09-21T03:44:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.217+0000 7f919b200640 1 -- 172.21.15.16:0/944280516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f919c103dc0 con 0x7f919c1092f0 2024-09-21T03:44:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.220+0000 7f9192a00640 1 -- 172.21.15.16:0/944280516 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f9188010a90 con 0x7f919c1092f0 2024-09-21T03:44:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.221+0000 7f9192a00640 1 --2- 172.21.15.16:0/944280516 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f91800b65a0 0x7f91800b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.221+0000 7f9192a00640 1 -- 172.21.15.16:0/944280516 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f91880e7fb0 con 0x7f919c1092f0 2024-09-21T03:44:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.221+0000 7f9199e00640 1 --2- 172.21.15.16:0/944280516 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f91800b65a0 0x7f91800b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.222+0000 7f9199e00640 1 --2- 172.21.15.16:0/944280516 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f91800b65a0 0x7f91800b8ab0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f918c009bd0 tx=0x7f918c005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.223+0000 7f9192a00640 1 -- 172.21.15.16:0/944280516 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f91880b2700 con 0x7f919c1092f0 2024-09-21T03:44:34.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.473+0000 7f919b200640 1 -- 172.21.15.16:0/944280516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f919c1a9130 con 0x7f919c1092f0 2024-09-21T03:44:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.479+0000 7f9192a00640 1 -- 172.21.15.16:0/944280516 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+296438 (secure 0 0 0) 0x7f91880b7640 con 0x7f919c1092f0 2024-09-21T03:44:34.485 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:44:34.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.484+0000 7f919b200640 1 -- 172.21.15.16:0/944280516 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f91800b65a0 msgr2=0x7f91800b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:34.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.484+0000 7f919b200640 1 --2- 172.21.15.16:0/944280516 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f91800b65a0 0x7f91800b8ab0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f918c009bd0 tx=0x7f918c005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:34.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.484+0000 7f919b200640 1 -- 172.21.15.16:0/944280516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919c1092f0 msgr2=0x7f919c19ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:34.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.484+0000 7f919b200640 1 --2- 172.21.15.16:0/944280516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919c1092f0 0x7f919c19ca70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f919c06b930 tx=0x7f918800ee50 comp rx=0 tx=0).stop 2024-09-21T03:44:34.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.484+0000 7f919b200640 1 -- 172.21.15.16:0/944280516 shutdown_connections 2024-09-21T03:44:34.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.484+0000 7f919b200640 1 --2- 172.21.15.16:0/944280516 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f91800b65a0 0x7f91800b8ab0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.484+0000 7f919b200640 1 --2- 172.21.15.16:0/944280516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f919c109c40 0x7f919c1a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.484+0000 7f919b200640 1 --2- 172.21.15.16:0/944280516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919c1092f0 0x7f919c19ca70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.484+0000 7f919b200640 1 --2- 172.21.15.16:0/944280516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f919c1039b0 0x7f919c19c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:34.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.484+0000 7f919b200640 1 -- 172.21.15.16:0/944280516 >> 172.21.15.16:0/944280516 conn(0x7f919c0ff400 msgr2=0x7f919c1068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:34.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.484+0000 7f919b200640 1 -- 172.21.15.16:0/944280516 shutdown_connections 2024-09-21T03:44:34.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:34.484+0000 7f919b200640 1 -- 172.21.15.16:0/944280516 wait complete. 2024-09-21T03:44:34.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:34 smithi016 ceph-mon[30375]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-21T03:44:34.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:34 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/944280516' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:44:34.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:34 smithi123 ceph-mon[34673]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-21T03:44:34.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:34 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/944280516' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:44:35.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:34 smithi134 ceph-mon[35031]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-21T03:44:35.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:34 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/944280516' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:44:35.438 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"flags":0,"active_gid":14154,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3097741949},{"type":"v1","addr":"172.21.15.16:6801","nonce":3097741949}]},"active_addr":"172.21.15.16:6801/3097741949","active_change":"2024-09-21T03:34:53.217379+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24175,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24197,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.16:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2016398981}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2041301734}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":4171842414}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":4135499316}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3559234944}]}]} 2024-09-21T03:44:35.444 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T03:44:35.444 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T03:44:35.444 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:44:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.531+0000 7f92f7000640 1 -- 172.21.15.16:0/1467668847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92f01094a0 msgr2=0x7f92f01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.531+0000 7f92f7000640 1 --2- 172.21.15.16:0/1467668847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92f01094a0 0x7f92f01098b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009900 tx=0x7f92d802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.531+0000 7f92f7000640 1 -- 172.21.15.16:0/1467668847 shutdown_connections 2024-09-21T03:44:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.531+0000 7f92f7000640 1 --2- 172.21.15.16:0/1467668847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92f0109df0 0x7f92f0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.531+0000 7f92f7000640 1 --2- 172.21.15.16:0/1467668847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92f01094a0 0x7f92f01098b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.531+0000 7f92f7000640 1 --2- 172.21.15.16:0/1467668847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0103b60 0x7f92f0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.531+0000 7f92f7000640 1 -- 172.21.15.16:0/1467668847 >> 172.21.15.16:0/1467668847 conn(0x7f92f007b850 msgr2=0x7f92f007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.531+0000 7f92f7000640 1 -- 172.21.15.16:0/1467668847 shutdown_connections 2024-09-21T03:44:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.531+0000 7f92f7000640 1 -- 172.21.15.16:0/1467668847 wait complete. 2024-09-21T03:44:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.533+0000 7f92f7000640 1 Processor -- start 2024-09-21T03:44:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.533+0000 7f92f7000640 1 -- start start 2024-09-21T03:44:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.533+0000 7f92f7000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92f0103b60 0x7f92f019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.533+0000 7f92f7000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f01094a0 0x7f92f019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.533+0000 7f92f7000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92f0109df0 0x7f92f01a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.533+0000 7f92f7000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0113b00 con 0x7f92f0109df0 2024-09-21T03:44:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.533+0000 7f92f7000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0113980 con 0x7f92f0103b60 2024-09-21T03:44:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.533+0000 7f92f7000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0113c80 con 0x7f92f01094a0 2024-09-21T03:44:36.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.534+0000 7f92e7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f01094a0 0x7f92f019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:36.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.534+0000 7f92efe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92f0103b60 0x7f92f019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:36.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.534+0000 7f92e7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f01094a0 0x7f92f019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:58372/0 (socket says 172.21.15.16:58372) 2024-09-21T03:44:36.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.534+0000 7f92e7e00640 1 -- 172.21.15.16:0/3225171016 learned_addr learned my addr 172.21.15.16:0/3225171016 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:36.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.534+0000 7f92f5000640 1 --2- 172.21.15.16:0/3225171016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92f0109df0 0x7f92f01a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:36.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.534+0000 7f92e7e00640 1 -- 172.21.15.16:0/3225171016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92f0103b60 msgr2=0x7f92f019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.534+0000 7f92e7e00640 1 --2- 172.21.15.16:0/3225171016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92f0103b60 0x7f92f019c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.534+0000 7f92e7e00640 1 -- 172.21.15.16:0/3225171016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92f0109df0 msgr2=0x7f92f01a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.534+0000 7f92e7e00640 1 --2- 172.21.15.16:0/3225171016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92f0109df0 0x7f92f01a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.534+0000 7f92e7e00640 1 -- 172.21.15.16:0/3225171016 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f01a2660 con 0x7f92f01094a0 2024-09-21T03:44:36.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.534+0000 7f92e7e00640 1 --2- 172.21.15.16:0/3225171016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f01094a0 0x7f92f019cc90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f92d8005ab0 tx=0x7f92d8003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:36.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.535+0000 7f92ed600640 1 -- 172.21.15.16:0/3225171016 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d803f070 con 0x7f92f01094a0 2024-09-21T03:44:36.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.535+0000 7f92f7000640 1 -- 172.21.15.16:0/3225171016 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f01a28f0 con 0x7f92f01094a0 2024-09-21T03:44:36.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.535+0000 7f92ed600640 1 -- 172.21.15.16:0/3225171016 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92d8031b50 con 0x7f92f01094a0 2024-09-21T03:44:36.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.535+0000 7f92ed600640 1 -- 172.21.15.16:0/3225171016 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8035030 con 0x7f92f01094a0 2024-09-21T03:44:36.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.535+0000 7f92f7000640 1 -- 172.21.15.16:0/3225171016 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f01a90b0 con 0x7f92f01094a0 2024-09-21T03:44:36.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.536+0000 7f92f7000640 1 -- 172.21.15.16:0/3225171016 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f0104de0 con 0x7f92f01094a0 2024-09-21T03:44:36.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.538+0000 7f92ed600640 1 -- 172.21.15.16:0/3225171016 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f92d8042070 con 0x7f92f01094a0 2024-09-21T03:44:36.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.540+0000 7f92ed600640 1 --2- 172.21.15.16:0/3225171016 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f92d40b65a0 0x7f92d40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:36.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.540+0000 7f92ed600640 1 -- 172.21.15.16:0/3225171016 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f92d8108200 con 0x7f92f01094a0 2024-09-21T03:44:36.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.540+0000 7f92efe00640 1 --2- 172.21.15.16:0/3225171016 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f92d40b65a0 0x7f92d40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:36.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.541+0000 7f92efe00640 1 --2- 172.21.15.16:0/3225171016 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f92d40b65a0 0x7f92d40b8ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009bd0 tx=0x7f92e0005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.542+0000 7f92ed600640 1 -- 172.21.15.16:0/3225171016 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f92d80ce570 con 0x7f92f01094a0 2024-09-21T03:44:36.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.743+0000 7f92f7000640 1 -- 172.21.15.16:0/3225171016 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92f01a9280 con 0x7f92f01094a0 2024-09-21T03:44:36.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.745+0000 7f92ed600640 1 -- 172.21.15.16:0/3225171016 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+24704 (secure 0 0 0) 0x7f92d80d2950 con 0x7f92f01094a0 2024-09-21T03:44:36.748 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:44:36.748 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:33:53.220358+0000","modified":"2024-09-21T03:43:54.450112+0000","last_up_change":"2024-09-21T03:42:06.614247+0000","last_in_change":"2024-09-21T03:41:43.379404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:37:55.708915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:43:03.567522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:43:06.048083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:43:16.061356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6a7e4b9e-078f-4ae9-ab9e-37be995e4deb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6803","nonce":311974551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6805","nonce":311974551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6809","nonce":311974551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6807","nonce":311974551}]},"public_addr":"172.21.15.16:6803/311974551","cluster_addr":"172.21.15.16:6805/311974551","heartbeat_back_addr":"172.21.15.16:6809/311974551","heartbeat_front_addr":"172.21.15.16:6807/311974551","state":["exists","up"]},{"osd":1,"uuid":"3a497a64-e1a8-43ef-8509-0a79515da0aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6801","nonce":2152486571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6803","nonce":2152486571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6807","nonce":2152486571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6805","nonce":2152486571}]},"public_addr":"172.21.15.123:6801/2152486571","cluster_addr":"172.21.15.123:6803/2152486571","heartbeat_back_addr":"172.21.15.123:6807/2152486571","heartbeat_front_addr":"172.21.15.123:6805/2152486571","state":["exists","up"]},{"osd":2,"uuid":"1c39518e-6cce-4a0f-9ee8-55c60ad325f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6801","nonce":2812351279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6803","nonce":2812351279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6807","nonce":2812351279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6805","nonce":2812351279}]},"public_addr":"172.21.15.134:6801/2812351279","cluster_addr":"172.21.15.134:6803/2812351279","heartbeat_back_addr":"172.21.15.134:6807/2812351279","heartbeat_front_addr":"172.21.15.134:6805/2812351279","state":["exists","up"]},{"osd":3,"uuid":"0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6811","nonce":2366216026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6813","nonce":2366216026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6817","nonce":2366216026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6815","nonce":2366216026}]},"public_addr":"172.21.15.16:6811/2366216026","cluster_addr":"172.21.15.16:6813/2366216026","heartbeat_back_addr":"172.21.15.16:6817/2366216026","heartbeat_front_addr":"172.21.15.16:6815/2366216026","state":["exists","up"]},{"osd":4,"uuid":"ad5e6d0e-31e8-4779-9e74-f215032777ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6809","nonce":2354279116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6811","nonce":2354279116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6815","nonce":2354279116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6813","nonce":2354279116}]},"public_addr":"172.21.15.123:6809/2354279116","cluster_addr":"172.21.15.123:6811/2354279116","heartbeat_back_addr":"172.21.15.123:6815/2354279116","heartbeat_front_addr":"172.21.15.123:6813/2354279116","state":["exists","up"]},{"osd":5,"uuid":"ca09bf88-1772-4dc0-8711-cbf9a5b31977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6809","nonce":2678416182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6811","nonce":2678416182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6815","nonce":2678416182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6813","nonce":2678416182}]},"public_addr":"172.21.15.134:6809/2678416182","cluster_addr":"172.21.15.134:6811/2678416182","heartbeat_back_addr":"172.21.15.134:6815/2678416182","heartbeat_front_addr":"172.21.15.134:6813/2678416182","state":["exists","up"]},{"osd":6,"uuid":"0d41ed4e-8699-4d88-9254-12d3b84941e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6819","nonce":3377807744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6821","nonce":3377807744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6825","nonce":3377807744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6823","nonce":3377807744}]},"public_addr":"172.21.15.16:6819/3377807744","cluster_addr":"172.21.15.16:6821/3377807744","heartbeat_back_addr":"172.21.15.16:6825/3377807744","heartbeat_front_addr":"172.21.15.16:6823/3377807744","state":["exists","up"]},{"osd":7,"uuid":"1541a272-c524-4f12-9b94-763f69bb1c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6817","nonce":1463412739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6819","nonce":1463412739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6823","nonce":1463412739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6821","nonce":1463412739}]},"public_addr":"172.21.15.123:6817/1463412739","cluster_addr":"172.21.15.123:6819/1463412739","heartbeat_back_addr":"172.21.15.123:6823/1463412739","heartbeat_front_addr":"172.21.15.123:6821/1463412739","state":["exists","up"]},{"osd":8,"uuid":"cc00014c-eb99-49ab-984c-0bc26ef85130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6817","nonce":2644021450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6819","nonce":2644021450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6823","nonce":2644021450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6821","nonce":2644021450}]},"public_addr":"172.21.15.134:6817/2644021450","cluster_addr":"172.21.15.134:6819/2644021450","heartbeat_back_addr":"172.21.15.134:6823/2644021450","heartbeat_front_addr":"172.21.15.134:6821/2644021450","state":["exists","up"]},{"osd":9,"uuid":"3cbfff74-31ff-4ffd-844b-54ec8563ba32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6827","nonce":1653164371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6829","nonce":1653164371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6833","nonce":1653164371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6831","nonce":1653164371}]},"public_addr":"172.21.15.16:6827/1653164371","cluster_addr":"172.21.15.16:6829/1653164371","heartbeat_back_addr":"172.21.15.16:6833/1653164371","heartbeat_front_addr":"172.21.15.16:6831/1653164371","state":["exists","up"]},{"osd":10,"uuid":"0a1caaad-70cd-4c2c-88d3-e9a5cc885655","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6825","nonce":3682234496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6827","nonce":3682234496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6831","nonce":3682234496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6829","nonce":3682234496}]},"public_addr":"172.21.15.123:6825/3682234496","cluster_addr":"172.21.15.123:6827/3682234496","heartbeat_back_addr":"172.21.15.123:6831/3682234496","heartbeat_front_addr":"172.21.15.123:6829/3682234496","state":["exists","up"]},{"osd":11,"uuid":"3aff3258-5a38-41ff-aac3-4a082feffdae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6825","nonce":584758685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6827","nonce":584758685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6831","nonce":584758685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6829","nonce":584758685}]},"public_addr":"172.21.15.134:6825/584758685","cluster_addr":"172.21.15.134:6827/584758685","heartbeat_back_addr":"172.21.15.134:6831/584758685","heartbeat_front_addr":"172.21.15.134:6829/584758685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:02.177731+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:28.847562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:54.272384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:24.338596+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:52.430371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:17.832737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:50.397890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:16.842245+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:42.102933+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:13.074732+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:40.057905+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:04.898190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/557742783":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/3238676548":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6801/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/2357336233":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6801/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6800/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/3806844703":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/2351581621":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6800/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/355731724":"2024-09-22T03:34:22.549156+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:44:36.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.747+0000 7f92f7000640 1 -- 172.21.15.16:0/3225171016 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f92d40b65a0 msgr2=0x7f92d40b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.747+0000 7f92f7000640 1 --2- 172.21.15.16:0/3225171016 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f92d40b65a0 0x7f92d40b8ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009bd0 tx=0x7f92e0005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.747+0000 7f92f7000640 1 -- 172.21.15.16:0/3225171016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f01094a0 msgr2=0x7f92f019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.747+0000 7f92f7000640 1 --2- 172.21.15.16:0/3225171016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f01094a0 0x7f92f019cc90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f92d8005ab0 tx=0x7f92d8003b90 comp rx=0 tx=0).stop 2024-09-21T03:44:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.748+0000 7f92f7000640 1 -- 172.21.15.16:0/3225171016 shutdown_connections 2024-09-21T03:44:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.748+0000 7f92f7000640 1 --2- 172.21.15.16:0/3225171016 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f92d40b65a0 0x7f92d40b8ab0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.748+0000 7f92f7000640 1 --2- 172.21.15.16:0/3225171016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92f0109df0 0x7f92f01a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.748+0000 7f92f7000640 1 --2- 172.21.15.16:0/3225171016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f01094a0 0x7f92f019cc90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.748+0000 7f92f7000640 1 --2- 172.21.15.16:0/3225171016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92f0103b60 0x7f92f019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.748+0000 7f92f7000640 1 -- 172.21.15.16:0/3225171016 >> 172.21.15.16:0/3225171016 conn(0x7f92f007b850 msgr2=0x7f92f0105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.748+0000 7f92f7000640 1 -- 172.21.15.16:0/3225171016 shutdown_connections 2024-09-21T03:44:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:36.748+0000 7f92f7000640 1 -- 172.21.15.16:0/3225171016 wait complete. 2024-09-21T03:44:36.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:36 smithi123 ceph-mon[34673]: pgmap v365: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-21T03:44:36.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:36 smithi016 ceph-mon[30375]: pgmap v365: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-21T03:44:37.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:36 smithi134 ceph-mon[35031]: pgmap v365: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-21T03:44:37.658 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T03:44:37.659 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:44:37.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:37 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3225171016' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:37.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:37 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3225171016' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:38.043 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:38.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:37 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3225171016' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:38.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.680+0000 7f73ea000640 1 -- 172.21.15.16:0/133165173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d80318a0 con 0x7f73ec1092f0 2024-09-21T03:44:38.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.680+0000 7f73f0a00640 1 -- 172.21.15.16:0/133165173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73ec1092f0 msgr2=0x7f73ec109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:38.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.680+0000 7f73f0a00640 1 --2- 172.21.15.16:0/133165173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73ec1092f0 0x7f73ec109700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009900 tx=0x7f73d802a410 comp rx=0 tx=0).stop 2024-09-21T03:44:38.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.680+0000 7f73f0a00640 1 -- 172.21.15.16:0/133165173 shutdown_connections 2024-09-21T03:44:38.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.680+0000 7f73f0a00640 1 --2- 172.21.15.16:0/133165173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73ec109c40 0x7f73ec110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.680+0000 7f73f0a00640 1 --2- 172.21.15.16:0/133165173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73ec1092f0 0x7f73ec109700 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.680+0000 7f73f0a00640 1 --2- 172.21.15.16:0/133165173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ec1039b0 0x7f73ec103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.680+0000 7f73f0a00640 1 -- 172.21.15.16:0/133165173 >> 172.21.15.16:0/133165173 conn(0x7f73ec0ff400 msgr2=0x7f73ec101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:38.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.680+0000 7f73f0a00640 1 -- 172.21.15.16:0/133165173 shutdown_connections 2024-09-21T03:44:38.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.680+0000 7f73f0a00640 1 -- 172.21.15.16:0/133165173 wait complete. 2024-09-21T03:44:38.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.681+0000 7f73f0a00640 1 Processor -- start 2024-09-21T03:44:38.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.682+0000 7f73f0a00640 1 -- start start 2024-09-21T03:44:38.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.682+0000 7f73f0a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ec1039b0 0x7f73ec19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.682+0000 7f73f0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73ec1092f0 0x7f73ec19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.682+0000 7f73f0a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73ec109c40 0x7f73ec19fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.682+0000 7f73f0a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec113950 con 0x7f73ec109c40 2024-09-21T03:44:38.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.682+0000 7f73f0a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1137d0 con 0x7f73ec1092f0 2024-09-21T03:44:38.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.682+0000 7f73f0a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec113ad0 con 0x7f73ec1039b0 2024-09-21T03:44:38.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.682+0000 7f73eb400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ec1039b0 0x7f73ec19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.682+0000 7f73eb400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ec1039b0 0x7f73ec19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:58400/0 (socket says 172.21.15.16:58400) 2024-09-21T03:44:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.682+0000 7f73eb400640 1 -- 172.21.15.16:0/1449309663 learned_addr learned my addr 172.21.15.16:0/1449309663 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.682+0000 7f73ebe00640 1 --2- 172.21.15.16:0/1449309663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73ec109c40 0x7f73ec19fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.683+0000 7f73eaa00640 1 --2- 172.21.15.16:0/1449309663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73ec1092f0 0x7f73ec19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.683+0000 7f73eb400640 1 -- 172.21.15.16:0/1449309663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73ec1092f0 msgr2=0x7f73ec19a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.683+0000 7f73eb400640 1 --2- 172.21.15.16:0/1449309663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73ec1092f0 0x7f73ec19a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.683+0000 7f73eb400640 1 -- 172.21.15.16:0/1449309663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73ec109c40 msgr2=0x7f73ec19fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.683+0000 7f73eb400640 1 --2- 172.21.15.16:0/1449309663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73ec109c40 0x7f73ec19fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.683+0000 7f73eb400640 1 -- 172.21.15.16:0/1449309663 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec1a0270 con 0x7f73ec1039b0 2024-09-21T03:44:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.683+0000 7f73eb400640 1 --2- 172.21.15.16:0/1449309663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ec1039b0 0x7f73ec19a300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00e9f0 tx=0x7f73dc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.684+0000 7f73e3e00640 1 -- 172.21.15.16:0/1449309663 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc00cd80 con 0x7f73ec1039b0 2024-09-21T03:44:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.684+0000 7f73e3e00640 1 -- 172.21.15.16:0/1449309663 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73dc0054d0 con 0x7f73ec1039b0 2024-09-21T03:44:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.684+0000 7f73e3e00640 1 -- 172.21.15.16:0/1449309663 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc010690 con 0x7f73ec1039b0 2024-09-21T03:44:38.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.684+0000 7f73f0a00640 1 -- 172.21.15.16:0/1449309663 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ec1a0560 con 0x7f73ec1039b0 2024-09-21T03:44:38.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.684+0000 7f73f0a00640 1 -- 172.21.15.16:0/1449309663 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ec1a6cc0 con 0x7f73ec1039b0 2024-09-21T03:44:38.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.685+0000 7f73f0a00640 1 -- 172.21.15.16:0/1449309663 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ec103dc0 con 0x7f73ec1039b0 2024-09-21T03:44:38.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.687+0000 7f73e3e00640 1 -- 172.21.15.16:0/1449309663 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f73dc004e90 con 0x7f73ec1039b0 2024-09-21T03:44:38.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.689+0000 7f73e3e00640 1 --2- 172.21.15.16:0/1449309663 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f73d00b6620 0x7f73d00b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.689+0000 7f73e3e00640 1 -- 172.21.15.16:0/1449309663 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f73dc0e7f10 con 0x7f73ec1039b0 2024-09-21T03:44:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.689+0000 7f73eaa00640 1 --2- 172.21.15.16:0/1449309663 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f73d00b6620 0x7f73d00b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.690+0000 7f73eaa00640 1 --2- 172.21.15.16:0/1449309663 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f73d00b6620 0x7f73d00b8b30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009230 tx=0x7f73d8031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:38.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.691+0000 7f73e3e00640 1 -- 172.21.15.16:0/1449309663 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f73dc0ae280 con 0x7f73ec1039b0 2024-09-21T03:44:38.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:38 smithi016 ceph-mon[30375]: pgmap v366: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-21T03:44:38.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.890+0000 7f73f0a00640 1 -- 172.21.15.16:0/1449309663 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73ec19b430 con 0x7f73ec1039b0 2024-09-21T03:44:38.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.892+0000 7f73e3e00640 1 -- 172.21.15.16:0/1449309663 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+24704 (secure 0 0 0) 0x7f73ec19b430 con 0x7f73ec1039b0 2024-09-21T03:44:38.896 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:44:38.896 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:33:53.220358+0000","modified":"2024-09-21T03:43:54.450112+0000","last_up_change":"2024-09-21T03:42:06.614247+0000","last_in_change":"2024-09-21T03:41:43.379404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:37:55.708915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:43:03.567522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:43:06.048083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:43:16.061356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6a7e4b9e-078f-4ae9-ab9e-37be995e4deb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6803","nonce":311974551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6805","nonce":311974551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6809","nonce":311974551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6807","nonce":311974551}]},"public_addr":"172.21.15.16:6803/311974551","cluster_addr":"172.21.15.16:6805/311974551","heartbeat_back_addr":"172.21.15.16:6809/311974551","heartbeat_front_addr":"172.21.15.16:6807/311974551","state":["exists","up"]},{"osd":1,"uuid":"3a497a64-e1a8-43ef-8509-0a79515da0aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6801","nonce":2152486571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6803","nonce":2152486571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6807","nonce":2152486571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6805","nonce":2152486571}]},"public_addr":"172.21.15.123:6801/2152486571","cluster_addr":"172.21.15.123:6803/2152486571","heartbeat_back_addr":"172.21.15.123:6807/2152486571","heartbeat_front_addr":"172.21.15.123:6805/2152486571","state":["exists","up"]},{"osd":2,"uuid":"1c39518e-6cce-4a0f-9ee8-55c60ad325f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6801","nonce":2812351279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6803","nonce":2812351279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6807","nonce":2812351279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6805","nonce":2812351279}]},"public_addr":"172.21.15.134:6801/2812351279","cluster_addr":"172.21.15.134:6803/2812351279","heartbeat_back_addr":"172.21.15.134:6807/2812351279","heartbeat_front_addr":"172.21.15.134:6805/2812351279","state":["exists","up"]},{"osd":3,"uuid":"0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6811","nonce":2366216026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6813","nonce":2366216026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6817","nonce":2366216026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6815","nonce":2366216026}]},"public_addr":"172.21.15.16:6811/2366216026","cluster_addr":"172.21.15.16:6813/2366216026","heartbeat_back_addr":"172.21.15.16:6817/2366216026","heartbeat_front_addr":"172.21.15.16:6815/2366216026","state":["exists","up"]},{"osd":4,"uuid":"ad5e6d0e-31e8-4779-9e74-f215032777ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6809","nonce":2354279116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6811","nonce":2354279116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6815","nonce":2354279116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6813","nonce":2354279116}]},"public_addr":"172.21.15.123:6809/2354279116","cluster_addr":"172.21.15.123:6811/2354279116","heartbeat_back_addr":"172.21.15.123:6815/2354279116","heartbeat_front_addr":"172.21.15.123:6813/2354279116","state":["exists","up"]},{"osd":5,"uuid":"ca09bf88-1772-4dc0-8711-cbf9a5b31977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6809","nonce":2678416182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6811","nonce":2678416182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6815","nonce":2678416182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6813","nonce":2678416182}]},"public_addr":"172.21.15.134:6809/2678416182","cluster_addr":"172.21.15.134:6811/2678416182","heartbeat_back_addr":"172.21.15.134:6815/2678416182","heartbeat_front_addr":"172.21.15.134:6813/2678416182","state":["exists","up"]},{"osd":6,"uuid":"0d41ed4e-8699-4d88-9254-12d3b84941e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6819","nonce":3377807744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6821","nonce":3377807744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6825","nonce":3377807744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6823","nonce":3377807744}]},"public_addr":"172.21.15.16:6819/3377807744","cluster_addr":"172.21.15.16:6821/3377807744","heartbeat_back_addr":"172.21.15.16:6825/3377807744","heartbeat_front_addr":"172.21.15.16:6823/3377807744","state":["exists","up"]},{"osd":7,"uuid":"1541a272-c524-4f12-9b94-763f69bb1c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6817","nonce":1463412739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6819","nonce":1463412739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6823","nonce":1463412739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6821","nonce":1463412739}]},"public_addr":"172.21.15.123:6817/1463412739","cluster_addr":"172.21.15.123:6819/1463412739","heartbeat_back_addr":"172.21.15.123:6823/1463412739","heartbeat_front_addr":"172.21.15.123:6821/1463412739","state":["exists","up"]},{"osd":8,"uuid":"cc00014c-eb99-49ab-984c-0bc26ef85130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6817","nonce":2644021450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6819","nonce":2644021450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6823","nonce":2644021450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6821","nonce":2644021450}]},"public_addr":"172.21.15.134:6817/2644021450","cluster_addr":"172.21.15.134:6819/2644021450","heartbeat_back_addr":"172.21.15.134:6823/2644021450","heartbeat_front_addr":"172.21.15.134:6821/2644021450","state":["exists","up"]},{"osd":9,"uuid":"3cbfff74-31ff-4ffd-844b-54ec8563ba32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6827","nonce":1653164371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6829","nonce":1653164371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6833","nonce":1653164371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6831","nonce":1653164371}]},"public_addr":"172.21.15.16:6827/1653164371","cluster_addr":"172.21.15.16:6829/1653164371","heartbeat_back_addr":"172.21.15.16:6833/1653164371","heartbeat_front_addr":"172.21.15.16:6831/1653164371","state":["exists","up"]},{"osd":10,"uuid":"0a1caaad-70cd-4c2c-88d3-e9a5cc885655","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6825","nonce":3682234496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6827","nonce":3682234496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6831","nonce":3682234496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6829","nonce":3682234496}]},"public_addr":"172.21.15.123:6825/3682234496","cluster_addr":"172.21.15.123:6827/3682234496","heartbeat_back_addr":"172.21.15.123:6831/3682234496","heartbeat_front_addr":"172.21.15.123:6829/3682234496","state":["exists","up"]},{"osd":11,"uuid":"3aff3258-5a38-41ff-aac3-4a082feffdae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6825","nonce":584758685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6827","nonce":584758685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6831","nonce":584758685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6829","nonce":584758685}]},"public_addr":"172.21.15.134:6825/584758685","cluster_addr":"172.21.15.134:6827/584758685","heartbeat_back_addr":"172.21.15.134:6831/584758685","heartbeat_front_addr":"172.21.15.134:6829/584758685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:02.177731+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:28.847562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:54.272384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:24.338596+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:52.430371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:17.832737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:50.397890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:16.842245+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:42.102933+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:13.074732+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:40.057905+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:04.898190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/557742783":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/3238676548":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6801/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/2357336233":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6801/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6800/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/3806844703":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/2351581621":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6800/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/355731724":"2024-09-22T03:34:22.549156+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:44:38.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.895+0000 7f73f0a00640 1 -- 172.21.15.16:0/1449309663 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f73d00b6620 msgr2=0x7f73d00b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:38.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.895+0000 7f73f0a00640 1 --2- 172.21.15.16:0/1449309663 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f73d00b6620 0x7f73d00b8b30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009230 tx=0x7f73d8031400 comp rx=0 tx=0).stop 2024-09-21T03:44:38.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.895+0000 7f73f0a00640 1 -- 172.21.15.16:0/1449309663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ec1039b0 msgr2=0x7f73ec19a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:38.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.895+0000 7f73f0a00640 1 --2- 172.21.15.16:0/1449309663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ec1039b0 0x7f73ec19a300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00e9f0 tx=0x7f73dc00edf0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.895+0000 7f73f0a00640 1 -- 172.21.15.16:0/1449309663 shutdown_connections 2024-09-21T03:44:38.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.895+0000 7f73f0a00640 1 --2- 172.21.15.16:0/1449309663 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f73d00b6620 0x7f73d00b8b30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.895+0000 7f73f0a00640 1 --2- 172.21.15.16:0/1449309663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73ec109c40 0x7f73ec19fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.895+0000 7f73f0a00640 1 --2- 172.21.15.16:0/1449309663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73ec1092f0 0x7f73ec19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.895+0000 7f73f0a00640 1 --2- 172.21.15.16:0/1449309663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ec1039b0 0x7f73ec19a300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:38.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.895+0000 7f73f0a00640 1 -- 172.21.15.16:0/1449309663 >> 172.21.15.16:0/1449309663 conn(0x7f73ec0ff400 msgr2=0x7f73ec101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:38.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.895+0000 7f73f0a00640 1 -- 172.21.15.16:0/1449309663 shutdown_connections 2024-09-21T03:44:38.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:38.895+0000 7f73f0a00640 1 -- 172.21.15.16:0/1449309663 wait complete. 2024-09-21T03:44:38.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:38 smithi123 ceph-mon[34673]: pgmap v366: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-21T03:44:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:38 smithi134 ceph-mon[35031]: pgmap v366: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-21T03:44:39.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T03:44:39.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T03:44:39.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T03:44:39.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T03:44:39.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T03:44:39.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T03:44:39.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T03:44:39.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T03:44:39.663 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T03:44:39.663 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T03:44:39.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T03:44:39.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T03:44:39.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:39 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1449309663' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:39.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:39 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1449309663' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:40.003 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:39 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1449309663' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:40.588 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:40.704 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:40.711 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:40.736 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:40.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:40 smithi123 ceph-mon[34673]: pgmap v367: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T03:44:40.909 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:40 smithi016 ceph-mon[30375]: pgmap v367: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T03:44:41.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:40 smithi134 ceph-mon[35031]: pgmap v367: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T03:44:41.261 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:41.515 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:41.537 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.591+0000 7fecea200640 1 -- 172.21.15.16:0/1158428685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fece4109c40 msgr2=0x7fece4110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.591+0000 7fecea200640 1 --2- 172.21.15.16:0/1158428685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fece4109c40 0x7fece4110470 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009ca0 tx=0x7fecdc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.592+0000 7fecea200640 1 -- 172.21.15.16:0/1158428685 shutdown_connections 2024-09-21T03:44:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.592+0000 7fecea200640 1 --2- 172.21.15.16:0/1158428685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fece4109c40 0x7fece4110470 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.592+0000 7fecea200640 1 --2- 172.21.15.16:0/1158428685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fece41092f0 0x7fece4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.592+0000 7fecea200640 1 --2- 172.21.15.16:0/1158428685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece41039b0 0x7fece4103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.592+0000 7fecea200640 1 -- 172.21.15.16:0/1158428685 >> 172.21.15.16:0/1158428685 conn(0x7fece40ff440 msgr2=0x7fece4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:41.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.592+0000 7fecea200640 1 -- 172.21.15.16:0/1158428685 shutdown_connections 2024-09-21T03:44:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.592+0000 7fecea200640 1 -- 172.21.15.16:0/1158428685 wait complete. 2024-09-21T03:44:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.593+0000 7fecea200640 1 Processor -- start 2024-09-21T03:44:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.593+0000 7fecea200640 1 -- start start 2024-09-21T03:44:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.593+0000 7fecea200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fece41039b0 0x7fece419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.594+0000 7fecea200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece41092f0 0x7fece419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.594+0000 7fecea200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fece4109c40 0x7fece41a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.594+0000 7fecea200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fece4113950 con 0x7fece41039b0 2024-09-21T03:44:41.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.594+0000 7fecea200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fece41137d0 con 0x7fece41092f0 2024-09-21T03:44:41.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.594+0000 7fecea200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fece4113ad0 con 0x7fece4109c40 2024-09-21T03:44:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.594+0000 7fece8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fece41039b0 0x7fece419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.594+0000 7fece8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fece41039b0 0x7fece419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48384/0 (socket says 172.21.15.16:48384) 2024-09-21T03:44:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.594+0000 7fece8e00640 1 -- 172.21.15.16:0/554443806 learned_addr learned my addr 172.21.15.16:0/554443806 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.594+0000 7fece9800640 1 --2- 172.21.15.16:0/554443806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fece4109c40 0x7fece41a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.594+0000 7fece3e00640 1 --2- 172.21.15.16:0/554443806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece41092f0 0x7fece419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.595+0000 7fece3e00640 1 -- 172.21.15.16:0/554443806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fece4109c40 msgr2=0x7fece41a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.595+0000 7fece3e00640 1 --2- 172.21.15.16:0/554443806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fece4109c40 0x7fece41a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.595+0000 7fece3e00640 1 -- 172.21.15.16:0/554443806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fece41039b0 msgr2=0x7fece419c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.595+0000 7fece3e00640 1 --2- 172.21.15.16:0/554443806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fece41039b0 0x7fece419c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.595+0000 7fece3e00640 1 -- 172.21.15.16:0/554443806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece41a2470 con 0x7fece41092f0 2024-09-21T03:44:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.595+0000 7fece3e00640 1 --2- 172.21.15.16:0/554443806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece41092f0 0x7fece419caa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fece406b990 tx=0x7fecd000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:41.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.596+0000 7fece1600640 1 -- 172.21.15.16:0/554443806 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd000cc40 con 0x7fece41092f0 2024-09-21T03:44:41.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.596+0000 7fece1600640 1 -- 172.21.15.16:0/554443806 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecd000cde0 con 0x7fece41092f0 2024-09-21T03:44:41.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.596+0000 7fecea200640 1 -- 172.21.15.16:0/554443806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece41a2760 con 0x7fece41092f0 2024-09-21T03:44:41.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.596+0000 7fece1600640 1 -- 172.21.15.16:0/554443806 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd0004a00 con 0x7fece41092f0 2024-09-21T03:44:41.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.596+0000 7fecea200640 1 -- 172.21.15.16:0/554443806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece41a8ec0 con 0x7fece41092f0 2024-09-21T03:44:41.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.597+0000 7fecea200640 1 -- 172.21.15.16:0/554443806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fece410fa90 con 0x7fece41092f0 2024-09-21T03:44:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.603+0000 7fece1600640 1 -- 172.21.15.16:0/554443806 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fecd0004ba0 con 0x7fece41092f0 2024-09-21T03:44:41.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.605+0000 7fece1600640 1 --2- 172.21.15.16:0/554443806 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7feccc0b66f0 0x7feccc0b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.605+0000 7fece1600640 1 -- 172.21.15.16:0/554443806 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7fecd00e7af0 con 0x7fece41092f0 2024-09-21T03:44:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.605+0000 7fece8e00640 1 --2- 172.21.15.16:0/554443806 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7feccc0b66f0 0x7feccc0b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.605+0000 7fece1600640 1 --2- 172.21.15.16:0/554443806 >> [v2:172.21.15.134:6808/2678416182,v1:172.21.15.134:6809/2678416182] conn(0x7feccc0c1800 0x7feccc0c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.605+0000 7fece1600640 1 -- 172.21.15.16:0/554443806 --> [v2:172.21.15.134:6808/2678416182,v1:172.21.15.134:6809/2678416182] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fecd000be50 con 0x7feccc0c1800 2024-09-21T03:44:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.605+0000 7fece1600640 1 -- 172.21.15.16:0/554443806 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fecd00e7ea0 con 0x7fece41092f0 2024-09-21T03:44:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.606+0000 7fece9800640 1 --2- 172.21.15.16:0/554443806 >> [v2:172.21.15.134:6808/2678416182,v1:172.21.15.134:6809/2678416182] conn(0x7feccc0c1800 0x7feccc0c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.606+0000 7fece8e00640 1 --2- 172.21.15.16:0/554443806 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7feccc0b66f0 0x7feccc0b8c00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fecd80095f0 tx=0x7fecd8005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.606+0000 7fece9800640 1 --2- 172.21.15.16:0/554443806 >> [v2:172.21.15.134:6808/2678416182,v1:172.21.15.134:6809/2678416182] conn(0x7feccc0c1800 0x7feccc0c3cf0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.607+0000 7fece1600640 1 -- 172.21.15.16:0/554443806 <== osd.5 v2:172.21.15.134:6808/2678416182 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fecd000be50 con 0x7feccc0c1800 2024-09-21T03:44:41.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.622+0000 7fecea200640 1 -- 172.21.15.16:0/554443806 --> [v2:172.21.15.134:6808/2678416182,v1:172.21.15.134:6809/2678416182] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fece4104c30 con 0x7feccc0c1800 2024-09-21T03:44:41.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.623+0000 7fece1600640 1 -- 172.21.15.16:0/554443806 <== osd.5 v2:172.21.15.134:6808/2678416182 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fece4104c30 con 0x7feccc0c1800 2024-09-21T03:44:41.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.623+0000 7fecea200640 1 -- 172.21.15.16:0/554443806 >> [v2:172.21.15.134:6808/2678416182,v1:172.21.15.134:6809/2678416182] conn(0x7feccc0c1800 msgr2=0x7feccc0c3cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.623+0000 7fecea200640 1 --2- 172.21.15.16:0/554443806 >> [v2:172.21.15.134:6808/2678416182,v1:172.21.15.134:6809/2678416182] conn(0x7feccc0c1800 0x7feccc0c3cf0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:41.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.623+0000 7fecea200640 1 -- 172.21.15.16:0/554443806 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7feccc0b66f0 msgr2=0x7feccc0b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.623+0000 7fecea200640 1 --2- 172.21.15.16:0/554443806 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7feccc0b66f0 0x7feccc0b8c00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fecd80095f0 tx=0x7fecd8005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:41.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.623+0000 7fecea200640 1 -- 172.21.15.16:0/554443806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece41092f0 msgr2=0x7fece419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:41.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.623+0000 7fecea200640 1 --2- 172.21.15.16:0/554443806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece41092f0 0x7fece419caa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fece406b990 tx=0x7fecd000ee50 comp rx=0 tx=0).stop 2024-09-21T03:44:41.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.623+0000 7fece9800640 1 -- 172.21.15.16:0/554443806 reap_dead start 2024-09-21T03:44:41.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.623+0000 7fecea200640 1 -- 172.21.15.16:0/554443806 shutdown_connections 2024-09-21T03:44:41.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.623+0000 7fecea200640 1 -- 172.21.15.16:0/554443806 >> 172.21.15.16:0/554443806 conn(0x7fece40ff440 msgr2=0x7fece41088d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:41.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.624+0000 7fecea200640 1 -- 172.21.15.16:0/554443806 shutdown_connections 2024-09-21T03:44:41.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:41.624+0000 7fecea200640 1 -- 172.21.15.16:0/554443806 wait complete. 2024-09-21T03:44:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.237+0000 7f0ef3e00640 1 -- 172.21.15.16:0/2552878948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef4076100 msgr2=0x7f0ef410f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.237+0000 7f0ef3e00640 1 --2- 172.21.15.16:0/2552878948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef4076100 0x7f0ef410f9d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009900 tx=0x7f0edc02a400 comp rx=0 tx=0).stop 2024-09-21T03:44:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.237+0000 7f0ef3e00640 1 -- 172.21.15.16:0/2552878948 shutdown_connections 2024-09-21T03:44:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.237+0000 7f0ef3e00640 1 --2- 172.21.15.16:0/2552878948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ef410ff10 0x7f0ef4112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.237+0000 7f0ef3e00640 1 --2- 172.21.15.16:0/2552878948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef4076100 0x7f0ef410f9d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.237+0000 7f0ef3e00640 1 --2- 172.21.15.16:0/2552878948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ef40757b0 0x7f0ef4075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.237+0000 7f0ef3e00640 1 -- 172.21.15.16:0/2552878948 >> 172.21.15.16:0/2552878948 conn(0x7f0ef40ff440 msgr2=0x7f0ef4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.237+0000 7f0ef3e00640 1 -- 172.21.15.16:0/2552878948 shutdown_connections 2024-09-21T03:44:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.237+0000 7f0ef3e00640 1 -- 172.21.15.16:0/2552878948 wait complete. 2024-09-21T03:44:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.238+0000 7f0ef3e00640 1 Processor -- start 2024-09-21T03:44:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.238+0000 7f0ef3e00640 1 -- start start 2024-09-21T03:44:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.238+0000 7f0ef3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ef40757b0 0x7f0ef419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.239+0000 7f0ef3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ef4076100 0x7f0ef419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.239+0000 7f0ef3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef410ff10 0x7f0ef41a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.239+0000 7f0ef3e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4115ac0 con 0x7f0ef40757b0 2024-09-21T03:44:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.239+0000 7f0ef3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4115940 con 0x7f0ef410ff10 2024-09-21T03:44:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.239+0000 7f0ef3e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4115c40 con 0x7f0ef4076100 2024-09-21T03:44:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.239+0000 7f0ef2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ef40757b0 0x7f0ef419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.239+0000 7f0ef2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ef40757b0 0x7f0ef419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60512/0 (socket says 172.21.15.16:60512) 2024-09-21T03:44:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.239+0000 7f0ef2a00640 1 -- 172.21.15.16:0/1110779118 learned_addr learned my addr 172.21.15.16:0/1110779118 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.239+0000 7f0eebe00640 1 --2- 172.21.15.16:0/1110779118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ef4076100 0x7f0ef419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.239+0000 7f0ef3400640 1 --2- 172.21.15.16:0/1110779118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef410ff10 0x7f0ef41a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.239+0000 7f0ef2a00640 1 -- 172.21.15.16:0/1110779118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ef4076100 msgr2=0x7f0ef419cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.239+0000 7f0ef2a00640 1 --2- 172.21.15.16:0/1110779118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ef4076100 0x7f0ef419cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.240+0000 7f0ef2a00640 1 -- 172.21.15.16:0/1110779118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef410ff10 msgr2=0x7f0ef41a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.240+0000 7f0ef2a00640 1 --2- 172.21.15.16:0/1110779118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef410ff10 0x7f0ef41a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.240+0000 7f0ef2a00640 1 -- 172.21.15.16:0/1110779118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef41a2480 con 0x7f0ef40757b0 2024-09-21T03:44:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.240+0000 7f0eebe00640 1 --2- 172.21.15.16:0/1110779118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ef4076100 0x7f0ef419cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.240+0000 7f0ef2a00640 1 --2- 172.21.15.16:0/1110779118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ef40757b0 0x7f0ef419c510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400ea60 tx=0x7f0ee400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.240+0000 7f0eeb400640 1 -- 172.21.15.16:0/1110779118 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400cde0 con 0x7f0ef40757b0 2024-09-21T03:44:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.240+0000 7f0ef3e00640 1 -- 172.21.15.16:0/1110779118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef41a2770 con 0x7f0ef40757b0 2024-09-21T03:44:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.240+0000 7f0eeb400640 1 -- 172.21.15.16:0/1110779118 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee40035f0 con 0x7f0ef40757b0 2024-09-21T03:44:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.240+0000 7f0ef3e00640 1 -- 172.21.15.16:0/1110779118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef41a8ed0 con 0x7f0ef40757b0 2024-09-21T03:44:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.240+0000 7f0eeb400640 1 -- 172.21.15.16:0/1110779118 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4010690 con 0x7f0ef40757b0 2024-09-21T03:44:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.241+0000 7f0ef3e00640 1 -- 172.21.15.16:0/1110779118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ed0001030 con 0x7f0ef40757b0 2024-09-21T03:44:42.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.244+0000 7f0eeb400640 1 -- 172.21.15.16:0/1110779118 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f0ee4003e90 con 0x7f0ef40757b0 2024-09-21T03:44:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.246+0000 7f0eeb400640 1 --2- 172.21.15.16:0/1110779118 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0ed80b6620 0x7f0ed80b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.246+0000 7f0eeb400640 1 -- 172.21.15.16:0/1110779118 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f0ee40e7f70 con 0x7f0ef40757b0 2024-09-21T03:44:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.246+0000 7f0eebe00640 1 --2- 172.21.15.16:0/1110779118 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0ed80b6620 0x7f0ed80b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.246+0000 7f0eeb400640 1 --2- 172.21.15.16:0/1110779118 >> [v2:172.21.15.134:6800/2812351279,v1:172.21.15.134:6801/2812351279] conn(0x7f0ed80c1730 0x7f0ed80c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.246+0000 7f0eeb400640 1 -- 172.21.15.16:0/1110779118 --> [v2:172.21.15.134:6800/2812351279,v1:172.21.15.134:6801/2812351279] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ee4015e80 con 0x7f0ed80c1730 2024-09-21T03:44:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.246+0000 7f0eeb400640 1 -- 172.21.15.16:0/1110779118 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f0ee40e8320 con 0x7f0ef40757b0 2024-09-21T03:44:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.247+0000 7f0ef3400640 1 --2- 172.21.15.16:0/1110779118 >> [v2:172.21.15.134:6800/2812351279,v1:172.21.15.134:6801/2812351279] conn(0x7f0ed80c1730 0x7f0ed80c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.247+0000 7f0eebe00640 1 --2- 172.21.15.16:0/1110779118 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0ed80b6620 0x7f0ed80b8b30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0edc000c00 tx=0x7f0edc031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.247+0000 7f0ef3400640 1 --2- 172.21.15.16:0/1110779118 >> [v2:172.21.15.134:6800/2812351279,v1:172.21.15.134:6801/2812351279] conn(0x7f0ed80c1730 0x7f0ed80c3c20 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.248+0000 7f0eeb400640 1 -- 172.21.15.16:0/1110779118 <== osd.2 v2:172.21.15.134:6800/2812351279 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0ee4015e80 con 0x7f0ed80c1730 2024-09-21T03:44:42.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.264+0000 7f0ef3e00640 1 -- 172.21.15.16:0/1110779118 --> [v2:172.21.15.134:6800/2812351279,v1:172.21.15.134:6801/2812351279] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0ed0002dd0 con 0x7f0ed80c1730 2024-09-21T03:44:42.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.264+0000 7f0eeb400640 1 -- 172.21.15.16:0/1110779118 <== osd.2 v2:172.21.15.134:6800/2812351279 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0ed0002dd0 con 0x7f0ed80c1730 2024-09-21T03:44:42.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.265+0000 7f0ef3e00640 1 -- 172.21.15.16:0/1110779118 >> [v2:172.21.15.134:6800/2812351279,v1:172.21.15.134:6801/2812351279] conn(0x7f0ed80c1730 msgr2=0x7f0ed80c3c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.265+0000 7f0ef3e00640 1 --2- 172.21.15.16:0/1110779118 >> [v2:172.21.15.134:6800/2812351279,v1:172.21.15.134:6801/2812351279] conn(0x7f0ed80c1730 0x7f0ed80c3c20 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.266+0000 7f0ef3e00640 1 -- 172.21.15.16:0/1110779118 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0ed80b6620 msgr2=0x7f0ed80b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.266+0000 7f0ef3e00640 1 --2- 172.21.15.16:0/1110779118 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0ed80b6620 0x7f0ed80b8b30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0edc000c00 tx=0x7f0edc031400 comp rx=0 tx=0).stop 2024-09-21T03:44:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.266+0000 7f0ef3e00640 1 -- 172.21.15.16:0/1110779118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ef40757b0 msgr2=0x7f0ef419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.266+0000 7f0ef3e00640 1 --2- 172.21.15.16:0/1110779118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ef40757b0 0x7f0ef419c510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400ea60 tx=0x7f0ee400ef20 comp rx=0 tx=0).stop 2024-09-21T03:44:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.266+0000 7f0ef3400640 1 -- 172.21.15.16:0/1110779118 reap_dead start 2024-09-21T03:44:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.266+0000 7f0ef3e00640 1 -- 172.21.15.16:0/1110779118 shutdown_connections 2024-09-21T03:44:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.266+0000 7f0ef3e00640 1 -- 172.21.15.16:0/1110779118 >> 172.21.15.16:0/1110779118 conn(0x7f0ef40ff440 msgr2=0x7f0ef4109af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.266+0000 7f0ef3e00640 1 -- 172.21.15.16:0/1110779118 shutdown_connections 2024-09-21T03:44:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.266+0000 7f0ef3e00640 1 -- 172.21.15.16:0/1110779118 wait complete. 2024-09-21T03:44:42.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:42 smithi016 ceph-mon[30375]: pgmap v368: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T03:44:42.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:42 smithi123 ceph-mon[34673]: pgmap v368: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T03:44:42.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.937+0000 7fa33c600640 1 -- 172.21.15.16:0/3083771752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3341094a0 msgr2=0x7fa3341098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.937+0000 7fa33c600640 1 --2- 172.21.15.16:0/3083771752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3341094a0 0x7fa3341098b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa320009900 tx=0x7fa32002a410 comp rx=0 tx=0).stop 2024-09-21T03:44:42.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.937+0000 7fa33c600640 1 -- 172.21.15.16:0/3083771752 shutdown_connections 2024-09-21T03:44:42.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.937+0000 7fa33c600640 1 --2- 172.21.15.16:0/3083771752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa334109df0 0x7fa334110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.937+0000 7fa33c600640 1 --2- 172.21.15.16:0/3083771752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3341094a0 0x7fa3341098b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.937+0000 7fa33c600640 1 --2- 172.21.15.16:0/3083771752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa334103b60 0x7fa334103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.937+0000 7fa33c600640 1 -- 172.21.15.16:0/3083771752 >> 172.21.15.16:0/3083771752 conn(0x7fa33407b850 msgr2=0x7fa33407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:42.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.937+0000 7fa33c600640 1 -- 172.21.15.16:0/3083771752 shutdown_connections 2024-09-21T03:44:42.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.937+0000 7fa33c600640 1 -- 172.21.15.16:0/3083771752 wait complete. 2024-09-21T03:44:42.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.938+0000 7fa33c600640 1 Processor -- start 2024-09-21T03:44:42.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.938+0000 7fa33c600640 1 -- start start 2024-09-21T03:44:42.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.939+0000 7fa33c600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa334103b60 0x7fa334075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.939+0000 7fa33c600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3341094a0 0x7fa334075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.939+0000 7fa33c600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa334109df0 0x7fa334076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.939+0000 7fa33c600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa334113b00 con 0x7fa334103b60 2024-09-21T03:44:42.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.939+0000 7fa33c600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa334113980 con 0x7fa334109df0 2024-09-21T03:44:42.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.939+0000 7fa33c600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa334113c80 con 0x7fa3341094a0 2024-09-21T03:44:42.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.939+0000 7fa339c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa334103b60 0x7fa334075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.939+0000 7fa339c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa334103b60 0x7fa334075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60532/0 (socket says 172.21.15.16:60532) 2024-09-21T03:44:42.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.939+0000 7fa339c00640 1 -- 172.21.15.16:0/4250055120 learned_addr learned my addr 172.21.15.16:0/4250055120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:42.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.939+0000 7fa33a600640 1 --2- 172.21.15.16:0/4250055120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa334109df0 0x7fa334076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.939+0000 7fa339200640 1 --2- 172.21.15.16:0/4250055120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3341094a0 0x7fa334075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.940+0000 7fa33a600640 1 -- 172.21.15.16:0/4250055120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3341094a0 msgr2=0x7fa334075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.940+0000 7fa33a600640 1 --2- 172.21.15.16:0/4250055120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3341094a0 0x7fa334075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.940+0000 7fa33a600640 1 -- 172.21.15.16:0/4250055120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa334103b60 msgr2=0x7fa334075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.940+0000 7fa33a600640 1 --2- 172.21.15.16:0/4250055120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa334103b60 0x7fa334075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.940+0000 7fa33a600640 1 -- 172.21.15.16:0/4250055120 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3341a8790 con 0x7fa334109df0 2024-09-21T03:44:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.940+0000 7fa339c00640 1 --2- 172.21.15.16:0/4250055120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa334103b60 0x7fa334075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.940+0000 7fa33a600640 1 --2- 172.21.15.16:0/4250055120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa334109df0 0x7fa334076190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009fd0 tx=0x7fa32c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.940+0000 7fa332000640 1 -- 172.21.15.16:0/4250055120 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c00ce20 con 0x7fa334109df0 2024-09-21T03:44:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.940+0000 7fa332000640 1 -- 172.21.15.16:0/4250055120 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa32c005020 con 0x7fa334109df0 2024-09-21T03:44:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.940+0000 7fa33c600640 1 -- 172.21.15.16:0/4250055120 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3341a8a80 con 0x7fa334109df0 2024-09-21T03:44:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.940+0000 7fa332000640 1 -- 172.21.15.16:0/4250055120 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c010690 con 0x7fa334109df0 2024-09-21T03:44:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.940+0000 7fa33c600640 1 -- 172.21.15.16:0/4250055120 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3341a8fe0 con 0x7fa334109df0 2024-09-21T03:44:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.941+0000 7fa33c600640 1 -- 172.21.15.16:0/4250055120 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa334063b50 con 0x7fa334109df0 2024-09-21T03:44:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.944+0000 7fa332000640 1 -- 172.21.15.16:0/4250055120 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fa32c0054d0 con 0x7fa334109df0 2024-09-21T03:44:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.946+0000 7fa332000640 1 --2- 172.21.15.16:0/4250055120 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa31c0b6620 0x7fa31c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.946+0000 7fa332000640 1 -- 172.21.15.16:0/4250055120 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7fa32c0e7fe0 con 0x7fa334109df0 2024-09-21T03:44:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.946+0000 7fa339c00640 1 --2- 172.21.15.16:0/4250055120 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa31c0b6620 0x7fa31c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.946+0000 7fa332000640 1 --2- 172.21.15.16:0/4250055120 >> [v2:172.21.15.123:6816/1463412739,v1:172.21.15.123:6817/1463412739] conn(0x7fa31c0c1730 0x7fa31c0c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.946+0000 7fa332000640 1 -- 172.21.15.16:0/4250055120 --> [v2:172.21.15.123:6816/1463412739,v1:172.21.15.123:6817/1463412739] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa32c016e50 con 0x7fa31c0c1730 2024-09-21T03:44:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.946+0000 7fa332000640 1 -- 172.21.15.16:0/4250055120 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fa32c0e8410 con 0x7fa334109df0 2024-09-21T03:44:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.947+0000 7fa339200640 1 --2- 172.21.15.16:0/4250055120 >> [v2:172.21.15.123:6816/1463412739,v1:172.21.15.123:6817/1463412739] conn(0x7fa31c0c1730 0x7fa31c0c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.947+0000 7fa339c00640 1 --2- 172.21.15.16:0/4250055120 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa31c0b6620 0x7fa31c0b8b30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa334077160 tx=0x7fa328005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.947+0000 7fa339200640 1 --2- 172.21.15.16:0/4250055120 >> [v2:172.21.15.123:6816/1463412739,v1:172.21.15.123:6817/1463412739] conn(0x7fa31c0c1730 0x7fa31c0c3c20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.948+0000 7fa332000640 1 -- 172.21.15.16:0/4250055120 <== osd.7 v2:172.21.15.123:6816/1463412739 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa32c016e50 con 0x7fa31c0c1730 2024-09-21T03:44:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.962+0000 7fa33c600640 1 -- 172.21.15.16:0/4250055120 --> [v2:172.21.15.123:6816/1463412739,v1:172.21.15.123:6817/1463412739] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa334063fd0 con 0x7fa31c0c1730 2024-09-21T03:44:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.963+0000 7fa332000640 1 -- 172.21.15.16:0/4250055120 <== osd.7 v2:172.21.15.123:6816/1463412739 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa334063fd0 con 0x7fa31c0c1730 2024-09-21T03:44:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.964+0000 7fa33c600640 1 -- 172.21.15.16:0/4250055120 >> [v2:172.21.15.123:6816/1463412739,v1:172.21.15.123:6817/1463412739] conn(0x7fa31c0c1730 msgr2=0x7fa31c0c3c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.964+0000 7fa33c600640 1 --2- 172.21.15.16:0/4250055120 >> [v2:172.21.15.123:6816/1463412739,v1:172.21.15.123:6817/1463412739] conn(0x7fa31c0c1730 0x7fa31c0c3c20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.964+0000 7fa33c600640 1 -- 172.21.15.16:0/4250055120 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa31c0b6620 msgr2=0x7fa31c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.964+0000 7fa33c600640 1 --2- 172.21.15.16:0/4250055120 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa31c0b6620 0x7fa31c0b8b30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa334077160 tx=0x7fa328005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.964+0000 7fa33c600640 1 -- 172.21.15.16:0/4250055120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa334109df0 msgr2=0x7fa334076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.964+0000 7fa33c600640 1 --2- 172.21.15.16:0/4250055120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa334109df0 0x7fa334076190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009fd0 tx=0x7fa32c00ef10 comp rx=0 tx=0).stop 2024-09-21T03:44:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.964+0000 7fa33a600640 1 -- 172.21.15.16:0/4250055120 reap_dead start 2024-09-21T03:44:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.964+0000 7fa33c600640 1 -- 172.21.15.16:0/4250055120 shutdown_connections 2024-09-21T03:44:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.964+0000 7fa33c600640 1 -- 172.21.15.16:0/4250055120 >> 172.21.15.16:0/4250055120 conn(0x7fa33407b850 msgr2=0x7fa3341054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.964+0000 7fa33c600640 1 -- 172.21.15.16:0/4250055120 shutdown_connections 2024-09-21T03:44:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:42.965+0000 7fa33c600640 1 -- 172.21.15.16:0/4250055120 wait complete. 2024-09-21T03:44:43.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:42 smithi134 ceph-mon[35031]: pgmap v368: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T03:44:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.554+0000 7f1339c00640 1 -- 172.21.15.16:0/4006645253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334103990 msgr2=0x7f1334103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.554+0000 7f1339c00640 1 --2- 172.21.15.16:0/4006645253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334103990 0x7f1334103da0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1324009900 tx=0x7f132402a410 comp rx=0 tx=0).stop 2024-09-21T03:44:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.555+0000 7f1339c00640 1 -- 172.21.15.16:0/4006645253 shutdown_connections 2024-09-21T03:44:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.555+0000 7f1339c00640 1 --2- 172.21.15.16:0/4006645253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1334109c20 0x7f1334110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.555+0000 7f1339c00640 1 --2- 172.21.15.16:0/4006645253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13341092d0 0x7f13341096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.555+0000 7f1339c00640 1 --2- 172.21.15.16:0/4006645253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334103990 0x7f1334103da0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.555+0000 7f1339c00640 1 -- 172.21.15.16:0/4006645253 >> 172.21.15.16:0/4006645253 conn(0x7f13340ff440 msgr2=0x7f1334101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.555+0000 7f1339c00640 1 -- 172.21.15.16:0/4006645253 shutdown_connections 2024-09-21T03:44:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.555+0000 7f1339c00640 1 -- 172.21.15.16:0/4006645253 wait complete. 2024-09-21T03:44:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.556+0000 7f1339c00640 1 Processor -- start 2024-09-21T03:44:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.556+0000 7f1339c00640 1 -- start start 2024-09-21T03:44:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.557+0000 7f1339c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1334103990 0x7f133419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.557+0000 7f1339c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13341092d0 0x7f133419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.557+0000 7f1339c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334109c20 0x7f133419fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.557+0000 7f1339c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13341138e0 con 0x7f1334109c20 2024-09-21T03:44:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.557+0000 7f1339c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1334113760 con 0x7f13341092d0 2024-09-21T03:44:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.557+0000 7f1339c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1334113a60 con 0x7f1334103990 2024-09-21T03:44:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.557+0000 7f1339200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334109c20 0x7f133419fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.557+0000 7f1339200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334109c20 0x7f133419fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60550/0 (socket says 172.21.15.16:60550) 2024-09-21T03:44:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.557+0000 7f1339200640 1 -- 172.21.15.16:0/101896931 learned_addr learned my addr 172.21.15.16:0/101896931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.557+0000 7f1333e00640 1 --2- 172.21.15.16:0/101896931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1334103990 0x7f133419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.557+0000 7f1333400640 1 --2- 172.21.15.16:0/101896931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13341092d0 0x7f133419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.558+0000 7f1339200640 1 -- 172.21.15.16:0/101896931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1334103990 msgr2=0x7f133419a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.558+0000 7f1339200640 1 --2- 172.21.15.16:0/101896931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1334103990 0x7f133419a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.558+0000 7f1339200640 1 -- 172.21.15.16:0/101896931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13341092d0 msgr2=0x7f133419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.558+0000 7f1339200640 1 --2- 172.21.15.16:0/101896931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13341092d0 0x7f133419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.558+0000 7f1339200640 1 -- 172.21.15.16:0/101896931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13341a0240 con 0x7f1334109c20 2024-09-21T03:44:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.558+0000 7f1333e00640 1 --2- 172.21.15.16:0/101896931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1334103990 0x7f133419a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.558+0000 7f1339200640 1 --2- 172.21.15.16:0/101896931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334109c20 0x7f133419fb10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f132c009fd0 tx=0x7f132c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.558+0000 7f1330c00640 1 -- 172.21.15.16:0/101896931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c017070 con 0x7f1334109c20 2024-09-21T03:44:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.558+0000 7f1339c00640 1 -- 172.21.15.16:0/101896931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13341a0530 con 0x7f1334109c20 2024-09-21T03:44:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.558+0000 7f1330c00640 1 -- 172.21.15.16:0/101896931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f132c003e20 con 0x7f1334109c20 2024-09-21T03:44:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.558+0000 7f1330c00640 1 -- 172.21.15.16:0/101896931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c010460 con 0x7f1334109c20 2024-09-21T03:44:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.558+0000 7f1339c00640 1 -- 172.21.15.16:0/101896931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13341a6c90 con 0x7f1334109c20 2024-09-21T03:44:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.559+0000 7f1339c00640 1 -- 172.21.15.16:0/101896931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1334063b50 con 0x7f1334109c20 2024-09-21T03:44:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.562+0000 7f1330c00640 1 -- 172.21.15.16:0/101896931 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f132c003970 con 0x7f1334109c20 2024-09-21T03:44:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.563+0000 7f1330c00640 1 --2- 172.21.15.16:0/101896931 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f13180b65a0 0x7f13180b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.564+0000 7f1330c00640 1 -- 172.21.15.16:0/101896931 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f132c0e7950 con 0x7f1334109c20 2024-09-21T03:44:43.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.564+0000 7f1333e00640 1 --2- 172.21.15.16:0/101896931 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f13180b65a0 0x7f13180b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.564+0000 7f1330c00640 1 --2- 172.21.15.16:0/101896931 >> [v2:172.21.15.134:6816/2644021450,v1:172.21.15.134:6817/2644021450] conn(0x7f13180c16b0 0x7f13180c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:43.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.564+0000 7f1330c00640 1 -- 172.21.15.16:0/101896931 --> [v2:172.21.15.134:6816/2644021450,v1:172.21.15.134:6817/2644021450] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13180c4270 con 0x7f13180c16b0 2024-09-21T03:44:43.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.564+0000 7f1330c00640 1 -- 172.21.15.16:0/101896931 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f132c0e7d00 con 0x7f1334109c20 2024-09-21T03:44:43.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.564+0000 7f1333400640 1 --2- 172.21.15.16:0/101896931 >> [v2:172.21.15.134:6816/2644021450,v1:172.21.15.134:6817/2644021450] conn(0x7f13180c16b0 0x7f13180c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:43.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.565+0000 7f1333e00640 1 --2- 172.21.15.16:0/101896931 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f13180b65a0 0x7f13180b8ab0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1324031440 tx=0x7f1324032ed0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:43.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.565+0000 7f1333400640 1 --2- 172.21.15.16:0/101896931 >> [v2:172.21.15.134:6816/2644021450,v1:172.21.15.134:6817/2644021450] conn(0x7f13180c16b0 0x7f13180c3ba0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:43.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.566+0000 7f1330c00640 1 -- 172.21.15.16:0/101896931 <== osd.8 v2:172.21.15.134:6816/2644021450 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13180c4270 con 0x7f13180c16b0 2024-09-21T03:44:43.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.582+0000 7f1339c00640 1 -- 172.21.15.16:0/101896931 --> [v2:172.21.15.134:6816/2644021450,v1:172.21.15.134:6817/2644021450] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1334063fd0 con 0x7f13180c16b0 2024-09-21T03:44:43.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.582+0000 7f1330c00640 1 -- 172.21.15.16:0/101896931 <== osd.8 v2:172.21.15.134:6816/2644021450 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1334063fd0 con 0x7f13180c16b0 2024-09-21T03:44:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.582+0000 7f1339c00640 1 -- 172.21.15.16:0/101896931 >> [v2:172.21.15.134:6816/2644021450,v1:172.21.15.134:6817/2644021450] conn(0x7f13180c16b0 msgr2=0x7f13180c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.582+0000 7f1339c00640 1 --2- 172.21.15.16:0/101896931 >> [v2:172.21.15.134:6816/2644021450,v1:172.21.15.134:6817/2644021450] conn(0x7f13180c16b0 0x7f13180c3ba0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.582+0000 7f1339c00640 1 -- 172.21.15.16:0/101896931 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f13180b65a0 msgr2=0x7f13180b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.582+0000 7f1339c00640 1 --2- 172.21.15.16:0/101896931 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f13180b65a0 0x7f13180b8ab0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1324031440 tx=0x7f1324032ed0 comp rx=0 tx=0).stop 2024-09-21T03:44:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.582+0000 7f1339c00640 1 -- 172.21.15.16:0/101896931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334109c20 msgr2=0x7f133419fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.582+0000 7f1339c00640 1 --2- 172.21.15.16:0/101896931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334109c20 0x7f133419fb10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f132c009fd0 tx=0x7f132c00b260 comp rx=0 tx=0).stop 2024-09-21T03:44:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.582+0000 7f1339200640 1 -- 172.21.15.16:0/101896931 reap_dead start 2024-09-21T03:44:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.582+0000 7f1339c00640 1 -- 172.21.15.16:0/101896931 shutdown_connections 2024-09-21T03:44:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.582+0000 7f1339c00640 1 -- 172.21.15.16:0/101896931 >> 172.21.15.16:0/101896931 conn(0x7f13340ff440 msgr2=0x7f1334101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.582+0000 7f1339c00640 1 -- 172.21.15.16:0/101896931 shutdown_connections 2024-09-21T03:44:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:43.582+0000 7f1339c00640 1 -- 172.21.15.16:0/101896931 wait complete. 2024-09-21T03:44:44.519 INFO:teuthology.orchestra.run.smithi016.stdout:73014444117 2024-09-21T03:44:44.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T03:44:44.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:44 smithi016 ceph-mon[30375]: pgmap v369: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:44:44.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:44 smithi123 ceph-mon[34673]: pgmap v369: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:44:45.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:44 smithi134 ceph-mon[35031]: pgmap v369: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:44:45.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.828+0000 7f8bc7e00640 1 -- 172.21.15.16:0/1360785069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc81039b0 msgr2=0x7f8bc8103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.828+0000 7f8bc7e00640 1 --2- 172.21.15.16:0/1360785069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc81039b0 0x7f8bc8103dc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009900 tx=0x7f8bb802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.828+0000 7f8bc7e00640 1 -- 172.21.15.16:0/1360785069 shutdown_connections 2024-09-21T03:44:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.828+0000 7f8bc7e00640 1 --2- 172.21.15.16:0/1360785069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc8109c40 0x7f8bc8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.828+0000 7f8bc7e00640 1 --2- 172.21.15.16:0/1360785069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc81092f0 0x7f8bc8109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.828+0000 7f8bc7e00640 1 --2- 172.21.15.16:0/1360785069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc81039b0 0x7f8bc8103dc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.828+0000 7f8bc7e00640 1 -- 172.21.15.16:0/1360785069 >> 172.21.15.16:0/1360785069 conn(0x7f8bc80ff400 msgr2=0x7f8bc8101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.828+0000 7f8bc7e00640 1 -- 172.21.15.16:0/1360785069 shutdown_connections 2024-09-21T03:44:45.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.828+0000 7f8bc7e00640 1 -- 172.21.15.16:0/1360785069 wait complete. 2024-09-21T03:44:45.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.830+0000 7f8bc7e00640 1 Processor -- start 2024-09-21T03:44:45.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.830+0000 7f8bc7e00640 1 -- start start 2024-09-21T03:44:45.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.830+0000 7f8bc7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc81039b0 0x7f8bc819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:45.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.830+0000 7f8bc7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc81092f0 0x7f8bc819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:45.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.830+0000 7f8bc7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc8109c40 0x7f8bc81a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:45.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.830+0000 7f8bc7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8113900 con 0x7f8bc8109c40 2024-09-21T03:44:45.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.830+0000 7f8bc7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8113780 con 0x7f8bc81039b0 2024-09-21T03:44:45.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.830+0000 7f8bc7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8113a80 con 0x7f8bc81092f0 2024-09-21T03:44:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.831+0000 7f8bc7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc8109c40 0x7f8bc81a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.831+0000 7f8bc6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc81039b0 0x7f8bc819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.831+0000 7f8bc7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc8109c40 0x7f8bc81a1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60618/0 (socket says 172.21.15.16:60618) 2024-09-21T03:44:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.831+0000 7f8bc7400640 1 -- 172.21.15.16:0/2026971805 learned_addr learned my addr 172.21.15.16:0/2026971805 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.831+0000 7f8bc6000640 1 --2- 172.21.15.16:0/2026971805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc81092f0 0x7f8bc819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.831+0000 7f8bc6a00640 1 -- 172.21.15.16:0/2026971805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc81092f0 msgr2=0x7f8bc819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.831+0000 7f8bc6a00640 1 --2- 172.21.15.16:0/2026971805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc81092f0 0x7f8bc819ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.831+0000 7f8bc6a00640 1 -- 172.21.15.16:0/2026971805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc8109c40 msgr2=0x7f8bc81a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.831+0000 7f8bc6a00640 1 --2- 172.21.15.16:0/2026971805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc8109c40 0x7f8bc81a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.831+0000 7f8bc6a00640 1 -- 172.21.15.16:0/2026971805 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc81a2440 con 0x7f8bc81039b0 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.831+0000 7f8bc6000640 1 --2- 172.21.15.16:0/2026971805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc81092f0 0x7f8bc819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.831+0000 7f8bc7400640 1 --2- 172.21.15.16:0/2026971805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc8109c40 0x7f8bc81a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.832+0000 7f8bc6a00640 1 --2- 172.21.15.16:0/2026971805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc81039b0 0x7f8bc819c4d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8bb802a910 tx=0x7f8bb8031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.832+0000 7f8bcca00640 1 -- 172.21.15.16:0/2026971805 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb803f030 con 0x7f8bc81039b0 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.832+0000 7f8bc7e00640 1 -- 172.21.15.16:0/2026971805 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc81a26d0 con 0x7f8bc81039b0 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.832+0000 7f8bcca00640 1 -- 172.21.15.16:0/2026971805 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb8004490 con 0x7f8bc81039b0 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.832+0000 7f8bcca00640 1 -- 172.21.15.16:0/2026971805 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8035070 con 0x7f8bc81039b0 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.832+0000 7f8bc7e00640 1 -- 172.21.15.16:0/2026971805 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc81a8e90 con 0x7f8bc81039b0 2024-09-21T03:44:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.833+0000 7f8bc7e00640 1 -- 172.21.15.16:0/2026971805 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8bc810fa70 con 0x7f8bc81039b0 2024-09-21T03:44:45.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.835+0000 7f8bcca00640 1 -- 172.21.15.16:0/2026971805 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f8bb8042030 con 0x7f8bc81039b0 2024-09-21T03:44:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.837+0000 7f8bcca00640 1 --2- 172.21.15.16:0/2026971805 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8bac0b6620 0x7f8bac0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.837+0000 7f8bcca00640 1 -- 172.21.15.16:0/2026971805 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f8bb8108140 con 0x7f8bc81039b0 2024-09-21T03:44:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.837+0000 7f8bc6000640 1 --2- 172.21.15.16:0/2026971805 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8bac0b6620 0x7f8bac0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.837+0000 7f8bcca00640 1 --2- 172.21.15.16:0/2026971805 >> [v2:172.21.15.123:6808/2354279116,v1:172.21.15.123:6809/2354279116] conn(0x7f8bac0c1730 0x7f8bac0c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.837+0000 7f8bcca00640 1 -- 172.21.15.16:0/2026971805 --> [v2:172.21.15.123:6808/2354279116,v1:172.21.15.123:6809/2354279116] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8bb8032970 con 0x7f8bac0c1730 2024-09-21T03:44:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.837+0000 7f8bcca00640 1 -- 172.21.15.16:0/2026971805 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f8bb81084f0 con 0x7f8bc81039b0 2024-09-21T03:44:45.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.838+0000 7f8bc7400640 1 --2- 172.21.15.16:0/2026971805 >> [v2:172.21.15.123:6808/2354279116,v1:172.21.15.123:6809/2354279116] conn(0x7f8bac0c1730 0x7f8bac0c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:45.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.838+0000 7f8bc6000640 1 --2- 172.21.15.16:0/2026971805 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8bac0b6620 0x7f8bac0b8b30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8105a40 tx=0x7f8bb4005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:45.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.838+0000 7f8bc7400640 1 --2- 172.21.15.16:0/2026971805 >> [v2:172.21.15.123:6808/2354279116,v1:172.21.15.123:6809/2354279116] conn(0x7f8bac0c1730 0x7f8bac0c3c20 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.840+0000 7f8bcca00640 1 -- 172.21.15.16:0/2026971805 <== osd.4 v2:172.21.15.123:6808/2354279116 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8bb8032970 con 0x7f8bac0c1730 2024-09-21T03:44:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.854+0000 7f8bc7e00640 1 -- 172.21.15.16:0/2026971805 --> [v2:172.21.15.123:6808/2354279116,v1:172.21.15.123:6809/2354279116] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8bc8104c30 con 0x7f8bac0c1730 2024-09-21T03:44:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.854+0000 7f8bcca00640 1 -- 172.21.15.16:0/2026971805 <== osd.4 v2:172.21.15.123:6808/2354279116 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8bc8104c30 con 0x7f8bac0c1730 2024-09-21T03:44:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.855+0000 7f8bc7e00640 1 -- 172.21.15.16:0/2026971805 >> [v2:172.21.15.123:6808/2354279116,v1:172.21.15.123:6809/2354279116] conn(0x7f8bac0c1730 msgr2=0x7f8bac0c3c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.855+0000 7f8bc7e00640 1 --2- 172.21.15.16:0/2026971805 >> [v2:172.21.15.123:6808/2354279116,v1:172.21.15.123:6809/2354279116] conn(0x7f8bac0c1730 0x7f8bac0c3c20 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.855+0000 7f8bc7e00640 1 -- 172.21.15.16:0/2026971805 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8bac0b6620 msgr2=0x7f8bac0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.855+0000 7f8bc7e00640 1 --2- 172.21.15.16:0/2026971805 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8bac0b6620 0x7f8bac0b8b30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8105a40 tx=0x7f8bb4005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.855+0000 7f8bc7e00640 1 -- 172.21.15.16:0/2026971805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc81039b0 msgr2=0x7f8bc819c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.855+0000 7f8bc7e00640 1 --2- 172.21.15.16:0/2026971805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc81039b0 0x7f8bc819c4d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8bb802a910 tx=0x7f8bb8031a20 comp rx=0 tx=0).stop 2024-09-21T03:44:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.855+0000 7f8bc7400640 1 -- 172.21.15.16:0/2026971805 reap_dead start 2024-09-21T03:44:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.855+0000 7f8bc7e00640 1 -- 172.21.15.16:0/2026971805 shutdown_connections 2024-09-21T03:44:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.855+0000 7f8bc7e00640 1 -- 172.21.15.16:0/2026971805 >> 172.21.15.16:0/2026971805 conn(0x7f8bc80ff400 msgr2=0x7f8bc8108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.855+0000 7f8bc7e00640 1 -- 172.21.15.16:0/2026971805 shutdown_connections 2024-09-21T03:44:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.856+0000 7f8bc7e00640 1 -- 172.21.15.16:0/2026971805 wait complete. 2024-09-21T03:44:46.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.042+0000 7f8c61a00640 1 -- 172.21.15.16:0/676313491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c103b60 msgr2=0x7f8c5c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:46.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.042+0000 7f8c61a00640 1 --2- 172.21.15.16:0/676313491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c103b60 0x7f8c5c103f70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009900 tx=0x7f8c4c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.042+0000 7f8c61a00640 1 -- 172.21.15.16:0/676313491 shutdown_connections 2024-09-21T03:44:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.042+0000 7f8c61a00640 1 --2- 172.21.15.16:0/676313491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c5c109df0 0x7f8c5c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.042+0000 7f8c61a00640 1 --2- 172.21.15.16:0/676313491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c5c1094a0 0x7f8c5c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.042+0000 7f8c61a00640 1 --2- 172.21.15.16:0/676313491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c103b60 0x7f8c5c103f70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.042+0000 7f8c61a00640 1 -- 172.21.15.16:0/676313491 >> 172.21.15.16:0/676313491 conn(0x7f8c5c07b850 msgr2=0x7f8c5c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.042+0000 7f8c61a00640 1 -- 172.21.15.16:0/676313491 shutdown_connections 2024-09-21T03:44:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.042+0000 7f8c61a00640 1 -- 172.21.15.16:0/676313491 wait complete. 2024-09-21T03:44:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.043+0000 7f8c61a00640 1 Processor -- start 2024-09-21T03:44:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.043+0000 7f8c61a00640 1 -- start start 2024-09-21T03:44:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.044+0000 7f8c61a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c103b60 0x7f8c5c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.044+0000 7f8c61a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c5c1094a0 0x7f8c5c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.044+0000 7f8c61a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c5c109df0 0x7f8c5c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:46.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.044+0000 7f8c61a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c113b00 con 0x7f8c5c103b60 2024-09-21T03:44:46.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.044+0000 7f8c61a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c113980 con 0x7f8c5c1094a0 2024-09-21T03:44:46.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.044+0000 7f8c61a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c113c80 con 0x7f8c5c109df0 2024-09-21T03:44:46.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.044+0000 7f8c5aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c103b60 0x7f8c5c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:46.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.044+0000 7f8c5aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c103b60 0x7f8c5c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60640/0 (socket says 172.21.15.16:60640) 2024-09-21T03:44:46.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.044+0000 7f8c5aa00640 1 -- 172.21.15.16:0/1537275073 learned_addr learned my addr 172.21.15.16:0/1537275073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:46.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.044+0000 7f8c53e00640 1 --2- 172.21.15.16:0/1537275073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c5c1094a0 0x7f8c5c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:46.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.044+0000 7f8c5b400640 1 --2- 172.21.15.16:0/1537275073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c5c109df0 0x7f8c5c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:46.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.044+0000 7f8c53e00640 1 -- 172.21.15.16:0/1537275073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c5c109df0 msgr2=0x7f8c5c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:46.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.044+0000 7f8c53e00640 1 --2- 172.21.15.16:0/1537275073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c5c109df0 0x7f8c5c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.044+0000 7f8c53e00640 1 -- 172.21.15.16:0/1537275073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c103b60 msgr2=0x7f8c5c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:46.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.044+0000 7f8c53e00640 1 --2- 172.21.15.16:0/1537275073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c103b60 0x7f8c5c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.044+0000 7f8c53e00640 1 -- 172.21.15.16:0/1537275073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5c1a8770 con 0x7f8c5c1094a0 2024-09-21T03:44:46.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.044+0000 7f8c5b400640 1 --2- 172.21.15.16:0/1537275073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c5c109df0 0x7f8c5c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:46.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.045+0000 7f8c53e00640 1 --2- 172.21.15.16:0/1537275073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c5c1094a0 0x7f8c5c075c50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8c44005fd0 tx=0x7f8c4400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:46.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.045+0000 7f8c53400640 1 -- 172.21.15.16:0/1537275073 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44009c00 con 0x7f8c5c1094a0 2024-09-21T03:44:46.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.045+0000 7f8c61a00640 1 -- 172.21.15.16:0/1537275073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5c1a8a60 con 0x7f8c5c1094a0 2024-09-21T03:44:46.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.045+0000 7f8c53400640 1 -- 172.21.15.16:0/1537275073 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c44009da0 con 0x7f8c5c1094a0 2024-09-21T03:44:46.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.045+0000 7f8c53400640 1 -- 172.21.15.16:0/1537275073 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44003e10 con 0x7f8c5c1094a0 2024-09-21T03:44:46.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.045+0000 7f8c61a00640 1 -- 172.21.15.16:0/1537275073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c5c1a8fa0 con 0x7f8c5c1094a0 2024-09-21T03:44:46.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.046+0000 7f8c61a00640 1 -- 172.21.15.16:0/1537275073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8c5c063b50 con 0x7f8c5c1094a0 2024-09-21T03:44:46.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.050+0000 7f8c53400640 1 -- 172.21.15.16:0/1537275073 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f8c44003fb0 con 0x7f8c5c1094a0 2024-09-21T03:44:46.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.051+0000 7f8c53400640 1 --2- 172.21.15.16:0/1537275073 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8c400b65a0 0x7f8c400b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:46.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.051+0000 7f8c53400640 1 -- 172.21.15.16:0/1537275073 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f8c440e7850 con 0x7f8c5c1094a0 2024-09-21T03:44:46.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.051+0000 7f8c5aa00640 1 --2- 172.21.15.16:0/1537275073 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8c400b65a0 0x7f8c400b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:46.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.052+0000 7f8c53400640 1 --2- 172.21.15.16:0/1537275073 >> [v2:172.21.15.16:6818/3377807744,v1:172.21.15.16:6819/3377807744] conn(0x7f8c400c16b0 0x7f8c400c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.052+0000 7f8c53400640 1 -- 172.21.15.16:0/1537275073 --> [v2:172.21.15.16:6818/3377807744,v1:172.21.15.16:6819/3377807744] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8c44015ec0 con 0x7f8c400c16b0 2024-09-21T03:44:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.052+0000 7f8c53400640 1 -- 172.21.15.16:0/1537275073 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f8c440e7c00 con 0x7f8c5c1094a0 2024-09-21T03:44:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.052+0000 7f8c5b400640 1 --2- 172.21.15.16:0/1537275073 >> [v2:172.21.15.16:6818/3377807744,v1:172.21.15.16:6819/3377807744] conn(0x7f8c400c16b0 0x7f8c400c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.052+0000 7f8c5aa00640 1 --2- 172.21.15.16:0/1537275073 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8c400b65a0 0x7f8c400b8ab0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c005ec0 tx=0x7f8c4c005a20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.053+0000 7f8c5b400640 1 --2- 172.21.15.16:0/1537275073 >> [v2:172.21.15.16:6818/3377807744,v1:172.21.15.16:6819/3377807744] conn(0x7f8c400c16b0 0x7f8c400c3ba0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.053+0000 7f8c53400640 1 -- 172.21.15.16:0/1537275073 <== osd.6 v2:172.21.15.16:6818/3377807744 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8c44015ec0 con 0x7f8c400c16b0 2024-09-21T03:44:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.068+0000 7f8c61a00640 1 -- 172.21.15.16:0/1537275073 --> [v2:172.21.15.16:6818/3377807744,v1:172.21.15.16:6819/3377807744] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8c5c063fd0 con 0x7f8c400c16b0 2024-09-21T03:44:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.068+0000 7f8c53400640 1 -- 172.21.15.16:0/1537275073 <== osd.6 v2:172.21.15.16:6818/3377807744 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8c5c063fd0 con 0x7f8c400c16b0 2024-09-21T03:44:46.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.069+0000 7f8c61a00640 1 -- 172.21.15.16:0/1537275073 >> [v2:172.21.15.16:6818/3377807744,v1:172.21.15.16:6819/3377807744] conn(0x7f8c400c16b0 msgr2=0x7f8c400c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:46.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.069+0000 7f8c61a00640 1 --2- 172.21.15.16:0/1537275073 >> [v2:172.21.15.16:6818/3377807744,v1:172.21.15.16:6819/3377807744] conn(0x7f8c400c16b0 0x7f8c400c3ba0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.069+0000 7f8c61a00640 1 -- 172.21.15.16:0/1537275073 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8c400b65a0 msgr2=0x7f8c400b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:46.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.069+0000 7f8c61a00640 1 --2- 172.21.15.16:0/1537275073 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8c400b65a0 0x7f8c400b8ab0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c005ec0 tx=0x7f8c4c005a20 comp rx=0 tx=0).stop 2024-09-21T03:44:46.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.069+0000 7f8c61a00640 1 -- 172.21.15.16:0/1537275073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c5c1094a0 msgr2=0x7f8c5c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:46.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.069+0000 7f8c61a00640 1 --2- 172.21.15.16:0/1537275073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c5c1094a0 0x7f8c5c075c50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8c44005fd0 tx=0x7f8c4400dde0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.069+0000 7f8c5b400640 1 -- 172.21.15.16:0/1537275073 reap_dead start 2024-09-21T03:44:46.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.069+0000 7f8c61a00640 1 -- 172.21.15.16:0/1537275073 shutdown_connections 2024-09-21T03:44:46.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.069+0000 7f8c61a00640 1 -- 172.21.15.16:0/1537275073 >> 172.21.15.16:0/1537275073 conn(0x7f8c5c07b850 msgr2=0x7f8c5c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:46.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.069+0000 7f8c61a00640 1 -- 172.21.15.16:0/1537275073 shutdown_connections 2024-09-21T03:44:46.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:46.069+0000 7f8c61a00640 1 -- 172.21.15.16:0/1537275073 wait complete. 2024-09-21T03:44:46.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.227+0000 7f1d46200640 1 -- 172.21.15.16:0/860864496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d401092f0 msgr2=0x7f1d40109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:46.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.227+0000 7f1d46200640 1 --2- 172.21.15.16:0/860864496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d401092f0 0x7f1d40109700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009900 tx=0x7f1d2c02a410 comp rx=0 tx=0).stop 2024-09-21T03:44:46.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.227+0000 7f1d46200640 1 -- 172.21.15.16:0/860864496 shutdown_connections 2024-09-21T03:44:46.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.227+0000 7f1d46200640 1 --2- 172.21.15.16:0/860864496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d40109c40 0x7f1d40110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.227+0000 7f1d46200640 1 --2- 172.21.15.16:0/860864496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d401092f0 0x7f1d40109700 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.227+0000 7f1d46200640 1 --2- 172.21.15.16:0/860864496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d401039b0 0x7f1d40103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.227+0000 7f1d46200640 1 -- 172.21.15.16:0/860864496 >> 172.21.15.16:0/860864496 conn(0x7f1d400ff440 msgr2=0x7f1d40101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:46.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.227+0000 7f1d46200640 1 -- 172.21.15.16:0/860864496 shutdown_connections 2024-09-21T03:44:46.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.227+0000 7f1d46200640 1 -- 172.21.15.16:0/860864496 wait complete. 2024-09-21T03:44:46.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.228+0000 7f1d46200640 1 Processor -- start 2024-09-21T03:44:46.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.229+0000 7f1d46200640 1 -- start start 2024-09-21T03:44:46.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.229+0000 7f1d46200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d401039b0 0x7f1d4019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:46.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.229+0000 7f1d46200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d401092f0 0x7f1d4019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.229+0000 7f1d46200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d40109c40 0x7f1d401a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.229+0000 7f1d46200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1d40113900 con 0x7f1d401039b0 2024-09-21T03:44:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.229+0000 7f1d46200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d40113780 con 0x7f1d401092f0 2024-09-21T03:44:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.229+0000 7f1d46200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1d40113a80 con 0x7f1d40109c40 2024-09-21T03:44:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.230+0000 7f1d44e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d401039b0 0x7f1d4019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.230+0000 7f1d44e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d401039b0 0x7f1d4019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60580/0 (socket says 172.21.15.16:60580) 2024-09-21T03:44:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.230+0000 7f1d44e00640 1 -- 172.21.15.16:0/3338663513 learned_addr learned my addr 172.21.15.16:0/3338663513 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.230+0000 7f1d3fe00640 1 --2- 172.21.15.16:0/3338663513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d401092f0 0x7f1d4019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.230+0000 7f1d45800640 1 --2- 172.21.15.16:0/3338663513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d40109c40 0x7f1d401a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.230+0000 7f1d44e00640 1 -- 172.21.15.16:0/3338663513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d40109c40 msgr2=0x7f1d401a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.230+0000 7f1d44e00640 1 --2- 172.21.15.16:0/3338663513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d40109c40 0x7f1d401a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.230+0000 7f1d44e00640 1 -- 172.21.15.16:0/3338663513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d401092f0 msgr2=0x7f1d4019cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.230+0000 7f1d44e00640 1 --2- 172.21.15.16:0/3338663513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d401092f0 0x7f1d4019cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.230+0000 7f1d44e00640 1 -- 172.21.15.16:0/3338663513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d401a2480 con 0x7f1d401039b0 2024-09-21T03:44:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.230+0000 7f1d44e00640 1 --2- 172.21.15.16:0/3338663513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d401039b0 0x7f1d4019c510 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400e9f0 tx=0x7f1d3400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.231+0000 7f1d3d600640 1 -- 172.21.15.16:0/3338663513 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3400cd80 con 0x7f1d401039b0 2024-09-21T03:44:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.231+0000 7f1d3d600640 1 -- 172.21.15.16:0/3338663513 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d340054d0 con 0x7f1d401039b0 2024-09-21T03:44:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.231+0000 7f1d3d600640 1 -- 172.21.15.16:0/3338663513 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34010690 con 0x7f1d401039b0 2024-09-21T03:44:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.231+0000 7f1d46200640 1 -- 172.21.15.16:0/3338663513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d401a2770 con 0x7f1d401039b0 2024-09-21T03:44:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.231+0000 7f1d46200640 1 -- 172.21.15.16:0/3338663513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d401a8ed0 con 0x7f1d401039b0 2024-09-21T03:44:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.232+0000 7f1d46200640 1 -- 172.21.15.16:0/3338663513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1d4010fa90 con 0x7f1d401039b0 2024-09-21T03:44:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.235+0000 7f1d3d600640 1 -- 172.21.15.16:0/3338663513 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f1d34004e30 con 0x7f1d401039b0 2024-09-21T03:44:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.236+0000 7f1d3d600640 1 --2- 172.21.15.16:0/3338663513 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1d280b6620 0x7f1d280b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.236+0000 7f1d3d600640 1 -- 172.21.15.16:0/3338663513 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f1d340e7f80 con 0x7f1d401039b0 2024-09-21T03:44:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.236+0000 7f1d3fe00640 1 --2- 172.21.15.16:0/3338663513 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1d280b6620 0x7f1d280b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.237+0000 7f1d3d600640 1 --2- 172.21.15.16:0/3338663513 >> [v2:172.21.15.123:6800/2152486571,v1:172.21.15.123:6801/2152486571] conn(0x7f1d280c1730 0x7f1d280c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.237+0000 7f1d3d600640 1 -- 172.21.15.16:0/3338663513 --> [v2:172.21.15.123:6800/2152486571,v1:172.21.15.123:6801/2152486571] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1d34015e80 con 0x7f1d280c1730 2024-09-21T03:44:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.237+0000 7f1d3d600640 1 -- 172.21.15.16:0/3338663513 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f1d340e83d0 con 0x7f1d401039b0 2024-09-21T03:44:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.237+0000 7f1d45800640 1 --2- 172.21.15.16:0/3338663513 >> [v2:172.21.15.123:6800/2152486571,v1:172.21.15.123:6801/2152486571] conn(0x7f1d280c1730 0x7f1d280c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:46.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.237+0000 7f1d3fe00640 1 --2- 172.21.15.16:0/3338663513 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1d280b6620 0x7f1d280b8b30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009230 tx=0x7f1d2c031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:46.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.238+0000 7f1d45800640 1 --2- 172.21.15.16:0/3338663513 >> [v2:172.21.15.123:6800/2152486571,v1:172.21.15.123:6801/2152486571] conn(0x7f1d280c1730 0x7f1d280c3c20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:46.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.239+0000 7f1d3d600640 1 -- 172.21.15.16:0/3338663513 <== osd.1 v2:172.21.15.123:6800/2152486571 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1d34015e80 con 0x7f1d280c1730 2024-09-21T03:44:46.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.254+0000 7f1d46200640 1 -- 172.21.15.16:0/3338663513 --> [v2:172.21.15.123:6800/2152486571,v1:172.21.15.123:6801/2152486571] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1d40104c30 con 0x7f1d280c1730 2024-09-21T03:44:46.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.254+0000 7f1d3d600640 1 -- 172.21.15.16:0/3338663513 <== osd.1 v2:172.21.15.123:6800/2152486571 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1d40104c30 con 0x7f1d280c1730 2024-09-21T03:44:46.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.255+0000 7f1d46200640 1 -- 172.21.15.16:0/3338663513 >> [v2:172.21.15.123:6800/2152486571,v1:172.21.15.123:6801/2152486571] conn(0x7f1d280c1730 msgr2=0x7f1d280c3c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:46.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.255+0000 7f1d46200640 1 --2- 172.21.15.16:0/3338663513 >> [v2:172.21.15.123:6800/2152486571,v1:172.21.15.123:6801/2152486571] conn(0x7f1d280c1730 0x7f1d280c3c20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.255+0000 7f1d46200640 1 -- 172.21.15.16:0/3338663513 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1d280b6620 msgr2=0x7f1d280b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:46.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.255+0000 7f1d46200640 1 --2- 172.21.15.16:0/3338663513 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1d280b6620 0x7f1d280b8b30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009230 tx=0x7f1d2c031400 comp rx=0 tx=0).stop 2024-09-21T03:44:46.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.255+0000 7f1d46200640 1 -- 172.21.15.16:0/3338663513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d401039b0 msgr2=0x7f1d4019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:46.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.255+0000 7f1d46200640 1 --2- 172.21.15.16:0/3338663513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d401039b0 0x7f1d4019c510 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400e9f0 tx=0x7f1d3400edf0 comp rx=0 tx=0).stop 2024-09-21T03:44:46.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.255+0000 7f1d45800640 1 -- 172.21.15.16:0/3338663513 reap_dead start 2024-09-21T03:44:46.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.255+0000 7f1d46200640 1 -- 172.21.15.16:0/3338663513 shutdown_connections 2024-09-21T03:44:46.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.255+0000 7f1d46200640 1 -- 172.21.15.16:0/3338663513 >> 172.21.15.16:0/3338663513 conn(0x7f1d400ff440 msgr2=0x7f1d401089a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:46.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.255+0000 7f1d46200640 1 -- 172.21.15.16:0/3338663513 shutdown_connections 2024-09-21T03:44:46.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.255+0000 7f1d46200640 1 -- 172.21.15.16:0/3338663513 wait complete. 2024-09-21T03:44:46.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:46 smithi123 ceph-mon[34673]: pgmap v370: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:44:46.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:46 smithi016 ceph-mon[30375]: pgmap v370: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:44:47.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:46 smithi134 ceph-mon[35031]: pgmap v370: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:44:47.450 INFO:teuthology.orchestra.run.smithi016.stdout:55834574938 2024-09-21T03:44:47.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T03:44:47.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.370+0000 7f5aa6000640 1 -- 172.21.15.16:0/3532175202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa01092f0 msgr2=0x7f5aa0109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.370+0000 7f5aa6000640 1 --2- 172.21.15.16:0/3532175202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa01092f0 0x7f5aa0109700 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009900 tx=0x7f5a8c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.371+0000 7f5aa6000640 1 -- 172.21.15.16:0/3532175202 shutdown_connections 2024-09-21T03:44:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.371+0000 7f5aa6000640 1 --2- 172.21.15.16:0/3532175202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5aa0109c40 0x7f5aa0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.371+0000 7f5aa6000640 1 --2- 172.21.15.16:0/3532175202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa01092f0 0x7f5aa0109700 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.371+0000 7f5aa6000640 1 --2- 172.21.15.16:0/3532175202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5aa01039b0 0x7f5aa0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.371+0000 7f5aa6000640 1 -- 172.21.15.16:0/3532175202 >> 172.21.15.16:0/3532175202 conn(0x7f5aa00ff440 msgr2=0x7f5aa0101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.371+0000 7f5aa6000640 1 -- 172.21.15.16:0/3532175202 shutdown_connections 2024-09-21T03:44:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.371+0000 7f5aa6000640 1 -- 172.21.15.16:0/3532175202 wait complete. 2024-09-21T03:44:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.372+0000 7f5aa6000640 1 Processor -- start 2024-09-21T03:44:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.372+0000 7f5aa6000640 1 -- start start 2024-09-21T03:44:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.372+0000 7f5aa6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5aa01039b0 0x7f5aa019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.372+0000 7f5aa6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5aa01092f0 0x7f5aa019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.372+0000 7f5aa6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa0109c40 0x7f5aa019fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.372+0000 7f5aa6000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0113900 con 0x7f5aa0109c40 2024-09-21T03:44:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.372+0000 7f5aa6000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0113780 con 0x7f5aa01092f0 2024-09-21T03:44:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.372+0000 7f5aa6000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0113a80 con 0x7f5aa01039b0 2024-09-21T03:44:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.373+0000 7f5aa4c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5aa01039b0 0x7f5aa019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.373+0000 7f5a9fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5aa01092f0 0x7f5aa019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.373+0000 7f5aa4c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5aa01039b0 0x7f5aa019a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:51994/0 (socket says 172.21.15.16:51994) 2024-09-21T03:44:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.373+0000 7f5aa5600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa0109c40 0x7f5aa019fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.373+0000 7f5aa4c00640 1 -- 172.21.15.16:0/3079066808 learned_addr learned my addr 172.21.15.16:0/3079066808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.373+0000 7f5a9fe00640 1 -- 172.21.15.16:0/3079066808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5aa01039b0 msgr2=0x7f5aa019a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.373+0000 7f5a9fe00640 1 --2- 172.21.15.16:0/3079066808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5aa01039b0 0x7f5aa019a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.373+0000 7f5a9fe00640 1 -- 172.21.15.16:0/3079066808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa0109c40 msgr2=0x7f5aa019fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.373+0000 7f5a9fe00640 1 --2- 172.21.15.16:0/3079066808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa0109c40 0x7f5aa019fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.373+0000 7f5a9fe00640 1 -- 172.21.15.16:0/3079066808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa01a0280 con 0x7f5aa01092f0 2024-09-21T03:44:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.373+0000 7f5aa4c00640 1 --2- 172.21.15.16:0/3079066808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5aa01039b0 0x7f5aa019a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.373+0000 7f5a9fe00640 1 --2- 172.21.15.16:0/3079066808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5aa01092f0 0x7f5aa019a8b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c02a910 tx=0x7f5a8c031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.374+0000 7f5a9d600640 1 -- 172.21.15.16:0/3079066808 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c03f070 con 0x7f5aa01092f0 2024-09-21T03:44:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.374+0000 7f5aa6000640 1 -- 172.21.15.16:0/3079066808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa01a0510 con 0x7f5aa01092f0 2024-09-21T03:44:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.374+0000 7f5aa6000640 1 -- 172.21.15.16:0/3079066808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa01a6cd0 con 0x7f5aa01092f0 2024-09-21T03:44:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.374+0000 7f5a9d600640 1 -- 172.21.15.16:0/3079066808 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a8c004490 con 0x7f5aa01092f0 2024-09-21T03:44:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.374+0000 7f5a9d600640 1 -- 172.21.15.16:0/3079066808 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c048060 con 0x7f5aa01092f0 2024-09-21T03:44:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.375+0000 7f5aa6000640 1 -- 172.21.15.16:0/3079066808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5aa0063b10 con 0x7f5aa01092f0 2024-09-21T03:44:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.378+0000 7f5a9d600640 1 -- 172.21.15.16:0/3079066808 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f5a8c03b460 con 0x7f5aa01092f0 2024-09-21T03:44:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.379+0000 7f5a9d600640 1 --2- 172.21.15.16:0/3079066808 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5a880b6620 0x7f5a880b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.379+0000 7f5a9d600640 1 -- 172.21.15.16:0/3079066808 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f5a8c107ee0 con 0x7f5aa01092f0 2024-09-21T03:44:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.379+0000 7f5aa4c00640 1 --2- 172.21.15.16:0/3079066808 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5a880b6620 0x7f5a880b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.379+0000 7f5a9d600640 1 --2- 172.21.15.16:0/3079066808 >> [v2:172.21.15.16:6810/2366216026,v1:172.21.15.16:6811/2366216026] conn(0x7f5a880c1730 0x7f5a880c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.380+0000 7f5a9d600640 1 -- 172.21.15.16:0/3079066808 --> [v2:172.21.15.16:6810/2366216026,v1:172.21.15.16:6811/2366216026] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a8c005640 con 0x7f5a880c1730 2024-09-21T03:44:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.380+0000 7f5a9d600640 1 -- 172.21.15.16:0/3079066808 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f5a8c108290 con 0x7f5aa01092f0 2024-09-21T03:44:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.380+0000 7f5aa5600640 1 --2- 172.21.15.16:0/3079066808 >> [v2:172.21.15.16:6810/2366216026,v1:172.21.15.16:6811/2366216026] conn(0x7f5a880c1730 0x7f5a880c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.380+0000 7f5aa4c00640 1 --2- 172.21.15.16:0/3079066808 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5a880b6620 0x7f5a880b8b30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5a940095f0 tx=0x7f5a94005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.380+0000 7f5aa5600640 1 --2- 172.21.15.16:0/3079066808 >> [v2:172.21.15.16:6810/2366216026,v1:172.21.15.16:6811/2366216026] conn(0x7f5a880c1730 0x7f5a880c3c20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.381+0000 7f5a9d600640 1 -- 172.21.15.16:0/3079066808 <== osd.3 v2:172.21.15.16:6810/2366216026 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a8c005640 con 0x7f5a880c1730 2024-09-21T03:44:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.398+0000 7f5aa6000640 1 -- 172.21.15.16:0/3079066808 --> [v2:172.21.15.16:6810/2366216026,v1:172.21.15.16:6811/2366216026] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5aa0063f10 con 0x7f5a880c1730 2024-09-21T03:44:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.398+0000 7f5a9d600640 1 -- 172.21.15.16:0/3079066808 <== osd.3 v2:172.21.15.16:6810/2366216026 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5aa0063f10 con 0x7f5a880c1730 2024-09-21T03:44:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.399+0000 7f5aa6000640 1 -- 172.21.15.16:0/3079066808 >> [v2:172.21.15.16:6810/2366216026,v1:172.21.15.16:6811/2366216026] conn(0x7f5a880c1730 msgr2=0x7f5a880c3c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.399+0000 7f5aa6000640 1 --2- 172.21.15.16:0/3079066808 >> [v2:172.21.15.16:6810/2366216026,v1:172.21.15.16:6811/2366216026] conn(0x7f5a880c1730 0x7f5a880c3c20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.399+0000 7f5aa6000640 1 -- 172.21.15.16:0/3079066808 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5a880b6620 msgr2=0x7f5a880b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.399+0000 7f5aa6000640 1 --2- 172.21.15.16:0/3079066808 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5a880b6620 0x7f5a880b8b30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5a940095f0 tx=0x7f5a94005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:47.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.399+0000 7f5aa6000640 1 -- 172.21.15.16:0/3079066808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5aa01092f0 msgr2=0x7f5aa019a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.399+0000 7f5aa6000640 1 --2- 172.21.15.16:0/3079066808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5aa01092f0 0x7f5aa019a8b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c02a910 tx=0x7f5a8c031a20 comp rx=0 tx=0).stop 2024-09-21T03:44:47.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.399+0000 7f5aa5600640 1 -- 172.21.15.16:0/3079066808 reap_dead start 2024-09-21T03:44:47.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.399+0000 7f5aa6000640 1 -- 172.21.15.16:0/3079066808 shutdown_connections 2024-09-21T03:44:47.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.399+0000 7f5aa6000640 1 -- 172.21.15.16:0/3079066808 >> 172.21.15.16:0/3079066808 conn(0x7f5aa00ff440 msgr2=0x7f5aa0101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:47.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.399+0000 7f5aa6000640 1 -- 172.21.15.16:0/3079066808 shutdown_connections 2024-09-21T03:44:47.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:45.399+0000 7f5aa6000640 1 -- 172.21.15.16:0/3079066808 wait complete. 2024-09-21T03:44:48.608 INFO:teuthology.orchestra.run.smithi016.stdout:219043332146 2024-09-21T03:44:48.608 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T03:44:48.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:48 smithi123 ceph-mon[34673]: pgmap v371: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-21T03:44:48.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:48 smithi016 ceph-mon[30375]: pgmap v371: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-21T03:44:48.956 INFO:teuthology.orchestra.run.smithi016.stdout:103079215182 2024-09-21T03:44:48.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T03:44:49.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:48 smithi134 ceph-mon[35031]: pgmap v371: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-21T03:44:49.557 INFO:teuthology.orchestra.run.smithi016.stdout:197568495672 2024-09-21T03:44:49.558 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T03:44:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:49.907 INFO:teuthology.orchestra.run.smithi016.stdout:150323855427 2024-09-21T03:44:49.908 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T03:44:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:50.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:50 smithi123 ceph-mon[34673]: pgmap v372: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:44:50.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:50 smithi016 ceph-mon[30375]: pgmap v372: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:44:51.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:50 smithi134 ceph-mon[35031]: pgmap v372: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:44:51.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.269+0000 7ff820200640 1 -- 172.21.15.16:0/97695700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff818075730 msgr2=0x7ff818075b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.269+0000 7ff820200640 1 --2- 172.21.15.16:0/97695700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff818075730 0x7ff818075b40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff80c009900 tx=0x7ff80c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.269+0000 7ff820200640 1 -- 172.21.15.16:0/97695700 shutdown_connections 2024-09-21T03:44:51.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.269+0000 7ff820200640 1 --2- 172.21.15.16:0/97695700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff818110160 0x7ff818112580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.269+0000 7ff820200640 1 --2- 172.21.15.16:0/97695700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff818076080 0x7ff81810fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.269+0000 7ff820200640 1 --2- 172.21.15.16:0/97695700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff818075730 0x7ff818075b40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.269+0000 7ff820200640 1 -- 172.21.15.16:0/97695700 >> 172.21.15.16:0/97695700 conn(0x7ff8180ff590 msgr2=0x7ff8181019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:51.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.270+0000 7ff820200640 1 -- 172.21.15.16:0/97695700 shutdown_connections 2024-09-21T03:44:51.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.270+0000 7ff820200640 1 -- 172.21.15.16:0/97695700 wait complete. 2024-09-21T03:44:51.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.271+0000 7ff820200640 1 Processor -- start 2024-09-21T03:44:51.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.271+0000 7ff820200640 1 -- start start 2024-09-21T03:44:51.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.272+0000 7ff820200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff818075730 0x7ff81819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.272+0000 7ff820200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff818076080 0x7ff81819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.272+0000 7ff820200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff818110160 0x7ff8181a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.272+0000 7ff820200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff818115cf0 con 0x7ff818076080 2024-09-21T03:44:51.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.272+0000 7ff820200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff818115b70 con 0x7ff818075730 2024-09-21T03:44:51.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.272+0000 7ff820200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff818115e70 con 0x7ff818110160 2024-09-21T03:44:51.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.272+0000 7ff81d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff818075730 0x7ff81819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.273+0000 7ff81e200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff818110160 0x7ff8181a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.273+0000 7ff81d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff818075730 0x7ff81819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:48732/0 (socket says 172.21.15.16:48732) 2024-09-21T03:44:51.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.273+0000 7ff81d800640 1 -- 172.21.15.16:0/113172658 learned_addr learned my addr 172.21.15.16:0/113172658 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:51.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.273+0000 7ff81ce00640 1 --2- 172.21.15.16:0/113172658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff818076080 0x7ff81819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.273+0000 7ff81e200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff818110160 0x7ff8181a1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52062/0 (socket says 172.21.15.16:52062) 2024-09-21T03:44:51.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.273+0000 7ff81d800640 1 -- 172.21.15.16:0/113172658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff818110160 msgr2=0x7ff8181a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.273+0000 7ff81d800640 1 --2- 172.21.15.16:0/113172658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff818110160 0x7ff8181a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.273+0000 7ff81d800640 1 -- 172.21.15.16:0/113172658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff818076080 msgr2=0x7ff81819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.273+0000 7ff81d800640 1 --2- 172.21.15.16:0/113172658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff818076080 0x7ff81819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.273+0000 7ff81d800640 1 -- 172.21.15.16:0/113172658 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8181079c0 con 0x7ff818075730 2024-09-21T03:44:51.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.273+0000 7ff81ce00640 1 --2- 172.21.15.16:0/113172658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff818076080 0x7ff81819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:51.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.274+0000 7ff81e200640 1 --2- 172.21.15.16:0/113172658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff818110160 0x7ff8181a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:51.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.274+0000 7ff81d800640 1 --2- 172.21.15.16:0/113172658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff818075730 0x7ff81819c6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff80c005ab0 tx=0x7ff80c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:51.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.274+0000 7ff816000640 1 -- 172.21.15.16:0/113172658 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c03f070 con 0x7ff818075730 2024-09-21T03:44:51.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.274+0000 7ff816000640 1 -- 172.21.15.16:0/113172658 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff80c031b60 con 0x7ff818075730 2024-09-21T03:44:51.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.274+0000 7ff816000640 1 -- 172.21.15.16:0/113172658 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c035030 con 0x7ff818075730 2024-09-21T03:44:51.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.275+0000 7ff820200640 1 -- 172.21.15.16:0/113172658 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8181a2500 con 0x7ff818075730 2024-09-21T03:44:51.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.275+0000 7ff820200640 1 -- 172.21.15.16:0/113172658 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8181a2960 con 0x7ff818075730 2024-09-21T03:44:51.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.276+0000 7ff820200640 1 -- 172.21.15.16:0/113172658 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff818076c40 con 0x7ff818075730 2024-09-21T03:44:51.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.279+0000 7ff816000640 1 -- 172.21.15.16:0/113172658 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7ff80c042070 con 0x7ff818075730 2024-09-21T03:44:51.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.280+0000 7ff816000640 1 --2- 172.21.15.16:0/113172658 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff8000b66a0 0x7ff8000b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.280+0000 7ff816000640 1 -- 172.21.15.16:0/113172658 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7ff80c1085c0 con 0x7ff818075730 2024-09-21T03:44:51.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.281+0000 7ff81ce00640 1 --2- 172.21.15.16:0/113172658 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff8000b66a0 0x7ff8000b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.281+0000 7ff816000640 1 --2- 172.21.15.16:0/113172658 >> [v2:172.21.15.134:6824/584758685,v1:172.21.15.134:6825/584758685] conn(0x7ff8000c17b0 0x7ff8000c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.281+0000 7ff816000640 1 -- 172.21.15.16:0/113172658 --> [v2:172.21.15.134:6824/584758685,v1:172.21.15.134:6825/584758685] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff80c03cec0 con 0x7ff8000c17b0 2024-09-21T03:44:51.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.281+0000 7ff816000640 1 -- 172.21.15.16:0/113172658 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7ff80c1089b0 con 0x7ff818075730 2024-09-21T03:44:51.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.281+0000 7ff81e200640 1 --2- 172.21.15.16:0/113172658 >> [v2:172.21.15.134:6824/584758685,v1:172.21.15.134:6825/584758685] conn(0x7ff8000c17b0 0x7ff8000c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.281+0000 7ff81ce00640 1 --2- 172.21.15.16:0/113172658 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff8000b66a0 0x7ff8000b8bb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff818102bc0 tx=0x7ff804005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:51.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.282+0000 7ff81e200640 1 --2- 172.21.15.16:0/113172658 >> [v2:172.21.15.134:6824/584758685,v1:172.21.15.134:6825/584758685] conn(0x7ff8000c17b0 0x7ff8000c3ca0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:51.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.283+0000 7ff816000640 1 -- 172.21.15.16:0/113172658 <== osd.11 v2:172.21.15.134:6824/584758685 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff80c03cec0 con 0x7ff8000c17b0 2024-09-21T03:44:51.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.297+0000 7ff820200640 1 -- 172.21.15.16:0/113172658 --> [v2:172.21.15.134:6824/584758685,v1:172.21.15.134:6825/584758685] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff818109670 con 0x7ff8000c17b0 2024-09-21T03:44:51.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.298+0000 7ff816000640 1 -- 172.21.15.16:0/113172658 <== osd.11 v2:172.21.15.134:6824/584758685 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff818109670 con 0x7ff8000c17b0 2024-09-21T03:44:51.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.298+0000 7ff820200640 1 -- 172.21.15.16:0/113172658 >> [v2:172.21.15.134:6824/584758685,v1:172.21.15.134:6825/584758685] conn(0x7ff8000c17b0 msgr2=0x7ff8000c3ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.298+0000 7ff820200640 1 --2- 172.21.15.16:0/113172658 >> [v2:172.21.15.134:6824/584758685,v1:172.21.15.134:6825/584758685] conn(0x7ff8000c17b0 0x7ff8000c3ca0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.298+0000 7ff820200640 1 -- 172.21.15.16:0/113172658 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff8000b66a0 msgr2=0x7ff8000b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.298+0000 7ff820200640 1 --2- 172.21.15.16:0/113172658 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff8000b66a0 0x7ff8000b8bb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff818102bc0 tx=0x7ff804005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:51.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.298+0000 7ff820200640 1 -- 172.21.15.16:0/113172658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff818075730 msgr2=0x7ff81819c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.298+0000 7ff820200640 1 --2- 172.21.15.16:0/113172658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff818075730 0x7ff81819c6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff80c005ab0 tx=0x7ff80c003c10 comp rx=0 tx=0).stop 2024-09-21T03:44:51.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.298+0000 7ff81e200640 1 -- 172.21.15.16:0/113172658 reap_dead start 2024-09-21T03:44:51.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.298+0000 7ff820200640 1 -- 172.21.15.16:0/113172658 shutdown_connections 2024-09-21T03:44:51.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.298+0000 7ff820200640 1 -- 172.21.15.16:0/113172658 >> 172.21.15.16:0/113172658 conn(0x7ff8180ff590 msgr2=0x7ff818107cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:51.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.299+0000 7ff820200640 1 -- 172.21.15.16:0/113172658 shutdown_connections 2024-09-21T03:44:51.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.299+0000 7ff820200640 1 -- 172.21.15.16:0/113172658 wait complete. 2024-09-21T03:44:51.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.549+0000 7f42baa00640 1 -- 172.21.15.16:0/149575773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42b4078290 msgr2=0x7f42b4114200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.549+0000 7f42baa00640 1 --2- 172.21.15.16:0/149575773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42b4078290 0x7f42b4114200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f429c009900 tx=0x7f429c02a400 comp rx=0 tx=0).stop 2024-09-21T03:44:51.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.549+0000 7f42baa00640 1 -- 172.21.15.16:0/149575773 shutdown_connections 2024-09-21T03:44:51.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.549+0000 7f42baa00640 1 --2- 172.21.15.16:0/149575773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42b4114740 0x7f42b4116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.549+0000 7f42baa00640 1 --2- 172.21.15.16:0/149575773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42b4078290 0x7f42b4114200 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.549+0000 7f42baa00640 1 --2- 172.21.15.16:0/149575773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42b4077940 0x7f42b4077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.549+0000 7f42baa00640 1 -- 172.21.15.16:0/149575773 >> 172.21.15.16:0/149575773 conn(0x7f42b4101810 msgr2=0x7f42b4103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.549+0000 7f42baa00640 1 -- 172.21.15.16:0/149575773 shutdown_connections 2024-09-21T03:44:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.549+0000 7f42baa00640 1 -- 172.21.15.16:0/149575773 wait complete. 2024-09-21T03:44:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.551+0000 7f42baa00640 1 Processor -- start 2024-09-21T03:44:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.551+0000 7f42baa00640 1 -- start start 2024-09-21T03:44:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.551+0000 7f42baa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42b4077940 0x7f42b41a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.551+0000 7f42baa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42b4078290 0x7f42b41a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.551+0000 7f42baa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42b4114740 0x7f42b41a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.551+0000 7f42baa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42b411a070 con 0x7f42b4114740 2024-09-21T03:44:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.551+0000 7f42baa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42b4119ef0 con 0x7f42b4077940 2024-09-21T03:44:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.551+0000 7f42baa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f42b411a1f0 con 0x7f42b4078290 2024-09-21T03:44:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.552+0000 7f42abe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42b4078290 0x7f42b41a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.552+0000 7f42b8a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42b4114740 0x7f42b41a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.552+0000 7f42abe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42b4078290 0x7f42b41a1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52072/0 (socket says 172.21.15.16:52072) 2024-09-21T03:44:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.552+0000 7f42abe00640 1 -- 172.21.15.16:0/3519780481 learned_addr learned my addr 172.21.15.16:0/3519780481 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.552+0000 7f42b3e00640 1 --2- 172.21.15.16:0/3519780481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42b4077940 0x7f42b41a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.552+0000 7f42b8a00640 1 -- 172.21.15.16:0/3519780481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42b4078290 msgr2=0x7f42b41a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.552+0000 7f42b8a00640 1 --2- 172.21.15.16:0/3519780481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42b4078290 0x7f42b41a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.552+0000 7f42b8a00640 1 -- 172.21.15.16:0/3519780481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42b4077940 msgr2=0x7f42b41a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.552+0000 7f42b8a00640 1 --2- 172.21.15.16:0/3519780481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42b4077940 0x7f42b41a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.552+0000 7f42b8a00640 1 -- 172.21.15.16:0/3519780481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b41a6a30 con 0x7f42b4114740 2024-09-21T03:44:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.552+0000 7f42abe00640 1 --2- 172.21.15.16:0/3519780481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42b4078290 0x7f42b41a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.552+0000 7f42b3e00640 1 --2- 172.21.15.16:0/3519780481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42b4077940 0x7f42b41a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.553+0000 7f42b8a00640 1 --2- 172.21.15.16:0/3519780481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42b4114740 0x7f42b41a6300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009fd0 tx=0x7f42ac00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.553+0000 7f42b1600640 1 -- 172.21.15.16:0/3519780481 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00ce20 con 0x7f42b4114740 2024-09-21T03:44:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.553+0000 7f42baa00640 1 -- 172.21.15.16:0/3519780481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42b41a6d20 con 0x7f42b4114740 2024-09-21T03:44:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.553+0000 7f42b1600640 1 -- 172.21.15.16:0/3519780481 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ac005020 con 0x7f42b4114740 2024-09-21T03:44:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.553+0000 7f42b1600640 1 -- 172.21.15.16:0/3519780481 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac010690 con 0x7f42b4114740 2024-09-21T03:44:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.553+0000 7f42baa00640 1 -- 172.21.15.16:0/3519780481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42b41ad480 con 0x7f42b4114740 2024-09-21T03:44:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.554+0000 7f42baa00640 1 -- 172.21.15.16:0/3519780481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f42b4063ad0 con 0x7f42b4114740 2024-09-21T03:44:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.557+0000 7f42b1600640 1 -- 172.21.15.16:0/3519780481 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f42ac0054d0 con 0x7f42b4114740 2024-09-21T03:44:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.559+0000 7f42b1600640 1 --2- 172.21.15.16:0/3519780481 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42980b65a0 0x7f42980b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.560+0000 7f42b1600640 1 -- 172.21.15.16:0/3519780481 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f42ac0e8090 con 0x7f42b4114740 2024-09-21T03:44:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.560+0000 7f42b3e00640 1 --2- 172.21.15.16:0/3519780481 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42980b65a0 0x7f42980b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.560+0000 7f42b1600640 1 --2- 172.21.15.16:0/3519780481 >> [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371] conn(0x7f42980c16b0 0x7f42980c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.560+0000 7f42b1600640 1 -- 172.21.15.16:0/3519780481 --> [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f42ac016e50 con 0x7f42980c16b0 2024-09-21T03:44:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.560+0000 7f42b1600640 1 -- 172.21.15.16:0/3519780481 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f42ac0e8440 con 0x7f42b4114740 2024-09-21T03:44:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.561+0000 7f42abe00640 1 --2- 172.21.15.16:0/3519780481 >> [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371] conn(0x7f42980c16b0 0x7f42980c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.561+0000 7f42b3e00640 1 --2- 172.21.15.16:0/3519780481 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42980b65a0 0x7f42980b8ab0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f42a40095f0 tx=0x7f42a4005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.562+0000 7f42abe00640 1 --2- 172.21.15.16:0/3519780481 >> [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371] conn(0x7f42980c16b0 0x7f42980c3ba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.563+0000 7f42b1600640 1 -- 172.21.15.16:0/3519780481 <== osd.9 v2:172.21.15.16:6826/1653164371 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42ac016e50 con 0x7f42980c16b0 2024-09-21T03:44:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.578+0000 7f42baa00640 1 -- 172.21.15.16:0/3519780481 --> [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f42b4063f50 con 0x7f42980c16b0 2024-09-21T03:44:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.579+0000 7f42b1600640 1 -- 172.21.15.16:0/3519780481 <== osd.9 v2:172.21.15.16:6826/1653164371 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f42b4063f50 con 0x7f42980c16b0 2024-09-21T03:44:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.579+0000 7f42baa00640 1 -- 172.21.15.16:0/3519780481 >> [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371] conn(0x7f42980c16b0 msgr2=0x7f42980c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.579+0000 7f42baa00640 1 --2- 172.21.15.16:0/3519780481 >> [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371] conn(0x7f42980c16b0 0x7f42980c3ba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.579+0000 7f42baa00640 1 -- 172.21.15.16:0/3519780481 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42980b65a0 msgr2=0x7f42980b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.579+0000 7f42baa00640 1 --2- 172.21.15.16:0/3519780481 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f42980b65a0 0x7f42980b8ab0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f42a40095f0 tx=0x7f42a4005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:51.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.579+0000 7f42baa00640 1 -- 172.21.15.16:0/3519780481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42b4114740 msgr2=0x7f42b41a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.579+0000 7f42baa00640 1 --2- 172.21.15.16:0/3519780481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42b4114740 0x7f42b41a6300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009fd0 tx=0x7f42ac00ef10 comp rx=0 tx=0).stop 2024-09-21T03:44:51.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.579+0000 7f42b8a00640 1 -- 172.21.15.16:0/3519780481 reap_dead start 2024-09-21T03:44:51.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.579+0000 7f42baa00640 1 -- 172.21.15.16:0/3519780481 shutdown_connections 2024-09-21T03:44:51.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.579+0000 7f42baa00640 1 -- 172.21.15.16:0/3519780481 >> 172.21.15.16:0/3519780481 conn(0x7f42b4101810 msgr2=0x7f42b4116840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:51.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.580+0000 7f42baa00640 1 -- 172.21.15.16:0/3519780481 shutdown_connections 2024-09-21T03:44:51.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.580+0000 7f42baa00640 1 -- 172.21.15.16:0/3519780481 wait complete. 2024-09-21T03:44:52.750 INFO:teuthology.orchestra.run.smithi016.stdout:244813135918 2024-09-21T03:44:52.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T03:44:52.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.165+0000 7f2b87e00640 1 -- 172.21.15.16:0/3311577244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b881092f0 msgr2=0x7f2b88109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:52.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.165+0000 7f2b87e00640 1 --2- 172.21.15.16:0/3311577244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b881092f0 0x7f2b88109700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009900 tx=0x7f2b7402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:44:52.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.165+0000 7f2b87e00640 1 -- 172.21.15.16:0/3311577244 shutdown_connections 2024-09-21T03:44:52.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.165+0000 7f2b87e00640 1 --2- 172.21.15.16:0/3311577244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b88109c40 0x7f2b88110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:52.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.165+0000 7f2b87e00640 1 --2- 172.21.15.16:0/3311577244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b881092f0 0x7f2b88109700 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:52.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.165+0000 7f2b87e00640 1 --2- 172.21.15.16:0/3311577244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b881039b0 0x7f2b88103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:52.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.165+0000 7f2b87e00640 1 -- 172.21.15.16:0/3311577244 >> 172.21.15.16:0/3311577244 conn(0x7f2b880ff440 msgr2=0x7f2b88101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:52.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.165+0000 7f2b87e00640 1 -- 172.21.15.16:0/3311577244 shutdown_connections 2024-09-21T03:44:52.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.165+0000 7f2b87e00640 1 -- 172.21.15.16:0/3311577244 wait complete. 2024-09-21T03:44:52.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.166+0000 7f2b87e00640 1 Processor -- start 2024-09-21T03:44:52.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.166+0000 7f2b87e00640 1 -- start start 2024-09-21T03:44:52.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.167+0000 7f2b87e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b881039b0 0x7f2b8819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:52.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.167+0000 7f2b87e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b881092f0 0x7f2b8819caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:52.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.167+0000 7f2b87e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b88109c40 0x7f2b881a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:52.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.167+0000 7f2b87e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b88113900 con 0x7f2b881092f0 2024-09-21T03:44:52.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.167+0000 7f2b87e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b88113780 con 0x7f2b88109c40 2024-09-21T03:44:52.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.167+0000 7f2b87e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b88113a80 con 0x7f2b881039b0 2024-09-21T03:44:52.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.167+0000 7f2b86000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b881092f0 0x7f2b8819caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:52.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.167+0000 7f2b86a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b881039b0 0x7f2b8819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:52.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.167+0000 7f2b86a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b881039b0 0x7f2b8819c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52046/0 (socket says 172.21.15.16:52046) 2024-09-21T03:44:52.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.167+0000 7f2b86a00640 1 -- 172.21.15.16:0/3014935784 learned_addr learned my addr 172.21.15.16:0/3014935784 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:52.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.167+0000 7f2b87400640 1 --2- 172.21.15.16:0/3014935784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b88109c40 0x7f2b881a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:52.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.168+0000 7f2b86a00640 1 -- 172.21.15.16:0/3014935784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b88109c40 msgr2=0x7f2b881a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:52.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.168+0000 7f2b86a00640 1 --2- 172.21.15.16:0/3014935784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b88109c40 0x7f2b881a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:52.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.168+0000 7f2b86a00640 1 -- 172.21.15.16:0/3014935784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b881092f0 msgr2=0x7f2b8819caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:52.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.168+0000 7f2b86a00640 1 --2- 172.21.15.16:0/3014935784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b881092f0 0x7f2b8819caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:52.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.168+0000 7f2b86a00640 1 -- 172.21.15.16:0/3014935784 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b881a24c0 con 0x7f2b881039b0 2024-09-21T03:44:52.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.168+0000 7f2b86a00640 1 --2- 172.21.15.16:0/3014935784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b881039b0 0x7f2b8819c550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800d950 tx=0x7f2b7800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:52.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.168+0000 7f2b7f400640 1 -- 172.21.15.16:0/3014935784 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b78017070 con 0x7f2b881039b0 2024-09-21T03:44:52.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.168+0000 7f2b87e00640 1 -- 172.21.15.16:0/3014935784 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b881a27b0 con 0x7f2b881039b0 2024-09-21T03:44:52.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.168+0000 7f2b7f400640 1 -- 172.21.15.16:0/3014935784 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b78009c30 con 0x7f2b881039b0 2024-09-21T03:44:52.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.168+0000 7f2b7f400640 1 -- 172.21.15.16:0/3014935784 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7800f620 con 0x7f2b881039b0 2024-09-21T03:44:52.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.169+0000 7f2b87e00640 1 -- 172.21.15.16:0/3014935784 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b881a8f10 con 0x7f2b881039b0 2024-09-21T03:44:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.169+0000 7f2b87e00640 1 -- 172.21.15.16:0/3014935784 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2b8810fa90 con 0x7f2b881039b0 2024-09-21T03:44:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.172+0000 7f2b7f400640 1 -- 172.21.15.16:0/3014935784 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f2b78010460 con 0x7f2b881039b0 2024-09-21T03:44:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.173+0000 7f2b7f400640 1 --2- 172.21.15.16:0/3014935784 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2b6c0b6620 0x7f2b6c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.174+0000 7f2b7f400640 1 -- 172.21.15.16:0/3014935784 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9092+0+0 (secure 0 0 0) 0x7f2b780e77a0 con 0x7f2b881039b0 2024-09-21T03:44:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.174+0000 7f2b86000640 1 --2- 172.21.15.16:0/3014935784 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2b6c0b6620 0x7f2b6c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.174+0000 7f2b7f400640 1 --2- 172.21.15.16:0/3014935784 >> [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551] conn(0x7f2b6c0c1730 0x7f2b6c0c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.174+0000 7f2b7f400640 1 -- 172.21.15.16:0/3014935784 --> [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2b78024ea0 con 0x7f2b6c0c1730 2024-09-21T03:44:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.174+0000 7f2b7f400640 1 -- 172.21.15.16:0/3014935784 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f2b780e7b50 con 0x7f2b881039b0 2024-09-21T03:44:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.174+0000 7f2b87400640 1 --2- 172.21.15.16:0/3014935784 >> [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551] conn(0x7f2b6c0c1730 0x7f2b6c0c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.175+0000 7f2b86000640 1 --2- 172.21.15.16:0/3014935784 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2b6c0b6620 0x7f2b6c0b8b30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009230 tx=0x7f2b74031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.175+0000 7f2b87400640 1 --2- 172.21.15.16:0/3014935784 >> [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551] conn(0x7f2b6c0c1730 0x7f2b6c0c3c20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.176+0000 7f2b7f400640 1 -- 172.21.15.16:0/3014935784 <== osd.0 v2:172.21.15.16:6802/311974551 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b78024ea0 con 0x7f2b6c0c1730 2024-09-21T03:44:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.191+0000 7f2b87e00640 1 -- 172.21.15.16:0/3014935784 --> [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2b88104c30 con 0x7f2b6c0c1730 2024-09-21T03:44:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.191+0000 7f2b7f400640 1 -- 172.21.15.16:0/3014935784 <== osd.0 v2:172.21.15.16:6802/311974551 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2b88104c30 con 0x7f2b6c0c1730 2024-09-21T03:44:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.192+0000 7f2b87e00640 1 -- 172.21.15.16:0/3014935784 >> [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551] conn(0x7f2b6c0c1730 msgr2=0x7f2b6c0c3c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.192+0000 7f2b87e00640 1 --2- 172.21.15.16:0/3014935784 >> [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551] conn(0x7f2b6c0c1730 0x7f2b6c0c3c20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.192+0000 7f2b87e00640 1 -- 172.21.15.16:0/3014935784 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2b6c0b6620 msgr2=0x7f2b6c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.192+0000 7f2b87e00640 1 --2- 172.21.15.16:0/3014935784 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2b6c0b6620 0x7f2b6c0b8b30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009230 tx=0x7f2b74031400 comp rx=0 tx=0).stop 2024-09-21T03:44:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.192+0000 7f2b87e00640 1 -- 172.21.15.16:0/3014935784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b881039b0 msgr2=0x7f2b8819c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.192+0000 7f2b87e00640 1 --2- 172.21.15.16:0/3014935784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b881039b0 0x7f2b8819c550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800d950 tx=0x7f2b7800de10 comp rx=0 tx=0).stop 2024-09-21T03:44:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.192+0000 7f2b87400640 1 -- 172.21.15.16:0/3014935784 reap_dead start 2024-09-21T03:44:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.192+0000 7f2b87e00640 1 -- 172.21.15.16:0/3014935784 shutdown_connections 2024-09-21T03:44:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.192+0000 7f2b87e00640 1 -- 172.21.15.16:0/3014935784 >> 172.21.15.16:0/3014935784 conn(0x7f2b880ff440 msgr2=0x7f2b881089f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.192+0000 7f2b87e00640 1 -- 172.21.15.16:0/3014935784 shutdown_connections 2024-09-21T03:44:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:51.193+0000 7f2b87e00640 1 -- 172.21.15.16:0/3014935784 wait complete. 2024-09-21T03:44:52.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:52 smithi123 ceph-mon[34673]: pgmap v373: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:44:52.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:52 smithi016 ceph-mon[30375]: pgmap v373: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:44:53.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:52 smithi134 ceph-mon[35031]: pgmap v373: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:44:53.557 INFO:teuthology.orchestra.run.smithi016.stdout:128849018954 2024-09-21T03:44:53.558 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T03:44:53.814 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]} : dispatch 2024-09-21T03:44:53.814 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-21T03:44:53.814 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-21T03:44:53.856 INFO:teuthology.orchestra.run.smithi016.stdout:34359738464 2024-09-21T03:44:53.856 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T03:44:53.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]} : dispatch 2024-09-21T03:44:53.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-21T03:44:53.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-21T03:44:54.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]} : dispatch 2024-09-21T03:44:54.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-21T03:44:54.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-21T03:44:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.220+0000 7f46f6600640 1 -- 172.21.15.16:0/3249923567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46f0075dd0 msgr2=0x7f46f00761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.220+0000 7f46f6600640 1 --2- 172.21.15.16:0/3249923567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46f0075dd0 0x7f46f00761e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009900 tx=0x7f46dc02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.220+0000 7f46f6600640 1 -- 172.21.15.16:0/3249923567 shutdown_connections 2024-09-21T03:44:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.220+0000 7f46f6600640 1 --2- 172.21.15.16:0/3249923567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46f0114280 0x7f46f0116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.220+0000 7f46f6600640 1 --2- 172.21.15.16:0/3249923567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46f0075dd0 0x7f46f00761e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.220+0000 7f46f6600640 1 --2- 172.21.15.16:0/3249923567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46f0077380 0x7f46f00757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.220+0000 7f46f6600640 1 -- 172.21.15.16:0/3249923567 >> 172.21.15.16:0/3249923567 conn(0x7f46f00ff440 msgr2=0x7f46f0101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.221+0000 7f46f6600640 1 -- 172.21.15.16:0/3249923567 shutdown_connections 2024-09-21T03:44:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.221+0000 7f46f6600640 1 -- 172.21.15.16:0/3249923567 wait complete. 2024-09-21T03:44:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.222+0000 7f46f6600640 1 Processor -- start 2024-09-21T03:44:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.222+0000 7f46f6600640 1 -- start start 2024-09-21T03:44:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.222+0000 7f46f6600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46f0075dd0 0x7f46f01a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.222+0000 7f46f6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46f0077380 0x7f46f01a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.222+0000 7f46f6600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46f0114280 0x7f46f01a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.222+0000 7f46f6600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f46f0119ec0 con 0x7f46f0077380 2024-09-21T03:44:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.222+0000 7f46f6600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46f0119d40 con 0x7f46f0114280 2024-09-21T03:44:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.222+0000 7f46f6600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f46f011a040 con 0x7f46f0075dd0 2024-09-21T03:44:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.222+0000 7f46efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46f0077380 0x7f46f01a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.223+0000 7f46efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46f0077380 0x7f46f01a0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37194/0 (socket says 172.21.15.16:37194) 2024-09-21T03:44:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.223+0000 7f46efe00640 1 -- 172.21.15.16:0/1327546098 learned_addr learned my addr 172.21.15.16:0/1327546098 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.223+0000 7f46f5200640 1 --2- 172.21.15.16:0/1327546098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46f0075dd0 0x7f46f01a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.223+0000 7f46f5c00640 1 --2- 172.21.15.16:0/1327546098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46f0114280 0x7f46f01a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.223+0000 7f46efe00640 1 -- 172.21.15.16:0/1327546098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46f0075dd0 msgr2=0x7f46f01a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.223+0000 7f46efe00640 1 --2- 172.21.15.16:0/1327546098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46f0075dd0 0x7f46f01a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.223+0000 7f46efe00640 1 -- 172.21.15.16:0/1327546098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46f0114280 msgr2=0x7f46f01a6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.223+0000 7f46efe00640 1 --2- 172.21.15.16:0/1327546098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46f0114280 0x7f46f01a6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.223+0000 7f46efe00640 1 -- 172.21.15.16:0/1327546098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f01a6860 con 0x7f46f0077380 2024-09-21T03:44:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.223+0000 7f46efe00640 1 --2- 172.21.15.16:0/1327546098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46f0077380 0x7f46f01a0e90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f46dc032fd0 tx=0x7f46dc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.224+0000 7f46ed600640 1 -- 172.21.15.16:0/1327546098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc03f030 con 0x7f46f0077380 2024-09-21T03:44:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.224+0000 7f46f6600640 1 -- 172.21.15.16:0/1327546098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f01a6af0 con 0x7f46f0077380 2024-09-21T03:44:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.224+0000 7f46f6600640 1 -- 172.21.15.16:0/1327546098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f01ad2b0 con 0x7f46f0077380 2024-09-21T03:44:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.224+0000 7f46ed600640 1 -- 172.21.15.16:0/1327546098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46dc004490 con 0x7f46f0077380 2024-09-21T03:44:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.224+0000 7f46ed600640 1 -- 172.21.15.16:0/1327546098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc035070 con 0x7f46f0077380 2024-09-21T03:44:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.225+0000 7f46e2a00640 1 -- 172.21.15.16:0/1327546098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f46f0063ad0 con 0x7f46f0077380 2024-09-21T03:44:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.227+0000 7f46ed600640 1 -- 172.21.15.16:0/1327546098 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f46dc042030 con 0x7f46f0077380 2024-09-21T03:44:54.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.229+0000 7f46ed600640 1 --2- 172.21.15.16:0/1327546098 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f46d80b6620 0x7f46d80b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.229+0000 7f46ed600640 1 -- 172.21.15.16:0/1327546098 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9179+0+0 (secure 0 0 0) 0x7f46dc108160 con 0x7f46f0077380 2024-09-21T03:44:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.229+0000 7f46ed600640 1 --2- 172.21.15.16:0/1327546098 >> [v2:172.21.15.123:6824/3682234496,v1:172.21.15.123:6825/3682234496] conn(0x7f46d80c18b0 0x7f46d80c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.229+0000 7f46ed600640 1 -- 172.21.15.16:0/1327546098 --> [v2:172.21.15.123:6824/3682234496,v1:172.21.15.123:6825/3682234496] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f46dc032970 con 0x7f46d80c18b0 2024-09-21T03:44:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.229+0000 7f46f5200640 1 --2- 172.21.15.16:0/1327546098 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f46d80b6620 0x7f46d80b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.229+0000 7f46ed600640 1 -- 172.21.15.16:0/1327546098 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f46dc108510 con 0x7f46f0077380 2024-09-21T03:44:54.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.230+0000 7f46f5c00640 1 --2- 172.21.15.16:0/1327546098 >> [v2:172.21.15.123:6824/3682234496,v1:172.21.15.123:6825/3682234496] conn(0x7f46d80c18b0 0x7f46d80c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:54.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.230+0000 7f46f5200640 1 --2- 172.21.15.16:0/1327546098 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f46d80b6620 0x7f46d80b8b30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f46e4005fd0 tx=0x7f46e400b390 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.231+0000 7f46f5c00640 1 --2- 172.21.15.16:0/1327546098 >> [v2:172.21.15.123:6824/3682234496,v1:172.21.15.123:6825/3682234496] conn(0x7f46d80c18b0 0x7f46d80c3da0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:54.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.231+0000 7f46ed600640 1 -- 172.21.15.16:0/1327546098 <== osd.10 v2:172.21.15.123:6824/3682234496 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f46dc032970 con 0x7f46d80c18b0 2024-09-21T03:44:54.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.246+0000 7f46e2a00640 1 -- 172.21.15.16:0/1327546098 --> [v2:172.21.15.123:6824/3682234496,v1:172.21.15.123:6825/3682234496] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f46f0063ed0 con 0x7f46d80c18b0 2024-09-21T03:44:54.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.246+0000 7f46ed600640 1 -- 172.21.15.16:0/1327546098 <== osd.10 v2:172.21.15.123:6824/3682234496 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f46f0063ed0 con 0x7f46d80c18b0 2024-09-21T03:44:54.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.247+0000 7f46e2a00640 1 -- 172.21.15.16:0/1327546098 >> [v2:172.21.15.123:6824/3682234496,v1:172.21.15.123:6825/3682234496] conn(0x7f46d80c18b0 msgr2=0x7f46d80c3da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.247+0000 7f46e2a00640 1 --2- 172.21.15.16:0/1327546098 >> [v2:172.21.15.123:6824/3682234496,v1:172.21.15.123:6825/3682234496] conn(0x7f46d80c18b0 0x7f46d80c3da0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:54.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.247+0000 7f46e2a00640 1 -- 172.21.15.16:0/1327546098 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f46d80b6620 msgr2=0x7f46d80b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.247+0000 7f46e2a00640 1 --2- 172.21.15.16:0/1327546098 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f46d80b6620 0x7f46d80b8b30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f46e4005fd0 tx=0x7f46e400b390 comp rx=0 tx=0).stop 2024-09-21T03:44:54.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.247+0000 7f46e2a00640 1 -- 172.21.15.16:0/1327546098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46f0077380 msgr2=0x7f46f01a0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:54.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.247+0000 7f46e2a00640 1 --2- 172.21.15.16:0/1327546098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46f0077380 0x7f46f01a0e90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f46dc032fd0 tx=0x7f46dc004270 comp rx=0 tx=0).stop 2024-09-21T03:44:54.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.247+0000 7f46f5c00640 1 -- 172.21.15.16:0/1327546098 reap_dead start 2024-09-21T03:44:54.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.247+0000 7f46e2a00640 1 -- 172.21.15.16:0/1327546098 shutdown_connections 2024-09-21T03:44:54.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.247+0000 7f46e2a00640 1 -- 172.21.15.16:0/1327546098 >> 172.21.15.16:0/1327546098 conn(0x7f46f00ff440 msgr2=0x7f46f0110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:54.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.247+0000 7f46e2a00640 1 -- 172.21.15.16:0/1327546098 shutdown_connections 2024-09-21T03:44:54.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:54.247+0000 7f46e2a00640 1 -- 172.21.15.16:0/1327546098 wait complete. 2024-09-21T03:44:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:54 smithi016 ceph-mon[30375]: pgmap v374: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:44:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-21T03:44:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-21T03:44:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:54 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-21T03:44:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:54 smithi016 ceph-mon[30375]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:44:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:54 smithi134 ceph-mon[35031]: pgmap v374: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:44:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-21T03:44:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-21T03:44:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:54 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-21T03:44:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:54 smithi134 ceph-mon[35031]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:44:55.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:54 smithi123 ceph-mon[34673]: pgmap v374: 129 pgs: 129 active+clean; 649 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:44:55.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-21T03:44:55.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-21T03:44:55.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:54 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-21T03:44:55.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:54 smithi123 ceph-mon[34673]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:44:55.234 INFO:teuthology.orchestra.run.smithi016.stdout:266287972394 2024-09-21T03:44:55.234 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T03:44:55.573 INFO:teuthology.orchestra.run.smithi016.stdout:176093659198 2024-09-21T03:44:55.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T03:44:55.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:55 smithi016 ceph-mon[30375]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:44:55.993 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:56.038 INFO:teuthology.orchestra.run.smithi016.stdout:287762808868 2024-09-21T03:44:56.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T03:44:56.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:55 smithi134 ceph-mon[35031]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:44:56.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:55 smithi123 ceph-mon[34673]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:44:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.750+0000 7fe0f1600640 1 -- 172.21.15.16:0/155755343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0ec10de30 msgr2=0x7fe0ec110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.750+0000 7fe0f1600640 1 --2- 172.21.15.16:0/155755343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0ec10de30 0x7fe0ec110320 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009ca0 tx=0x7fe0e402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.750+0000 7fe0f1600640 1 -- 172.21.15.16:0/155755343 shutdown_connections 2024-09-21T03:44:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.750+0000 7fe0f1600640 1 --2- 172.21.15.16:0/155755343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0ec10de30 0x7fe0ec110320 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.750+0000 7fe0f1600640 1 --2- 172.21.15.16:0/155755343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0ec105320 0x7fe0ec10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.750+0000 7fe0f1600640 1 --2- 172.21.15.16:0/155755343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0ec1049d0 0x7fe0ec104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.750+0000 7fe0f1600640 1 -- 172.21.15.16:0/155755343 >> 172.21.15.16:0/155755343 conn(0x7fe0ec079740 msgr2=0x7fe0ec079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.751+0000 7fe0f1600640 1 -- 172.21.15.16:0/155755343 shutdown_connections 2024-09-21T03:44:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.751+0000 7fe0f1600640 1 -- 172.21.15.16:0/155755343 wait complete. 2024-09-21T03:44:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.752+0000 7fe0f1600640 1 Processor -- start 2024-09-21T03:44:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.752+0000 7fe0f1600640 1 -- start start 2024-09-21T03:44:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.753+0000 7fe0f1600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0ec1049d0 0x7fe0ec19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.753+0000 7fe0f1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0ec105320 0x7fe0ec19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.753+0000 7fe0f1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0ec10de30 0x7fe0ec1a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.753+0000 7fe0f1600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec113ae0 con 0x7fe0ec10de30 2024-09-21T03:44:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.753+0000 7fe0f1600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec113960 con 0x7fe0ec105320 2024-09-21T03:44:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.753+0000 7fe0f1600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec113c60 con 0x7fe0ec1049d0 2024-09-21T03:44:56.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.753+0000 7fe0eaa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0ec1049d0 0x7fe0ec19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.754+0000 7fe0eaa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0ec1049d0 0x7fe0ec19c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:58132/0 (socket says 172.21.15.16:58132) 2024-09-21T03:44:56.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.754+0000 7fe0eaa00640 1 -- 172.21.15.16:0/2905730711 learned_addr learned my addr 172.21.15.16:0/2905730711 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:56.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.754+0000 7fe0eb400640 1 --2- 172.21.15.16:0/2905730711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0ec10de30 0x7fe0ec1a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.754+0000 7fe0eaa00640 1 -- 172.21.15.16:0/2905730711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0ec105320 msgr2=0x7fe0ec19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.754+0000 7fe0eaa00640 1 --2- 172.21.15.16:0/2905730711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0ec105320 0x7fe0ec19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.754+0000 7fe0eaa00640 1 -- 172.21.15.16:0/2905730711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0ec10de30 msgr2=0x7fe0ec1a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.754+0000 7fe0eaa00640 1 --2- 172.21.15.16:0/2905730711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0ec10de30 0x7fe0ec1a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.754+0000 7fe0eaa00640 1 -- 172.21.15.16:0/2905730711 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ec1a2600 con 0x7fe0ec1049d0 2024-09-21T03:44:56.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.754+0000 7fe0eaa00640 1 --2- 172.21.15.16:0/2905730711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0ec1049d0 0x7fe0ec19c690 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00e9c0 tx=0x7fe0dc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:56.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.755+0000 7fe0e3400640 1 -- 172.21.15.16:0/2905730711 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc00cd80 con 0x7fe0ec1049d0 2024-09-21T03:44:56.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.756+0000 7fe0e3400640 1 -- 172.21.15.16:0/2905730711 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0dc0035f0 con 0x7fe0ec1049d0 2024-09-21T03:44:56.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.756+0000 7fe0e3400640 1 -- 172.21.15.16:0/2905730711 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc010690 con 0x7fe0ec1049d0 2024-09-21T03:44:56.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.756+0000 7fe0f1600640 1 -- 172.21.15.16:0/2905730711 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ec1a28f0 con 0x7fe0ec1049d0 2024-09-21T03:44:56.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.756+0000 7fe0f1600640 1 -- 172.21.15.16:0/2905730711 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ec1a9050 con 0x7fe0ec1049d0 2024-09-21T03:44:56.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.757+0000 7fe0f1600640 1 -- 172.21.15.16:0/2905730711 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0ec06c240 con 0x7fe0ec1049d0 2024-09-21T03:44:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.762+0000 7fe0e3400640 1 -- 172.21.15.16:0/2905730711 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fe0dc004000 con 0x7fe0ec1049d0 2024-09-21T03:44:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.764+0000 7fe0e3400640 1 --2- 172.21.15.16:0/2905730711 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe0d00b66a0 0x7fe0d00b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.765+0000 7fe0e3400640 1 -- 172.21.15.16:0/2905730711 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fe0dc0e8020 con 0x7fe0ec1049d0 2024-09-21T03:44:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.765+0000 7fe0e3400640 1 -- 172.21.15.16:0/2905730711 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe0dc0e8440 con 0x7fe0ec1049d0 2024-09-21T03:44:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.765+0000 7fe0e3e00640 1 --2- 172.21.15.16:0/2905730711 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe0d00b66a0 0x7fe0d00b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:56.767+0000 7fe0e3e00640 1 --2- 172.21.15.16:0/2905730711 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe0d00b66a0 0x7fe0d00b8bb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec10fa80 tx=0x7fe0d4009590 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:56.786 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:56.787 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:56.789 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:56.822 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:56.845 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:56.887 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:56.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:56 smithi016 ceph-mon[30375]: pgmap v377: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:44:56.963 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:57.083+0000 7fe0f1600640 1 -- 172.21.15.16:0/2905730711 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe0ec104de0 con 0x7fe0ec1049d0 2024-09-21T03:44:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:57.084+0000 7fe0e3400640 1 -- 172.21.15.16:0/2905730711 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe0dc0ae340 con 0x7fe0ec1049d0 2024-09-21T03:44:57.086 INFO:teuthology.orchestra.run.smithi016.stdout:55834574940 2024-09-21T03:44:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:57.086+0000 7fe0f1600640 1 -- 172.21.15.16:0/2905730711 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe0d00b66a0 msgr2=0x7fe0d00b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:57.086+0000 7fe0f1600640 1 --2- 172.21.15.16:0/2905730711 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe0d00b66a0 0x7fe0d00b8bb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec10fa80 tx=0x7fe0d4009590 comp rx=0 tx=0).stop 2024-09-21T03:44:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:57.086+0000 7fe0f1600640 1 -- 172.21.15.16:0/2905730711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0ec1049d0 msgr2=0x7fe0ec19c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:57.086+0000 7fe0f1600640 1 --2- 172.21.15.16:0/2905730711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0ec1049d0 0x7fe0ec19c690 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00e9c0 tx=0x7fe0dc00ee80 comp rx=0 tx=0).stop 2024-09-21T03:44:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:57.087+0000 7fe0f1600640 1 -- 172.21.15.16:0/2905730711 shutdown_connections 2024-09-21T03:44:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:57.087+0000 7fe0f1600640 1 --2- 172.21.15.16:0/2905730711 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe0d00b66a0 0x7fe0d00b8bb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:57.087+0000 7fe0f1600640 1 --2- 172.21.15.16:0/2905730711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0ec10de30 0x7fe0ec1a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:57.087+0000 7fe0f1600640 1 --2- 172.21.15.16:0/2905730711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0ec105320 0x7fe0ec19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:57.087+0000 7fe0f1600640 1 --2- 172.21.15.16:0/2905730711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0ec1049d0 0x7fe0ec19c690 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:57.087+0000 7fe0f1600640 1 -- 172.21.15.16:0/2905730711 >> 172.21.15.16:0/2905730711 conn(0x7fe0ec079740 msgr2=0x7fe0ec10cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:57.087+0000 7fe0f1600640 1 -- 172.21.15.16:0/2905730711 shutdown_connections 2024-09-21T03:44:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:57.087+0000 7fe0f1600640 1 -- 172.21.15.16:0/2905730711 wait complete. 2024-09-21T03:44:57.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:56 smithi134 ceph-mon[35031]: pgmap v377: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:44:57.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:56 smithi123 ceph-mon[34673]: pgmap v377: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:44:57.703 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:57 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2905730711' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:44:58.112 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:57 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2905730711' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:44:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.140+0000 7ff36f200640 1 -- 172.21.15.16:0/2269108454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3701092f0 msgr2=0x7ff370109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.140+0000 7ff36f200640 1 --2- 172.21.15.16:0/2269108454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3701092f0 0x7ff370109700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff35c009900 tx=0x7ff35c02a410 comp rx=0 tx=0).stop 2024-09-21T03:44:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.141+0000 7ff36f200640 1 -- 172.21.15.16:0/2269108454 shutdown_connections 2024-09-21T03:44:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.141+0000 7ff36f200640 1 --2- 172.21.15.16:0/2269108454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff370109c40 0x7ff370110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.141+0000 7ff36f200640 1 --2- 172.21.15.16:0/2269108454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3701092f0 0x7ff370109700 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.141+0000 7ff36f200640 1 --2- 172.21.15.16:0/2269108454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3701039b0 0x7ff370103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.141+0000 7ff36f200640 1 -- 172.21.15.16:0/2269108454 >> 172.21.15.16:0/2269108454 conn(0x7ff3700ff400 msgr2=0x7ff370101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.141+0000 7ff36f200640 1 -- 172.21.15.16:0/2269108454 shutdown_connections 2024-09-21T03:44:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.141+0000 7ff36f200640 1 -- 172.21.15.16:0/2269108454 wait complete. 2024-09-21T03:44:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.142+0000 7ff36f200640 1 Processor -- start 2024-09-21T03:44:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.142+0000 7ff36f200640 1 -- start start 2024-09-21T03:44:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.142+0000 7ff36f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3701039b0 0x7ff37019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.142+0000 7ff36f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3701092f0 0x7ff37019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.142+0000 7ff36f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff370109c40 0x7ff37019fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.142+0000 7ff36f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff370113900 con 0x7ff3701039b0 2024-09-21T03:44:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.142+0000 7ff36f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff370113780 con 0x7ff3701092f0 2024-09-21T03:44:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.142+0000 7ff36f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff370113a80 con 0x7ff370109c40 2024-09-21T03:44:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.143+0000 7ff36de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3701039b0 0x7ff37019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.143+0000 7ff36de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3701039b0 0x7ff37019a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37234/0 (socket says 172.21.15.16:37234) 2024-09-21T03:44:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.143+0000 7ff36de00640 1 -- 172.21.15.16:0/2077555005 learned_addr learned my addr 172.21.15.16:0/2077555005 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.143+0000 7ff36d400640 1 --2- 172.21.15.16:0/2077555005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3701092f0 0x7ff37019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.143+0000 7ff36e800640 1 --2- 172.21.15.16:0/2077555005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff370109c40 0x7ff37019fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.143+0000 7ff36de00640 1 -- 172.21.15.16:0/2077555005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff370109c40 msgr2=0x7ff37019fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.143+0000 7ff36de00640 1 --2- 172.21.15.16:0/2077555005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff370109c40 0x7ff37019fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.143+0000 7ff36de00640 1 -- 172.21.15.16:0/2077555005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3701092f0 msgr2=0x7ff37019a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.143+0000 7ff36de00640 1 --2- 172.21.15.16:0/2077555005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3701092f0 0x7ff37019a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.143+0000 7ff36de00640 1 -- 172.21.15.16:0/2077555005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3701a0280 con 0x7ff3701039b0 2024-09-21T03:44:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.143+0000 7ff36d400640 1 --2- 172.21.15.16:0/2077555005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3701092f0 0x7ff37019a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.143+0000 7ff36de00640 1 --2- 172.21.15.16:0/2077555005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3701039b0 0x7ff37019a310 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff36000e9f0 tx=0x7ff36000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.144+0000 7ff366a00640 1 -- 172.21.15.16:0/2077555005 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36000cd80 con 0x7ff3701039b0 2024-09-21T03:44:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.144+0000 7ff366a00640 1 -- 172.21.15.16:0/2077555005 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3600108e0 con 0x7ff3701039b0 2024-09-21T03:44:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.144+0000 7ff36f200640 1 -- 172.21.15.16:0/2077555005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3701a0570 con 0x7ff3701039b0 2024-09-21T03:44:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.144+0000 7ff366a00640 1 -- 172.21.15.16:0/2077555005 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360018980 con 0x7ff3701039b0 2024-09-21T03:44:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.144+0000 7ff36f200640 1 -- 172.21.15.16:0/2077555005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3701a6cd0 con 0x7ff3701039b0 2024-09-21T03:44:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.145+0000 7ff36f200640 1 -- 172.21.15.16:0/2077555005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff370103dc0 con 0x7ff3701039b0 2024-09-21T03:44:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.149+0000 7ff366a00640 1 -- 172.21.15.16:0/2077555005 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7ff360010430 con 0x7ff3701039b0 2024-09-21T03:44:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.151+0000 7ff366a00640 1 --2- 172.21.15.16:0/2077555005 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff3540b65a0 0x7ff3540b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.151+0000 7ff366a00640 1 -- 172.21.15.16:0/2077555005 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7ff3600e8060 con 0x7ff3701039b0 2024-09-21T03:44:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.151+0000 7ff36d400640 1 --2- 172.21.15.16:0/2077555005 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff3540b65a0 0x7ff3540b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.152+0000 7ff36d400640 1 --2- 172.21.15.16:0/2077555005 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff3540b65a0 0x7ff3540b8ab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff35c009230 tx=0x7ff35c031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.153+0000 7ff366a00640 1 -- 172.21.15.16:0/2077555005 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff3600b2760 con 0x7ff3701039b0 2024-09-21T03:44:58.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:57 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2905730711' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:44:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.356+0000 7ff36f200640 1 -- 172.21.15.16:0/2077555005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff37019b440 con 0x7ff3701039b0 2024-09-21T03:44:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.356+0000 7ff366a00640 1 -- 172.21.15.16:0/2077555005 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff3600b76a0 con 0x7ff3701039b0 2024-09-21T03:44:58.358 INFO:teuthology.orchestra.run.smithi016.stdout:34359738465 2024-09-21T03:44:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.358+0000 7ff36f200640 1 -- 172.21.15.16:0/2077555005 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff3540b65a0 msgr2=0x7ff3540b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.358+0000 7ff36f200640 1 --2- 172.21.15.16:0/2077555005 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff3540b65a0 0x7ff3540b8ab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff35c009230 tx=0x7ff35c031400 comp rx=0 tx=0).stop 2024-09-21T03:44:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.358+0000 7ff36f200640 1 -- 172.21.15.16:0/2077555005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3701039b0 msgr2=0x7ff37019a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.358+0000 7ff36f200640 1 --2- 172.21.15.16:0/2077555005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3701039b0 0x7ff37019a310 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff36000e9f0 tx=0x7ff36000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.359+0000 7ff36f200640 1 -- 172.21.15.16:0/2077555005 shutdown_connections 2024-09-21T03:44:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.359+0000 7ff36f200640 1 --2- 172.21.15.16:0/2077555005 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff3540b65a0 0x7ff3540b8ab0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.359+0000 7ff36f200640 1 --2- 172.21.15.16:0/2077555005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff370109c40 0x7ff37019fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.359+0000 7ff36f200640 1 --2- 172.21.15.16:0/2077555005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3701092f0 0x7ff37019a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.359+0000 7ff36f200640 1 --2- 172.21.15.16:0/2077555005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3701039b0 0x7ff37019a310 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.359+0000 7ff36f200640 1 -- 172.21.15.16:0/2077555005 >> 172.21.15.16:0/2077555005 conn(0x7ff3700ff400 msgr2=0x7ff370101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.359+0000 7ff36f200640 1 -- 172.21.15.16:0/2077555005 shutdown_connections 2024-09-21T03:44:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.359+0000 7ff36f200640 1 -- 172.21.15.16:0/2077555005 wait complete. 2024-09-21T03:44:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.622+0000 7fbb84600640 1 -- 172.21.15.16:0/1284263496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb7c1100e0 msgr2=0x7fbb7c112520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.622+0000 7fbb84600640 1 --2- 172.21.15.16:0/1284263496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb7c1100e0 0x7fbb7c112520 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009be0 tx=0x7fbb7402a6c0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.622+0000 7fbb84600640 1 -- 172.21.15.16:0/1284263496 shutdown_connections 2024-09-21T03:44:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.622+0000 7fbb84600640 1 --2- 172.21.15.16:0/1284263496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb7c1100e0 0x7fbb7c112520 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.622+0000 7fbb84600640 1 --2- 172.21.15.16:0/1284263496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb7c076100 0x7fbb7c10fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.622+0000 7fbb84600640 1 --2- 172.21.15.16:0/1284263496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb7c0757b0 0x7fbb7c075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.622+0000 7fbb84600640 1 -- 172.21.15.16:0/1284263496 >> 172.21.15.16:0/1284263496 conn(0x7fbb7c0ff610 msgr2=0x7fbb7c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.623+0000 7fbb84600640 1 -- 172.21.15.16:0/1284263496 shutdown_connections 2024-09-21T03:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.623+0000 7fbb84600640 1 -- 172.21.15.16:0/1284263496 wait complete. 2024-09-21T03:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.625+0000 7fbb84600640 1 Processor -- start 2024-09-21T03:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.625+0000 7fbb84600640 1 -- start start 2024-09-21T03:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.625+0000 7fbb84600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb7c0757b0 0x7fbb7c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.625+0000 7fbb84600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb7c076100 0x7fbb7c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.626+0000 7fbb84600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb7c1100e0 0x7fbb7c1a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.626+0000 7fbb84600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c115c90 con 0x7fbb7c076100 2024-09-21T03:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.626+0000 7fbb84600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c115b10 con 0x7fbb7c1100e0 2024-09-21T03:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.626+0000 7fbb84600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c115e10 con 0x7fbb7c0757b0 2024-09-21T03:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.626+0000 7fbb81200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb7c076100 0x7fbb7c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.626+0000 7fbb81c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb7c0757b0 0x7fbb7c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.626+0000 7fbb81200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb7c076100 0x7fbb7c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37250/0 (socket says 172.21.15.16:37250) 2024-09-21T03:44:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.626+0000 7fbb81200640 1 -- 172.21.15.16:0/739421441 learned_addr learned my addr 172.21.15.16:0/739421441 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:44:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.626+0000 7fbb82600640 1 --2- 172.21.15.16:0/739421441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb7c1100e0 0x7fbb7c1a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.627+0000 7fbb81c00640 1 -- 172.21.15.16:0/739421441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb7c1100e0 msgr2=0x7fbb7c1a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.627+0000 7fbb81c00640 1 --2- 172.21.15.16:0/739421441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb7c1100e0 0x7fbb7c1a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.627+0000 7fbb81c00640 1 -- 172.21.15.16:0/739421441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb7c076100 msgr2=0x7fbb7c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.627+0000 7fbb81c00640 1 --2- 172.21.15.16:0/739421441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb7c076100 0x7fbb7c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.627+0000 7fbb81c00640 1 -- 172.21.15.16:0/739421441 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c1a25d0 con 0x7fbb7c0757b0 2024-09-21T03:44:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.627+0000 7fbb82600640 1 --2- 172.21.15.16:0/739421441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb7c1100e0 0x7fbb7c1a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.627+0000 7fbb81c00640 1 --2- 172.21.15.16:0/739421441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb7c0757b0 0x7fbb7c19c660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbb70005fd0 tx=0x7fbb7000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.628+0000 7fbb7a000640 1 -- 172.21.15.16:0/739421441 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7000cd00 con 0x7fbb7c0757b0 2024-09-21T03:44:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.628+0000 7fbb7a000640 1 -- 172.21.15.16:0/739421441 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb700035f0 con 0x7fbb7c0757b0 2024-09-21T03:44:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.628+0000 7fbb7a000640 1 -- 172.21.15.16:0/739421441 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70010690 con 0x7fbb7c0757b0 2024-09-21T03:44:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.628+0000 7fbb84600640 1 -- 172.21.15.16:0/739421441 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb7c1a28c0 con 0x7fbb7c0757b0 2024-09-21T03:44:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.628+0000 7fbb84600640 1 -- 172.21.15.16:0/739421441 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb7c1a9020 con 0x7fbb7c0757b0 2024-09-21T03:44:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.630+0000 7fbb84600640 1 -- 172.21.15.16:0/739421441 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb7c076cc0 con 0x7fbb7c0757b0 2024-09-21T03:44:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.635+0000 7fbb7a000640 1 -- 172.21.15.16:0/739421441 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fbb70003a80 con 0x7fbb7c0757b0 2024-09-21T03:44:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.637+0000 7fbb7a000640 1 --2- 172.21.15.16:0/739421441 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbb640b6620 0x7fbb640b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.637+0000 7fbb7a000640 1 -- 172.21.15.16:0/739421441 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fbb700e7d60 con 0x7fbb7c0757b0 2024-09-21T03:44:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.637+0000 7fbb81200640 1 --2- 172.21.15.16:0/739421441 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbb640b6620 0x7fbb640b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.637+0000 7fbb7a000640 1 -- 172.21.15.16:0/739421441 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbb700e8260 con 0x7fbb7c0757b0 2024-09-21T03:44:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.638+0000 7fbb81200640 1 --2- 172.21.15.16:0/739421441 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbb640b6620 0x7fbb640b8b30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c1121c0 tx=0x7fbb68005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:58.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.839+0000 7fbb84600640 1 -- 172.21.15.16:0/739421441 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fbb7c075bc0 con 0x7fbb7c0757b0 2024-09-21T03:44:58.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.840+0000 7fbb7a000640 1 -- 172.21.15.16:0/739421441 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbb700ae0e0 con 0x7fbb7c0757b0 2024-09-21T03:44:58.842 INFO:teuthology.orchestra.run.smithi016.stdout:244813135919 2024-09-21T03:44:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.843+0000 7fbb84600640 1 -- 172.21.15.16:0/739421441 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbb640b6620 msgr2=0x7fbb640b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.843+0000 7fbb84600640 1 --2- 172.21.15.16:0/739421441 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbb640b6620 0x7fbb640b8b30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c1121c0 tx=0x7fbb68005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.843+0000 7fbb84600640 1 -- 172.21.15.16:0/739421441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb7c0757b0 msgr2=0x7fbb7c19c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.843+0000 7fbb84600640 1 --2- 172.21.15.16:0/739421441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb7c0757b0 0x7fbb7c19c660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbb70005fd0 tx=0x7fbb7000ee50 comp rx=0 tx=0).stop 2024-09-21T03:44:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.843+0000 7fbb84600640 1 -- 172.21.15.16:0/739421441 shutdown_connections 2024-09-21T03:44:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.843+0000 7fbb84600640 1 --2- 172.21.15.16:0/739421441 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbb640b6620 0x7fbb640b8b30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.843+0000 7fbb84600640 1 --2- 172.21.15.16:0/739421441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb7c1100e0 0x7fbb7c1a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.843+0000 7fbb84600640 1 --2- 172.21.15.16:0/739421441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb7c076100 0x7fbb7c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.843+0000 7fbb84600640 1 --2- 172.21.15.16:0/739421441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb7c0757b0 0x7fbb7c19c660 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.843+0000 7fbb84600640 1 -- 172.21.15.16:0/739421441 >> 172.21.15.16:0/739421441 conn(0x7fbb7c0ff610 msgr2=0x7fbb7c107b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.843+0000 7fbb84600640 1 -- 172.21.15.16:0/739421441 shutdown_connections 2024-09-21T03:44:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:44:58.843+0000 7fbb84600640 1 -- 172.21.15.16:0/739421441 wait complete. 2024-09-21T03:44:58.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:58 smithi016 ceph-mon[30375]: pgmap v378: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-09-21T03:44:58.879 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:58 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2077555005' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:44:59.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:58 smithi134 ceph-mon[35031]: pgmap v378: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-09-21T03:44:59.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:58 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2077555005' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:44:59.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:58 smithi123 ceph-mon[34673]: pgmap v378: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-09-21T03:44:59.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:58 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2077555005' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:44:59.466 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574938 got 55834574940 for osd.1 2024-09-21T03:44:59.466 DEBUG:teuthology.parallel:result is None 2024-09-21T03:44:59.948 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:44:59 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/739421441' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:45:00.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:44:59 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/739421441' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:45:00.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:44:59 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/739421441' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:45:00.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.122+0000 7f9364000640 1 -- 172.21.15.16:0/684280263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1094c0 msgr2=0x7f935c1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.122+0000 7f9364000640 1 --2- 172.21.15.16:0/684280263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1094c0 0x7f935c1098d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9348009900 tx=0x7f934802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.122+0000 7f9364000640 1 -- 172.21.15.16:0/684280263 shutdown_connections 2024-09-21T03:45:00.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.122+0000 7f9364000640 1 --2- 172.21.15.16:0/684280263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f935c109e10 0x7f935c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.122+0000 7f9364000640 1 --2- 172.21.15.16:0/684280263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1094c0 0x7f935c1098d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.122+0000 7f9364000640 1 --2- 172.21.15.16:0/684280263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f935c103b80 0x7f935c103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.122+0000 7f9364000640 1 -- 172.21.15.16:0/684280263 >> 172.21.15.16:0/684280263 conn(0x7f935c0ff5d0 msgr2=0x7f935c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:00.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.122+0000 7f9364000640 1 -- 172.21.15.16:0/684280263 shutdown_connections 2024-09-21T03:45:00.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.122+0000 7f9364000640 1 -- 172.21.15.16:0/684280263 wait complete. 2024-09-21T03:45:00.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.123+0000 7f9364000640 1 Processor -- start 2024-09-21T03:45:00.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.123+0000 7f9364000640 1 -- start start 2024-09-21T03:45:00.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.124+0000 7f9364000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f935c103b80 0x7f935c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.124+0000 7f9364000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1094c0 0x7f935c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.124+0000 7f9364000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f935c109e10 0x7f935c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.124+0000 7f9364000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f935c113b20 con 0x7f935c1094c0 2024-09-21T03:45:00.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.124+0000 7f9364000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1139a0 con 0x7f935c109e10 2024-09-21T03:45:00.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.124+0000 7f9364000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f935c113ca0 con 0x7f935c103b80 2024-09-21T03:45:00.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.124+0000 7f9360c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1094c0 0x7f935c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.124+0000 7f9361600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f935c103b80 0x7f935c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.124+0000 7f9360c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1094c0 0x7f935c19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37256/0 (socket says 172.21.15.16:37256) 2024-09-21T03:45:00.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.124+0000 7f9361600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f935c103b80 0x7f935c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:58180/0 (socket says 172.21.15.16:58180) 2024-09-21T03:45:00.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.124+0000 7f9360c00640 1 -- 172.21.15.16:0/2184726525 learned_addr learned my addr 172.21.15.16:0/2184726525 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:00.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.124+0000 7f9362000640 1 --2- 172.21.15.16:0/2184726525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f935c109e10 0x7f935c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.125+0000 7f9362000640 1 -- 172.21.15.16:0/2184726525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f935c103b80 msgr2=0x7f935c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.125+0000 7f9362000640 1 --2- 172.21.15.16:0/2184726525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f935c103b80 0x7f935c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.125+0000 7f9362000640 1 -- 172.21.15.16:0/2184726525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1094c0 msgr2=0x7f935c19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.125+0000 7f9362000640 1 --2- 172.21.15.16:0/2184726525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1094c0 0x7f935c19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.125+0000 7f9362000640 1 -- 172.21.15.16:0/2184726525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c1a0420 con 0x7f935c109e10 2024-09-21T03:45:00.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.125+0000 7f9360c00640 1 --2- 172.21.15.16:0/2184726525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1094c0 0x7f935c19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:00.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.125+0000 7f9362000640 1 --2- 172.21.15.16:0/2184726525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f935c109e10 0x7f935c19fcf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9354009fd0 tx=0x7f935400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:00.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.125+0000 7f935a000640 1 -- 172.21.15.16:0/2184726525 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935400ce20 con 0x7f935c109e10 2024-09-21T03:45:00.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.125+0000 7f9364000640 1 -- 172.21.15.16:0/2184726525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935c1a0710 con 0x7f935c109e10 2024-09-21T03:45:00.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.126+0000 7f935a000640 1 -- 172.21.15.16:0/2184726525 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9354005020 con 0x7f935c109e10 2024-09-21T03:45:00.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.126+0000 7f935a000640 1 -- 172.21.15.16:0/2184726525 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9354010690 con 0x7f935c109e10 2024-09-21T03:45:00.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.126+0000 7f9364000640 1 -- 172.21.15.16:0/2184726525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935c0755f0 con 0x7f935c109e10 2024-09-21T03:45:00.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.127+0000 7f9364000640 1 -- 172.21.15.16:0/2184726525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f935c103f90 con 0x7f935c109e10 2024-09-21T03:45:00.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.129+0000 7f935a000640 1 -- 172.21.15.16:0/2184726525 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f93540054d0 con 0x7f935c109e10 2024-09-21T03:45:00.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.131+0000 7f935a000640 1 --2- 172.21.15.16:0/2184726525 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93440b65a0 0x7f93440b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.131+0000 7f935a000640 1 -- 172.21.15.16:0/2184726525 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f93540e7f00 con 0x7f935c109e10 2024-09-21T03:45:00.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.131+0000 7f9361600640 1 --2- 172.21.15.16:0/2184726525 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93440b65a0 0x7f93440b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.132+0000 7f9361600640 1 --2- 172.21.15.16:0/2184726525 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93440b65a0 0x7f93440b8ab0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9350010d80 tx=0x7f9350012040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:00.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.133+0000 7f935a000640 1 -- 172.21.15.16:0/2184726525 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f93540ae280 con 0x7f935c109e10 2024-09-21T03:45:00.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.337+0000 7f9364000640 1 -- 172.21.15.16:0/2184726525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f935c19b5e0 con 0x7f935c109e10 2024-09-21T03:45:00.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.338+0000 7f935a000640 1 -- 172.21.15.16:0/2184726525 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f935c19b5e0 con 0x7f935c109e10 2024-09-21T03:45:00.339 INFO:teuthology.orchestra.run.smithi016.stdout:197568495675 2024-09-21T03:45:00.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.340+0000 7f9364000640 1 -- 172.21.15.16:0/2184726525 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93440b65a0 msgr2=0x7f93440b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.340+0000 7f9364000640 1 --2- 172.21.15.16:0/2184726525 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93440b65a0 0x7f93440b8ab0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9350010d80 tx=0x7f9350012040 comp rx=0 tx=0).stop 2024-09-21T03:45:00.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.340+0000 7f9364000640 1 -- 172.21.15.16:0/2184726525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f935c109e10 msgr2=0x7f935c19fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.340+0000 7f9364000640 1 --2- 172.21.15.16:0/2184726525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f935c109e10 0x7f935c19fcf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9354009fd0 tx=0x7f935400ef10 comp rx=0 tx=0).stop 2024-09-21T03:45:00.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.340+0000 7f9364000640 1 -- 172.21.15.16:0/2184726525 shutdown_connections 2024-09-21T03:45:00.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.340+0000 7f9364000640 1 --2- 172.21.15.16:0/2184726525 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93440b65a0 0x7f93440b8ab0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.340+0000 7f9364000640 1 --2- 172.21.15.16:0/2184726525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f935c109e10 0x7f935c19fcf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.340+0000 7f9364000640 1 --2- 172.21.15.16:0/2184726525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1094c0 0x7f935c19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.340+0000 7f9364000640 1 --2- 172.21.15.16:0/2184726525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f935c103b80 0x7f935c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.340+0000 7f9364000640 1 -- 172.21.15.16:0/2184726525 >> 172.21.15.16:0/2184726525 conn(0x7f935c0ff5d0 msgr2=0x7f935c1019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:00.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.340+0000 7f9364000640 1 -- 172.21.15.16:0/2184726525 shutdown_connections 2024-09-21T03:45:00.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:00.340+0000 7f9364000640 1 -- 172.21.15.16:0/2184726525 wait complete. 2024-09-21T03:45:00.532 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738464 got 34359738465 for osd.0 2024-09-21T03:45:00.532 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:01.050 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:00 smithi016 ceph-mon[30375]: pgmap v379: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-09-21T03:45:01.050 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:00 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2184726525' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:45:01.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:00 smithi134 ceph-mon[35031]: pgmap v379: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-09-21T03:45:01.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:00 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2184726525' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:45:01.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:00 smithi123 ceph-mon[34673]: pgmap v379: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-09-21T03:45:01.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:00 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2184726525' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:45:01.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.419+0000 7f8837200640 1 -- 172.21.15.16:0/3202839030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88300757b0 msgr2=0x7f8830075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.419+0000 7f8837200640 1 --2- 172.21.15.16:0/3202839030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88300757b0 0x7f8830075bc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8820009900 tx=0x7f882002a410 comp rx=0 tx=0).stop 2024-09-21T03:45:01.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.419+0000 7f8837200640 1 -- 172.21.15.16:0/3202839030 shutdown_connections 2024-09-21T03:45:01.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.419+0000 7f8837200640 1 --2- 172.21.15.16:0/3202839030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88301100e0 0x7f8830112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.419+0000 7f8837200640 1 --2- 172.21.15.16:0/3202839030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8830076100 0x7f883010fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.419+0000 7f8837200640 1 --2- 172.21.15.16:0/3202839030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88300757b0 0x7f8830075bc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.419+0000 7f8837200640 1 -- 172.21.15.16:0/3202839030 >> 172.21.15.16:0/3202839030 conn(0x7f88300ff610 msgr2=0x7f8830101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.419+0000 7f8837200640 1 -- 172.21.15.16:0/3202839030 shutdown_connections 2024-09-21T03:45:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.419+0000 7f8837200640 1 -- 172.21.15.16:0/3202839030 wait complete. 2024-09-21T03:45:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.420+0000 7f8837200640 1 Processor -- start 2024-09-21T03:45:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.420+0000 7f8837200640 1 -- start start 2024-09-21T03:45:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.421+0000 7f8837200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88300757b0 0x7f883019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.421+0000 7f8837200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8830076100 0x7f883019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.421+0000 7f8837200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88301100e0 0x7f88301a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.421+0000 7f8837200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8830115c90 con 0x7f88301100e0 2024-09-21T03:45:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.421+0000 7f8837200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8830115b10 con 0x7f8830076100 2024-09-21T03:45:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.421+0000 7f8837200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8830115e10 con 0x7f88300757b0 2024-09-21T03:45:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.421+0000 7f882fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88300757b0 0x7f883019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.421+0000 7f8835200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88301100e0 0x7f88301a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.421+0000 7f8827e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8830076100 0x7f883019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.421+0000 7f882fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88300757b0 0x7f883019c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:58210/0 (socket says 172.21.15.16:58210) 2024-09-21T03:45:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.421+0000 7f8835200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88301100e0 0x7f88301a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37276/0 (socket says 172.21.15.16:37276) 2024-09-21T03:45:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.421+0000 7f882fe00640 1 -- 172.21.15.16:0/1911945814 learned_addr learned my addr 172.21.15.16:0/1911945814 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.422+0000 7f882fe00640 1 -- 172.21.15.16:0/1911945814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8830076100 msgr2=0x7f883019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.422+0000 7f882fe00640 1 --2- 172.21.15.16:0/1911945814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8830076100 0x7f883019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.422+0000 7f882fe00640 1 -- 172.21.15.16:0/1911945814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88301100e0 msgr2=0x7f88301a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.422+0000 7f882fe00640 1 --2- 172.21.15.16:0/1911945814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88301100e0 0x7f88301a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.422+0000 7f882fe00640 1 -- 172.21.15.16:0/1911945814 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88301a25b0 con 0x7f88300757b0 2024-09-21T03:45:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.422+0000 7f882fe00640 1 --2- 172.21.15.16:0/1911945814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88300757b0 0x7f883019c640 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8820005940 tx=0x7f88200040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.422+0000 7f882d600640 1 -- 172.21.15.16:0/1911945814 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882003f030 con 0x7f88300757b0 2024-09-21T03:45:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.423+0000 7f8837200640 1 -- 172.21.15.16:0/1911945814 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88301a2840 con 0x7f88300757b0 2024-09-21T03:45:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.423+0000 7f882d600640 1 -- 172.21.15.16:0/1911945814 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88200044c0 con 0x7f88300757b0 2024-09-21T03:45:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.423+0000 7f882d600640 1 -- 172.21.15.16:0/1911945814 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820035070 con 0x7f88300757b0 2024-09-21T03:45:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.423+0000 7f8837200640 1 -- 172.21.15.16:0/1911945814 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88301a9000 con 0x7f88300757b0 2024-09-21T03:45:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.424+0000 7f8837200640 1 -- 172.21.15.16:0/1911945814 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8830076cc0 con 0x7f88300757b0 2024-09-21T03:45:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.426+0000 7f882d600640 1 -- 172.21.15.16:0/1911945814 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f8820042030 con 0x7f88300757b0 2024-09-21T03:45:01.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.428+0000 7f882d600640 1 --2- 172.21.15.16:0/1911945814 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f88140b65a0 0x7f88140b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:01.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.428+0000 7f882d600640 1 -- 172.21.15.16:0/1911945814 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f8820107e40 con 0x7f88300757b0 2024-09-21T03:45:01.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.428+0000 7f8827e00640 1 --2- 172.21.15.16:0/1911945814 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f88140b65a0 0x7f88140b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:01.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.429+0000 7f8827e00640 1 --2- 172.21.15.16:0/1911945814 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f88140b65a0 0x7f88140b8ab0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f88180095f0 tx=0x7f8818005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:01.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.430+0000 7f882d600640 1 -- 172.21.15.16:0/1911945814 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f88200ce160 con 0x7f88300757b0 2024-09-21T03:45:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.642+0000 7f8837200640 1 -- 172.21.15.16:0/1911945814 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f88301a9200 con 0x7f88300757b0 2024-09-21T03:45:01.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.643+0000 7f882d600640 1 -- 172.21.15.16:0/1911945814 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f88200d2540 con 0x7f88300757b0 2024-09-21T03:45:01.645 INFO:teuthology.orchestra.run.smithi016.stdout:73014444120 2024-09-21T03:45:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.645+0000 7f8837200640 1 -- 172.21.15.16:0/1911945814 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f88140b65a0 msgr2=0x7f88140b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.645+0000 7f8837200640 1 --2- 172.21.15.16:0/1911945814 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f88140b65a0 0x7f88140b8ab0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f88180095f0 tx=0x7f8818005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.645+0000 7f8837200640 1 -- 172.21.15.16:0/1911945814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88300757b0 msgr2=0x7f883019c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.645+0000 7f8837200640 1 --2- 172.21.15.16:0/1911945814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88300757b0 0x7f883019c640 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8820005940 tx=0x7f88200040c0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.645+0000 7f8837200640 1 -- 172.21.15.16:0/1911945814 shutdown_connections 2024-09-21T03:45:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.645+0000 7f8837200640 1 --2- 172.21.15.16:0/1911945814 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f88140b65a0 0x7f88140b8ab0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.645+0000 7f8837200640 1 --2- 172.21.15.16:0/1911945814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88301100e0 0x7f88301a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.645+0000 7f8837200640 1 --2- 172.21.15.16:0/1911945814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8830076100 0x7f883019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.645+0000 7f8837200640 1 --2- 172.21.15.16:0/1911945814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88300757b0 0x7f883019c640 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.645+0000 7f8837200640 1 -- 172.21.15.16:0/1911945814 >> 172.21.15.16:0/1911945814 conn(0x7f88300ff610 msgr2=0x7f8830109c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.646+0000 7f8837200640 1 -- 172.21.15.16:0/1911945814 shutdown_connections 2024-09-21T03:45:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:01.646+0000 7f8837200640 1 -- 172.21.15.16:0/1911945814 wait complete. 2024-09-21T03:45:01.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:01 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1911945814' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:45:02.044 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495672 got 197568495675 for osd.7 2024-09-21T03:45:02.044 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:02.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:01 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1911945814' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:45:02.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:01 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1911945814' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:45:02.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.463+0000 7f32b2400640 1 -- 172.21.15.16:0/2916415897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32ac109c40 msgr2=0x7f32ac110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.463+0000 7f32b2400640 1 --2- 172.21.15.16:0/2916415897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32ac109c40 0x7f32ac110470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009be0 tx=0x7f32a402a6d0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.463+0000 7f32b2400640 1 -- 172.21.15.16:0/2916415897 shutdown_connections 2024-09-21T03:45:02.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.463+0000 7f32b2400640 1 --2- 172.21.15.16:0/2916415897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32ac109c40 0x7f32ac110470 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.463+0000 7f32b2400640 1 --2- 172.21.15.16:0/2916415897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1092f0 0x7f32ac109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.463+0000 7f32b2400640 1 --2- 172.21.15.16:0/2916415897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32ac1039b0 0x7f32ac103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.463+0000 7f32b2400640 1 -- 172.21.15.16:0/2916415897 >> 172.21.15.16:0/2916415897 conn(0x7f32ac0ff440 msgr2=0x7f32ac101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.463+0000 7f32b2400640 1 -- 172.21.15.16:0/2916415897 shutdown_connections 2024-09-21T03:45:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.463+0000 7f32b2400640 1 -- 172.21.15.16:0/2916415897 wait complete. 2024-09-21T03:45:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.465+0000 7f32b2400640 1 Processor -- start 2024-09-21T03:45:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.465+0000 7f32b2400640 1 -- start start 2024-09-21T03:45:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.465+0000 7f32b2400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32ac1039b0 0x7f32ac19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.465+0000 7f32b2400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1092f0 0x7f32ac19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.465+0000 7f32b2400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32ac109c40 0x7f32ac19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.465+0000 7f32b2400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac113900 con 0x7f32ac1092f0 2024-09-21T03:45:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.465+0000 7f32b2400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac113780 con 0x7f32ac1039b0 2024-09-21T03:45:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.465+0000 7f32b2400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac113a80 con 0x7f32ac109c40 2024-09-21T03:45:02.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.466+0000 7f32abe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1092f0 0x7f32ac19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:02.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.466+0000 7f32b1000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32ac1039b0 0x7f32ac19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:02.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.466+0000 7f32abe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1092f0 0x7f32ac19a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60708/0 (socket says 172.21.15.16:60708) 2024-09-21T03:45:02.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.466+0000 7f32abe00640 1 -- 172.21.15.16:0/420355524 learned_addr learned my addr 172.21.15.16:0/420355524 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:02.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.466+0000 7f32b1a00640 1 --2- 172.21.15.16:0/420355524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32ac109c40 0x7f32ac19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:02.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.466+0000 7f32b1a00640 1 -- 172.21.15.16:0/420355524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32ac1039b0 msgr2=0x7f32ac19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.466+0000 7f32b1a00640 1 --2- 172.21.15.16:0/420355524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32ac1039b0 0x7f32ac19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.466+0000 7f32b1a00640 1 -- 172.21.15.16:0/420355524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1092f0 msgr2=0x7f32ac19a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.466+0000 7f32b1a00640 1 --2- 172.21.15.16:0/420355524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1092f0 0x7f32ac19a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.466+0000 7f32b1a00640 1 -- 172.21.15.16:0/420355524 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac1a0260 con 0x7f32ac109c40 2024-09-21T03:45:02.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.466+0000 7f32abe00640 1 --2- 172.21.15.16:0/420355524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1092f0 0x7f32ac19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:02.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.466+0000 7f32b1000640 1 --2- 172.21.15.16:0/420355524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32ac1039b0 0x7f32ac19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:02.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.467+0000 7f32b1a00640 1 --2- 172.21.15.16:0/420355524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32ac109c40 0x7f32ac19fb30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f32a402abf0 tx=0x7f32a4003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:02.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.467+0000 7f32a9600640 1 -- 172.21.15.16:0/420355524 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4041020 con 0x7f32ac109c40 2024-09-21T03:45:02.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.467+0000 7f32a9600640 1 -- 172.21.15.16:0/420355524 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a4003e10 con 0x7f32ac109c40 2024-09-21T03:45:02.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.467+0000 7f32b2400640 1 -- 172.21.15.16:0/420355524 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ac1a04f0 con 0x7f32ac109c40 2024-09-21T03:45:02.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.467+0000 7f32a9600640 1 -- 172.21.15.16:0/420355524 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4036040 con 0x7f32ac109c40 2024-09-21T03:45:02.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.467+0000 7f32b2400640 1 -- 172.21.15.16:0/420355524 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ac1a6cb0 con 0x7f32ac109c40 2024-09-21T03:45:02.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.468+0000 7f32b2400640 1 -- 172.21.15.16:0/420355524 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32ac103dc0 con 0x7f32ac109c40 2024-09-21T03:45:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.471+0000 7f32a9600640 1 -- 172.21.15.16:0/420355524 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f32a4044020 con 0x7f32ac109c40 2024-09-21T03:45:02.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.472+0000 7f32a9600640 1 --2- 172.21.15.16:0/420355524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f32940b6620 0x7f32940b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:02.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.473+0000 7f32a9600640 1 -- 172.21.15.16:0/420355524 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f32a4108700 con 0x7f32ac109c40 2024-09-21T03:45:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.473+0000 7f32b1000640 1 --2- 172.21.15.16:0/420355524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f32940b6620 0x7f32940b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.474+0000 7f32b1000640 1 --2- 172.21.15.16:0/420355524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f32940b6620 0x7f32940b8b30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009fc0 tx=0x7f32a0005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.474+0000 7f32a9600640 1 -- 172.21.15.16:0/420355524 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f32a404b020 con 0x7f32ac109c40 2024-09-21T03:45:02.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.680+0000 7f32b2400640 1 -- 172.21.15.16:0/420355524 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f32ac10ba30 con 0x7f32ac109c40 2024-09-21T03:45:02.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.681+0000 7f32a9600640 1 -- 172.21.15.16:0/420355524 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f32a40cea20 con 0x7f32ac109c40 2024-09-21T03:45:02.683 INFO:teuthology.orchestra.run.smithi016.stdout:103079215185 2024-09-21T03:45:02.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.683+0000 7f32b2400640 1 -- 172.21.15.16:0/420355524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f32940b6620 msgr2=0x7f32940b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.683+0000 7f32b2400640 1 --2- 172.21.15.16:0/420355524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f32940b6620 0x7f32940b8b30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009fc0 tx=0x7f32a0005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:02.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.683+0000 7f32b2400640 1 -- 172.21.15.16:0/420355524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32ac109c40 msgr2=0x7f32ac19fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:02.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.683+0000 7f32b2400640 1 --2- 172.21.15.16:0/420355524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32ac109c40 0x7f32ac19fb30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f32a402abf0 tx=0x7f32a4003c20 comp rx=0 tx=0).stop 2024-09-21T03:45:02.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.683+0000 7f32b2400640 1 -- 172.21.15.16:0/420355524 shutdown_connections 2024-09-21T03:45:02.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.683+0000 7f32b2400640 1 --2- 172.21.15.16:0/420355524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f32940b6620 0x7f32940b8b30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.683+0000 7f32b2400640 1 --2- 172.21.15.16:0/420355524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32ac109c40 0x7f32ac19fb30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.683+0000 7f32b2400640 1 --2- 172.21.15.16:0/420355524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1092f0 0x7f32ac19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.683+0000 7f32b2400640 1 --2- 172.21.15.16:0/420355524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32ac1039b0 0x7f32ac19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:02.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.683+0000 7f32b2400640 1 -- 172.21.15.16:0/420355524 >> 172.21.15.16:0/420355524 conn(0x7f32ac0ff440 msgr2=0x7f32ac1068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:02.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.683+0000 7f32b2400640 1 -- 172.21.15.16:0/420355524 shutdown_connections 2024-09-21T03:45:02.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:02.683+0000 7f32b2400640 1 -- 172.21.15.16:0/420355524 wait complete. 2024-09-21T03:45:03.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:02 smithi016 ceph-mon[30375]: pgmap v380: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-09-21T03:45:03.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:02 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/420355524' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:45:03.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:02 smithi134 ceph-mon[35031]: pgmap v380: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-09-21T03:45:03.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:02 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/420355524' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:45:03.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:02 smithi123 ceph-mon[34673]: pgmap v380: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-09-21T03:45:03.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:02 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/420355524' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:45:03.324 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:03.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.388+0000 7f1401400640 1 -- 172.21.15.16:0/4036485076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13fc10de10 msgr2=0x7f13fc110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.388+0000 7f1401400640 1 --2- 172.21.15.16:0/4036485076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13fc10de10 0x7f13fc110300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009be0 tx=0x7f13f402a6c0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.388+0000 7f1401400640 1 -- 172.21.15.16:0/4036485076 shutdown_connections 2024-09-21T03:45:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.388+0000 7f1401400640 1 --2- 172.21.15.16:0/4036485076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13fc10de10 0x7f13fc110300 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.388+0000 7f1401400640 1 --2- 172.21.15.16:0/4036485076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13fc105300 0x7f13fc10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.388+0000 7f1401400640 1 --2- 172.21.15.16:0/4036485076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13fc1049b0 0x7f13fc104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.388+0000 7f1401400640 1 -- 172.21.15.16:0/4036485076 >> 172.21.15.16:0/4036485076 conn(0x7f13fc079760 msgr2=0x7f13fc079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.388+0000 7f1401400640 1 -- 172.21.15.16:0/4036485076 shutdown_connections 2024-09-21T03:45:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.388+0000 7f1401400640 1 -- 172.21.15.16:0/4036485076 wait complete. 2024-09-21T03:45:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.389+0000 7f1401400640 1 Processor -- start 2024-09-21T03:45:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.390+0000 7f1401400640 1 -- start start 2024-09-21T03:45:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.390+0000 7f1401400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13fc1049b0 0x7f13fc1026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.390+0000 7f1401400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13fc105300 0x7f13fc102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.390+0000 7f1401400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13fc10de10 0x7f13fc103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.390+0000 7f1401400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc113a90 con 0x7f13fc1049b0 2024-09-21T03:45:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.390+0000 7f1401400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc113910 con 0x7f13fc105300 2024-09-21T03:45:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.390+0000 7f1401400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc113c10 con 0x7f13fc10de10 2024-09-21T03:45:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.390+0000 7f13f3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13fc105300 0x7f13fc102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.390+0000 7f13f3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13fc105300 0x7f13fc102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:53136/0 (socket says 172.21.15.16:53136) 2024-09-21T03:45:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.390+0000 7f13f3e00640 1 -- 172.21.15.16:0/2308039497 learned_addr learned my addr 172.21.15.16:0/2308039497 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.390+0000 7f13fa800640 1 --2- 172.21.15.16:0/2308039497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13fc1049b0 0x7f13fc1026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.390+0000 7f13fb200640 1 --2- 172.21.15.16:0/2308039497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13fc10de10 0x7f13fc103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.391+0000 7f13f3e00640 1 -- 172.21.15.16:0/2308039497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13fc10de10 msgr2=0x7f13fc103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.391+0000 7f13f3e00640 1 --2- 172.21.15.16:0/2308039497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13fc10de10 0x7f13fc103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.391+0000 7f13f3e00640 1 -- 172.21.15.16:0/2308039497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13fc1049b0 msgr2=0x7f13fc1026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.391+0000 7f13f3e00640 1 --2- 172.21.15.16:0/2308039497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13fc1049b0 0x7f13fc1026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.391+0000 7f13f3e00640 1 -- 172.21.15.16:0/2308039497 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13fc1a86e0 con 0x7f13fc105300 2024-09-21T03:45:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.391+0000 7f13f3e00640 1 --2- 172.21.15.16:0/2308039497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13fc105300 0x7f13fc102c30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f13e40058c0 tx=0x7f13e400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.391+0000 7f13f3400640 1 -- 172.21.15.16:0/2308039497 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e400cd30 con 0x7f13fc105300 2024-09-21T03:45:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.391+0000 7f13f3400640 1 -- 172.21.15.16:0/2308039497 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e40035f0 con 0x7f13fc105300 2024-09-21T03:45:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.391+0000 7f1401400640 1 -- 172.21.15.16:0/2308039497 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13fc1a89d0 con 0x7f13fc105300 2024-09-21T03:45:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.392+0000 7f13f3400640 1 -- 172.21.15.16:0/2308039497 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e40044e0 con 0x7f13fc105300 2024-09-21T03:45:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.392+0000 7f1401400640 1 -- 172.21.15.16:0/2308039497 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13fc1a8f10 con 0x7f13fc105300 2024-09-21T03:45:03.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.393+0000 7f1401400640 1 -- 172.21.15.16:0/2308039497 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13fc06c2c0 con 0x7f13fc105300 2024-09-21T03:45:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.395+0000 7f13f3400640 1 -- 172.21.15.16:0/2308039497 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f13e4003aa0 con 0x7f13fc105300 2024-09-21T03:45:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.397+0000 7f13f3400640 1 --2- 172.21.15.16:0/2308039497 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f13e00b65a0 0x7f13e00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.397+0000 7f13f3400640 1 -- 172.21.15.16:0/2308039497 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f13e40e7bd0 con 0x7f13fc105300 2024-09-21T03:45:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.397+0000 7f13fa800640 1 --2- 172.21.15.16:0/2308039497 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f13e00b65a0 0x7f13e00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.398+0000 7f13fa800640 1 --2- 172.21.15.16:0/2308039497 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f13e00b65a0 0x7f13e00b8ab0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f13fc104140 tx=0x7f13ec005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.398+0000 7f13f3400640 1 -- 172.21.15.16:0/2308039497 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f13e40adef0 con 0x7f13fc105300 2024-09-21T03:45:03.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.598+0000 7f1401400640 1 -- 172.21.15.16:0/2308039497 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f13fc1039f0 con 0x7f13fc105300 2024-09-21T03:45:03.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.599+0000 7f13f3400640 1 -- 172.21.15.16:0/2308039497 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f13e40b22d0 con 0x7f13fc105300 2024-09-21T03:45:03.601 INFO:teuthology.orchestra.run.smithi016.stdout:176093659201 2024-09-21T03:45:03.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.601+0000 7f1401400640 1 -- 172.21.15.16:0/2308039497 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f13e00b65a0 msgr2=0x7f13e00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.601+0000 7f1401400640 1 --2- 172.21.15.16:0/2308039497 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f13e00b65a0 0x7f13e00b8ab0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f13fc104140 tx=0x7f13ec005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:03.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.601+0000 7f1401400640 1 -- 172.21.15.16:0/2308039497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13fc105300 msgr2=0x7f13fc102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.601+0000 7f1401400640 1 --2- 172.21.15.16:0/2308039497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13fc105300 0x7f13fc102c30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f13e40058c0 tx=0x7f13e400ee50 comp rx=0 tx=0).stop 2024-09-21T03:45:03.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.601+0000 7f1401400640 1 -- 172.21.15.16:0/2308039497 shutdown_connections 2024-09-21T03:45:03.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.601+0000 7f1401400640 1 --2- 172.21.15.16:0/2308039497 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f13e00b65a0 0x7f13e00b8ab0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.601+0000 7f1401400640 1 --2- 172.21.15.16:0/2308039497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13fc10de10 0x7f13fc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.601+0000 7f1401400640 1 --2- 172.21.15.16:0/2308039497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13fc105300 0x7f13fc102c30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.601+0000 7f1401400640 1 --2- 172.21.15.16:0/2308039497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13fc1049b0 0x7f13fc1026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.601+0000 7f1401400640 1 -- 172.21.15.16:0/2308039497 >> 172.21.15.16:0/2308039497 conn(0x7f13fc079760 msgr2=0x7f13fc107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:03.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.601+0000 7f1401400640 1 -- 172.21.15.16:0/2308039497 shutdown_connections 2024-09-21T03:45:03.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.602+0000 7f1401400640 1 -- 172.21.15.16:0/2308039497 wait complete. 2024-09-21T03:45:03.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:03 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2308039497' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:45:03.957 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135918 got 244813135919 for osd.9 2024-09-21T03:45:03.957 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:03.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.974+0000 7f8213000640 1 -- 172.21.15.16:0/1560435820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f820c105d80 msgr2=0x7f820c106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.974+0000 7f8213000640 1 --2- 172.21.15.16:0/1560435820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f820c105d80 0x7f820c106190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009900 tx=0x7f81fc02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.974+0000 7f8213000640 1 -- 172.21.15.16:0/1560435820 shutdown_connections 2024-09-21T03:45:03.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.974+0000 7f8213000640 1 --2- 172.21.15.16:0/1560435820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f820c10c010 0x7f820c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.974+0000 7f8213000640 1 --2- 172.21.15.16:0/1560435820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f820c10b6c0 0x7f820c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.974+0000 7f8213000640 1 --2- 172.21.15.16:0/1560435820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f820c105d80 0x7f820c106190 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.974+0000 7f8213000640 1 -- 172.21.15.16:0/1560435820 >> 172.21.15.16:0/1560435820 conn(0x7f820c101810 msgr2=0x7f820c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:03.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.974+0000 7f8213000640 1 -- 172.21.15.16:0/1560435820 shutdown_connections 2024-09-21T03:45:03.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.974+0000 7f8213000640 1 -- 172.21.15.16:0/1560435820 wait complete. 2024-09-21T03:45:03.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.975+0000 7f8213000640 1 Processor -- start 2024-09-21T03:45:03.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.975+0000 7f8213000640 1 -- start start 2024-09-21T03:45:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.975+0000 7f8213000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f820c105d80 0x7f820c19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.975+0000 7f8213000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f820c10b6c0 0x7f820c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.976+0000 7f8213000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f820c10c010 0x7f820c1a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.976+0000 7f8213000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f820c115cd0 con 0x7f820c10b6c0 2024-09-21T03:45:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.976+0000 7f8213000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f820c115b50 con 0x7f820c10c010 2024-09-21T03:45:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.976+0000 7f8213000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f820c115e50 con 0x7f820c105d80 2024-09-21T03:45:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.976+0000 7f8203e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f820c10b6c0 0x7f820c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.976+0000 7f8203e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f820c10b6c0 0x7f820c19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60744/0 (socket says 172.21.15.16:60744) 2024-09-21T03:45:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.976+0000 7f8203e00640 1 -- 172.21.15.16:0/577576266 learned_addr learned my addr 172.21.15.16:0/577576266 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.976+0000 7f820be00640 1 --2- 172.21.15.16:0/577576266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f820c105d80 0x7f820c19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.976+0000 7f8211000640 1 --2- 172.21.15.16:0/577576266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f820c10c010 0x7f820c1a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.976+0000 7f8203e00640 1 -- 172.21.15.16:0/577576266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f820c105d80 msgr2=0x7f820c19c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.976+0000 7f8203e00640 1 --2- 172.21.15.16:0/577576266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f820c105d80 0x7f820c19c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.976+0000 7f8203e00640 1 -- 172.21.15.16:0/577576266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f820c10c010 msgr2=0x7f820c1a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.976+0000 7f8203e00640 1 --2- 172.21.15.16:0/577576266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f820c10c010 0x7f820c1a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.976+0000 7f8203e00640 1 -- 172.21.15.16:0/577576266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820c1a2610 con 0x7f820c10b6c0 2024-09-21T03:45:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.977+0000 7f8203e00640 1 --2- 172.21.15.16:0/577576266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f820c10b6c0 0x7f820c19cc40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f81f400e9f0 tx=0x7f81f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.977+0000 7f8209600640 1 -- 172.21.15.16:0/577576266 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f400ccc0 con 0x7f820c10b6c0 2024-09-21T03:45:03.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.977+0000 7f8209600640 1 -- 172.21.15.16:0/577576266 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81f40108e0 con 0x7f820c10b6c0 2024-09-21T03:45:03.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.977+0000 7f8213000640 1 -- 172.21.15.16:0/577576266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f820c1a2900 con 0x7f820c10b6c0 2024-09-21T03:45:03.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.977+0000 7f8209600640 1 -- 172.21.15.16:0/577576266 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4018860 con 0x7f820c10b6c0 2024-09-21T03:45:03.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.977+0000 7f8213000640 1 -- 172.21.15.16:0/577576266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f820c1a9060 con 0x7f820c10b6c0 2024-09-21T03:45:03.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.978+0000 7f8213000640 1 -- 172.21.15.16:0/577576266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f820c106190 con 0x7f820c10b6c0 2024-09-21T03:45:03.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.981+0000 7f8209600640 1 -- 172.21.15.16:0/577576266 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f81f4010430 con 0x7f820c10b6c0 2024-09-21T03:45:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.983+0000 7f8209600640 1 --2- 172.21.15.16:0/577576266 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f81f00b65a0 0x7f81f00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.984+0000 7f8209600640 1 -- 172.21.15.16:0/577576266 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f81f40e8240 con 0x7f820c10b6c0 2024-09-21T03:45:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.984+0000 7f820be00640 1 --2- 172.21.15.16:0/577576266 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f81f00b65a0 0x7f81f00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:03.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.984+0000 7f820be00640 1 --2- 172.21.15.16:0/577576266 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f81f00b65a0 0x7f81f00b8ab0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f81fc031440 tx=0x7f81fc009560 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:03.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:03.986+0000 7f8209600640 1 -- 172.21.15.16:0/577576266 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f81f40b2940 con 0x7f820c10b6c0 2024-09-21T03:45:04.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:03 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2308039497' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:45:04.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:03 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2308039497' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:45:04.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:04.190+0000 7f8213000640 1 -- 172.21.15.16:0/577576266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f820c02e640 con 0x7f820c10b6c0 2024-09-21T03:45:04.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:04.190+0000 7f8209600640 1 -- 172.21.15.16:0/577576266 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f81f40b7880 con 0x7f820c10b6c0 2024-09-21T03:45:04.192 INFO:teuthology.orchestra.run.smithi016.stdout:128849018957 2024-09-21T03:45:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:04.193+0000 7f8213000640 1 -- 172.21.15.16:0/577576266 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f81f00b65a0 msgr2=0x7f81f00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:04.193+0000 7f8213000640 1 --2- 172.21.15.16:0/577576266 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f81f00b65a0 0x7f81f00b8ab0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f81fc031440 tx=0x7f81fc009560 comp rx=0 tx=0).stop 2024-09-21T03:45:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:04.193+0000 7f8213000640 1 -- 172.21.15.16:0/577576266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f820c10b6c0 msgr2=0x7f820c19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:04.193+0000 7f8213000640 1 --2- 172.21.15.16:0/577576266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f820c10b6c0 0x7f820c19cc40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f81f400e9f0 tx=0x7f81f400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:45:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:04.193+0000 7f8213000640 1 -- 172.21.15.16:0/577576266 shutdown_connections 2024-09-21T03:45:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:04.193+0000 7f8213000640 1 --2- 172.21.15.16:0/577576266 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f81f00b65a0 0x7f81f00b8ab0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:04.193+0000 7f8213000640 1 --2- 172.21.15.16:0/577576266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f820c10c010 0x7f820c1a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:04.193+0000 7f8213000640 1 --2- 172.21.15.16:0/577576266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f820c10b6c0 0x7f820c19cc40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:04.193+0000 7f8213000640 1 --2- 172.21.15.16:0/577576266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f820c105d80 0x7f820c19c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:04.193+0000 7f8213000640 1 -- 172.21.15.16:0/577576266 >> 172.21.15.16:0/577576266 conn(0x7f820c101810 msgr2=0x7f820c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:04.193+0000 7f8213000640 1 -- 172.21.15.16:0/577576266 shutdown_connections 2024-09-21T03:45:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:04.193+0000 7f8213000640 1 -- 172.21.15.16:0/577576266 wait complete. 2024-09-21T03:45:04.892 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:04 smithi016 ceph-mon[30375]: pgmap v381: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T03:45:04.892 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:04 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/577576266' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:45:04.931 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659198 got 176093659201 for osd.6 2024-09-21T03:45:04.931 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:05.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:04 smithi134 ceph-mon[35031]: pgmap v381: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T03:45:05.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:04 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/577576266' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:45:05.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:04 smithi123 ceph-mon[34673]: pgmap v381: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T03:45:05.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:04 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/577576266' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:45:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.243+0000 7f9be1000640 1 -- 172.21.15.16:0/2112930289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bdc103b80 msgr2=0x7f9bdc103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.243+0000 7f9be1000640 1 --2- 172.21.15.16:0/2112930289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bdc103b80 0x7f9bdc103f90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc009860 tx=0x7f9bcc02a340 comp rx=0 tx=0).stop 2024-09-21T03:45:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.243+0000 7f9be1000640 1 -- 172.21.15.16:0/2112930289 shutdown_connections 2024-09-21T03:45:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.243+0000 7f9be1000640 1 --2- 172.21.15.16:0/2112930289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bdc109e10 0x7f9bdc110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.243+0000 7f9be1000640 1 --2- 172.21.15.16:0/2112930289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bdc1094c0 0x7f9bdc1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.243+0000 7f9be1000640 1 --2- 172.21.15.16:0/2112930289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bdc103b80 0x7f9bdc103f90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.243+0000 7f9be1000640 1 -- 172.21.15.16:0/2112930289 >> 172.21.15.16:0/2112930289 conn(0x7f9bdc0ff590 msgr2=0x7f9bdc1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.243+0000 7f9be1000640 1 -- 172.21.15.16:0/2112930289 shutdown_connections 2024-09-21T03:45:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.243+0000 7f9be1000640 1 -- 172.21.15.16:0/2112930289 wait complete. 2024-09-21T03:45:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.244+0000 7f9be1000640 1 Processor -- start 2024-09-21T03:45:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.245+0000 7f9be1000640 1 -- start start 2024-09-21T03:45:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.245+0000 7f9be1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bdc103b80 0x7f9bdc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.245+0000 7f9be1000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bdc1094c0 0x7f9bdc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.245+0000 7f9be1000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bdc109e10 0x7f9bdc076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.245+0000 7f9be1000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc113ad0 con 0x7f9bdc103b80 2024-09-21T03:45:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.245+0000 7f9be1000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc113950 con 0x7f9bdc109e10 2024-09-21T03:45:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.245+0000 7f9be1000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc113c50 con 0x7f9bdc1094c0 2024-09-21T03:45:05.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.245+0000 7f9bd9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bdc103b80 0x7f9bdc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:05.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.245+0000 7f9bd9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bdc103b80 0x7f9bdc075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60748/0 (socket says 172.21.15.16:60748) 2024-09-21T03:45:05.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.245+0000 7f9bd9e00640 1 -- 172.21.15.16:0/3138353736 learned_addr learned my addr 172.21.15.16:0/3138353736 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:05.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.245+0000 7f9bd3e00640 1 --2- 172.21.15.16:0/3138353736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bdc1094c0 0x7f9bdc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:05.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.245+0000 7f9bda800640 1 --2- 172.21.15.16:0/3138353736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bdc109e10 0x7f9bdc076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:05.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.246+0000 7f9bd3e00640 1 -- 172.21.15.16:0/3138353736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bdc109e10 msgr2=0x7f9bdc076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:05.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.246+0000 7f9bd3e00640 1 --2- 172.21.15.16:0/3138353736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bdc109e10 0x7f9bdc076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:05.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.246+0000 7f9bd3e00640 1 -- 172.21.15.16:0/3138353736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bdc103b80 msgr2=0x7f9bdc075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:05.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.246+0000 7f9bd3e00640 1 --2- 172.21.15.16:0/3138353736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bdc103b80 0x7f9bdc075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:05.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.246+0000 7f9bd3e00640 1 -- 172.21.15.16:0/3138353736 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bdc1a87a0 con 0x7f9bdc1094c0 2024-09-21T03:45:05.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.246+0000 7f9bd9e00640 1 --2- 172.21.15.16:0/3138353736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bdc103b80 0x7f9bdc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:05.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.246+0000 7f9bda800640 1 --2- 172.21.15.16:0/3138353736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bdc109e10 0x7f9bdc076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:05.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.246+0000 7f9bd3e00640 1 --2- 172.21.15.16:0/3138353736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bdc1094c0 0x7f9bdc075c50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc06aa90 tx=0x7f9bc400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:05.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.247+0000 7f9bd2a00640 1 -- 172.21.15.16:0/3138353736 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc400cc40 con 0x7f9bdc1094c0 2024-09-21T03:45:05.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.247+0000 7f9bd2a00640 1 -- 172.21.15.16:0/3138353736 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc400cde0 con 0x7f9bdc1094c0 2024-09-21T03:45:05.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.247+0000 7f9be1000640 1 -- 172.21.15.16:0/3138353736 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bdc1a8a90 con 0x7f9bdc1094c0 2024-09-21T03:45:05.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.247+0000 7f9bd2a00640 1 -- 172.21.15.16:0/3138353736 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4018780 con 0x7f9bdc1094c0 2024-09-21T03:45:05.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.247+0000 7f9be1000640 1 -- 172.21.15.16:0/3138353736 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bdc1a9020 con 0x7f9bdc1094c0 2024-09-21T03:45:05.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.248+0000 7f9be1000640 1 -- 172.21.15.16:0/3138353736 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bdc103f90 con 0x7f9bdc1094c0 2024-09-21T03:45:05.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.252+0000 7f9bd2a00640 1 -- 172.21.15.16:0/3138353736 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f9bc4010a90 con 0x7f9bdc1094c0 2024-09-21T03:45:05.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.254+0000 7f9bd2a00640 1 --2- 172.21.15.16:0/3138353736 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9bc00b6620 0x7f9bc00b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.254+0000 7f9bd2a00640 1 -- 172.21.15.16:0/3138353736 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f9bc40e7d10 con 0x7f9bdc1094c0 2024-09-21T03:45:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.254+0000 7f9bd9e00640 1 --2- 172.21.15.16:0/3138353736 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9bc00b6620 0x7f9bc00b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.254+0000 7f9bd2a00640 1 -- 172.21.15.16:0/3138353736 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9bc40e8170 con 0x7f9bdc1094c0 2024-09-21T03:45:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.255+0000 7f9bd9e00640 1 --2- 172.21.15.16:0/3138353736 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9bc00b6620 0x7f9bc00b8b30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0057c0 tx=0x7f9bcc030340 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:05.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.475+0000 7f9be1000640 1 -- 172.21.15.16:0/3138353736 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9bdc02e640 con 0x7f9bdc1094c0 2024-09-21T03:45:05.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.476+0000 7f9bd2a00640 1 -- 172.21.15.16:0/3138353736 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9bc40b2410 con 0x7f9bdc1094c0 2024-09-21T03:45:05.478 INFO:teuthology.orchestra.run.smithi016.stdout:150323855431 2024-09-21T03:45:05.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.478+0000 7f9be1000640 1 -- 172.21.15.16:0/3138353736 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9bc00b6620 msgr2=0x7f9bc00b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:05.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.478+0000 7f9be1000640 1 --2- 172.21.15.16:0/3138353736 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9bc00b6620 0x7f9bc00b8b30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0057c0 tx=0x7f9bcc030340 comp rx=0 tx=0).stop 2024-09-21T03:45:05.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.478+0000 7f9be1000640 1 -- 172.21.15.16:0/3138353736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bdc1094c0 msgr2=0x7f9bdc075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:05.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.478+0000 7f9be1000640 1 --2- 172.21.15.16:0/3138353736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bdc1094c0 0x7f9bdc075c50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc06aa90 tx=0x7f9bc400ee50 comp rx=0 tx=0).stop 2024-09-21T03:45:05.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.478+0000 7f9be1000640 1 -- 172.21.15.16:0/3138353736 shutdown_connections 2024-09-21T03:45:05.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.478+0000 7f9be1000640 1 --2- 172.21.15.16:0/3138353736 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9bc00b6620 0x7f9bc00b8b30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:05.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.478+0000 7f9be1000640 1 --2- 172.21.15.16:0/3138353736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bdc109e10 0x7f9bdc076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:05.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.478+0000 7f9be1000640 1 --2- 172.21.15.16:0/3138353736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bdc1094c0 0x7f9bdc075c50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:05.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.478+0000 7f9be1000640 1 --2- 172.21.15.16:0/3138353736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bdc103b80 0x7f9bdc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:05.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.478+0000 7f9be1000640 1 -- 172.21.15.16:0/3138353736 >> 172.21.15.16:0/3138353736 conn(0x7f9bdc0ff590 msgr2=0x7f9bdc102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:05.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.479+0000 7f9be1000640 1 -- 172.21.15.16:0/3138353736 shutdown_connections 2024-09-21T03:45:05.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:05.479+0000 7f9be1000640 1 -- 172.21.15.16:0/3138353736 wait complete. 2024-09-21T03:45:06.071 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:06.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:45:06.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:06.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:05 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:06.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:05 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3138353736' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:45:06.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:45:06.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:06.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:05 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:06.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:05 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3138353736' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:45:06.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:45:06.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:06.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:05 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:06.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:05 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3138353736' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:45:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.536+0000 7f29a8e00640 1 -- 172.21.15.16:0/4075701875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a410be40 msgr2=0x7f29a4112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.536+0000 7f29a8e00640 1 --2- 172.21.15.16:0/4075701875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a410be40 0x7f29a4112670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f299c009ca0 tx=0x7f299c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.537+0000 7f29a8e00640 1 -- 172.21.15.16:0/4075701875 shutdown_connections 2024-09-21T03:45:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.537+0000 7f29a8e00640 1 --2- 172.21.15.16:0/4075701875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a410be40 0x7f29a4112670 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.537+0000 7f29a8e00640 1 --2- 172.21.15.16:0/4075701875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a410b4f0 0x7f29a410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.537+0000 7f29a8e00640 1 --2- 172.21.15.16:0/4075701875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a4105bb0 0x7f29a4105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.537+0000 7f29a8e00640 1 -- 172.21.15.16:0/4075701875 >> 172.21.15.16:0/4075701875 conn(0x7f29a4101680 msgr2=0x7f29a4103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:06.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.537+0000 7f29a8e00640 1 -- 172.21.15.16:0/4075701875 shutdown_connections 2024-09-21T03:45:06.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.537+0000 7f29a8e00640 1 -- 172.21.15.16:0/4075701875 wait complete. 2024-09-21T03:45:06.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.538+0000 7f29a8e00640 1 Processor -- start 2024-09-21T03:45:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.538+0000 7f29a8e00640 1 -- start start 2024-09-21T03:45:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.539+0000 7f29a8e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a4105bb0 0x7f29a419c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.539+0000 7f29a8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a410b4f0 0x7f29a419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.539+0000 7f29a8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a410be40 0x7f29a41a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.539+0000 7f29a8e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4115b00 con 0x7f29a410b4f0 2024-09-21T03:45:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.539+0000 7f29a8e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4115980 con 0x7f29a410be40 2024-09-21T03:45:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.539+0000 7f29a8e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4115c80 con 0x7f29a4105bb0 2024-09-21T03:45:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.539+0000 7f299be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a410b4f0 0x7f29a419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.539+0000 7f29a3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a4105bb0 0x7f29a419c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.539+0000 7f299be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a410b4f0 0x7f29a419ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60772/0 (socket says 172.21.15.16:60772) 2024-09-21T03:45:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.539+0000 7f299be00640 1 -- 172.21.15.16:0/2008052780 learned_addr learned my addr 172.21.15.16:0/2008052780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.539+0000 7f29a3e00640 1 --2- 172.21.15.16:0/2008052780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a410be40 0x7f29a41a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.540+0000 7f29a3400640 1 -- 172.21.15.16:0/2008052780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a410be40 msgr2=0x7f29a41a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.540+0000 7f29a3400640 1 --2- 172.21.15.16:0/2008052780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a410be40 0x7f29a41a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.540+0000 7f29a3400640 1 -- 172.21.15.16:0/2008052780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a410b4f0 msgr2=0x7f29a419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.540+0000 7f29a3400640 1 --2- 172.21.15.16:0/2008052780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a410b4f0 0x7f29a419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.540+0000 7f29a3400640 1 -- 172.21.15.16:0/2008052780 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a41a2420 con 0x7f29a4105bb0 2024-09-21T03:45:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.540+0000 7f299be00640 1 --2- 172.21.15.16:0/2008052780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a410b4f0 0x7f29a419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.540+0000 7f29a3400640 1 --2- 172.21.15.16:0/2008052780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a4105bb0 0x7f29a419c4b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f299400e9f0 tx=0x7f299400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.540+0000 7f29a0c00640 1 -- 172.21.15.16:0/2008052780 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299400cd80 con 0x7f29a4105bb0 2024-09-21T03:45:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.540+0000 7f29a0c00640 1 -- 172.21.15.16:0/2008052780 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29940035f0 con 0x7f29a4105bb0 2024-09-21T03:45:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.540+0000 7f29a0c00640 1 -- 172.21.15.16:0/2008052780 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994010670 con 0x7f29a4105bb0 2024-09-21T03:45:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.540+0000 7f29a8e00640 1 -- 172.21.15.16:0/2008052780 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a41a2710 con 0x7f29a4105bb0 2024-09-21T03:45:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.540+0000 7f29a8e00640 1 -- 172.21.15.16:0/2008052780 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a41a8e70 con 0x7f29a4105bb0 2024-09-21T03:45:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.542+0000 7f29a8e00640 1 -- 172.21.15.16:0/2008052780 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a4105fc0 con 0x7f29a4105bb0 2024-09-21T03:45:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.544+0000 7f29a0c00640 1 -- 172.21.15.16:0/2008052780 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f29940037a0 con 0x7f29a4105bb0 2024-09-21T03:45:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.546+0000 7f29a0c00640 1 --2- 172.21.15.16:0/2008052780 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f29880b6620 0x7f29880b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.546+0000 7f29a0c00640 1 -- 172.21.15.16:0/2008052780 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f29940e7eb0 con 0x7f29a4105bb0 2024-09-21T03:45:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.546+0000 7f299be00640 1 --2- 172.21.15.16:0/2008052780 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f29880b6620 0x7f29880b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.547+0000 7f299be00640 1 --2- 172.21.15.16:0/2008052780 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f29880b6620 0x7f29880b8b30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f29a4107e80 tx=0x7f298c005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.547+0000 7f29a0c00640 1 -- 172.21.15.16:0/2008052780 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f29940ae1d0 con 0x7f29a4105bb0 2024-09-21T03:45:06.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.747+0000 7f29a8e00640 1 -- 172.21.15.16:0/2008052780 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f29a419d5e0 con 0x7f29a4105bb0 2024-09-21T03:45:06.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.748+0000 7f29a0c00640 1 -- 172.21.15.16:0/2008052780 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f29940b25b0 con 0x7f29a4105bb0 2024-09-21T03:45:06.750 INFO:teuthology.orchestra.run.smithi016.stdout:266287972396 2024-09-21T03:45:06.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.750+0000 7f29a8e00640 1 -- 172.21.15.16:0/2008052780 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f29880b6620 msgr2=0x7f29880b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.750+0000 7f29a8e00640 1 --2- 172.21.15.16:0/2008052780 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f29880b6620 0x7f29880b8b30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f29a4107e80 tx=0x7f298c005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.750+0000 7f29a8e00640 1 -- 172.21.15.16:0/2008052780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a4105bb0 msgr2=0x7f29a419c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.750+0000 7f29a8e00640 1 --2- 172.21.15.16:0/2008052780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a4105bb0 0x7f29a419c4b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f299400e9f0 tx=0x7f299400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.750+0000 7f29a8e00640 1 -- 172.21.15.16:0/2008052780 shutdown_connections 2024-09-21T03:45:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.750+0000 7f29a8e00640 1 --2- 172.21.15.16:0/2008052780 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f29880b6620 0x7f29880b8b30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.750+0000 7f29a8e00640 1 --2- 172.21.15.16:0/2008052780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a410be40 0x7f29a41a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.750+0000 7f29a8e00640 1 --2- 172.21.15.16:0/2008052780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a410b4f0 0x7f29a419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.750+0000 7f29a8e00640 1 --2- 172.21.15.16:0/2008052780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a4105bb0 0x7f29a419c4b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.750+0000 7f29a8e00640 1 -- 172.21.15.16:0/2008052780 >> 172.21.15.16:0/2008052780 conn(0x7f29a4101680 msgr2=0x7f29a410abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.751+0000 7f29a8e00640 1 -- 172.21.15.16:0/2008052780 shutdown_connections 2024-09-21T03:45:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:06.751+0000 7f29a8e00640 1 -- 172.21.15.16:0/2008052780 wait complete. 2024-09-21T03:45:06.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:06 smithi016 ceph-mon[30375]: pgmap v382: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T03:45:06.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:06 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:06 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:06 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:06 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:06.975 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215182 got 103079215185 for osd.3 2024-09-21T03:45:06.975 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:07.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:06 smithi134 ceph-mon[35031]: pgmap v382: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T03:45:07.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:06 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:07.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:06 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:07.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:06 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:07.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:06 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:07.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:06 smithi123 ceph-mon[34673]: pgmap v382: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T03:45:07.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:06 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:07.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:06 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:07.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:06 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:07.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:06 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:07.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.571+0000 7f3139600640 1 -- 172.21.15.16:0/3121545660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f313410c010 msgr2=0x7f3134112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.571+0000 7f3139600640 1 --2- 172.21.15.16:0/3121545660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f313410c010 0x7f3134112840 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f312c009ca0 tx=0x7f312c02a8b0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.571+0000 7f3139600640 1 -- 172.21.15.16:0/3121545660 shutdown_connections 2024-09-21T03:45:07.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.571+0000 7f3139600640 1 --2- 172.21.15.16:0/3121545660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f313410c010 0x7f3134112840 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.571+0000 7f3139600640 1 --2- 172.21.15.16:0/3121545660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f313410b6c0 0x7f313410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.571+0000 7f3139600640 1 --2- 172.21.15.16:0/3121545660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3134105d80 0x7f3134106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.571+0000 7f3139600640 1 -- 172.21.15.16:0/3121545660 >> 172.21.15.16:0/3121545660 conn(0x7f3134101810 msgr2=0x7f3134103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.571+0000 7f3139600640 1 -- 172.21.15.16:0/3121545660 shutdown_connections 2024-09-21T03:45:07.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.571+0000 7f3139600640 1 -- 172.21.15.16:0/3121545660 wait complete. 2024-09-21T03:45:07.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.572+0000 7f3139600640 1 Processor -- start 2024-09-21T03:45:07.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.572+0000 7f3139600640 1 -- start start 2024-09-21T03:45:07.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.573+0000 7f3139600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3134105d80 0x7f313419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.573+0000 7f3139600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f313410b6c0 0x7f313419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.573+0000 7f3139600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f313410c010 0x7f31341a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.573+0000 7f3139600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3134115d20 con 0x7f313410b6c0 2024-09-21T03:45:07.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.573+0000 7f3139600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3134115ba0 con 0x7f3134105d80 2024-09-21T03:45:07.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.573+0000 7f3139600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3134115ea0 con 0x7f313410c010 2024-09-21T03:45:07.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.573+0000 7f312a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f313410b6c0 0x7f313419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.573+0000 7f3132a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3134105d80 0x7f313419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.573+0000 7f312a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f313410b6c0 0x7f313419cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60798/0 (socket says 172.21.15.16:60798) 2024-09-21T03:45:07.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.573+0000 7f312a000640 1 -- 172.21.15.16:0/3944342945 learned_addr learned my addr 172.21.15.16:0/3944342945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.573+0000 7f3133400640 1 --2- 172.21.15.16:0/3944342945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f313410c010 0x7f31341a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.573+0000 7f312a000640 1 -- 172.21.15.16:0/3944342945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f313410c010 msgr2=0x7f31341a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.573+0000 7f312a000640 1 --2- 172.21.15.16:0/3944342945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f313410c010 0x7f31341a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.573+0000 7f312a000640 1 -- 172.21.15.16:0/3944342945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3134105d80 msgr2=0x7f313419c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.573+0000 7f312a000640 1 --2- 172.21.15.16:0/3944342945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3134105d80 0x7f313419c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.574+0000 7f312a000640 1 -- 172.21.15.16:0/3944342945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31341a25c0 con 0x7f313410b6c0 2024-09-21T03:45:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.574+0000 7f3132a00640 1 --2- 172.21.15.16:0/3944342945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3134105d80 0x7f313419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.574+0000 7f312a000640 1 --2- 172.21.15.16:0/3944342945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f313410b6c0 0x7f313419cbf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f311c00d740 tx=0x7f311c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.574+0000 7f312be00640 1 -- 172.21.15.16:0/3944342945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c017070 con 0x7f313410b6c0 2024-09-21T03:45:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.574+0000 7f312be00640 1 -- 172.21.15.16:0/3944342945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f311c00de50 con 0x7f313410b6c0 2024-09-21T03:45:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.574+0000 7f312be00640 1 -- 172.21.15.16:0/3944342945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c00f620 con 0x7f313410b6c0 2024-09-21T03:45:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.574+0000 7f3139600640 1 -- 172.21.15.16:0/3944342945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31341a28b0 con 0x7f313410b6c0 2024-09-21T03:45:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.574+0000 7f3139600640 1 -- 172.21.15.16:0/3944342945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31341a9010 con 0x7f313410b6c0 2024-09-21T03:45:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.575+0000 7f3139600640 1 -- 172.21.15.16:0/3944342945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3134106190 con 0x7f313410b6c0 2024-09-21T03:45:07.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.578+0000 7f312be00640 1 -- 172.21.15.16:0/3944342945 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f311c01a070 con 0x7f313410b6c0 2024-09-21T03:45:07.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.580+0000 7f312be00640 1 --2- 172.21.15.16:0/3944342945 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f31180b6620 0x7f31180b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.580+0000 7f312be00640 1 -- 172.21.15.16:0/3944342945 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f311c0e7130 con 0x7f313410b6c0 2024-09-21T03:45:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.580+0000 7f3132a00640 1 --2- 172.21.15.16:0/3944342945 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f31180b6620 0x7f31180b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.581+0000 7f3132a00640 1 --2- 172.21.15.16:0/3944342945 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f31180b6620 0x7f31180b8b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f312400bfd0 tx=0x7f312400b390 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.582+0000 7f312be00640 1 -- 172.21.15.16:0/3944342945 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f311c0b18b0 con 0x7f313410b6c0 2024-09-21T03:45:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.785+0000 7f3139600640 1 -- 172.21.15.16:0/3944342945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f313419d780 con 0x7f313410b6c0 2024-09-21T03:45:07.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.785+0000 7f312be00640 1 -- 172.21.15.16:0/3944342945 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f313419d780 con 0x7f313410b6c0 2024-09-21T03:45:07.787 INFO:teuthology.orchestra.run.smithi016.stdout:219043332151 2024-09-21T03:45:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.788+0000 7f3139600640 1 -- 172.21.15.16:0/3944342945 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f31180b6620 msgr2=0x7f31180b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.788+0000 7f3139600640 1 --2- 172.21.15.16:0/3944342945 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f31180b6620 0x7f31180b8b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f312400bfd0 tx=0x7f312400b390 comp rx=0 tx=0).stop 2024-09-21T03:45:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.788+0000 7f3139600640 1 -- 172.21.15.16:0/3944342945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f313410b6c0 msgr2=0x7f313419cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.788+0000 7f3139600640 1 --2- 172.21.15.16:0/3944342945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f313410b6c0 0x7f313419cbf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f311c00d740 tx=0x7f311c00dc00 comp rx=0 tx=0).stop 2024-09-21T03:45:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.788+0000 7f3139600640 1 -- 172.21.15.16:0/3944342945 shutdown_connections 2024-09-21T03:45:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.788+0000 7f3139600640 1 --2- 172.21.15.16:0/3944342945 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f31180b6620 0x7f31180b8b30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.788+0000 7f3139600640 1 --2- 172.21.15.16:0/3944342945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f313410c010 0x7f31341a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.788+0000 7f3139600640 1 --2- 172.21.15.16:0/3944342945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f313410b6c0 0x7f313419cbf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.788+0000 7f3139600640 1 --2- 172.21.15.16:0/3944342945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3134105d80 0x7f313419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.788+0000 7f3139600640 1 -- 172.21.15.16:0/3944342945 >> 172.21.15.16:0/3944342945 conn(0x7f3134101810 msgr2=0x7f3134108c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.788+0000 7f3139600640 1 -- 172.21.15.16:0/3944342945 shutdown_connections 2024-09-21T03:45:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:07.788+0000 7f3139600640 1 -- 172.21.15.16:0/3944342945 wait complete. 2024-09-21T03:45:07.796 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:07 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2008052780' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:45:08.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:07 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2008052780' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:45:08.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:07 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2008052780' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:45:08.178 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018954 got 128849018957 for osd.4 2024-09-21T03:45:08.178 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:08.506 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972394 got 266287972396 for osd.10 2024-09-21T03:45:08.506 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:09.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:08 smithi016 ceph-mon[30375]: pgmap v383: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T03:45:09.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:08 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3944342945' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:45:09.101 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855427 got 150323855431 for osd.5 2024-09-21T03:45:09.102 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:09.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:08 smithi134 ceph-mon[35031]: pgmap v383: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T03:45:09.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:08 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3944342945' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:45:09.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:08 smithi123 ceph-mon[34673]: pgmap v383: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T03:45:09.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:08 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3944342945' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:45:09.500 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444117 got 73014444120 for osd.2 2024-09-21T03:45:09.500 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:10.279 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332146 got 219043332151 for osd.8 2024-09-21T03:45:10.279 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.568+0000 7fd357e00640 1 -- 172.21.15.16:0/434461078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd358076100 msgr2=0x7fd35810f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.568+0000 7fd357e00640 1 --2- 172.21.15.16:0/434461078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd358076100 0x7fd35810f9d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd344009860 tx=0x7fd34402a340 comp rx=0 tx=0).stop 2024-09-21T03:45:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.568+0000 7fd357e00640 1 -- 172.21.15.16:0/434461078 shutdown_connections 2024-09-21T03:45:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.568+0000 7fd357e00640 1 --2- 172.21.15.16:0/434461078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd35810ff10 0x7fd358112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.568+0000 7fd357e00640 1 --2- 172.21.15.16:0/434461078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd358076100 0x7fd35810f9d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.568+0000 7fd357e00640 1 --2- 172.21.15.16:0/434461078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3580757b0 0x7fd358075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.568+0000 7fd357e00640 1 -- 172.21.15.16:0/434461078 >> 172.21.15.16:0/434461078 conn(0x7fd3580ff440 msgr2=0x7fd358101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:10.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.568+0000 7fd357e00640 1 -- 172.21.15.16:0/434461078 shutdown_connections 2024-09-21T03:45:10.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.568+0000 7fd357e00640 1 -- 172.21.15.16:0/434461078 wait complete. 2024-09-21T03:45:10.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.570+0000 7fd357e00640 1 Processor -- start 2024-09-21T03:45:10.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.570+0000 7fd357e00640 1 -- start start 2024-09-21T03:45:10.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.570+0000 7fd357e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3580757b0 0x7fd35819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.570+0000 7fd357e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd358076100 0x7fd35819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.570+0000 7fd357e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd35810ff10 0x7fd3581a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.570+0000 7fd357e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd358115a80 con 0x7fd358076100 2024-09-21T03:45:10.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.570+0000 7fd357e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd358115900 con 0x7fd3580757b0 2024-09-21T03:45:10.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.570+0000 7fd357e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd358115c00 con 0x7fd35810ff10 2024-09-21T03:45:10.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.571+0000 7fd356a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3580757b0 0x7fd35819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.571+0000 7fd356000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd358076100 0x7fd35819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.571+0000 7fd356000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd358076100 0x7fd35819ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60814/0 (socket says 172.21.15.16:60814) 2024-09-21T03:45:10.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.571+0000 7fd356000640 1 -- 172.21.15.16:0/3342720434 learned_addr learned my addr 172.21.15.16:0/3342720434 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:10.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.571+0000 7fd357400640 1 --2- 172.21.15.16:0/3342720434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd35810ff10 0x7fd3581a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.571+0000 7fd356a00640 1 -- 172.21.15.16:0/3342720434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd35810ff10 msgr2=0x7fd3581a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.571+0000 7fd356a00640 1 --2- 172.21.15.16:0/3342720434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd35810ff10 0x7fd3581a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.571+0000 7fd356a00640 1 -- 172.21.15.16:0/3342720434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd358076100 msgr2=0x7fd35819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.571+0000 7fd356a00640 1 --2- 172.21.15.16:0/3342720434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd358076100 0x7fd35819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.571+0000 7fd356a00640 1 -- 172.21.15.16:0/3342720434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3581a2420 con 0x7fd3580757b0 2024-09-21T03:45:10.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.572+0000 7fd356000640 1 --2- 172.21.15.16:0/3342720434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd358076100 0x7fd35819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:10.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.572+0000 7fd357400640 1 --2- 172.21.15.16:0/3342720434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd35810ff10 0x7fd3581a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:10.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.572+0000 7fd356a00640 1 --2- 172.21.15.16:0/3342720434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3580757b0 0x7fd35819c4b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd34800e790 tx=0x7fd34800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:10.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.572+0000 7fd34f400640 1 -- 172.21.15.16:0/3342720434 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34800eea0 con 0x7fd3580757b0 2024-09-21T03:45:10.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.572+0000 7fd34f400640 1 -- 172.21.15.16:0/3342720434 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd34800ccc0 con 0x7fd3580757b0 2024-09-21T03:45:10.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.572+0000 7fd34f400640 1 -- 172.21.15.16:0/3342720434 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348018660 con 0x7fd3580757b0 2024-09-21T03:45:10.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.572+0000 7fd357e00640 1 -- 172.21.15.16:0/3342720434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3581a2710 con 0x7fd3580757b0 2024-09-21T03:45:10.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.573+0000 7fd357e00640 1 -- 172.21.15.16:0/3342720434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3581a8e70 con 0x7fd3580757b0 2024-09-21T03:45:10.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.574+0000 7fd357e00640 1 -- 172.21.15.16:0/3342720434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd358075bc0 con 0x7fd3580757b0 2024-09-21T03:45:10.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.577+0000 7fd34f400640 1 -- 172.21.15.16:0/3342720434 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fd348005030 con 0x7fd3580757b0 2024-09-21T03:45:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.578+0000 7fd34f400640 1 --2- 172.21.15.16:0/3342720434 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd33c0b65a0 0x7fd33c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.579+0000 7fd34f400640 1 -- 172.21.15.16:0/3342720434 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fd3480e7f30 con 0x7fd3580757b0 2024-09-21T03:45:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.579+0000 7fd356000640 1 --2- 172.21.15.16:0/3342720434 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd33c0b65a0 0x7fd33c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.580+0000 7fd356000640 1 --2- 172.21.15.16:0/3342720434 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd33c0b65a0 0x7fd33c0b8ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd3581122c0 tx=0x7fd344005d30 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.580+0000 7fd34f400640 1 -- 172.21.15.16:0/3342720434 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd3480b2630 con 0x7fd3580757b0 2024-09-21T03:45:10.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.795+0000 7fd357e00640 1 -- 172.21.15.16:0/3342720434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fd3581a9110 con 0x7fd3580757b0 2024-09-21T03:45:10.798 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:10 smithi016 ceph-mon[30375]: pgmap v384: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-21T03:45:10.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.796+0000 7fd34f400640 1 -- 172.21.15.16:0/3342720434 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd3480b7570 con 0x7fd3580757b0 2024-09-21T03:45:10.799 INFO:teuthology.orchestra.run.smithi016.stdout:287762808872 2024-09-21T03:45:10.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.799+0000 7fd357e00640 1 -- 172.21.15.16:0/3342720434 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd33c0b65a0 msgr2=0x7fd33c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.799+0000 7fd357e00640 1 --2- 172.21.15.16:0/3342720434 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd33c0b65a0 0x7fd33c0b8ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd3581122c0 tx=0x7fd344005d30 comp rx=0 tx=0).stop 2024-09-21T03:45:10.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.799+0000 7fd357e00640 1 -- 172.21.15.16:0/3342720434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3580757b0 msgr2=0x7fd35819c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:10.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.799+0000 7fd357e00640 1 --2- 172.21.15.16:0/3342720434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3580757b0 0x7fd35819c4b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd34800e790 tx=0x7fd34800ec50 comp rx=0 tx=0).stop 2024-09-21T03:45:10.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.799+0000 7fd357e00640 1 -- 172.21.15.16:0/3342720434 shutdown_connections 2024-09-21T03:45:10.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.799+0000 7fd357e00640 1 --2- 172.21.15.16:0/3342720434 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd33c0b65a0 0x7fd33c0b8ab0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.799+0000 7fd357e00640 1 --2- 172.21.15.16:0/3342720434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd35810ff10 0x7fd3581a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.799+0000 7fd357e00640 1 --2- 172.21.15.16:0/3342720434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd358076100 0x7fd35819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.799+0000 7fd357e00640 1 --2- 172.21.15.16:0/3342720434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3580757b0 0x7fd35819c4b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:10.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.799+0000 7fd357e00640 1 -- 172.21.15.16:0/3342720434 >> 172.21.15.16:0/3342720434 conn(0x7fd3580ff440 msgr2=0x7fd358109a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:10.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.799+0000 7fd357e00640 1 -- 172.21.15.16:0/3342720434 shutdown_connections 2024-09-21T03:45:10.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:10.799+0000 7fd357e00640 1 -- 172.21.15.16:0/3342720434 wait complete. 2024-09-21T03:45:11.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:10 smithi134 ceph-mon[35031]: pgmap v384: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-21T03:45:11.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:10 smithi123 ceph-mon[34673]: pgmap v384: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-21T03:45:12.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:11 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3342720434' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:45:12.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:11 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3342720434' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:45:12.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:11 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3342720434' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:45:12.239 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808868 got 287762808872 for osd.11 2024-09-21T03:45:12.239 DEBUG:teuthology.parallel:result is None 2024-09-21T03:45:12.239 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T03:45:12.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:45:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:13.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:12 smithi134 ceph-mon[35031]: pgmap v385: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:45:13.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:12 smithi016 ceph-mon[30375]: pgmap v385: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:45:13.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:12 smithi123 ceph-mon[34673]: pgmap v385: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:45:13.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.294+0000 7f5260e00640 1 -- 172.21.15.16:0/3616505789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f525c109c40 msgr2=0x7f525c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:13.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.294+0000 7f5260e00640 1 --2- 172.21.15.16:0/3616505789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f525c109c40 0x7f525c110470 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5254009ca0 tx=0x7f525402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.294+0000 7f5260e00640 1 -- 172.21.15.16:0/3616505789 shutdown_connections 2024-09-21T03:45:13.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.294+0000 7f5260e00640 1 --2- 172.21.15.16:0/3616505789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f525c109c40 0x7f525c110470 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.294+0000 7f5260e00640 1 --2- 172.21.15.16:0/3616505789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f525c1092f0 0x7f525c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.294+0000 7f5260e00640 1 --2- 172.21.15.16:0/3616505789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f525c1039b0 0x7f525c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.294+0000 7f5260e00640 1 -- 172.21.15.16:0/3616505789 >> 172.21.15.16:0/3616505789 conn(0x7f525c0ff440 msgr2=0x7f525c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:13.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.294+0000 7f5260e00640 1 -- 172.21.15.16:0/3616505789 shutdown_connections 2024-09-21T03:45:13.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.294+0000 7f5260e00640 1 -- 172.21.15.16:0/3616505789 wait complete. 2024-09-21T03:45:13.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.296+0000 7f5260e00640 1 Processor -- start 2024-09-21T03:45:13.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.296+0000 7f5260e00640 1 -- start start 2024-09-21T03:45:13.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.296+0000 7f5260e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f525c1039b0 0x7f525c19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:13.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.297+0000 7f5260e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f525c1092f0 0x7f525c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:13.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.297+0000 7f5260e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f525c109c40 0x7f525c1a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:13.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.297+0000 7f5260e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f525c113950 con 0x7f525c109c40 2024-09-21T03:45:13.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.297+0000 7f5260e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f525c1137d0 con 0x7f525c1039b0 2024-09-21T03:45:13.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.297+0000 7f5260e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f525c113ad0 con 0x7f525c1092f0 2024-09-21T03:45:13.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.297+0000 7f525be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f525c109c40 0x7f525c1a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:13.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.297+0000 7f525be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f525c109c40 0x7f525c1a1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46144/0 (socket says 172.21.15.16:46144) 2024-09-21T03:45:13.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.297+0000 7f525be00640 1 -- 172.21.15.16:0/1397436091 learned_addr learned my addr 172.21.15.16:0/1397436091 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:13.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.297+0000 7f5252a00640 1 --2- 172.21.15.16:0/1397436091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f525c1092f0 0x7f525c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:13.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.297+0000 7f525b400640 1 --2- 172.21.15.16:0/1397436091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f525c1039b0 0x7f525c19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:13.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.297+0000 7f5252a00640 1 -- 172.21.15.16:0/1397436091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f525c1039b0 msgr2=0x7f525c19c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:13.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.297+0000 7f5252a00640 1 --2- 172.21.15.16:0/1397436091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f525c1039b0 0x7f525c19c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.297+0000 7f5252a00640 1 -- 172.21.15.16:0/1397436091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f525c109c40 msgr2=0x7f525c1a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:13.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.297+0000 7f5252a00640 1 --2- 172.21.15.16:0/1397436091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f525c109c40 0x7f525c1a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.297+0000 7f5252a00640 1 -- 172.21.15.16:0/1397436091 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525c1a2450 con 0x7f525c1092f0 2024-09-21T03:45:13.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.298+0000 7f5252a00640 1 --2- 172.21.15.16:0/1397436091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f525c1092f0 0x7f525c19ca80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f525c06b990 tx=0x7f524400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:13.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.298+0000 7f5258c00640 1 -- 172.21.15.16:0/1397436091 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524400cc40 con 0x7f525c1092f0 2024-09-21T03:45:13.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.298+0000 7f5258c00640 1 -- 172.21.15.16:0/1397436091 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f524400cde0 con 0x7f525c1092f0 2024-09-21T03:45:13.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.298+0000 7f5260e00640 1 -- 172.21.15.16:0/1397436091 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f525c1a2740 con 0x7f525c1092f0 2024-09-21T03:45:13.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.298+0000 7f5258c00640 1 -- 172.21.15.16:0/1397436091 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5244018780 con 0x7f525c1092f0 2024-09-21T03:45:13.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.298+0000 7f5260e00640 1 -- 172.21.15.16:0/1397436091 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f525c1a8ea0 con 0x7f525c1092f0 2024-09-21T03:45:13.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.300+0000 7f5260e00640 1 -- 172.21.15.16:0/1397436091 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f525c103dc0 con 0x7f525c1092f0 2024-09-21T03:45:13.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.302+0000 7f5258c00640 1 -- 172.21.15.16:0/1397436091 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f5244010a90 con 0x7f525c1092f0 2024-09-21T03:45:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.305+0000 7f5258c00640 1 --2- 172.21.15.16:0/1397436091 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f52400b65a0 0x7f52400b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:13.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.305+0000 7f5258c00640 1 -- 172.21.15.16:0/1397436091 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f52440e7d90 con 0x7f525c1092f0 2024-09-21T03:45:13.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.305+0000 7f525b400640 1 --2- 172.21.15.16:0/1397436091 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f52400b65a0 0x7f52400b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:13.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.306+0000 7f5258c00640 1 -- 172.21.15.16:0/1397436091 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f52440b2490 con 0x7f525c1092f0 2024-09-21T03:45:13.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.306+0000 7f525b400640 1 --2- 172.21.15.16:0/1397436091 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f52400b65a0 0x7f52400b8ab0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f524c009bd0 tx=0x7f524c005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:13.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.516+0000 7f5260e00640 1 -- 172.21.15.16:0/1397436091 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f525c10f740 con 0x7f52400b65a0 2024-09-21T03:45:13.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.523+0000 7f5258c00640 1 -- 172.21.15.16:0/1397436091 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340407 (secure 0 0 0) 0x7f525c10f740 con 0x7f52400b65a0 2024-09-21T03:45:13.529 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:45:13.530 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-21T03:45:13.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.528+0000 7f5260e00640 1 -- 172.21.15.16:0/1397436091 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f52400b65a0 msgr2=0x7f52400b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:13.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.528+0000 7f5260e00640 1 --2- 172.21.15.16:0/1397436091 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f52400b65a0 0x7f52400b8ab0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f524c009bd0 tx=0x7f524c005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:13.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.528+0000 7f5260e00640 1 -- 172.21.15.16:0/1397436091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f525c1092f0 msgr2=0x7f525c19ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:13.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.528+0000 7f5260e00640 1 --2- 172.21.15.16:0/1397436091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f525c1092f0 0x7f525c19ca80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f525c06b990 tx=0x7f524400ee50 comp rx=0 tx=0).stop 2024-09-21T03:45:13.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.528+0000 7f5260e00640 1 -- 172.21.15.16:0/1397436091 shutdown_connections 2024-09-21T03:45:13.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.528+0000 7f5260e00640 1 --2- 172.21.15.16:0/1397436091 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f52400b65a0 0x7f52400b8ab0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.528+0000 7f5260e00640 1 --2- 172.21.15.16:0/1397436091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f525c109c40 0x7f525c1a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.528+0000 7f5260e00640 1 --2- 172.21.15.16:0/1397436091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f525c1092f0 0x7f525c19ca80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.528+0000 7f5260e00640 1 --2- 172.21.15.16:0/1397436091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f525c1039b0 0x7f525c19c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:13.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.528+0000 7f5260e00640 1 -- 172.21.15.16:0/1397436091 >> 172.21.15.16:0/1397436091 conn(0x7f525c0ff440 msgr2=0x7f525c106850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:13.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.528+0000 7f5260e00640 1 -- 172.21.15.16:0/1397436091 shutdown_connections 2024-09-21T03:45:13.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:13.528+0000 7f5260e00640 1 -- 172.21.15.16:0/1397436091 wait complete. 2024-09-21T03:45:14.322 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-09-21T03:45:13.364216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664482,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":768,"num_read_kb":494,"num_write":374,"num_write_kb":2297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2951952,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379832,"kb_used_data":13868,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561992,"statfs":{"total":1151940427776,"available":1151551479808,"internally_reserved":0,"allocated":14200832,"data_stored":7969604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":129,"num_read_kb":69,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003644"},"pg_stats":[{"pgid":"2.1f","version":"84'13","reported_seq":124,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:09.646552+0000","last_change":"2024-09-21T03:43:04.980132+0000","last_active":"2024-09-21T03:45:09.646552+0000","last_peered":"2024-09-21T03:45:09.646552+0000","last_clean":"2024-09-21T03:45:09.646552+0000","last_became_active":"2024-09-21T03:43:04.979274+0000","last_became_peered":"2024-09-21T03:43:04.979274+0000","last_unstale":"2024-09-21T03:45:09.646552+0000","last_undegraded":"2024-09-21T03:45:09.646552+0000","last_fullsized":"2024-09-21T03:45:09.646552+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:35.047129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.672221+0000","last_change":"2024-09-21T03:43:08.011576+0000","last_active":"2024-09-21T03:44:54.672221+0000","last_peered":"2024-09-21T03:44:54.672221+0000","last_clean":"2024-09-21T03:44:54.672221+0000","last_became_active":"2024-09-21T03:43:08.011188+0000","last_became_peered":"2024-09-21T03:43:08.011188+0000","last_unstale":"2024-09-21T03:44:54.672221+0000","last_undegraded":"2024-09-21T03:44:54.672221+0000","last_fullsized":"2024-09-21T03:44:54.672221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:34.477639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677992+0000","last_change":"2024-09-21T03:43:54.493183+0000","last_active":"2024-09-21T03:44:54.677992+0000","last_peered":"2024-09-21T03:44:54.677992+0000","last_clean":"2024-09-21T03:44:54.677992+0000","last_became_active":"2024-09-21T03:43:54.492605+0000","last_became_peered":"2024-09-21T03:43:54.492605+0000","last_unstale":"2024-09-21T03:44:54.677992+0000","last_undegraded":"2024-09-21T03:44:54.677992+0000","last_fullsized":"2024-09-21T03:44:54.677992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:26.634042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.672352+0000","last_change":"2024-09-21T03:43:05.141790+0000","last_active":"2024-09-21T03:44:54.672352+0000","last_peered":"2024-09-21T03:44:54.672352+0000","last_clean":"2024-09-21T03:44:54.672352+0000","last_became_active":"2024-09-21T03:43:05.141299+0000","last_became_peered":"2024-09-21T03:43:05.141299+0000","last_unstale":"2024-09-21T03:44:54.672352+0000","last_undegraded":"2024-09-21T03:44:54.672352+0000","last_fullsized":"2024-09-21T03:44:54.672352+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:48.844615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679988+0000","last_change":"2024-09-21T03:43:08.018245+0000","last_active":"2024-09-21T03:44:54.679988+0000","last_peered":"2024-09-21T03:44:54.679988+0000","last_clean":"2024-09-21T03:44:54.679988+0000","last_became_active":"2024-09-21T03:43:08.015719+0000","last_became_peered":"2024-09-21T03:43:08.015719+0000","last_unstale":"2024-09-21T03:44:54.679988+0000","last_undegraded":"2024-09-21T03:44:54.679988+0000","last_fullsized":"2024-09-21T03:44:54.679988+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:39.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766917+0000","last_change":"2024-09-21T03:43:54.493279+0000","last_active":"2024-09-21T03:44:54.766917+0000","last_peered":"2024-09-21T03:44:54.766917+0000","last_clean":"2024-09-21T03:44:54.766917+0000","last_became_active":"2024-09-21T03:43:54.492592+0000","last_became_peered":"2024-09-21T03:43:54.492592+0000","last_unstale":"2024-09-21T03:44:54.766917+0000","last_undegraded":"2024-09-21T03:44:54.766917+0000","last_fullsized":"2024-09-21T03:44:54.766917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:28.520990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.693998+0000","last_change":"2024-09-21T03:44:54.693431+0000","last_active":"2024-09-21T03:44:54.693998+0000","last_peered":"2024-09-21T03:44:54.693998+0000","last_clean":"2024-09-21T03:44:54.693998+0000","last_became_active":"2024-09-21T03:44:54.680324+0000","last_became_peered":"2024-09-21T03:44:54.680324+0000","last_unstale":"2024-09-21T03:44:54.693998+0000","last_undegraded":"2024-09-21T03:44:54.693998+0000","last_fullsized":"2024-09-21T03:44:54.693998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:43.730697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767573+0000","last_change":"2024-09-21T03:43:08.009018+0000","last_active":"2024-09-21T03:44:54.767573+0000","last_peered":"2024-09-21T03:44:54.767573+0000","last_clean":"2024-09-21T03:44:54.767573+0000","last_became_active":"2024-09-21T03:43:08.008614+0000","last_became_peered":"2024-09-21T03:43:08.008614+0000","last_unstale":"2024-09-21T03:44:54.767573+0000","last_undegraded":"2024-09-21T03:44:54.767573+0000","last_fullsized":"2024-09-21T03:44:54.767573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:31.000616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768700+0000","last_change":"2024-09-21T03:43:18.092750+0000","last_active":"2024-09-21T03:44:54.768700+0000","last_peered":"2024-09-21T03:44:54.768700+0000","last_clean":"2024-09-21T03:44:54.768700+0000","last_became_active":"2024-09-21T03:43:18.092122+0000","last_became_peered":"2024-09-21T03:43:18.092122+0000","last_unstale":"2024-09-21T03:44:54.768700+0000","last_undegraded":"2024-09-21T03:44:54.768700+0000","last_fullsized":"2024-09-21T03:44:54.768700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:15.032388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675502+0000","last_change":"2024-09-21T03:43:04.978045+0000","last_active":"2024-09-21T03:44:54.675502+0000","last_peered":"2024-09-21T03:44:54.675502+0000","last_clean":"2024-09-21T03:44:54.675502+0000","last_became_active":"2024-09-21T03:43:04.977565+0000","last_became_peered":"2024-09-21T03:43:04.977565+0000","last_unstale":"2024-09-21T03:44:54.675502+0000","last_undegraded":"2024-09-21T03:44:54.675502+0000","last_fullsized":"2024-09-21T03:44:54.675502+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:42.537892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766856+0000","last_change":"2024-09-21T03:43:08.007094+0000","last_active":"2024-09-21T03:44:54.766856+0000","last_peered":"2024-09-21T03:44:54.766856+0000","last_clean":"2024-09-21T03:44:54.766856+0000","last_became_active":"2024-09-21T03:43:08.006666+0000","last_became_peered":"2024-09-21T03:43:08.006666+0000","last_unstale":"2024-09-21T03:44:54.766856+0000","last_undegraded":"2024-09-21T03:44:54.766856+0000","last_fullsized":"2024-09-21T03:44:54.766856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:59.247459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767074+0000","last_change":"2024-09-21T03:43:18.081141+0000","last_active":"2024-09-21T03:44:54.767074+0000","last_peered":"2024-09-21T03:44:54.767074+0000","last_clean":"2024-09-21T03:44:54.767074+0000","last_became_active":"2024-09-21T03:43:18.080623+0000","last_became_peered":"2024-09-21T03:43:18.080623+0000","last_unstale":"2024-09-21T03:44:54.767074+0000","last_undegraded":"2024-09-21T03:44:54.767074+0000","last_fullsized":"2024-09-21T03:44:54.767074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:44.932673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678247+0000","last_change":"2024-09-21T03:43:08.008602+0000","last_active":"2024-09-21T03:44:54.678247+0000","last_peered":"2024-09-21T03:44:54.678247+0000","last_clean":"2024-09-21T03:44:54.678247+0000","last_became_active":"2024-09-21T03:43:08.008220+0000","last_became_peered":"2024-09-21T03:43:08.008220+0000","last_unstale":"2024-09-21T03:44:54.678247+0000","last_undegraded":"2024-09-21T03:44:54.678247+0000","last_fullsized":"2024-09-21T03:44:54.678247+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:28.286016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":63,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.774006+0000","last_change":"2024-09-21T03:44:54.773918+0000","last_active":"2024-09-21T03:44:54.774006+0000","last_peered":"2024-09-21T03:44:54.774006+0000","last_clean":"2024-09-21T03:44:54.774006+0000","last_became_active":"2024-09-21T03:44:54.769721+0000","last_became_peered":"2024-09-21T03:44:54.769721+0000","last_unstale":"2024-09-21T03:44:54.774006+0000","last_undegraded":"2024-09-21T03:44:54.774006+0000","last_fullsized":"2024-09-21T03:44:54.774006+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:05.656051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767123+0000","last_change":"2024-09-21T03:43:18.483523+0000","last_active":"2024-09-21T03:44:54.767123+0000","last_peered":"2024-09-21T03:44:54.767123+0000","last_clean":"2024-09-21T03:44:54.767123+0000","last_became_active":"2024-09-21T03:43:18.482833+0000","last_became_peered":"2024-09-21T03:43:18.482833+0000","last_unstale":"2024-09-21T03:44:54.767123+0000","last_undegraded":"2024-09-21T03:44:54.767123+0000","last_fullsized":"2024-09-21T03:44:54.767123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:48.236569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671717+0000","last_change":"2024-09-21T03:43:08.022592+0000","last_active":"2024-09-21T03:44:54.671717+0000","last_peered":"2024-09-21T03:44:54.671717+0000","last_clean":"2024-09-21T03:44:54.671717+0000","last_became_active":"2024-09-21T03:43:08.018065+0000","last_became_peered":"2024-09-21T03:43:08.018065+0000","last_unstale":"2024-09-21T03:44:54.671717+0000","last_undegraded":"2024-09-21T03:44:54.671717+0000","last_fullsized":"2024-09-21T03:44:54.671717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:41.524402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679199+0000","last_change":"2024-09-21T03:43:04.978788+0000","last_active":"2024-09-21T03:44:54.679199+0000","last_peered":"2024-09-21T03:44:54.679199+0000","last_clean":"2024-09-21T03:44:54.679199+0000","last_became_active":"2024-09-21T03:43:04.978382+0000","last_became_peered":"2024-09-21T03:43:04.978382+0000","last_unstale":"2024-09-21T03:44:54.679199+0000","last_undegraded":"2024-09-21T03:44:54.679199+0000","last_fullsized":"2024-09-21T03:44:54.679199+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:12.571217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768191+0000","last_change":"2024-09-21T03:43:54.493347+0000","last_active":"2024-09-21T03:44:54.768191+0000","last_peered":"2024-09-21T03:44:54.768191+0000","last_clean":"2024-09-21T03:44:54.768191+0000","last_became_active":"2024-09-21T03:43:54.492598+0000","last_became_peered":"2024-09-21T03:43:54.492598+0000","last_unstale":"2024-09-21T03:44:54.768191+0000","last_undegraded":"2024-09-21T03:44:54.768191+0000","last_fullsized":"2024-09-21T03:44:54.768191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:34.788296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678063+0000","last_change":"2024-09-21T03:43:08.014368+0000","last_active":"2024-09-21T03:44:54.678063+0000","last_peered":"2024-09-21T03:44:54.678063+0000","last_clean":"2024-09-21T03:44:54.678063+0000","last_became_active":"2024-09-21T03:43:08.013952+0000","last_became_peered":"2024-09-21T03:43:08.013952+0000","last_unstale":"2024-09-21T03:44:54.678063+0000","last_undegraded":"2024-09-21T03:44:54.678063+0000","last_fullsized":"2024-09-21T03:44:54.678063+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:16.893588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671939+0000","last_change":"2024-09-21T03:43:04.976162+0000","last_active":"2024-09-21T03:44:54.671939+0000","last_peered":"2024-09-21T03:44:54.671939+0000","last_clean":"2024-09-21T03:44:54.671939+0000","last_became_active":"2024-09-21T03:43:04.973790+0000","last_became_peered":"2024-09-21T03:43:04.973790+0000","last_unstale":"2024-09-21T03:44:54.671939+0000","last_undegraded":"2024-09-21T03:44:54.671939+0000","last_fullsized":"2024-09-21T03:44:54.671939+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:47.838229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670284+0000","last_change":"2024-09-21T03:43:18.084344+0000","last_active":"2024-09-21T03:44:54.670284+0000","last_peered":"2024-09-21T03:44:54.670284+0000","last_clean":"2024-09-21T03:44:54.670284+0000","last_became_active":"2024-09-21T03:43:18.083232+0000","last_became_peered":"2024-09-21T03:43:18.083232+0000","last_unstale":"2024-09-21T03:44:54.670284+0000","last_undegraded":"2024-09-21T03:44:54.670284+0000","last_fullsized":"2024-09-21T03:44:54.670284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:46.254405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.674815+0000","last_change":"2024-09-21T03:43:08.009932+0000","last_active":"2024-09-21T03:44:54.674815+0000","last_peered":"2024-09-21T03:44:54.674815+0000","last_clean":"2024-09-21T03:44:54.674815+0000","last_became_active":"2024-09-21T03:43:08.009492+0000","last_became_peered":"2024-09-21T03:43:08.009492+0000","last_unstale":"2024-09-21T03:44:54.674815+0000","last_undegraded":"2024-09-21T03:44:54.674815+0000","last_fullsized":"2024-09-21T03:44:54.674815+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:58.679348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767989+0000","last_change":"2024-09-21T03:43:05.141927+0000","last_active":"2024-09-21T03:44:54.767989+0000","last_peered":"2024-09-21T03:44:54.767989+0000","last_clean":"2024-09-21T03:44:54.767989+0000","last_became_active":"2024-09-21T03:43:05.141516+0000","last_became_peered":"2024-09-21T03:43:05.141516+0000","last_unstale":"2024-09-21T03:44:54.767989+0000","last_undegraded":"2024-09-21T03:44:54.767989+0000","last_fullsized":"2024-09-21T03:44:54.767989+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:35.666981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767323+0000","last_change":"2024-09-21T03:43:18.484478+0000","last_active":"2024-09-21T03:44:54.767323+0000","last_peered":"2024-09-21T03:44:54.767323+0000","last_clean":"2024-09-21T03:44:54.767323+0000","last_became_active":"2024-09-21T03:43:18.483890+0000","last_became_peered":"2024-09-21T03:43:18.483890+0000","last_unstale":"2024-09-21T03:44:54.767323+0000","last_undegraded":"2024-09-21T03:44:54.767323+0000","last_fullsized":"2024-09-21T03:44:54.767323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:03.542783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679792+0000","last_change":"2024-09-21T03:43:08.017785+0000","last_active":"2024-09-21T03:44:54.679792+0000","last_peered":"2024-09-21T03:44:54.679792+0000","last_clean":"2024-09-21T03:44:54.679792+0000","last_became_active":"2024-09-21T03:43:08.017389+0000","last_became_peered":"2024-09-21T03:43:08.017389+0000","last_unstale":"2024-09-21T03:44:54.679792+0000","last_undegraded":"2024-09-21T03:44:54.679792+0000","last_fullsized":"2024-09-21T03:44:54.679792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:37.557061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767595+0000","last_change":"2024-09-21T03:43:05.567829+0000","last_active":"2024-09-21T03:44:54.767595+0000","last_peered":"2024-09-21T03:44:54.767595+0000","last_clean":"2024-09-21T03:44:54.767595+0000","last_became_active":"2024-09-21T03:43:05.567401+0000","last_became_peered":"2024-09-21T03:43:05.567401+0000","last_unstale":"2024-09-21T03:44:54.767595+0000","last_undegraded":"2024-09-21T03:44:54.767595+0000","last_fullsized":"2024-09-21T03:44:54.767595+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:03.954280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678801+0000","last_change":"2024-09-21T03:43:18.080064+0000","last_active":"2024-09-21T03:44:54.678801+0000","last_peered":"2024-09-21T03:44:54.678801+0000","last_clean":"2024-09-21T03:44:54.678801+0000","last_became_active":"2024-09-21T03:43:18.079569+0000","last_became_peered":"2024-09-21T03:43:18.079569+0000","last_unstale":"2024-09-21T03:44:54.678801+0000","last_undegraded":"2024-09-21T03:44:54.678801+0000","last_fullsized":"2024-09-21T03:44:54.678801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:29.568476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.672366+0000","last_change":"2024-09-21T03:43:08.011603+0000","last_active":"2024-09-21T03:44:54.672366+0000","last_peered":"2024-09-21T03:44:54.672366+0000","last_clean":"2024-09-21T03:44:54.672366+0000","last_became_active":"2024-09-21T03:43:08.011078+0000","last_became_peered":"2024-09-21T03:43:08.011078+0000","last_unstale":"2024-09-21T03:44:54.672366+0000","last_undegraded":"2024-09-21T03:44:54.672366+0000","last_fullsized":"2024-09-21T03:44:54.672366+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:44.764713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'15","reported_seq":60,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670668+0000","last_change":"2024-09-21T03:43:04.979223+0000","last_active":"2024-09-21T03:44:54.670668+0000","last_peered":"2024-09-21T03:44:54.670668+0000","last_clean":"2024-09-21T03:44:54.670668+0000","last_became_active":"2024-09-21T03:43:04.978675+0000","last_became_peered":"2024-09-21T03:43:04.978675+0000","last_unstale":"2024-09-21T03:44:54.670668+0000","last_undegraded":"2024-09-21T03:44:54.670668+0000","last_fullsized":"2024-09-21T03:44:54.670668+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:09.717921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671303+0000","last_change":"2024-09-21T03:43:18.482334+0000","last_active":"2024-09-21T03:44:54.671303+0000","last_peered":"2024-09-21T03:44:54.671303+0000","last_clean":"2024-09-21T03:44:54.671303+0000","last_became_active":"2024-09-21T03:43:18.481881+0000","last_became_peered":"2024-09-21T03:43:18.481881+0000","last_unstale":"2024-09-21T03:44:54.671303+0000","last_undegraded":"2024-09-21T03:44:54.671303+0000","last_fullsized":"2024-09-21T03:44:54.671303+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:02:32.497348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767021+0000","last_change":"2024-09-21T03:43:07.993771+0000","last_active":"2024-09-21T03:44:54.767021+0000","last_peered":"2024-09-21T03:44:54.767021+0000","last_clean":"2024-09-21T03:44:54.767021+0000","last_became_active":"2024-09-21T03:43:07.993268+0000","last_became_peered":"2024-09-21T03:43:07.993268+0000","last_unstale":"2024-09-21T03:44:54.767021+0000","last_undegraded":"2024-09-21T03:44:54.767021+0000","last_fullsized":"2024-09-21T03:44:54.767021+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:42.348561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675923+0000","last_change":"2024-09-21T03:43:05.141786+0000","last_active":"2024-09-21T03:44:54.675923+0000","last_peered":"2024-09-21T03:44:54.675923+0000","last_clean":"2024-09-21T03:44:54.675923+0000","last_became_active":"2024-09-21T03:43:05.141307+0000","last_became_peered":"2024-09-21T03:43:05.141307+0000","last_unstale":"2024-09-21T03:44:54.675923+0000","last_undegraded":"2024-09-21T03:44:54.675923+0000","last_fullsized":"2024-09-21T03:44:54.675923+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:12.139398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677302+0000","last_change":"2024-09-21T03:43:18.087848+0000","last_active":"2024-09-21T03:44:54.677302+0000","last_peered":"2024-09-21T03:44:54.677302+0000","last_clean":"2024-09-21T03:44:54.677302+0000","last_became_active":"2024-09-21T03:43:18.087400+0000","last_became_peered":"2024-09-21T03:43:18.087400+0000","last_unstale":"2024-09-21T03:44:54.677302+0000","last_undegraded":"2024-09-21T03:44:54.677302+0000","last_fullsized":"2024-09-21T03:44:54.677302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:46.902979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768961+0000","last_change":"2024-09-21T03:43:08.010051+0000","last_active":"2024-09-21T03:44:54.768961+0000","last_peered":"2024-09-21T03:44:54.768961+0000","last_clean":"2024-09-21T03:44:54.768961+0000","last_became_active":"2024-09-21T03:43:08.007700+0000","last_became_peered":"2024-09-21T03:43:08.007700+0000","last_unstale":"2024-09-21T03:44:54.768961+0000","last_undegraded":"2024-09-21T03:44:54.768961+0000","last_fullsized":"2024-09-21T03:44:54.768961+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:31.828193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670543+0000","last_change":"2024-09-21T03:43:05.140490+0000","last_active":"2024-09-21T03:44:54.670543+0000","last_peered":"2024-09-21T03:44:54.670543+0000","last_clean":"2024-09-21T03:44:54.670543+0000","last_became_active":"2024-09-21T03:43:05.139979+0000","last_became_peered":"2024-09-21T03:43:05.139979+0000","last_unstale":"2024-09-21T03:44:54.670543+0000","last_undegraded":"2024-09-21T03:44:54.670543+0000","last_fullsized":"2024-09-21T03:44:54.670543+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:20.957596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.674815+0000","last_change":"2024-09-21T03:43:18.482777+0000","last_active":"2024-09-21T03:44:54.674815+0000","last_peered":"2024-09-21T03:44:54.674815+0000","last_clean":"2024-09-21T03:44:54.674815+0000","last_became_active":"2024-09-21T03:43:18.482244+0000","last_became_peered":"2024-09-21T03:43:18.482244+0000","last_unstale":"2024-09-21T03:44:54.674815+0000","last_undegraded":"2024-09-21T03:44:54.674815+0000","last_fullsized":"2024-09-21T03:44:54.674815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:41.997309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768352+0000","last_change":"2024-09-21T03:43:08.019087+0000","last_active":"2024-09-21T03:44:54.768352+0000","last_peered":"2024-09-21T03:44:54.768352+0000","last_clean":"2024-09-21T03:44:54.768352+0000","last_became_active":"2024-09-21T03:43:08.018467+0000","last_became_peered":"2024-09-21T03:43:08.018467+0000","last_unstale":"2024-09-21T03:44:54.768352+0000","last_undegraded":"2024-09-21T03:44:54.768352+0000","last_fullsized":"2024-09-21T03:44:54.768352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:25.824955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"84'5","reported_seq":116,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:09.647247+0000","last_change":"2024-09-21T03:43:04.973433+0000","last_active":"2024-09-21T03:45:09.647247+0000","last_peered":"2024-09-21T03:45:09.647247+0000","last_clean":"2024-09-21T03:45:09.647247+0000","last_became_active":"2024-09-21T03:43:04.973033+0000","last_became_peered":"2024-09-21T03:43:04.973033+0000","last_unstale":"2024-09-21T03:45:09.647247+0000","last_undegraded":"2024-09-21T03:45:09.647247+0000","last_fullsized":"2024-09-21T03:45:09.647247+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:25.462635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766845+0000","last_change":"2024-09-21T03:43:18.082385+0000","last_active":"2024-09-21T03:44:54.766845+0000","last_peered":"2024-09-21T03:44:54.766845+0000","last_clean":"2024-09-21T03:44:54.766845+0000","last_became_active":"2024-09-21T03:43:18.081958+0000","last_became_peered":"2024-09-21T03:43:18.081958+0000","last_unstale":"2024-09-21T03:44:54.766845+0000","last_undegraded":"2024-09-21T03:44:54.766845+0000","last_fullsized":"2024-09-21T03:44:54.766845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:36.554838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766522+0000","last_change":"2024-09-21T03:43:08.008307+0000","last_active":"2024-09-21T03:44:54.766522+0000","last_peered":"2024-09-21T03:44:54.766522+0000","last_clean":"2024-09-21T03:44:54.766522+0000","last_became_active":"2024-09-21T03:43:08.006736+0000","last_became_peered":"2024-09-21T03:43:08.006736+0000","last_unstale":"2024-09-21T03:44:54.766522+0000","last_undegraded":"2024-09-21T03:44:54.766522+0000","last_fullsized":"2024-09-21T03:44:54.766522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:31.694655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":119,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:12.651467+0000","last_change":"2024-09-21T03:43:05.140263+0000","last_active":"2024-09-21T03:45:12.651467+0000","last_peered":"2024-09-21T03:45:12.651467+0000","last_clean":"2024-09-21T03:45:12.651467+0000","last_became_active":"2024-09-21T03:43:05.139869+0000","last_became_peered":"2024-09-21T03:43:05.139869+0000","last_unstale":"2024-09-21T03:45:12.651467+0000","last_undegraded":"2024-09-21T03:45:12.651467+0000","last_fullsized":"2024-09-21T03:45:12.651467+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:19.313677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766813+0000","last_change":"2024-09-21T03:43:18.483795+0000","last_active":"2024-09-21T03:44:54.766813+0000","last_peered":"2024-09-21T03:44:54.766813+0000","last_clean":"2024-09-21T03:44:54.766813+0000","last_became_active":"2024-09-21T03:43:18.483159+0000","last_became_peered":"2024-09-21T03:43:18.483159+0000","last_unstale":"2024-09-21T03:44:54.766813+0000","last_undegraded":"2024-09-21T03:44:54.766813+0000","last_fullsized":"2024-09-21T03:44:54.766813+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:56.277956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766443+0000","last_change":"2024-09-21T03:43:08.001661+0000","last_active":"2024-09-21T03:44:54.766443+0000","last_peered":"2024-09-21T03:44:54.766443+0000","last_clean":"2024-09-21T03:44:54.766443+0000","last_became_active":"2024-09-21T03:43:08.001254+0000","last_became_peered":"2024-09-21T03:43:08.001254+0000","last_unstale":"2024-09-21T03:44:54.766443+0000","last_undegraded":"2024-09-21T03:44:54.766443+0000","last_fullsized":"2024-09-21T03:44:54.766443+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:19.020542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678080+0000","last_change":"2024-09-21T03:43:04.979127+0000","last_active":"2024-09-21T03:44:54.678080+0000","last_peered":"2024-09-21T03:44:54.678080+0000","last_clean":"2024-09-21T03:44:54.678080+0000","last_became_active":"2024-09-21T03:43:04.978705+0000","last_became_peered":"2024-09-21T03:43:04.978705+0000","last_unstale":"2024-09-21T03:44:54.678080+0000","last_undegraded":"2024-09-21T03:44:54.678080+0000","last_fullsized":"2024-09-21T03:44:54.678080+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:48.548002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766916+0000","last_change":"2024-09-21T03:43:18.483682+0000","last_active":"2024-09-21T03:44:54.766916+0000","last_peered":"2024-09-21T03:44:54.766916+0000","last_clean":"2024-09-21T03:44:54.766916+0000","last_became_active":"2024-09-21T03:43:18.483124+0000","last_became_peered":"2024-09-21T03:43:18.483124+0000","last_unstale":"2024-09-21T03:44:54.766916+0000","last_undegraded":"2024-09-21T03:44:54.766916+0000","last_fullsized":"2024-09-21T03:44:54.766916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:12.513795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671793+0000","last_change":"2024-09-21T03:43:08.013193+0000","last_active":"2024-09-21T03:44:54.671793+0000","last_peered":"2024-09-21T03:44:54.671793+0000","last_clean":"2024-09-21T03:44:54.671793+0000","last_became_active":"2024-09-21T03:43:08.010360+0000","last_became_peered":"2024-09-21T03:43:08.010360+0000","last_unstale":"2024-09-21T03:44:54.671793+0000","last_undegraded":"2024-09-21T03:44:54.671793+0000","last_fullsized":"2024-09-21T03:44:54.671793+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:28.101646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768945+0000","last_change":"2024-09-21T03:43:04.971782+0000","last_active":"2024-09-21T03:44:54.768945+0000","last_peered":"2024-09-21T03:44:54.768945+0000","last_clean":"2024-09-21T03:44:54.768945+0000","last_became_active":"2024-09-21T03:43:04.971375+0000","last_became_peered":"2024-09-21T03:43:04.971375+0000","last_unstale":"2024-09-21T03:44:54.768945+0000","last_undegraded":"2024-09-21T03:44:54.768945+0000","last_fullsized":"2024-09-21T03:44:54.768945+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:49.573490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'92","reported_seq":115,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678174+0000","last_change":"2024-09-21T03:41:42.914435+0000","last_active":"2024-09-21T03:44:54.678174+0000","last_peered":"2024-09-21T03:44:54.678174+0000","last_clean":"2024-09-21T03:44:54.678174+0000","last_became_active":"2024-09-21T03:41:42.909490+0000","last_became_peered":"2024-09-21T03:41:42.909490+0000","last_unstale":"2024-09-21T03:44:54.678174+0000","last_undegraded":"2024-09-21T03:44:54.678174+0000","last_fullsized":"2024-09-21T03:44:54.678174+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:56.507027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:56.507027+0000","last_clean_scrub_stamp":"2024-09-21T03:37:56.507027+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:34.564062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678228+0000","last_change":"2024-09-21T03:43:18.084701+0000","last_active":"2024-09-21T03:44:54.678228+0000","last_peered":"2024-09-21T03:44:54.678228+0000","last_clean":"2024-09-21T03:44:54.678228+0000","last_became_active":"2024-09-21T03:43:18.084251+0000","last_became_peered":"2024-09-21T03:43:18.084251+0000","last_unstale":"2024-09-21T03:44:54.678228+0000","last_undegraded":"2024-09-21T03:44:54.678228+0000","last_fullsized":"2024-09-21T03:44:54.678228+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:15.660276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678714+0000","last_change":"2024-09-21T03:43:08.012195+0000","last_active":"2024-09-21T03:44:54.678714+0000","last_peered":"2024-09-21T03:44:54.678714+0000","last_clean":"2024-09-21T03:44:54.678714+0000","last_became_active":"2024-09-21T03:43:08.011670+0000","last_became_peered":"2024-09-21T03:43:08.011670+0000","last_unstale":"2024-09-21T03:44:54.678714+0000","last_undegraded":"2024-09-21T03:44:54.678714+0000","last_fullsized":"2024-09-21T03:44:54.678714+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:14.290074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":119,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:09.647333+0000","last_change":"2024-09-21T03:43:05.567898+0000","last_active":"2024-09-21T03:45:09.647333+0000","last_peered":"2024-09-21T03:45:09.647333+0000","last_clean":"2024-09-21T03:45:09.647333+0000","last_became_active":"2024-09-21T03:43:05.567367+0000","last_became_peered":"2024-09-21T03:43:05.567367+0000","last_unstale":"2024-09-21T03:45:09.647333+0000","last_undegraded":"2024-09-21T03:45:09.647333+0000","last_fullsized":"2024-09-21T03:45:09.647333+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:55.120838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768128+0000","last_change":"2024-09-21T03:43:18.073942+0000","last_active":"2024-09-21T03:44:54.768128+0000","last_peered":"2024-09-21T03:44:54.768128+0000","last_clean":"2024-09-21T03:44:54.768128+0000","last_became_active":"2024-09-21T03:43:18.073323+0000","last_became_peered":"2024-09-21T03:43:18.073323+0000","last_unstale":"2024-09-21T03:44:54.768128+0000","last_undegraded":"2024-09-21T03:44:54.768128+0000","last_fullsized":"2024-09-21T03:44:54.768128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:41.677143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678249+0000","last_change":"2024-09-21T03:43:08.016636+0000","last_active":"2024-09-21T03:44:54.678249+0000","last_peered":"2024-09-21T03:44:54.678249+0000","last_clean":"2024-09-21T03:44:54.678249+0000","last_became_active":"2024-09-21T03:43:08.016235+0000","last_became_peered":"2024-09-21T03:43:08.016235+0000","last_unstale":"2024-09-21T03:44:54.678249+0000","last_undegraded":"2024-09-21T03:44:54.678249+0000","last_fullsized":"2024-09-21T03:44:54.678249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:35.365830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767087+0000","last_change":"2024-09-21T03:43:04.980911+0000","last_active":"2024-09-21T03:44:54.767087+0000","last_peered":"2024-09-21T03:44:54.767087+0000","last_clean":"2024-09-21T03:44:54.767087+0000","last_became_active":"2024-09-21T03:43:04.980512+0000","last_became_peered":"2024-09-21T03:43:04.980512+0000","last_unstale":"2024-09-21T03:44:54.767087+0000","last_undegraded":"2024-09-21T03:44:54.767087+0000","last_fullsized":"2024-09-21T03:44:54.767087+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:04.136345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679872+0000","last_change":"2024-09-21T03:43:18.085874+0000","last_active":"2024-09-21T03:44:54.679872+0000","last_peered":"2024-09-21T03:44:54.679872+0000","last_clean":"2024-09-21T03:44:54.679872+0000","last_became_active":"2024-09-21T03:43:18.085381+0000","last_became_peered":"2024-09-21T03:43:18.085381+0000","last_unstale":"2024-09-21T03:44:54.679872+0000","last_undegraded":"2024-09-21T03:44:54.679872+0000","last_fullsized":"2024-09-21T03:44:54.679872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:02.829202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678621+0000","last_change":"2024-09-21T03:43:08.014776+0000","last_active":"2024-09-21T03:44:54.678621+0000","last_peered":"2024-09-21T03:44:54.678621+0000","last_clean":"2024-09-21T03:44:54.678621+0000","last_became_active":"2024-09-21T03:43:08.014191+0000","last_became_peered":"2024-09-21T03:43:08.014191+0000","last_unstale":"2024-09-21T03:44:54.678621+0000","last_undegraded":"2024-09-21T03:44:54.678621+0000","last_fullsized":"2024-09-21T03:44:54.678621+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:19.114838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766891+0000","last_change":"2024-09-21T03:43:04.973714+0000","last_active":"2024-09-21T03:44:54.766891+0000","last_peered":"2024-09-21T03:44:54.766891+0000","last_clean":"2024-09-21T03:44:54.766891+0000","last_became_active":"2024-09-21T03:43:04.973348+0000","last_became_peered":"2024-09-21T03:43:04.973348+0000","last_unstale":"2024-09-21T03:44:54.766891+0000","last_undegraded":"2024-09-21T03:44:54.766891+0000","last_fullsized":"2024-09-21T03:44:54.766891+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:00.872589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766289+0000","last_change":"2024-09-21T03:43:18.082643+0000","last_active":"2024-09-21T03:44:54.766289+0000","last_peered":"2024-09-21T03:44:54.766289+0000","last_clean":"2024-09-21T03:44:54.766289+0000","last_became_active":"2024-09-21T03:43:18.082157+0000","last_became_peered":"2024-09-21T03:43:18.082157+0000","last_unstale":"2024-09-21T03:44:54.766289+0000","last_undegraded":"2024-09-21T03:44:54.766289+0000","last_fullsized":"2024-09-21T03:44:54.766289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:52.652014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677306+0000","last_change":"2024-09-21T03:43:08.010913+0000","last_active":"2024-09-21T03:44:54.677306+0000","last_peered":"2024-09-21T03:44:54.677306+0000","last_clean":"2024-09-21T03:44:54.677306+0000","last_became_active":"2024-09-21T03:43:08.010483+0000","last_became_peered":"2024-09-21T03:43:08.010483+0000","last_unstale":"2024-09-21T03:44:54.677306+0000","last_undegraded":"2024-09-21T03:44:54.677306+0000","last_fullsized":"2024-09-21T03:44:54.677306+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:52.158346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675912+0000","last_change":"2024-09-21T03:43:04.980260+0000","last_active":"2024-09-21T03:44:54.675912+0000","last_peered":"2024-09-21T03:44:54.675912+0000","last_clean":"2024-09-21T03:44:54.675912+0000","last_became_active":"2024-09-21T03:43:04.978999+0000","last_became_peered":"2024-09-21T03:43:04.978999+0000","last_unstale":"2024-09-21T03:44:54.675912+0000","last_undegraded":"2024-09-21T03:44:54.675912+0000","last_fullsized":"2024-09-21T03:44:54.675912+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:33.682591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766304+0000","last_change":"2024-09-21T03:43:18.083665+0000","last_active":"2024-09-21T03:44:54.766304+0000","last_peered":"2024-09-21T03:44:54.766304+0000","last_clean":"2024-09-21T03:44:54.766304+0000","last_became_active":"2024-09-21T03:43:18.083215+0000","last_became_peered":"2024-09-21T03:43:18.083215+0000","last_unstale":"2024-09-21T03:44:54.766304+0000","last_undegraded":"2024-09-21T03:44:54.766304+0000","last_fullsized":"2024-09-21T03:44:54.766304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:55.968968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766486+0000","last_change":"2024-09-21T03:43:08.010570+0000","last_active":"2024-09-21T03:44:54.766486+0000","last_peered":"2024-09-21T03:44:54.766486+0000","last_clean":"2024-09-21T03:44:54.766486+0000","last_became_active":"2024-09-21T03:43:08.010163+0000","last_became_peered":"2024-09-21T03:43:08.010163+0000","last_unstale":"2024-09-21T03:44:54.766486+0000","last_undegraded":"2024-09-21T03:44:54.766486+0000","last_fullsized":"2024-09-21T03:44:54.766486+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:17.368534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.707142+0000","last_change":"2024-09-21T03:44:54.706609+0000","last_active":"2024-09-21T03:44:54.707142+0000","last_peered":"2024-09-21T03:44:54.707142+0000","last_clean":"2024-09-21T03:44:54.707142+0000","last_became_active":"2024-09-21T03:44:54.681060+0000","last_became_peered":"2024-09-21T03:44:54.681060+0000","last_unstale":"2024-09-21T03:44:54.707142+0000","last_undegraded":"2024-09-21T03:44:54.707142+0000","last_fullsized":"2024-09-21T03:44:54.707142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:49.797107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766891+0000","last_change":"2024-09-21T03:43:18.089516+0000","last_active":"2024-09-21T03:44:54.766891+0000","last_peered":"2024-09-21T03:44:54.766891+0000","last_clean":"2024-09-21T03:44:54.766891+0000","last_became_active":"2024-09-21T03:43:18.089097+0000","last_became_peered":"2024-09-21T03:43:18.089097+0000","last_unstale":"2024-09-21T03:44:54.766891+0000","last_undegraded":"2024-09-21T03:44:54.766891+0000","last_fullsized":"2024-09-21T03:44:54.766891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:50.428401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677900+0000","last_change":"2024-09-21T03:43:08.015762+0000","last_active":"2024-09-21T03:44:54.677900+0000","last_peered":"2024-09-21T03:44:54.677900+0000","last_clean":"2024-09-21T03:44:54.677900+0000","last_became_active":"2024-09-21T03:43:08.015389+0000","last_became_peered":"2024-09-21T03:43:08.015389+0000","last_unstale":"2024-09-21T03:44:54.677900+0000","last_undegraded":"2024-09-21T03:44:54.677900+0000","last_fullsized":"2024-09-21T03:44:54.677900+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:00.341479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'5","reported_seq":116,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:08.645587+0000","last_change":"2024-09-21T03:43:04.973173+0000","last_active":"2024-09-21T03:45:08.645587+0000","last_peered":"2024-09-21T03:45:08.645587+0000","last_clean":"2024-09-21T03:45:08.645587+0000","last_became_active":"2024-09-21T03:43:04.972740+0000","last_became_peered":"2024-09-21T03:43:04.972740+0000","last_unstale":"2024-09-21T03:45:08.645587+0000","last_undegraded":"2024-09-21T03:45:08.645587+0000","last_fullsized":"2024-09-21T03:45:08.645587+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:26.329220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670457+0000","last_change":"2024-09-21T03:43:18.093730+0000","last_active":"2024-09-21T03:44:54.670457+0000","last_peered":"2024-09-21T03:44:54.670457+0000","last_clean":"2024-09-21T03:44:54.670457+0000","last_became_active":"2024-09-21T03:43:18.089621+0000","last_became_peered":"2024-09-21T03:43:18.089621+0000","last_unstale":"2024-09-21T03:44:54.670457+0000","last_undegraded":"2024-09-21T03:44:54.670457+0000","last_fullsized":"2024-09-21T03:44:54.670457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:05.227038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678343+0000","last_change":"2024-09-21T03:43:54.465751+0000","last_active":"2024-09-21T03:44:54.678343+0000","last_peered":"2024-09-21T03:44:54.678343+0000","last_clean":"2024-09-21T03:44:54.678343+0000","last_became_active":"2024-09-21T03:43:54.465333+0000","last_became_peered":"2024-09-21T03:43:54.465333+0000","last_unstale":"2024-09-21T03:44:54.678343+0000","last_undegraded":"2024-09-21T03:44:54.678343+0000","last_fullsized":"2024-09-21T03:44:54.678343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:34.307699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766650+0000","last_change":"2024-09-21T03:43:04.979385+0000","last_active":"2024-09-21T03:44:54.766650+0000","last_peered":"2024-09-21T03:44:54.766650+0000","last_clean":"2024-09-21T03:44:54.766650+0000","last_became_active":"2024-09-21T03:43:04.978764+0000","last_became_peered":"2024-09-21T03:43:04.978764+0000","last_unstale":"2024-09-21T03:44:54.766650+0000","last_undegraded":"2024-09-21T03:44:54.766650+0000","last_fullsized":"2024-09-21T03:44:54.766650+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:34.256966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677391+0000","last_change":"2024-09-21T03:43:18.095937+0000","last_active":"2024-09-21T03:44:54.677391+0000","last_peered":"2024-09-21T03:44:54.677391+0000","last_clean":"2024-09-21T03:44:54.677391+0000","last_became_active":"2024-09-21T03:43:18.095531+0000","last_became_peered":"2024-09-21T03:43:18.095531+0000","last_unstale":"2024-09-21T03:44:54.677391+0000","last_undegraded":"2024-09-21T03:44:54.677391+0000","last_fullsized":"2024-09-21T03:44:54.677391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:18.089661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766550+0000","last_change":"2024-09-21T03:43:08.011342+0000","last_active":"2024-09-21T03:44:54.766550+0000","last_peered":"2024-09-21T03:44:54.766550+0000","last_clean":"2024-09-21T03:44:54.766550+0000","last_became_active":"2024-09-21T03:43:08.010963+0000","last_became_peered":"2024-09-21T03:43:08.010963+0000","last_unstale":"2024-09-21T03:44:54.766550+0000","last_undegraded":"2024-09-21T03:44:54.766550+0000","last_fullsized":"2024-09-21T03:44:54.766550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:05.970863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679486+0000","last_change":"2024-09-21T03:43:04.972896+0000","last_active":"2024-09-21T03:44:54.679486+0000","last_peered":"2024-09-21T03:44:54.679486+0000","last_clean":"2024-09-21T03:44:54.679486+0000","last_became_active":"2024-09-21T03:43:04.972535+0000","last_became_peered":"2024-09-21T03:43:04.972535+0000","last_unstale":"2024-09-21T03:44:54.679486+0000","last_undegraded":"2024-09-21T03:44:54.679486+0000","last_fullsized":"2024-09-21T03:44:54.679486+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:36.362019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766858+0000","last_change":"2024-09-21T03:43:18.081300+0000","last_active":"2024-09-21T03:44:54.766858+0000","last_peered":"2024-09-21T03:44:54.766858+0000","last_clean":"2024-09-21T03:44:54.766858+0000","last_became_active":"2024-09-21T03:43:18.080874+0000","last_became_peered":"2024-09-21T03:43:18.080874+0000","last_unstale":"2024-09-21T03:44:54.766858+0000","last_undegraded":"2024-09-21T03:44:54.766858+0000","last_fullsized":"2024-09-21T03:44:54.766858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:13.246348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675450+0000","last_change":"2024-09-21T03:43:08.014954+0000","last_active":"2024-09-21T03:44:54.675450+0000","last_peered":"2024-09-21T03:44:54.675450+0000","last_clean":"2024-09-21T03:44:54.675450+0000","last_became_active":"2024-09-21T03:43:08.014576+0000","last_became_peered":"2024-09-21T03:43:08.014576+0000","last_unstale":"2024-09-21T03:44:54.675450+0000","last_undegraded":"2024-09-21T03:44:54.675450+0000","last_fullsized":"2024-09-21T03:44:54.675450+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:13.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"84'35","reported_seq":223,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:12.651517+0000","last_change":"2024-09-21T03:43:04.973305+0000","last_active":"2024-09-21T03:45:12.651517+0000","last_peered":"2024-09-21T03:45:12.651517+0000","last_clean":"2024-09-21T03:45:12.651517+0000","last_became_active":"2024-09-21T03:43:04.972891+0000","last_became_peered":"2024-09-21T03:43:04.972891+0000","last_unstale":"2024-09-21T03:45:12.651517+0000","last_undegraded":"2024-09-21T03:45:12.651517+0000","last_fullsized":"2024-09-21T03:45:12.651517+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:15.653769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67067,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":143,"num_read_kb":137,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670013+0000","last_change":"2024-09-21T03:43:18.093613+0000","last_active":"2024-09-21T03:44:54.670013+0000","last_peered":"2024-09-21T03:44:54.670013+0000","last_clean":"2024-09-21T03:44:54.670013+0000","last_became_active":"2024-09-21T03:43:18.093195+0000","last_became_peered":"2024-09-21T03:43:18.093195+0000","last_unstale":"2024-09-21T03:44:54.670013+0000","last_undegraded":"2024-09-21T03:44:54.670013+0000","last_fullsized":"2024-09-21T03:44:54.670013+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:17.825071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767766+0000","last_change":"2024-09-21T03:43:08.014314+0000","last_active":"2024-09-21T03:44:54.767766+0000","last_peered":"2024-09-21T03:44:54.767766+0000","last_clean":"2024-09-21T03:44:54.767766+0000","last_became_active":"2024-09-21T03:43:08.012925+0000","last_became_peered":"2024-09-21T03:43:08.012925+0000","last_unstale":"2024-09-21T03:44:54.767766+0000","last_undegraded":"2024-09-21T03:44:54.767766+0000","last_fullsized":"2024-09-21T03:44:54.767766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:47:28.599444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767033+0000","last_change":"2024-09-21T03:43:05.141584+0000","last_active":"2024-09-21T03:44:54.767033+0000","last_peered":"2024-09-21T03:44:54.767033+0000","last_clean":"2024-09-21T03:44:54.767033+0000","last_became_active":"2024-09-21T03:43:05.141200+0000","last_became_peered":"2024-09-21T03:43:05.141200+0000","last_unstale":"2024-09-21T03:44:54.767033+0000","last_undegraded":"2024-09-21T03:44:54.767033+0000","last_fullsized":"2024-09-21T03:44:54.767033+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:04.357970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768505+0000","last_change":"2024-09-21T03:43:18.092184+0000","last_active":"2024-09-21T03:44:54.768505+0000","last_peered":"2024-09-21T03:44:54.768505+0000","last_clean":"2024-09-21T03:44:54.768505+0000","last_became_active":"2024-09-21T03:43:18.091629+0000","last_became_peered":"2024-09-21T03:43:18.091629+0000","last_unstale":"2024-09-21T03:44:54.768505+0000","last_undegraded":"2024-09-21T03:44:54.768505+0000","last_fullsized":"2024-09-21T03:44:54.768505+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:39.448009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.680071+0000","last_change":"2024-09-21T03:43:08.018335+0000","last_active":"2024-09-21T03:44:54.680071+0000","last_peered":"2024-09-21T03:44:54.680071+0000","last_clean":"2024-09-21T03:44:54.680071+0000","last_became_active":"2024-09-21T03:43:08.014005+0000","last_became_peered":"2024-09-21T03:43:08.014005+0000","last_unstale":"2024-09-21T03:44:54.680071+0000","last_undegraded":"2024-09-21T03:44:54.680071+0000","last_fullsized":"2024-09-21T03:44:54.680071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:35.749381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678484+0000","last_change":"2024-09-21T03:43:04.978609+0000","last_active":"2024-09-21T03:44:54.678484+0000","last_peered":"2024-09-21T03:44:54.678484+0000","last_clean":"2024-09-21T03:44:54.678484+0000","last_became_active":"2024-09-21T03:43:04.978146+0000","last_became_peered":"2024-09-21T03:43:04.978146+0000","last_unstale":"2024-09-21T03:44:54.678484+0000","last_undegraded":"2024-09-21T03:44:54.678484+0000","last_fullsized":"2024-09-21T03:44:54.678484+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:41.651745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.669874+0000","last_change":"2024-09-21T03:43:18.089399+0000","last_active":"2024-09-21T03:44:54.669874+0000","last_peered":"2024-09-21T03:44:54.669874+0000","last_clean":"2024-09-21T03:44:54.669874+0000","last_became_active":"2024-09-21T03:43:18.086156+0000","last_became_peered":"2024-09-21T03:43:18.086156+0000","last_unstale":"2024-09-21T03:44:54.669874+0000","last_undegraded":"2024-09-21T03:44:54.669874+0000","last_fullsized":"2024-09-21T03:44:54.669874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:41.955341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768619+0000","last_change":"2024-09-21T03:43:08.008773+0000","last_active":"2024-09-21T03:44:54.768619+0000","last_peered":"2024-09-21T03:44:54.768619+0000","last_clean":"2024-09-21T03:44:54.768619+0000","last_became_active":"2024-09-21T03:43:08.008270+0000","last_became_peered":"2024-09-21T03:43:08.008270+0000","last_unstale":"2024-09-21T03:44:54.768619+0000","last_undegraded":"2024-09-21T03:44:54.768619+0000","last_fullsized":"2024-09-21T03:44:54.768619+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:57.901701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":183,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:12.651417+0000","last_change":"2024-09-21T03:43:04.978296+0000","last_active":"2024-09-21T03:45:12.651417+0000","last_peered":"2024-09-21T03:45:12.651417+0000","last_clean":"2024-09-21T03:45:12.651417+0000","last_became_active":"2024-09-21T03:43:04.977782+0000","last_became_peered":"2024-09-21T03:43:04.977782+0000","last_unstale":"2024-09-21T03:45:12.651417+0000","last_undegraded":"2024-09-21T03:45:12.651417+0000","last_fullsized":"2024-09-21T03:45:12.651417+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:57.748737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":134,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671480+0000","last_change":"2024-09-21T03:43:54.495536+0000","last_active":"2024-09-21T03:44:54.671480+0000","last_peered":"2024-09-21T03:44:54.671480+0000","last_clean":"2024-09-21T03:44:54.671480+0000","last_became_active":"2024-09-21T03:43:54.493492+0000","last_became_peered":"2024-09-21T03:43:54.493492+0000","last_unstale":"2024-09-21T03:44:54.671480+0000","last_undegraded":"2024-09-21T03:44:54.671480+0000","last_fullsized":"2024-09-21T03:44:54.671480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:11.291911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679520+0000","last_change":"2024-09-21T03:43:18.077741+0000","last_active":"2024-09-21T03:44:54.679520+0000","last_peered":"2024-09-21T03:44:54.679520+0000","last_clean":"2024-09-21T03:44:54.679520+0000","last_became_active":"2024-09-21T03:43:18.076619+0000","last_became_peered":"2024-09-21T03:43:18.076619+0000","last_unstale":"2024-09-21T03:44:54.679520+0000","last_undegraded":"2024-09-21T03:44:54.679520+0000","last_fullsized":"2024-09-21T03:44:54.679520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:33.297020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.669875+0000","last_change":"2024-09-21T03:43:08.008838+0000","last_active":"2024-09-21T03:44:54.669875+0000","last_peered":"2024-09-21T03:44:54.669875+0000","last_clean":"2024-09-21T03:44:54.669875+0000","last_became_active":"2024-09-21T03:43:08.008422+0000","last_became_peered":"2024-09-21T03:43:08.008422+0000","last_unstale":"2024-09-21T03:44:54.669875+0000","last_undegraded":"2024-09-21T03:44:54.669875+0000","last_fullsized":"2024-09-21T03:44:54.669875+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:00:17.881443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766577+0000","last_change":"2024-09-21T03:43:04.979289+0000","last_active":"2024-09-21T03:44:54.766577+0000","last_peered":"2024-09-21T03:44:54.766577+0000","last_clean":"2024-09-21T03:44:54.766577+0000","last_became_active":"2024-09-21T03:43:04.978843+0000","last_became_peered":"2024-09-21T03:43:04.978843+0000","last_unstale":"2024-09-21T03:44:54.766577+0000","last_undegraded":"2024-09-21T03:44:54.766577+0000","last_fullsized":"2024-09-21T03:44:54.766577+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:50.560607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677180+0000","last_change":"2024-09-21T03:43:18.096870+0000","last_active":"2024-09-21T03:44:54.677180+0000","last_peered":"2024-09-21T03:44:54.677180+0000","last_clean":"2024-09-21T03:44:54.677180+0000","last_became_active":"2024-09-21T03:43:18.096424+0000","last_became_peered":"2024-09-21T03:43:18.096424+0000","last_unstale":"2024-09-21T03:44:54.677180+0000","last_undegraded":"2024-09-21T03:44:54.677180+0000","last_fullsized":"2024-09-21T03:44:54.677180+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:39.532368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.669916+0000","last_change":"2024-09-21T03:43:54.492499+0000","last_active":"2024-09-21T03:44:54.669916+0000","last_peered":"2024-09-21T03:44:54.669916+0000","last_clean":"2024-09-21T03:44:54.669916+0000","last_became_active":"2024-09-21T03:43:54.492043+0000","last_became_peered":"2024-09-21T03:43:54.492043+0000","last_unstale":"2024-09-21T03:44:54.669916+0000","last_undegraded":"2024-09-21T03:44:54.669916+0000","last_fullsized":"2024-09-21T03:44:54.669916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:25.687090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766389+0000","last_change":"2024-09-21T03:43:04.974265+0000","last_active":"2024-09-21T03:44:54.766389+0000","last_peered":"2024-09-21T03:44:54.766389+0000","last_clean":"2024-09-21T03:44:54.766389+0000","last_became_active":"2024-09-21T03:43:04.973383+0000","last_became_peered":"2024-09-21T03:43:04.973383+0000","last_unstale":"2024-09-21T03:44:54.766389+0000","last_undegraded":"2024-09-21T03:44:54.766389+0000","last_fullsized":"2024-09-21T03:44:54.766389+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:02.312378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768029+0000","last_change":"2024-09-21T03:43:04.980353+0000","last_active":"2024-09-21T03:44:54.768029+0000","last_peered":"2024-09-21T03:44:54.768029+0000","last_clean":"2024-09-21T03:44:54.768029+0000","last_became_active":"2024-09-21T03:43:04.979922+0000","last_became_peered":"2024-09-21T03:43:04.979922+0000","last_unstale":"2024-09-21T03:44:54.768029+0000","last_undegraded":"2024-09-21T03:44:54.768029+0000","last_fullsized":"2024-09-21T03:44:54.768029+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:17.636536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766686+0000","last_change":"2024-09-21T03:43:08.009225+0000","last_active":"2024-09-21T03:44:54.766686+0000","last_peered":"2024-09-21T03:44:54.766686+0000","last_clean":"2024-09-21T03:44:54.766686+0000","last_became_active":"2024-09-21T03:43:08.008830+0000","last_became_peered":"2024-09-21T03:43:08.008830+0000","last_unstale":"2024-09-21T03:44:54.766686+0000","last_undegraded":"2024-09-21T03:44:54.766686+0000","last_fullsized":"2024-09-21T03:44:54.766686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:38.047763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679485+0000","last_change":"2024-09-21T03:43:18.089302+0000","last_active":"2024-09-21T03:44:54.679485+0000","last_peered":"2024-09-21T03:44:54.679485+0000","last_clean":"2024-09-21T03:44:54.679485+0000","last_became_active":"2024-09-21T03:43:18.088867+0000","last_became_peered":"2024-09-21T03:43:18.088867+0000","last_unstale":"2024-09-21T03:44:54.679485+0000","last_undegraded":"2024-09-21T03:44:54.679485+0000","last_fullsized":"2024-09-21T03:44:54.679485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:08.960044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":122,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:07.644360+0000","last_change":"2024-09-21T03:43:54.464317+0000","last_active":"2024-09-21T03:45:07.644360+0000","last_peered":"2024-09-21T03:45:07.644360+0000","last_clean":"2024-09-21T03:45:07.644360+0000","last_became_active":"2024-09-21T03:43:54.463870+0000","last_became_peered":"2024-09-21T03:43:54.463870+0000","last_unstale":"2024-09-21T03:45:07.644360+0000","last_undegraded":"2024-09-21T03:45:07.644360+0000","last_fullsized":"2024-09-21T03:45:07.644360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:14.581726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.674905+0000","last_change":"2024-09-21T03:43:08.009398+0000","last_active":"2024-09-21T03:44:54.674905+0000","last_peered":"2024-09-21T03:44:54.674905+0000","last_clean":"2024-09-21T03:44:54.674905+0000","last_became_active":"2024-09-21T03:43:08.009000+0000","last_became_peered":"2024-09-21T03:43:08.009000+0000","last_unstale":"2024-09-21T03:44:54.674905+0000","last_undegraded":"2024-09-21T03:44:54.674905+0000","last_fullsized":"2024-09-21T03:44:54.674905+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:07.278613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679818+0000","last_change":"2024-09-21T03:43:18.081834+0000","last_active":"2024-09-21T03:44:54.679818+0000","last_peered":"2024-09-21T03:44:54.679818+0000","last_clean":"2024-09-21T03:44:54.679818+0000","last_became_active":"2024-09-21T03:43:18.081328+0000","last_became_peered":"2024-09-21T03:43:18.081328+0000","last_unstale":"2024-09-21T03:44:54.679818+0000","last_undegraded":"2024-09-21T03:44:54.679818+0000","last_fullsized":"2024-09-21T03:44:54.679818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:37.638826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678021+0000","last_change":"2024-09-21T03:43:08.005144+0000","last_active":"2024-09-21T03:44:54.678021+0000","last_peered":"2024-09-21T03:44:54.678021+0000","last_clean":"2024-09-21T03:44:54.678021+0000","last_became_active":"2024-09-21T03:43:08.003895+0000","last_became_peered":"2024-09-21T03:43:08.003895+0000","last_unstale":"2024-09-21T03:44:54.678021+0000","last_undegraded":"2024-09-21T03:44:54.678021+0000","last_fullsized":"2024-09-21T03:44:54.678021+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:25.822315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679102+0000","last_change":"2024-09-21T03:43:08.000218+0000","last_active":"2024-09-21T03:44:54.679102+0000","last_peered":"2024-09-21T03:44:54.679102+0000","last_clean":"2024-09-21T03:44:54.679102+0000","last_became_active":"2024-09-21T03:43:07.999840+0000","last_became_peered":"2024-09-21T03:43:07.999840+0000","last_unstale":"2024-09-21T03:44:54.679102+0000","last_undegraded":"2024-09-21T03:44:54.679102+0000","last_fullsized":"2024-09-21T03:44:54.679102+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:55.391706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679062+0000","last_change":"2024-09-21T03:43:08.012075+0000","last_active":"2024-09-21T03:44:54.679062+0000","last_peered":"2024-09-21T03:44:54.679062+0000","last_clean":"2024-09-21T03:44:54.679062+0000","last_became_active":"2024-09-21T03:43:08.011666+0000","last_became_peered":"2024-09-21T03:43:08.011666+0000","last_unstale":"2024-09-21T03:44:54.679062+0000","last_undegraded":"2024-09-21T03:44:54.679062+0000","last_fullsized":"2024-09-21T03:44:54.679062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:42.124948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679945+0000","last_change":"2024-09-21T03:43:08.021082+0000","last_active":"2024-09-21T03:44:54.679945+0000","last_peered":"2024-09-21T03:44:54.679945+0000","last_clean":"2024-09-21T03:44:54.679945+0000","last_became_active":"2024-09-21T03:43:08.020674+0000","last_became_peered":"2024-09-21T03:43:08.020674+0000","last_unstale":"2024-09-21T03:44:54.679945+0000","last_undegraded":"2024-09-21T03:44:54.679945+0000","last_fullsized":"2024-09-21T03:44:54.679945+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:13.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766965+0000","last_change":"2024-09-21T03:43:08.004199+0000","last_active":"2024-09-21T03:44:54.766965+0000","last_peered":"2024-09-21T03:44:54.766965+0000","last_clean":"2024-09-21T03:44:54.766965+0000","last_became_active":"2024-09-21T03:43:08.003806+0000","last_became_peered":"2024-09-21T03:43:08.003806+0000","last_unstale":"2024-09-21T03:44:54.766965+0000","last_undegraded":"2024-09-21T03:44:54.766965+0000","last_fullsized":"2024-09-21T03:44:54.766965+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:13.941130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678414+0000","last_change":"2024-09-21T03:43:08.007237+0000","last_active":"2024-09-21T03:44:54.678414+0000","last_peered":"2024-09-21T03:44:54.678414+0000","last_clean":"2024-09-21T03:44:54.678414+0000","last_became_active":"2024-09-21T03:43:08.006805+0000","last_became_peered":"2024-09-21T03:43:08.006805+0000","last_unstale":"2024-09-21T03:44:54.678414+0000","last_undegraded":"2024-09-21T03:44:54.678414+0000","last_fullsized":"2024-09-21T03:44:54.678414+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:26.274494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.680363+0000","last_change":"2024-09-21T03:43:08.007934+0000","last_active":"2024-09-21T03:44:54.680363+0000","last_peered":"2024-09-21T03:44:54.680363+0000","last_clean":"2024-09-21T03:44:54.680363+0000","last_became_active":"2024-09-21T03:43:08.007549+0000","last_became_peered":"2024-09-21T03:43:08.007549+0000","last_unstale":"2024-09-21T03:44:54.680363+0000","last_undegraded":"2024-09-21T03:44:54.680363+0000","last_fullsized":"2024-09-21T03:44:54.680363+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:58.158255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.672360+0000","last_change":"2024-09-21T03:43:54.493665+0000","last_active":"2024-09-21T03:44:54.672360+0000","last_peered":"2024-09-21T03:44:54.672360+0000","last_clean":"2024-09-21T03:44:54.672360+0000","last_became_active":"2024-09-21T03:43:54.493245+0000","last_became_peered":"2024-09-21T03:43:54.493245+0000","last_unstale":"2024-09-21T03:44:54.672360+0000","last_undegraded":"2024-09-21T03:44:54.672360+0000","last_fullsized":"2024-09-21T03:44:54.672360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:35.449852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766343+0000","last_change":"2024-09-21T03:43:54.463419+0000","last_active":"2024-09-21T03:44:54.766343+0000","last_peered":"2024-09-21T03:44:54.766343+0000","last_clean":"2024-09-21T03:44:54.766343+0000","last_became_active":"2024-09-21T03:43:54.462951+0000","last_became_peered":"2024-09-21T03:43:54.462951+0000","last_unstale":"2024-09-21T03:44:54.766343+0000","last_undegraded":"2024-09-21T03:44:54.766343+0000","last_fullsized":"2024-09-21T03:44:54.766343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:23.620275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671805+0000","last_change":"2024-09-21T03:43:08.011436+0000","last_active":"2024-09-21T03:44:54.671805+0000","last_peered":"2024-09-21T03:44:54.671805+0000","last_clean":"2024-09-21T03:44:54.671805+0000","last_became_active":"2024-09-21T03:43:08.011025+0000","last_became_peered":"2024-09-21T03:43:08.011025+0000","last_unstale":"2024-09-21T03:44:54.671805+0000","last_undegraded":"2024-09-21T03:44:54.671805+0000","last_fullsized":"2024-09-21T03:44:54.671805+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:31.411858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768311+0000","last_change":"2024-09-21T03:43:08.016378+0000","last_active":"2024-09-21T03:44:54.768311+0000","last_peered":"2024-09-21T03:44:54.768311+0000","last_clean":"2024-09-21T03:44:54.768311+0000","last_became_active":"2024-09-21T03:43:08.016013+0000","last_became_peered":"2024-09-21T03:43:08.016013+0000","last_unstale":"2024-09-21T03:44:54.768311+0000","last_undegraded":"2024-09-21T03:44:54.768311+0000","last_fullsized":"2024-09-21T03:44:54.768311+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:01.865669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766706+0000","last_change":"2024-09-21T03:43:08.007862+0000","last_active":"2024-09-21T03:44:54.766706+0000","last_peered":"2024-09-21T03:44:54.766706+0000","last_clean":"2024-09-21T03:44:54.766706+0000","last_became_active":"2024-09-21T03:43:08.007433+0000","last_became_peered":"2024-09-21T03:43:08.007433+0000","last_unstale":"2024-09-21T03:44:54.766706+0000","last_undegraded":"2024-09-21T03:44:54.766706+0000","last_fullsized":"2024-09-21T03:44:54.766706+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:22:03.364767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677476+0000","last_change":"2024-09-21T03:43:08.010542+0000","last_active":"2024-09-21T03:44:54.677476+0000","last_peered":"2024-09-21T03:44:54.677476+0000","last_clean":"2024-09-21T03:44:54.677476+0000","last_became_active":"2024-09-21T03:43:08.010195+0000","last_became_peered":"2024-09-21T03:43:08.010195+0000","last_unstale":"2024-09-21T03:44:54.677476+0000","last_undegraded":"2024-09-21T03:44:54.677476+0000","last_fullsized":"2024-09-21T03:44:54.677476+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:52.850657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.680060+0000","last_change":"2024-09-21T03:43:08.006168+0000","last_active":"2024-09-21T03:44:54.680060+0000","last_peered":"2024-09-21T03:44:54.680060+0000","last_clean":"2024-09-21T03:44:54.680060+0000","last_became_active":"2024-09-21T03:43:08.005283+0000","last_became_peered":"2024-09-21T03:43:08.005283+0000","last_unstale":"2024-09-21T03:44:54.680060+0000","last_undegraded":"2024-09-21T03:44:54.680060+0000","last_fullsized":"2024-09-21T03:44:54.680060+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:59.852412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670656+0000","last_change":"2024-09-21T03:43:08.005870+0000","last_active":"2024-09-21T03:44:54.670656+0000","last_peered":"2024-09-21T03:44:54.670656+0000","last_clean":"2024-09-21T03:44:54.670656+0000","last_became_active":"2024-09-21T03:43:08.005485+0000","last_became_peered":"2024-09-21T03:43:08.005485+0000","last_unstale":"2024-09-21T03:44:54.670656+0000","last_undegraded":"2024-09-21T03:44:54.670656+0000","last_fullsized":"2024-09-21T03:44:54.670656+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:41.387865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677705+0000","last_change":"2024-09-21T03:43:08.015176+0000","last_active":"2024-09-21T03:44:54.677705+0000","last_peered":"2024-09-21T03:44:54.677705+0000","last_clean":"2024-09-21T03:44:54.677705+0000","last_became_active":"2024-09-21T03:43:08.014451+0000","last_became_peered":"2024-09-21T03:43:08.014451+0000","last_unstale":"2024-09-21T03:44:54.677705+0000","last_undegraded":"2024-09-21T03:44:54.677705+0000","last_fullsized":"2024-09-21T03:44:54.677705+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:49.524800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767481+0000","last_change":"2024-09-21T03:43:08.016798+0000","last_active":"2024-09-21T03:44:54.767481+0000","last_peered":"2024-09-21T03:44:54.767481+0000","last_clean":"2024-09-21T03:44:54.767481+0000","last_became_active":"2024-09-21T03:43:08.014842+0000","last_became_peered":"2024-09-21T03:43:08.014842+0000","last_unstale":"2024-09-21T03:44:54.767481+0000","last_undegraded":"2024-09-21T03:44:54.767481+0000","last_fullsized":"2024-09-21T03:44:54.767481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:19.290618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679434+0000","last_change":"2024-09-21T03:43:08.021285+0000","last_active":"2024-09-21T03:44:54.679434+0000","last_peered":"2024-09-21T03:44:54.679434+0000","last_clean":"2024-09-21T03:44:54.679434+0000","last_became_active":"2024-09-21T03:43:08.020904+0000","last_became_peered":"2024-09-21T03:43:08.020904+0000","last_unstale":"2024-09-21T03:44:54.679434+0000","last_undegraded":"2024-09-21T03:44:54.679434+0000","last_fullsized":"2024-09-21T03:44:54.679434+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:56.589089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679378+0000","last_change":"2024-09-21T03:43:08.017599+0000","last_active":"2024-09-21T03:44:54.679378+0000","last_peered":"2024-09-21T03:44:54.679378+0000","last_clean":"2024-09-21T03:44:54.679378+0000","last_became_active":"2024-09-21T03:43:08.016138+0000","last_became_peered":"2024-09-21T03:43:08.016138+0000","last_unstale":"2024-09-21T03:44:54.679378+0000","last_undegraded":"2024-09-21T03:44:54.679378+0000","last_fullsized":"2024-09-21T03:44:54.679378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:09.902246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766674+0000","last_change":"2024-09-21T03:43:08.009598+0000","last_active":"2024-09-21T03:44:54.766674+0000","last_peered":"2024-09-21T03:44:54.766674+0000","last_clean":"2024-09-21T03:44:54.766674+0000","last_became_active":"2024-09-21T03:43:08.007488+0000","last_became_peered":"2024-09-21T03:43:08.007488+0000","last_unstale":"2024-09-21T03:44:54.766674+0000","last_undegraded":"2024-09-21T03:44:54.766674+0000","last_fullsized":"2024-09-21T03:44:54.766674+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:23.192761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768416+0000","last_change":"2024-09-21T03:43:08.016450+0000","last_active":"2024-09-21T03:44:54.768416+0000","last_peered":"2024-09-21T03:44:54.768416+0000","last_clean":"2024-09-21T03:44:54.768416+0000","last_became_active":"2024-09-21T03:43:08.016037+0000","last_became_peered":"2024-09-21T03:43:08.016037+0000","last_unstale":"2024-09-21T03:44:54.768416+0000","last_undegraded":"2024-09-21T03:44:54.768416+0000","last_fullsized":"2024-09-21T03:44:54.768416+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:39.109408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678614+0000","last_change":"2024-09-21T03:43:08.010298+0000","last_active":"2024-09-21T03:44:54.678614+0000","last_peered":"2024-09-21T03:44:54.678614+0000","last_clean":"2024-09-21T03:44:54.678614+0000","last_became_active":"2024-09-21T03:43:08.009847+0000","last_became_peered":"2024-09-21T03:43:08.009847+0000","last_unstale":"2024-09-21T03:44:54.678614+0000","last_undegraded":"2024-09-21T03:44:54.678614+0000","last_fullsized":"2024-09-21T03:44:54.678614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:37.463456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768895+0000","last_change":"2024-09-21T03:43:08.017834+0000","last_active":"2024-09-21T03:44:54.768895+0000","last_peered":"2024-09-21T03:44:54.768895+0000","last_clean":"2024-09-21T03:44:54.768895+0000","last_became_active":"2024-09-21T03:43:08.016910+0000","last_became_peered":"2024-09-21T03:43:08.016910+0000","last_unstale":"2024-09-21T03:44:54.768895+0000","last_undegraded":"2024-09-21T03:44:54.768895+0000","last_fullsized":"2024-09-21T03:44:54.768895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:50.528651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768591+0000","last_change":"2024-09-21T03:43:08.013945+0000","last_active":"2024-09-21T03:44:54.768591+0000","last_peered":"2024-09-21T03:44:54.768591+0000","last_clean":"2024-09-21T03:44:54.768591+0000","last_became_active":"2024-09-21T03:43:08.013598+0000","last_became_peered":"2024-09-21T03:43:08.013598+0000","last_unstale":"2024-09-21T03:44:54.768591+0000","last_undegraded":"2024-09-21T03:44:54.768591+0000","last_fullsized":"2024-09-21T03:44:54.768591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:58.182938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671440+0000","last_change":"2024-09-21T03:43:08.018824+0000","last_active":"2024-09-21T03:44:54.671440+0000","last_peered":"2024-09-21T03:44:54.671440+0000","last_clean":"2024-09-21T03:44:54.671440+0000","last_became_active":"2024-09-21T03:43:08.017196+0000","last_became_peered":"2024-09-21T03:43:08.017196+0000","last_unstale":"2024-09-21T03:44:54.671440+0000","last_undegraded":"2024-09-21T03:44:54.671440+0000","last_fullsized":"2024-09-21T03:44:54.671440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:02.357523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677945+0000","last_change":"2024-09-21T03:43:08.003514+0000","last_active":"2024-09-21T03:44:54.677945+0000","last_peered":"2024-09-21T03:44:54.677945+0000","last_clean":"2024-09-21T03:44:54.677945+0000","last_became_active":"2024-09-21T03:43:08.003131+0000","last_became_peered":"2024-09-21T03:43:08.003131+0000","last_unstale":"2024-09-21T03:44:54.677945+0000","last_undegraded":"2024-09-21T03:44:54.677945+0000","last_fullsized":"2024-09-21T03:44:54.677945+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:23.170302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767907+0000","last_change":"2024-09-21T03:43:08.015890+0000","last_active":"2024-09-21T03:44:54.767907+0000","last_peered":"2024-09-21T03:44:54.767907+0000","last_clean":"2024-09-21T03:44:54.767907+0000","last_became_active":"2024-09-21T03:43:08.015519+0000","last_became_peered":"2024-09-21T03:43:08.015519+0000","last_unstale":"2024-09-21T03:44:54.767907+0000","last_undegraded":"2024-09-21T03:44:54.767907+0000","last_fullsized":"2024-09-21T03:44:54.767907+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:10.875923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675017+0000","last_change":"2024-09-21T03:43:08.012261+0000","last_active":"2024-09-21T03:44:54.675017+0000","last_peered":"2024-09-21T03:44:54.675017+0000","last_clean":"2024-09-21T03:44:54.675017+0000","last_became_active":"2024-09-21T03:43:08.011783+0000","last_became_peered":"2024-09-21T03:43:08.011783+0000","last_unstale":"2024-09-21T03:44:54.675017+0000","last_undegraded":"2024-09-21T03:44:54.675017+0000","last_fullsized":"2024-09-21T03:44:54.675017+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:07.011340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670137+0000","last_change":"2024-09-21T03:43:54.494826+0000","last_active":"2024-09-21T03:44:54.670137+0000","last_peered":"2024-09-21T03:44:54.670137+0000","last_clean":"2024-09-21T03:44:54.670137+0000","last_became_active":"2024-09-21T03:43:54.494404+0000","last_became_peered":"2024-09-21T03:43:54.494404+0000","last_unstale":"2024-09-21T03:44:54.670137+0000","last_undegraded":"2024-09-21T03:44:54.670137+0000","last_fullsized":"2024-09-21T03:44:54.670137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:42.109053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679632+0000","last_change":"2024-09-21T03:43:08.016214+0000","last_active":"2024-09-21T03:44:54.679632+0000","last_peered":"2024-09-21T03:44:54.679632+0000","last_clean":"2024-09-21T03:44:54.679632+0000","last_became_active":"2024-09-21T03:43:08.014815+0000","last_became_peered":"2024-09-21T03:43:08.014815+0000","last_unstale":"2024-09-21T03:44:54.679632+0000","last_undegraded":"2024-09-21T03:44:54.679632+0000","last_fullsized":"2024-09-21T03:44:54.679632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:04.742829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679672+0000","last_change":"2024-09-21T03:43:08.019269+0000","last_active":"2024-09-21T03:44:54.679672+0000","last_peered":"2024-09-21T03:44:54.679672+0000","last_clean":"2024-09-21T03:44:54.679672+0000","last_became_active":"2024-09-21T03:43:08.017962+0000","last_became_peered":"2024-09-21T03:43:08.017962+0000","last_unstale":"2024-09-21T03:44:54.679672+0000","last_undegraded":"2024-09-21T03:44:54.679672+0000","last_fullsized":"2024-09-21T03:44:54.679672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:05.950697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677702+0000","last_change":"2024-09-21T03:43:08.014790+0000","last_active":"2024-09-21T03:44:54.677702+0000","last_peered":"2024-09-21T03:44:54.677702+0000","last_clean":"2024-09-21T03:44:54.677702+0000","last_became_active":"2024-09-21T03:43:08.013891+0000","last_became_peered":"2024-09-21T03:43:08.013891+0000","last_unstale":"2024-09-21T03:44:54.677702+0000","last_undegraded":"2024-09-21T03:44:54.677702+0000","last_fullsized":"2024-09-21T03:44:54.677702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:46.045675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73880,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":692,"num_read_kb":430,"num_write":249,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":221640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808873,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1007616,"data_stored":501132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972398,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1703936,"data_stored":1158608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135922,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1024000,"data_stored":500559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332152,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1011712,"data_stored":500825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495678,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36932,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708220,"statfs":{"total":95995035648,"available":95957217280,"internally_reserved":0,"allocated":1617920,"data_stored":1097043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659204,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":999424,"data_stored":496063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855433,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1626112,"data_stored":1092625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018959,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1015808,"data_stored":500759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215188,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1011712,"data_stored":495771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444123,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1085440,"data_stored":564149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1073152,"data_stored":564160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1024000,"data_stored":497910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:45:14.327 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:45:14.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:14 smithi016 ceph-mon[30375]: pgmap v386: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T03:45:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:14 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:14 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:14 smithi016 ceph-mon[30375]: from='client.25114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:45:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:14 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:45:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:14 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:45:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:14 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:14.713 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:14.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:14 smithi134 ceph-mon[35031]: pgmap v386: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T03:45:14.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:14 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:14.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:14 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:14.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:14 smithi134 ceph-mon[35031]: from='client.25114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:45:14.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:14 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:45:14.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:14 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:45:14.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:14 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:14.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:14 smithi123 ceph-mon[34673]: pgmap v386: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T03:45:14.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:14 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:14.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:14 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:14.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:14 smithi123 ceph-mon[34673]: from='client.25114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:45:14.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:14 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:45:14.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:14 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:45:14.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:14 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:45:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.359+0000 7fe61f200640 1 -- 172.21.15.16:0/256538801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61810dc20 msgr2=0x7fe618110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.359+0000 7fe61f200640 1 --2- 172.21.15.16:0/256538801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61810dc20 0x7fe618110110 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe614009ca0 tx=0x7fe61402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.359+0000 7fe61f200640 1 -- 172.21.15.16:0/256538801 shutdown_connections 2024-09-21T03:45:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.359+0000 7fe61f200640 1 --2- 172.21.15.16:0/256538801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61810dc20 0x7fe618110110 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.359+0000 7fe61f200640 1 --2- 172.21.15.16:0/256538801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe618105110 0x7fe61810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.359+0000 7fe61f200640 1 --2- 172.21.15.16:0/256538801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6181047c0 0x7fe618104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.359+0000 7fe61f200640 1 -- 172.21.15.16:0/256538801 >> 172.21.15.16:0/256538801 conn(0x7fe6180fe2c0 msgr2=0x7fe618100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.359+0000 7fe61f200640 1 -- 172.21.15.16:0/256538801 shutdown_connections 2024-09-21T03:45:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.359+0000 7fe61f200640 1 -- 172.21.15.16:0/256538801 wait complete. 2024-09-21T03:45:15.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.360+0000 7fe61f200640 1 Processor -- start 2024-09-21T03:45:15.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.360+0000 7fe61f200640 1 -- start start 2024-09-21T03:45:15.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.361+0000 7fe61f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6181047c0 0x7fe61819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:15.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.361+0000 7fe61f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe618105110 0x7fe61819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:15.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.361+0000 7fe61f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe61810dc20 0x7fe61819faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:15.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.361+0000 7fe61f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6181138f0 con 0x7fe6181047c0 2024-09-21T03:45:15.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.361+0000 7fe61f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe618113770 con 0x7fe61810dc20 2024-09-21T03:45:15.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.361+0000 7fe61f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe618113a70 con 0x7fe618105110 2024-09-21T03:45:15.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.361+0000 7fe61d400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe618105110 0x7fe61819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:15.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.361+0000 7fe61d400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe618105110 0x7fe61819a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:49912/0 (socket says 172.21.15.16:49912) 2024-09-21T03:45:15.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.361+0000 7fe61de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6181047c0 0x7fe61819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:15.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.361+0000 7fe61d400640 1 -- 172.21.15.16:0/4024216570 learned_addr learned my addr 172.21.15.16:0/4024216570 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:15.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.361+0000 7fe61e800640 1 --2- 172.21.15.16:0/4024216570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe61810dc20 0x7fe61819faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.362+0000 7fe61d400640 1 -- 172.21.15.16:0/4024216570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe61810dc20 msgr2=0x7fe61819faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.362+0000 7fe61d400640 1 --2- 172.21.15.16:0/4024216570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe61810dc20 0x7fe61819faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.362+0000 7fe61d400640 1 -- 172.21.15.16:0/4024216570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6181047c0 msgr2=0x7fe61819a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.362+0000 7fe61d400640 1 --2- 172.21.15.16:0/4024216570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6181047c0 0x7fe61819a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.362+0000 7fe61d400640 1 -- 172.21.15.16:0/4024216570 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6181a01d0 con 0x7fe618105110 2024-09-21T03:45:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.362+0000 7fe61d400640 1 --2- 172.21.15.16:0/4024216570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe618105110 0x7fe61819a800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe60800e970 tx=0x7fe60800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.362+0000 7fe612a00640 1 -- 172.21.15.16:0/4024216570 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60800cd80 con 0x7fe618105110 2024-09-21T03:45:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.362+0000 7fe61f200640 1 -- 172.21.15.16:0/4024216570 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6181a04c0 con 0x7fe618105110 2024-09-21T03:45:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.362+0000 7fe612a00640 1 -- 172.21.15.16:0/4024216570 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6080035f0 con 0x7fe618105110 2024-09-21T03:45:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.362+0000 7fe612a00640 1 -- 172.21.15.16:0/4024216570 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608010690 con 0x7fe618105110 2024-09-21T03:45:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.362+0000 7fe61f200640 1 -- 172.21.15.16:0/4024216570 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6181a6c20 con 0x7fe618105110 2024-09-21T03:45:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.363+0000 7fe61f200640 1 -- 172.21.15.16:0/4024216570 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe618104bd0 con 0x7fe618105110 2024-09-21T03:45:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.367+0000 7fe612a00640 1 -- 172.21.15.16:0/4024216570 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fe608003aa0 con 0x7fe618105110 2024-09-21T03:45:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.369+0000 7fe612a00640 1 --2- 172.21.15.16:0/4024216570 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe6000b6620 0x7fe6000b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.369+0000 7fe612a00640 1 -- 172.21.15.16:0/4024216570 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fe6080e7cd0 con 0x7fe618105110 2024-09-21T03:45:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.369+0000 7fe61de00640 1 --2- 172.21.15.16:0/4024216570 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe6000b6620 0x7fe6000b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.369+0000 7fe612a00640 1 -- 172.21.15.16:0/4024216570 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe608056710 con 0x7fe618105110 2024-09-21T03:45:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.370+0000 7fe61de00640 1 --2- 172.21.15.16:0/4024216570 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe6000b6620 0x7fe6000b8b30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009bd0 tx=0x7fe60c005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.563+0000 7fe61f200640 1 -- 172.21.15.16:0/4024216570 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe618107dc0 con 0x7fe6000b6620 2024-09-21T03:45:15.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.569+0000 7fe612a00640 1 -- 172.21.15.16:0/4024216570 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340407 (secure 0 0 0) 0x7fe618107dc0 con 0x7fe6000b6620 2024-09-21T03:45:15.574 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:45:15.578 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-21T03:45:15.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.574+0000 7fe61f200640 1 -- 172.21.15.16:0/4024216570 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe6000b6620 msgr2=0x7fe6000b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:15.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.574+0000 7fe61f200640 1 --2- 172.21.15.16:0/4024216570 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe6000b6620 0x7fe6000b8b30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009bd0 tx=0x7fe60c005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:15.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.574+0000 7fe61f200640 1 -- 172.21.15.16:0/4024216570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe618105110 msgr2=0x7fe61819a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:15.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.574+0000 7fe61f200640 1 --2- 172.21.15.16:0/4024216570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe618105110 0x7fe61819a800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe60800e970 tx=0x7fe60800ee30 comp rx=0 tx=0).stop 2024-09-21T03:45:15.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.574+0000 7fe61f200640 1 -- 172.21.15.16:0/4024216570 shutdown_connections 2024-09-21T03:45:15.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.574+0000 7fe61f200640 1 --2- 172.21.15.16:0/4024216570 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe6000b6620 0x7fe6000b8b30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:15.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.574+0000 7fe61f200640 1 --2- 172.21.15.16:0/4024216570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe61810dc20 0x7fe61819faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:15.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.574+0000 7fe61f200640 1 --2- 172.21.15.16:0/4024216570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe618105110 0x7fe61819a800 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:15.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.574+0000 7fe61f200640 1 --2- 172.21.15.16:0/4024216570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6181047c0 0x7fe61819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:15.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.574+0000 7fe61f200640 1 -- 172.21.15.16:0/4024216570 >> 172.21.15.16:0/4024216570 conn(0x7fe6180fe2c0 msgr2=0x7fe6181076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:15.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.575+0000 7fe61f200640 1 -- 172.21.15.16:0/4024216570 shutdown_connections 2024-09-21T03:45:15.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:15.575+0000 7fe61f200640 1 -- 172.21.15.16:0/4024216570 wait complete. 2024-09-21T03:45:16.336 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-21T03:45:15.364952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664482,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":783,"num_read_kb":494,"num_write":374,"num_write_kb":2297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2951952,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379832,"kb_used_data":13868,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561992,"statfs":{"total":1151940427776,"available":1151551479808,"internally_reserved":0,"allocated":14200832,"data_stored":7969604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":49,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003836"},"pg_stats":[{"pgid":"2.1f","version":"84'13","reported_seq":124,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:09.646552+0000","last_change":"2024-09-21T03:43:04.980132+0000","last_active":"2024-09-21T03:45:09.646552+0000","last_peered":"2024-09-21T03:45:09.646552+0000","last_clean":"2024-09-21T03:45:09.646552+0000","last_became_active":"2024-09-21T03:43:04.979274+0000","last_became_peered":"2024-09-21T03:43:04.979274+0000","last_unstale":"2024-09-21T03:45:09.646552+0000","last_undegraded":"2024-09-21T03:45:09.646552+0000","last_fullsized":"2024-09-21T03:45:09.646552+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:35.047129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.672221+0000","last_change":"2024-09-21T03:43:08.011576+0000","last_active":"2024-09-21T03:44:54.672221+0000","last_peered":"2024-09-21T03:44:54.672221+0000","last_clean":"2024-09-21T03:44:54.672221+0000","last_became_active":"2024-09-21T03:43:08.011188+0000","last_became_peered":"2024-09-21T03:43:08.011188+0000","last_unstale":"2024-09-21T03:44:54.672221+0000","last_undegraded":"2024-09-21T03:44:54.672221+0000","last_fullsized":"2024-09-21T03:44:54.672221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:34.477639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677992+0000","last_change":"2024-09-21T03:43:54.493183+0000","last_active":"2024-09-21T03:44:54.677992+0000","last_peered":"2024-09-21T03:44:54.677992+0000","last_clean":"2024-09-21T03:44:54.677992+0000","last_became_active":"2024-09-21T03:43:54.492605+0000","last_became_peered":"2024-09-21T03:43:54.492605+0000","last_unstale":"2024-09-21T03:44:54.677992+0000","last_undegraded":"2024-09-21T03:44:54.677992+0000","last_fullsized":"2024-09-21T03:44:54.677992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:26.634042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.672352+0000","last_change":"2024-09-21T03:43:05.141790+0000","last_active":"2024-09-21T03:44:54.672352+0000","last_peered":"2024-09-21T03:44:54.672352+0000","last_clean":"2024-09-21T03:44:54.672352+0000","last_became_active":"2024-09-21T03:43:05.141299+0000","last_became_peered":"2024-09-21T03:43:05.141299+0000","last_unstale":"2024-09-21T03:44:54.672352+0000","last_undegraded":"2024-09-21T03:44:54.672352+0000","last_fullsized":"2024-09-21T03:44:54.672352+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:48.844615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.680045+0000","last_change":"2024-09-21T03:43:08.018245+0000","last_active":"2024-09-21T03:44:54.680045+0000","last_peered":"2024-09-21T03:44:54.680045+0000","last_clean":"2024-09-21T03:44:54.680045+0000","last_became_active":"2024-09-21T03:43:08.015719+0000","last_became_peered":"2024-09-21T03:43:08.015719+0000","last_unstale":"2024-09-21T03:44:54.680045+0000","last_undegraded":"2024-09-21T03:44:54.680045+0000","last_fullsized":"2024-09-21T03:44:54.680045+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:39.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766917+0000","last_change":"2024-09-21T03:43:54.493279+0000","last_active":"2024-09-21T03:44:54.766917+0000","last_peered":"2024-09-21T03:44:54.766917+0000","last_clean":"2024-09-21T03:44:54.766917+0000","last_became_active":"2024-09-21T03:43:54.492592+0000","last_became_peered":"2024-09-21T03:43:54.492592+0000","last_unstale":"2024-09-21T03:44:54.766917+0000","last_undegraded":"2024-09-21T03:44:54.766917+0000","last_fullsized":"2024-09-21T03:44:54.766917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:28.520990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.693998+0000","last_change":"2024-09-21T03:44:54.693431+0000","last_active":"2024-09-21T03:44:54.693998+0000","last_peered":"2024-09-21T03:44:54.693998+0000","last_clean":"2024-09-21T03:44:54.693998+0000","last_became_active":"2024-09-21T03:44:54.680324+0000","last_became_peered":"2024-09-21T03:44:54.680324+0000","last_unstale":"2024-09-21T03:44:54.693998+0000","last_undegraded":"2024-09-21T03:44:54.693998+0000","last_fullsized":"2024-09-21T03:44:54.693998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:43.730697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767573+0000","last_change":"2024-09-21T03:43:08.009018+0000","last_active":"2024-09-21T03:44:54.767573+0000","last_peered":"2024-09-21T03:44:54.767573+0000","last_clean":"2024-09-21T03:44:54.767573+0000","last_became_active":"2024-09-21T03:43:08.008614+0000","last_became_peered":"2024-09-21T03:43:08.008614+0000","last_unstale":"2024-09-21T03:44:54.767573+0000","last_undegraded":"2024-09-21T03:44:54.767573+0000","last_fullsized":"2024-09-21T03:44:54.767573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:31.000616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768700+0000","last_change":"2024-09-21T03:43:18.092750+0000","last_active":"2024-09-21T03:44:54.768700+0000","last_peered":"2024-09-21T03:44:54.768700+0000","last_clean":"2024-09-21T03:44:54.768700+0000","last_became_active":"2024-09-21T03:43:18.092122+0000","last_became_peered":"2024-09-21T03:43:18.092122+0000","last_unstale":"2024-09-21T03:44:54.768700+0000","last_undegraded":"2024-09-21T03:44:54.768700+0000","last_fullsized":"2024-09-21T03:44:54.768700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:15.032388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675502+0000","last_change":"2024-09-21T03:43:04.978045+0000","last_active":"2024-09-21T03:44:54.675502+0000","last_peered":"2024-09-21T03:44:54.675502+0000","last_clean":"2024-09-21T03:44:54.675502+0000","last_became_active":"2024-09-21T03:43:04.977565+0000","last_became_peered":"2024-09-21T03:43:04.977565+0000","last_unstale":"2024-09-21T03:44:54.675502+0000","last_undegraded":"2024-09-21T03:44:54.675502+0000","last_fullsized":"2024-09-21T03:44:54.675502+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:42.537892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766856+0000","last_change":"2024-09-21T03:43:08.007094+0000","last_active":"2024-09-21T03:44:54.766856+0000","last_peered":"2024-09-21T03:44:54.766856+0000","last_clean":"2024-09-21T03:44:54.766856+0000","last_became_active":"2024-09-21T03:43:08.006666+0000","last_became_peered":"2024-09-21T03:43:08.006666+0000","last_unstale":"2024-09-21T03:44:54.766856+0000","last_undegraded":"2024-09-21T03:44:54.766856+0000","last_fullsized":"2024-09-21T03:44:54.766856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:59.247459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767074+0000","last_change":"2024-09-21T03:43:18.081141+0000","last_active":"2024-09-21T03:44:54.767074+0000","last_peered":"2024-09-21T03:44:54.767074+0000","last_clean":"2024-09-21T03:44:54.767074+0000","last_became_active":"2024-09-21T03:43:18.080623+0000","last_became_peered":"2024-09-21T03:43:18.080623+0000","last_unstale":"2024-09-21T03:44:54.767074+0000","last_undegraded":"2024-09-21T03:44:54.767074+0000","last_fullsized":"2024-09-21T03:44:54.767074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:44.932673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678247+0000","last_change":"2024-09-21T03:43:08.008602+0000","last_active":"2024-09-21T03:44:54.678247+0000","last_peered":"2024-09-21T03:44:54.678247+0000","last_clean":"2024-09-21T03:44:54.678247+0000","last_became_active":"2024-09-21T03:43:08.008220+0000","last_became_peered":"2024-09-21T03:43:08.008220+0000","last_unstale":"2024-09-21T03:44:54.678247+0000","last_undegraded":"2024-09-21T03:44:54.678247+0000","last_fullsized":"2024-09-21T03:44:54.678247+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:28.286016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":63,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.774006+0000","last_change":"2024-09-21T03:44:54.773918+0000","last_active":"2024-09-21T03:44:54.774006+0000","last_peered":"2024-09-21T03:44:54.774006+0000","last_clean":"2024-09-21T03:44:54.774006+0000","last_became_active":"2024-09-21T03:44:54.769721+0000","last_became_peered":"2024-09-21T03:44:54.769721+0000","last_unstale":"2024-09-21T03:44:54.774006+0000","last_undegraded":"2024-09-21T03:44:54.774006+0000","last_fullsized":"2024-09-21T03:44:54.774006+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:05.656051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767123+0000","last_change":"2024-09-21T03:43:18.483523+0000","last_active":"2024-09-21T03:44:54.767123+0000","last_peered":"2024-09-21T03:44:54.767123+0000","last_clean":"2024-09-21T03:44:54.767123+0000","last_became_active":"2024-09-21T03:43:18.482833+0000","last_became_peered":"2024-09-21T03:43:18.482833+0000","last_unstale":"2024-09-21T03:44:54.767123+0000","last_undegraded":"2024-09-21T03:44:54.767123+0000","last_fullsized":"2024-09-21T03:44:54.767123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:48.236569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671717+0000","last_change":"2024-09-21T03:43:08.022592+0000","last_active":"2024-09-21T03:44:54.671717+0000","last_peered":"2024-09-21T03:44:54.671717+0000","last_clean":"2024-09-21T03:44:54.671717+0000","last_became_active":"2024-09-21T03:43:08.018065+0000","last_became_peered":"2024-09-21T03:43:08.018065+0000","last_unstale":"2024-09-21T03:44:54.671717+0000","last_undegraded":"2024-09-21T03:44:54.671717+0000","last_fullsized":"2024-09-21T03:44:54.671717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:41.524402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679199+0000","last_change":"2024-09-21T03:43:04.978788+0000","last_active":"2024-09-21T03:44:54.679199+0000","last_peered":"2024-09-21T03:44:54.679199+0000","last_clean":"2024-09-21T03:44:54.679199+0000","last_became_active":"2024-09-21T03:43:04.978382+0000","last_became_peered":"2024-09-21T03:43:04.978382+0000","last_unstale":"2024-09-21T03:44:54.679199+0000","last_undegraded":"2024-09-21T03:44:54.679199+0000","last_fullsized":"2024-09-21T03:44:54.679199+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:12.571217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768191+0000","last_change":"2024-09-21T03:43:54.493347+0000","last_active":"2024-09-21T03:44:54.768191+0000","last_peered":"2024-09-21T03:44:54.768191+0000","last_clean":"2024-09-21T03:44:54.768191+0000","last_became_active":"2024-09-21T03:43:54.492598+0000","last_became_peered":"2024-09-21T03:43:54.492598+0000","last_unstale":"2024-09-21T03:44:54.768191+0000","last_undegraded":"2024-09-21T03:44:54.768191+0000","last_fullsized":"2024-09-21T03:44:54.768191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:34.788296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678063+0000","last_change":"2024-09-21T03:43:08.014368+0000","last_active":"2024-09-21T03:44:54.678063+0000","last_peered":"2024-09-21T03:44:54.678063+0000","last_clean":"2024-09-21T03:44:54.678063+0000","last_became_active":"2024-09-21T03:43:08.013952+0000","last_became_peered":"2024-09-21T03:43:08.013952+0000","last_unstale":"2024-09-21T03:44:54.678063+0000","last_undegraded":"2024-09-21T03:44:54.678063+0000","last_fullsized":"2024-09-21T03:44:54.678063+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:16.893588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671939+0000","last_change":"2024-09-21T03:43:04.976162+0000","last_active":"2024-09-21T03:44:54.671939+0000","last_peered":"2024-09-21T03:44:54.671939+0000","last_clean":"2024-09-21T03:44:54.671939+0000","last_became_active":"2024-09-21T03:43:04.973790+0000","last_became_peered":"2024-09-21T03:43:04.973790+0000","last_unstale":"2024-09-21T03:44:54.671939+0000","last_undegraded":"2024-09-21T03:44:54.671939+0000","last_fullsized":"2024-09-21T03:44:54.671939+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:47.838229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670284+0000","last_change":"2024-09-21T03:43:18.084344+0000","last_active":"2024-09-21T03:44:54.670284+0000","last_peered":"2024-09-21T03:44:54.670284+0000","last_clean":"2024-09-21T03:44:54.670284+0000","last_became_active":"2024-09-21T03:43:18.083232+0000","last_became_peered":"2024-09-21T03:43:18.083232+0000","last_unstale":"2024-09-21T03:44:54.670284+0000","last_undegraded":"2024-09-21T03:44:54.670284+0000","last_fullsized":"2024-09-21T03:44:54.670284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:46.254405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.674815+0000","last_change":"2024-09-21T03:43:08.009932+0000","last_active":"2024-09-21T03:44:54.674815+0000","last_peered":"2024-09-21T03:44:54.674815+0000","last_clean":"2024-09-21T03:44:54.674815+0000","last_became_active":"2024-09-21T03:43:08.009492+0000","last_became_peered":"2024-09-21T03:43:08.009492+0000","last_unstale":"2024-09-21T03:44:54.674815+0000","last_undegraded":"2024-09-21T03:44:54.674815+0000","last_fullsized":"2024-09-21T03:44:54.674815+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:58.679348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767989+0000","last_change":"2024-09-21T03:43:05.141927+0000","last_active":"2024-09-21T03:44:54.767989+0000","last_peered":"2024-09-21T03:44:54.767989+0000","last_clean":"2024-09-21T03:44:54.767989+0000","last_became_active":"2024-09-21T03:43:05.141516+0000","last_became_peered":"2024-09-21T03:43:05.141516+0000","last_unstale":"2024-09-21T03:44:54.767989+0000","last_undegraded":"2024-09-21T03:44:54.767989+0000","last_fullsized":"2024-09-21T03:44:54.767989+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:35.666981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767323+0000","last_change":"2024-09-21T03:43:18.484478+0000","last_active":"2024-09-21T03:44:54.767323+0000","last_peered":"2024-09-21T03:44:54.767323+0000","last_clean":"2024-09-21T03:44:54.767323+0000","last_became_active":"2024-09-21T03:43:18.483890+0000","last_became_peered":"2024-09-21T03:43:18.483890+0000","last_unstale":"2024-09-21T03:44:54.767323+0000","last_undegraded":"2024-09-21T03:44:54.767323+0000","last_fullsized":"2024-09-21T03:44:54.767323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:03.542783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679792+0000","last_change":"2024-09-21T03:43:08.017785+0000","last_active":"2024-09-21T03:44:54.679792+0000","last_peered":"2024-09-21T03:44:54.679792+0000","last_clean":"2024-09-21T03:44:54.679792+0000","last_became_active":"2024-09-21T03:43:08.017389+0000","last_became_peered":"2024-09-21T03:43:08.017389+0000","last_unstale":"2024-09-21T03:44:54.679792+0000","last_undegraded":"2024-09-21T03:44:54.679792+0000","last_fullsized":"2024-09-21T03:44:54.679792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:37.557061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":54,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767655+0000","last_change":"2024-09-21T03:43:05.567829+0000","last_active":"2024-09-21T03:44:54.767655+0000","last_peered":"2024-09-21T03:44:54.767655+0000","last_clean":"2024-09-21T03:44:54.767655+0000","last_became_active":"2024-09-21T03:43:05.567401+0000","last_became_peered":"2024-09-21T03:43:05.567401+0000","last_unstale":"2024-09-21T03:44:54.767655+0000","last_undegraded":"2024-09-21T03:44:54.767655+0000","last_fullsized":"2024-09-21T03:44:54.767655+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:03.954280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678801+0000","last_change":"2024-09-21T03:43:18.080064+0000","last_active":"2024-09-21T03:44:54.678801+0000","last_peered":"2024-09-21T03:44:54.678801+0000","last_clean":"2024-09-21T03:44:54.678801+0000","last_became_active":"2024-09-21T03:43:18.079569+0000","last_became_peered":"2024-09-21T03:43:18.079569+0000","last_unstale":"2024-09-21T03:44:54.678801+0000","last_undegraded":"2024-09-21T03:44:54.678801+0000","last_fullsized":"2024-09-21T03:44:54.678801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:29.568476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.672366+0000","last_change":"2024-09-21T03:43:08.011603+0000","last_active":"2024-09-21T03:44:54.672366+0000","last_peered":"2024-09-21T03:44:54.672366+0000","last_clean":"2024-09-21T03:44:54.672366+0000","last_became_active":"2024-09-21T03:43:08.011078+0000","last_became_peered":"2024-09-21T03:43:08.011078+0000","last_unstale":"2024-09-21T03:44:54.672366+0000","last_undegraded":"2024-09-21T03:44:54.672366+0000","last_fullsized":"2024-09-21T03:44:54.672366+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:44.764713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'15","reported_seq":60,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670668+0000","last_change":"2024-09-21T03:43:04.979223+0000","last_active":"2024-09-21T03:44:54.670668+0000","last_peered":"2024-09-21T03:44:54.670668+0000","last_clean":"2024-09-21T03:44:54.670668+0000","last_became_active":"2024-09-21T03:43:04.978675+0000","last_became_peered":"2024-09-21T03:43:04.978675+0000","last_unstale":"2024-09-21T03:44:54.670668+0000","last_undegraded":"2024-09-21T03:44:54.670668+0000","last_fullsized":"2024-09-21T03:44:54.670668+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:09.717921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671303+0000","last_change":"2024-09-21T03:43:18.482334+0000","last_active":"2024-09-21T03:44:54.671303+0000","last_peered":"2024-09-21T03:44:54.671303+0000","last_clean":"2024-09-21T03:44:54.671303+0000","last_became_active":"2024-09-21T03:43:18.481881+0000","last_became_peered":"2024-09-21T03:43:18.481881+0000","last_unstale":"2024-09-21T03:44:54.671303+0000","last_undegraded":"2024-09-21T03:44:54.671303+0000","last_fullsized":"2024-09-21T03:44:54.671303+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:02:32.497348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767021+0000","last_change":"2024-09-21T03:43:07.993771+0000","last_active":"2024-09-21T03:44:54.767021+0000","last_peered":"2024-09-21T03:44:54.767021+0000","last_clean":"2024-09-21T03:44:54.767021+0000","last_became_active":"2024-09-21T03:43:07.993268+0000","last_became_peered":"2024-09-21T03:43:07.993268+0000","last_unstale":"2024-09-21T03:44:54.767021+0000","last_undegraded":"2024-09-21T03:44:54.767021+0000","last_fullsized":"2024-09-21T03:44:54.767021+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:42.348561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675923+0000","last_change":"2024-09-21T03:43:05.141786+0000","last_active":"2024-09-21T03:44:54.675923+0000","last_peered":"2024-09-21T03:44:54.675923+0000","last_clean":"2024-09-21T03:44:54.675923+0000","last_became_active":"2024-09-21T03:43:05.141307+0000","last_became_peered":"2024-09-21T03:43:05.141307+0000","last_unstale":"2024-09-21T03:44:54.675923+0000","last_undegraded":"2024-09-21T03:44:54.675923+0000","last_fullsized":"2024-09-21T03:44:54.675923+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:12.139398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677302+0000","last_change":"2024-09-21T03:43:18.087848+0000","last_active":"2024-09-21T03:44:54.677302+0000","last_peered":"2024-09-21T03:44:54.677302+0000","last_clean":"2024-09-21T03:44:54.677302+0000","last_became_active":"2024-09-21T03:43:18.087400+0000","last_became_peered":"2024-09-21T03:43:18.087400+0000","last_unstale":"2024-09-21T03:44:54.677302+0000","last_undegraded":"2024-09-21T03:44:54.677302+0000","last_fullsized":"2024-09-21T03:44:54.677302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:46.902979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768961+0000","last_change":"2024-09-21T03:43:08.010051+0000","last_active":"2024-09-21T03:44:54.768961+0000","last_peered":"2024-09-21T03:44:54.768961+0000","last_clean":"2024-09-21T03:44:54.768961+0000","last_became_active":"2024-09-21T03:43:08.007700+0000","last_became_peered":"2024-09-21T03:43:08.007700+0000","last_unstale":"2024-09-21T03:44:54.768961+0000","last_undegraded":"2024-09-21T03:44:54.768961+0000","last_fullsized":"2024-09-21T03:44:54.768961+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:31.828193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670543+0000","last_change":"2024-09-21T03:43:05.140490+0000","last_active":"2024-09-21T03:44:54.670543+0000","last_peered":"2024-09-21T03:44:54.670543+0000","last_clean":"2024-09-21T03:44:54.670543+0000","last_became_active":"2024-09-21T03:43:05.139979+0000","last_became_peered":"2024-09-21T03:43:05.139979+0000","last_unstale":"2024-09-21T03:44:54.670543+0000","last_undegraded":"2024-09-21T03:44:54.670543+0000","last_fullsized":"2024-09-21T03:44:54.670543+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:20.957596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.674815+0000","last_change":"2024-09-21T03:43:18.482777+0000","last_active":"2024-09-21T03:44:54.674815+0000","last_peered":"2024-09-21T03:44:54.674815+0000","last_clean":"2024-09-21T03:44:54.674815+0000","last_became_active":"2024-09-21T03:43:18.482244+0000","last_became_peered":"2024-09-21T03:43:18.482244+0000","last_unstale":"2024-09-21T03:44:54.674815+0000","last_undegraded":"2024-09-21T03:44:54.674815+0000","last_fullsized":"2024-09-21T03:44:54.674815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:41.997309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768352+0000","last_change":"2024-09-21T03:43:08.019087+0000","last_active":"2024-09-21T03:44:54.768352+0000","last_peered":"2024-09-21T03:44:54.768352+0000","last_clean":"2024-09-21T03:44:54.768352+0000","last_became_active":"2024-09-21T03:43:08.018467+0000","last_became_peered":"2024-09-21T03:43:08.018467+0000","last_unstale":"2024-09-21T03:44:54.768352+0000","last_undegraded":"2024-09-21T03:44:54.768352+0000","last_fullsized":"2024-09-21T03:44:54.768352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:25.824955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"84'5","reported_seq":121,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:13.653158+0000","last_change":"2024-09-21T03:43:04.973433+0000","last_active":"2024-09-21T03:45:13.653158+0000","last_peered":"2024-09-21T03:45:13.653158+0000","last_clean":"2024-09-21T03:45:13.653158+0000","last_became_active":"2024-09-21T03:43:04.973033+0000","last_became_peered":"2024-09-21T03:43:04.973033+0000","last_unstale":"2024-09-21T03:45:13.653158+0000","last_undegraded":"2024-09-21T03:45:13.653158+0000","last_fullsized":"2024-09-21T03:45:13.653158+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:25.462635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766845+0000","last_change":"2024-09-21T03:43:18.082385+0000","last_active":"2024-09-21T03:44:54.766845+0000","last_peered":"2024-09-21T03:44:54.766845+0000","last_clean":"2024-09-21T03:44:54.766845+0000","last_became_active":"2024-09-21T03:43:18.081958+0000","last_became_peered":"2024-09-21T03:43:18.081958+0000","last_unstale":"2024-09-21T03:44:54.766845+0000","last_undegraded":"2024-09-21T03:44:54.766845+0000","last_fullsized":"2024-09-21T03:44:54.766845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:36.554838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766522+0000","last_change":"2024-09-21T03:43:08.008307+0000","last_active":"2024-09-21T03:44:54.766522+0000","last_peered":"2024-09-21T03:44:54.766522+0000","last_clean":"2024-09-21T03:44:54.766522+0000","last_became_active":"2024-09-21T03:43:08.006736+0000","last_became_peered":"2024-09-21T03:43:08.006736+0000","last_unstale":"2024-09-21T03:44:54.766522+0000","last_undegraded":"2024-09-21T03:44:54.766522+0000","last_fullsized":"2024-09-21T03:44:54.766522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:31.694655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":119,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:12.651467+0000","last_change":"2024-09-21T03:43:05.140263+0000","last_active":"2024-09-21T03:45:12.651467+0000","last_peered":"2024-09-21T03:45:12.651467+0000","last_clean":"2024-09-21T03:45:12.651467+0000","last_became_active":"2024-09-21T03:43:05.139869+0000","last_became_peered":"2024-09-21T03:43:05.139869+0000","last_unstale":"2024-09-21T03:45:12.651467+0000","last_undegraded":"2024-09-21T03:45:12.651467+0000","last_fullsized":"2024-09-21T03:45:12.651467+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:19.313677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766813+0000","last_change":"2024-09-21T03:43:18.483795+0000","last_active":"2024-09-21T03:44:54.766813+0000","last_peered":"2024-09-21T03:44:54.766813+0000","last_clean":"2024-09-21T03:44:54.766813+0000","last_became_active":"2024-09-21T03:43:18.483159+0000","last_became_peered":"2024-09-21T03:43:18.483159+0000","last_unstale":"2024-09-21T03:44:54.766813+0000","last_undegraded":"2024-09-21T03:44:54.766813+0000","last_fullsized":"2024-09-21T03:44:54.766813+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:56.277956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766443+0000","last_change":"2024-09-21T03:43:08.001661+0000","last_active":"2024-09-21T03:44:54.766443+0000","last_peered":"2024-09-21T03:44:54.766443+0000","last_clean":"2024-09-21T03:44:54.766443+0000","last_became_active":"2024-09-21T03:43:08.001254+0000","last_became_peered":"2024-09-21T03:43:08.001254+0000","last_unstale":"2024-09-21T03:44:54.766443+0000","last_undegraded":"2024-09-21T03:44:54.766443+0000","last_fullsized":"2024-09-21T03:44:54.766443+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:19.020542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678080+0000","last_change":"2024-09-21T03:43:04.979127+0000","last_active":"2024-09-21T03:44:54.678080+0000","last_peered":"2024-09-21T03:44:54.678080+0000","last_clean":"2024-09-21T03:44:54.678080+0000","last_became_active":"2024-09-21T03:43:04.978705+0000","last_became_peered":"2024-09-21T03:43:04.978705+0000","last_unstale":"2024-09-21T03:44:54.678080+0000","last_undegraded":"2024-09-21T03:44:54.678080+0000","last_fullsized":"2024-09-21T03:44:54.678080+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:48.548002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766916+0000","last_change":"2024-09-21T03:43:18.483682+0000","last_active":"2024-09-21T03:44:54.766916+0000","last_peered":"2024-09-21T03:44:54.766916+0000","last_clean":"2024-09-21T03:44:54.766916+0000","last_became_active":"2024-09-21T03:43:18.483124+0000","last_became_peered":"2024-09-21T03:43:18.483124+0000","last_unstale":"2024-09-21T03:44:54.766916+0000","last_undegraded":"2024-09-21T03:44:54.766916+0000","last_fullsized":"2024-09-21T03:44:54.766916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:12.513795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671793+0000","last_change":"2024-09-21T03:43:08.013193+0000","last_active":"2024-09-21T03:44:54.671793+0000","last_peered":"2024-09-21T03:44:54.671793+0000","last_clean":"2024-09-21T03:44:54.671793+0000","last_became_active":"2024-09-21T03:43:08.010360+0000","last_became_peered":"2024-09-21T03:43:08.010360+0000","last_unstale":"2024-09-21T03:44:54.671793+0000","last_undegraded":"2024-09-21T03:44:54.671793+0000","last_fullsized":"2024-09-21T03:44:54.671793+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:28.101646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768945+0000","last_change":"2024-09-21T03:43:04.971782+0000","last_active":"2024-09-21T03:44:54.768945+0000","last_peered":"2024-09-21T03:44:54.768945+0000","last_clean":"2024-09-21T03:44:54.768945+0000","last_became_active":"2024-09-21T03:43:04.971375+0000","last_became_peered":"2024-09-21T03:43:04.971375+0000","last_unstale":"2024-09-21T03:44:54.768945+0000","last_undegraded":"2024-09-21T03:44:54.768945+0000","last_fullsized":"2024-09-21T03:44:54.768945+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:49.573490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'92","reported_seq":115,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678174+0000","last_change":"2024-09-21T03:41:42.914435+0000","last_active":"2024-09-21T03:44:54.678174+0000","last_peered":"2024-09-21T03:44:54.678174+0000","last_clean":"2024-09-21T03:44:54.678174+0000","last_became_active":"2024-09-21T03:41:42.909490+0000","last_became_peered":"2024-09-21T03:41:42.909490+0000","last_unstale":"2024-09-21T03:44:54.678174+0000","last_undegraded":"2024-09-21T03:44:54.678174+0000","last_fullsized":"2024-09-21T03:44:54.678174+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:56.507027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:56.507027+0000","last_clean_scrub_stamp":"2024-09-21T03:37:56.507027+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:34.564062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678228+0000","last_change":"2024-09-21T03:43:18.084701+0000","last_active":"2024-09-21T03:44:54.678228+0000","last_peered":"2024-09-21T03:44:54.678228+0000","last_clean":"2024-09-21T03:44:54.678228+0000","last_became_active":"2024-09-21T03:43:18.084251+0000","last_became_peered":"2024-09-21T03:43:18.084251+0000","last_unstale":"2024-09-21T03:44:54.678228+0000","last_undegraded":"2024-09-21T03:44:54.678228+0000","last_fullsized":"2024-09-21T03:44:54.678228+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:15.660276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678714+0000","last_change":"2024-09-21T03:43:08.012195+0000","last_active":"2024-09-21T03:44:54.678714+0000","last_peered":"2024-09-21T03:44:54.678714+0000","last_clean":"2024-09-21T03:44:54.678714+0000","last_became_active":"2024-09-21T03:43:08.011670+0000","last_became_peered":"2024-09-21T03:43:08.011670+0000","last_unstale":"2024-09-21T03:44:54.678714+0000","last_undegraded":"2024-09-21T03:44:54.678714+0000","last_fullsized":"2024-09-21T03:44:54.678714+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:14.290074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":119,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:09.647333+0000","last_change":"2024-09-21T03:43:05.567898+0000","last_active":"2024-09-21T03:45:09.647333+0000","last_peered":"2024-09-21T03:45:09.647333+0000","last_clean":"2024-09-21T03:45:09.647333+0000","last_became_active":"2024-09-21T03:43:05.567367+0000","last_became_peered":"2024-09-21T03:43:05.567367+0000","last_unstale":"2024-09-21T03:45:09.647333+0000","last_undegraded":"2024-09-21T03:45:09.647333+0000","last_fullsized":"2024-09-21T03:45:09.647333+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:55.120838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768128+0000","last_change":"2024-09-21T03:43:18.073942+0000","last_active":"2024-09-21T03:44:54.768128+0000","last_peered":"2024-09-21T03:44:54.768128+0000","last_clean":"2024-09-21T03:44:54.768128+0000","last_became_active":"2024-09-21T03:43:18.073323+0000","last_became_peered":"2024-09-21T03:43:18.073323+0000","last_unstale":"2024-09-21T03:44:54.768128+0000","last_undegraded":"2024-09-21T03:44:54.768128+0000","last_fullsized":"2024-09-21T03:44:54.768128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:41.677143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678249+0000","last_change":"2024-09-21T03:43:08.016636+0000","last_active":"2024-09-21T03:44:54.678249+0000","last_peered":"2024-09-21T03:44:54.678249+0000","last_clean":"2024-09-21T03:44:54.678249+0000","last_became_active":"2024-09-21T03:43:08.016235+0000","last_became_peered":"2024-09-21T03:43:08.016235+0000","last_unstale":"2024-09-21T03:44:54.678249+0000","last_undegraded":"2024-09-21T03:44:54.678249+0000","last_fullsized":"2024-09-21T03:44:54.678249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:35.365830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767087+0000","last_change":"2024-09-21T03:43:04.980911+0000","last_active":"2024-09-21T03:44:54.767087+0000","last_peered":"2024-09-21T03:44:54.767087+0000","last_clean":"2024-09-21T03:44:54.767087+0000","last_became_active":"2024-09-21T03:43:04.980512+0000","last_became_peered":"2024-09-21T03:43:04.980512+0000","last_unstale":"2024-09-21T03:44:54.767087+0000","last_undegraded":"2024-09-21T03:44:54.767087+0000","last_fullsized":"2024-09-21T03:44:54.767087+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:04.136345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679872+0000","last_change":"2024-09-21T03:43:18.085874+0000","last_active":"2024-09-21T03:44:54.679872+0000","last_peered":"2024-09-21T03:44:54.679872+0000","last_clean":"2024-09-21T03:44:54.679872+0000","last_became_active":"2024-09-21T03:43:18.085381+0000","last_became_peered":"2024-09-21T03:43:18.085381+0000","last_unstale":"2024-09-21T03:44:54.679872+0000","last_undegraded":"2024-09-21T03:44:54.679872+0000","last_fullsized":"2024-09-21T03:44:54.679872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:02.829202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678621+0000","last_change":"2024-09-21T03:43:08.014776+0000","last_active":"2024-09-21T03:44:54.678621+0000","last_peered":"2024-09-21T03:44:54.678621+0000","last_clean":"2024-09-21T03:44:54.678621+0000","last_became_active":"2024-09-21T03:43:08.014191+0000","last_became_peered":"2024-09-21T03:43:08.014191+0000","last_unstale":"2024-09-21T03:44:54.678621+0000","last_undegraded":"2024-09-21T03:44:54.678621+0000","last_fullsized":"2024-09-21T03:44:54.678621+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:19.114838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766891+0000","last_change":"2024-09-21T03:43:04.973714+0000","last_active":"2024-09-21T03:44:54.766891+0000","last_peered":"2024-09-21T03:44:54.766891+0000","last_clean":"2024-09-21T03:44:54.766891+0000","last_became_active":"2024-09-21T03:43:04.973348+0000","last_became_peered":"2024-09-21T03:43:04.973348+0000","last_unstale":"2024-09-21T03:44:54.766891+0000","last_undegraded":"2024-09-21T03:44:54.766891+0000","last_fullsized":"2024-09-21T03:44:54.766891+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:00.872589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766289+0000","last_change":"2024-09-21T03:43:18.082643+0000","last_active":"2024-09-21T03:44:54.766289+0000","last_peered":"2024-09-21T03:44:54.766289+0000","last_clean":"2024-09-21T03:44:54.766289+0000","last_became_active":"2024-09-21T03:43:18.082157+0000","last_became_peered":"2024-09-21T03:43:18.082157+0000","last_unstale":"2024-09-21T03:44:54.766289+0000","last_undegraded":"2024-09-21T03:44:54.766289+0000","last_fullsized":"2024-09-21T03:44:54.766289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:52.652014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677306+0000","last_change":"2024-09-21T03:43:08.010913+0000","last_active":"2024-09-21T03:44:54.677306+0000","last_peered":"2024-09-21T03:44:54.677306+0000","last_clean":"2024-09-21T03:44:54.677306+0000","last_became_active":"2024-09-21T03:43:08.010483+0000","last_became_peered":"2024-09-21T03:43:08.010483+0000","last_unstale":"2024-09-21T03:44:54.677306+0000","last_undegraded":"2024-09-21T03:44:54.677306+0000","last_fullsized":"2024-09-21T03:44:54.677306+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:52.158346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675912+0000","last_change":"2024-09-21T03:43:04.980260+0000","last_active":"2024-09-21T03:44:54.675912+0000","last_peered":"2024-09-21T03:44:54.675912+0000","last_clean":"2024-09-21T03:44:54.675912+0000","last_became_active":"2024-09-21T03:43:04.978999+0000","last_became_peered":"2024-09-21T03:43:04.978999+0000","last_unstale":"2024-09-21T03:44:54.675912+0000","last_undegraded":"2024-09-21T03:44:54.675912+0000","last_fullsized":"2024-09-21T03:44:54.675912+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:33.682591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766304+0000","last_change":"2024-09-21T03:43:18.083665+0000","last_active":"2024-09-21T03:44:54.766304+0000","last_peered":"2024-09-21T03:44:54.766304+0000","last_clean":"2024-09-21T03:44:54.766304+0000","last_became_active":"2024-09-21T03:43:18.083215+0000","last_became_peered":"2024-09-21T03:43:18.083215+0000","last_unstale":"2024-09-21T03:44:54.766304+0000","last_undegraded":"2024-09-21T03:44:54.766304+0000","last_fullsized":"2024-09-21T03:44:54.766304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:55.968968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766486+0000","last_change":"2024-09-21T03:43:08.010570+0000","last_active":"2024-09-21T03:44:54.766486+0000","last_peered":"2024-09-21T03:44:54.766486+0000","last_clean":"2024-09-21T03:44:54.766486+0000","last_became_active":"2024-09-21T03:43:08.010163+0000","last_became_peered":"2024-09-21T03:43:08.010163+0000","last_unstale":"2024-09-21T03:44:54.766486+0000","last_undegraded":"2024-09-21T03:44:54.766486+0000","last_fullsized":"2024-09-21T03:44:54.766486+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:17.368534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.707142+0000","last_change":"2024-09-21T03:44:54.706609+0000","last_active":"2024-09-21T03:44:54.707142+0000","last_peered":"2024-09-21T03:44:54.707142+0000","last_clean":"2024-09-21T03:44:54.707142+0000","last_became_active":"2024-09-21T03:44:54.681060+0000","last_became_peered":"2024-09-21T03:44:54.681060+0000","last_unstale":"2024-09-21T03:44:54.707142+0000","last_undegraded":"2024-09-21T03:44:54.707142+0000","last_fullsized":"2024-09-21T03:44:54.707142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:49.797107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766891+0000","last_change":"2024-09-21T03:43:18.089516+0000","last_active":"2024-09-21T03:44:54.766891+0000","last_peered":"2024-09-21T03:44:54.766891+0000","last_clean":"2024-09-21T03:44:54.766891+0000","last_became_active":"2024-09-21T03:43:18.089097+0000","last_became_peered":"2024-09-21T03:43:18.089097+0000","last_unstale":"2024-09-21T03:44:54.766891+0000","last_undegraded":"2024-09-21T03:44:54.766891+0000","last_fullsized":"2024-09-21T03:44:54.766891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:50.428401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677900+0000","last_change":"2024-09-21T03:43:08.015762+0000","last_active":"2024-09-21T03:44:54.677900+0000","last_peered":"2024-09-21T03:44:54.677900+0000","last_clean":"2024-09-21T03:44:54.677900+0000","last_became_active":"2024-09-21T03:43:08.015389+0000","last_became_peered":"2024-09-21T03:43:08.015389+0000","last_unstale":"2024-09-21T03:44:54.677900+0000","last_undegraded":"2024-09-21T03:44:54.677900+0000","last_fullsized":"2024-09-21T03:44:54.677900+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:00.341479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'5","reported_seq":121,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:13.653619+0000","last_change":"2024-09-21T03:43:04.973173+0000","last_active":"2024-09-21T03:45:13.653619+0000","last_peered":"2024-09-21T03:45:13.653619+0000","last_clean":"2024-09-21T03:45:13.653619+0000","last_became_active":"2024-09-21T03:43:04.972740+0000","last_became_peered":"2024-09-21T03:43:04.972740+0000","last_unstale":"2024-09-21T03:45:13.653619+0000","last_undegraded":"2024-09-21T03:45:13.653619+0000","last_fullsized":"2024-09-21T03:45:13.653619+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:26.329220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670457+0000","last_change":"2024-09-21T03:43:18.093730+0000","last_active":"2024-09-21T03:44:54.670457+0000","last_peered":"2024-09-21T03:44:54.670457+0000","last_clean":"2024-09-21T03:44:54.670457+0000","last_became_active":"2024-09-21T03:43:18.089621+0000","last_became_peered":"2024-09-21T03:43:18.089621+0000","last_unstale":"2024-09-21T03:44:54.670457+0000","last_undegraded":"2024-09-21T03:44:54.670457+0000","last_fullsized":"2024-09-21T03:44:54.670457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:05.227038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678343+0000","last_change":"2024-09-21T03:43:54.465751+0000","last_active":"2024-09-21T03:44:54.678343+0000","last_peered":"2024-09-21T03:44:54.678343+0000","last_clean":"2024-09-21T03:44:54.678343+0000","last_became_active":"2024-09-21T03:43:54.465333+0000","last_became_peered":"2024-09-21T03:43:54.465333+0000","last_unstale":"2024-09-21T03:44:54.678343+0000","last_undegraded":"2024-09-21T03:44:54.678343+0000","last_fullsized":"2024-09-21T03:44:54.678343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:34.307699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766650+0000","last_change":"2024-09-21T03:43:04.979385+0000","last_active":"2024-09-21T03:44:54.766650+0000","last_peered":"2024-09-21T03:44:54.766650+0000","last_clean":"2024-09-21T03:44:54.766650+0000","last_became_active":"2024-09-21T03:43:04.978764+0000","last_became_peered":"2024-09-21T03:43:04.978764+0000","last_unstale":"2024-09-21T03:44:54.766650+0000","last_undegraded":"2024-09-21T03:44:54.766650+0000","last_fullsized":"2024-09-21T03:44:54.766650+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:34.256966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677391+0000","last_change":"2024-09-21T03:43:18.095937+0000","last_active":"2024-09-21T03:44:54.677391+0000","last_peered":"2024-09-21T03:44:54.677391+0000","last_clean":"2024-09-21T03:44:54.677391+0000","last_became_active":"2024-09-21T03:43:18.095531+0000","last_became_peered":"2024-09-21T03:43:18.095531+0000","last_unstale":"2024-09-21T03:44:54.677391+0000","last_undegraded":"2024-09-21T03:44:54.677391+0000","last_fullsized":"2024-09-21T03:44:54.677391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:18.089661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766550+0000","last_change":"2024-09-21T03:43:08.011342+0000","last_active":"2024-09-21T03:44:54.766550+0000","last_peered":"2024-09-21T03:44:54.766550+0000","last_clean":"2024-09-21T03:44:54.766550+0000","last_became_active":"2024-09-21T03:43:08.010963+0000","last_became_peered":"2024-09-21T03:43:08.010963+0000","last_unstale":"2024-09-21T03:44:54.766550+0000","last_undegraded":"2024-09-21T03:44:54.766550+0000","last_fullsized":"2024-09-21T03:44:54.766550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:05.970863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679486+0000","last_change":"2024-09-21T03:43:04.972896+0000","last_active":"2024-09-21T03:44:54.679486+0000","last_peered":"2024-09-21T03:44:54.679486+0000","last_clean":"2024-09-21T03:44:54.679486+0000","last_became_active":"2024-09-21T03:43:04.972535+0000","last_became_peered":"2024-09-21T03:43:04.972535+0000","last_unstale":"2024-09-21T03:44:54.679486+0000","last_undegraded":"2024-09-21T03:44:54.679486+0000","last_fullsized":"2024-09-21T03:44:54.679486+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:36.362019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766858+0000","last_change":"2024-09-21T03:43:18.081300+0000","last_active":"2024-09-21T03:44:54.766858+0000","last_peered":"2024-09-21T03:44:54.766858+0000","last_clean":"2024-09-21T03:44:54.766858+0000","last_became_active":"2024-09-21T03:43:18.080874+0000","last_became_peered":"2024-09-21T03:43:18.080874+0000","last_unstale":"2024-09-21T03:44:54.766858+0000","last_undegraded":"2024-09-21T03:44:54.766858+0000","last_fullsized":"2024-09-21T03:44:54.766858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:13.246348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675450+0000","last_change":"2024-09-21T03:43:08.014954+0000","last_active":"2024-09-21T03:44:54.675450+0000","last_peered":"2024-09-21T03:44:54.675450+0000","last_clean":"2024-09-21T03:44:54.675450+0000","last_became_active":"2024-09-21T03:43:08.014576+0000","last_became_peered":"2024-09-21T03:43:08.014576+0000","last_unstale":"2024-09-21T03:44:54.675450+0000","last_undegraded":"2024-09-21T03:44:54.675450+0000","last_fullsized":"2024-09-21T03:44:54.675450+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:13.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"84'35","reported_seq":223,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:12.651517+0000","last_change":"2024-09-21T03:43:04.973305+0000","last_active":"2024-09-21T03:45:12.651517+0000","last_peered":"2024-09-21T03:45:12.651517+0000","last_clean":"2024-09-21T03:45:12.651517+0000","last_became_active":"2024-09-21T03:43:04.972891+0000","last_became_peered":"2024-09-21T03:43:04.972891+0000","last_unstale":"2024-09-21T03:45:12.651517+0000","last_undegraded":"2024-09-21T03:45:12.651517+0000","last_fullsized":"2024-09-21T03:45:12.651517+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:15.653769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67067,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":143,"num_read_kb":137,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670013+0000","last_change":"2024-09-21T03:43:18.093613+0000","last_active":"2024-09-21T03:44:54.670013+0000","last_peered":"2024-09-21T03:44:54.670013+0000","last_clean":"2024-09-21T03:44:54.670013+0000","last_became_active":"2024-09-21T03:43:18.093195+0000","last_became_peered":"2024-09-21T03:43:18.093195+0000","last_unstale":"2024-09-21T03:44:54.670013+0000","last_undegraded":"2024-09-21T03:44:54.670013+0000","last_fullsized":"2024-09-21T03:44:54.670013+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:17.825071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767766+0000","last_change":"2024-09-21T03:43:08.014314+0000","last_active":"2024-09-21T03:44:54.767766+0000","last_peered":"2024-09-21T03:44:54.767766+0000","last_clean":"2024-09-21T03:44:54.767766+0000","last_became_active":"2024-09-21T03:43:08.012925+0000","last_became_peered":"2024-09-21T03:43:08.012925+0000","last_unstale":"2024-09-21T03:44:54.767766+0000","last_undegraded":"2024-09-21T03:44:54.767766+0000","last_fullsized":"2024-09-21T03:44:54.767766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:47:28.599444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767033+0000","last_change":"2024-09-21T03:43:05.141584+0000","last_active":"2024-09-21T03:44:54.767033+0000","last_peered":"2024-09-21T03:44:54.767033+0000","last_clean":"2024-09-21T03:44:54.767033+0000","last_became_active":"2024-09-21T03:43:05.141200+0000","last_became_peered":"2024-09-21T03:43:05.141200+0000","last_unstale":"2024-09-21T03:44:54.767033+0000","last_undegraded":"2024-09-21T03:44:54.767033+0000","last_fullsized":"2024-09-21T03:44:54.767033+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:04.357970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768505+0000","last_change":"2024-09-21T03:43:18.092184+0000","last_active":"2024-09-21T03:44:54.768505+0000","last_peered":"2024-09-21T03:44:54.768505+0000","last_clean":"2024-09-21T03:44:54.768505+0000","last_became_active":"2024-09-21T03:43:18.091629+0000","last_became_peered":"2024-09-21T03:43:18.091629+0000","last_unstale":"2024-09-21T03:44:54.768505+0000","last_undegraded":"2024-09-21T03:44:54.768505+0000","last_fullsized":"2024-09-21T03:44:54.768505+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:39.448009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.680071+0000","last_change":"2024-09-21T03:43:08.018335+0000","last_active":"2024-09-21T03:44:54.680071+0000","last_peered":"2024-09-21T03:44:54.680071+0000","last_clean":"2024-09-21T03:44:54.680071+0000","last_became_active":"2024-09-21T03:43:08.014005+0000","last_became_peered":"2024-09-21T03:43:08.014005+0000","last_unstale":"2024-09-21T03:44:54.680071+0000","last_undegraded":"2024-09-21T03:44:54.680071+0000","last_fullsized":"2024-09-21T03:44:54.680071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:35.749381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678484+0000","last_change":"2024-09-21T03:43:04.978609+0000","last_active":"2024-09-21T03:44:54.678484+0000","last_peered":"2024-09-21T03:44:54.678484+0000","last_clean":"2024-09-21T03:44:54.678484+0000","last_became_active":"2024-09-21T03:43:04.978146+0000","last_became_peered":"2024-09-21T03:43:04.978146+0000","last_unstale":"2024-09-21T03:44:54.678484+0000","last_undegraded":"2024-09-21T03:44:54.678484+0000","last_fullsized":"2024-09-21T03:44:54.678484+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:41.651745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.669874+0000","last_change":"2024-09-21T03:43:18.089399+0000","last_active":"2024-09-21T03:44:54.669874+0000","last_peered":"2024-09-21T03:44:54.669874+0000","last_clean":"2024-09-21T03:44:54.669874+0000","last_became_active":"2024-09-21T03:43:18.086156+0000","last_became_peered":"2024-09-21T03:43:18.086156+0000","last_unstale":"2024-09-21T03:44:54.669874+0000","last_undegraded":"2024-09-21T03:44:54.669874+0000","last_fullsized":"2024-09-21T03:44:54.669874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:41.955341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768619+0000","last_change":"2024-09-21T03:43:08.008773+0000","last_active":"2024-09-21T03:44:54.768619+0000","last_peered":"2024-09-21T03:44:54.768619+0000","last_clean":"2024-09-21T03:44:54.768619+0000","last_became_active":"2024-09-21T03:43:08.008270+0000","last_became_peered":"2024-09-21T03:43:08.008270+0000","last_unstale":"2024-09-21T03:44:54.768619+0000","last_undegraded":"2024-09-21T03:44:54.768619+0000","last_fullsized":"2024-09-21T03:44:54.768619+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:57.901701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":183,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:12.651417+0000","last_change":"2024-09-21T03:43:04.978296+0000","last_active":"2024-09-21T03:45:12.651417+0000","last_peered":"2024-09-21T03:45:12.651417+0000","last_clean":"2024-09-21T03:45:12.651417+0000","last_became_active":"2024-09-21T03:43:04.977782+0000","last_became_peered":"2024-09-21T03:43:04.977782+0000","last_unstale":"2024-09-21T03:45:12.651417+0000","last_undegraded":"2024-09-21T03:45:12.651417+0000","last_fullsized":"2024-09-21T03:45:12.651417+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:57.748737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":134,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671480+0000","last_change":"2024-09-21T03:43:54.495536+0000","last_active":"2024-09-21T03:44:54.671480+0000","last_peered":"2024-09-21T03:44:54.671480+0000","last_clean":"2024-09-21T03:44:54.671480+0000","last_became_active":"2024-09-21T03:43:54.493492+0000","last_became_peered":"2024-09-21T03:43:54.493492+0000","last_unstale":"2024-09-21T03:44:54.671480+0000","last_undegraded":"2024-09-21T03:44:54.671480+0000","last_fullsized":"2024-09-21T03:44:54.671480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:11.291911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679520+0000","last_change":"2024-09-21T03:43:18.077741+0000","last_active":"2024-09-21T03:44:54.679520+0000","last_peered":"2024-09-21T03:44:54.679520+0000","last_clean":"2024-09-21T03:44:54.679520+0000","last_became_active":"2024-09-21T03:43:18.076619+0000","last_became_peered":"2024-09-21T03:43:18.076619+0000","last_unstale":"2024-09-21T03:44:54.679520+0000","last_undegraded":"2024-09-21T03:44:54.679520+0000","last_fullsized":"2024-09-21T03:44:54.679520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:33.297020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.669875+0000","last_change":"2024-09-21T03:43:08.008838+0000","last_active":"2024-09-21T03:44:54.669875+0000","last_peered":"2024-09-21T03:44:54.669875+0000","last_clean":"2024-09-21T03:44:54.669875+0000","last_became_active":"2024-09-21T03:43:08.008422+0000","last_became_peered":"2024-09-21T03:43:08.008422+0000","last_unstale":"2024-09-21T03:44:54.669875+0000","last_undegraded":"2024-09-21T03:44:54.669875+0000","last_fullsized":"2024-09-21T03:44:54.669875+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:00:17.881443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766577+0000","last_change":"2024-09-21T03:43:04.979289+0000","last_active":"2024-09-21T03:44:54.766577+0000","last_peered":"2024-09-21T03:44:54.766577+0000","last_clean":"2024-09-21T03:44:54.766577+0000","last_became_active":"2024-09-21T03:43:04.978843+0000","last_became_peered":"2024-09-21T03:43:04.978843+0000","last_unstale":"2024-09-21T03:44:54.766577+0000","last_undegraded":"2024-09-21T03:44:54.766577+0000","last_fullsized":"2024-09-21T03:44:54.766577+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:50.560607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677180+0000","last_change":"2024-09-21T03:43:18.096870+0000","last_active":"2024-09-21T03:44:54.677180+0000","last_peered":"2024-09-21T03:44:54.677180+0000","last_clean":"2024-09-21T03:44:54.677180+0000","last_became_active":"2024-09-21T03:43:18.096424+0000","last_became_peered":"2024-09-21T03:43:18.096424+0000","last_unstale":"2024-09-21T03:44:54.677180+0000","last_undegraded":"2024-09-21T03:44:54.677180+0000","last_fullsized":"2024-09-21T03:44:54.677180+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:39.532368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.669916+0000","last_change":"2024-09-21T03:43:54.492499+0000","last_active":"2024-09-21T03:44:54.669916+0000","last_peered":"2024-09-21T03:44:54.669916+0000","last_clean":"2024-09-21T03:44:54.669916+0000","last_became_active":"2024-09-21T03:43:54.492043+0000","last_became_peered":"2024-09-21T03:43:54.492043+0000","last_unstale":"2024-09-21T03:44:54.669916+0000","last_undegraded":"2024-09-21T03:44:54.669916+0000","last_fullsized":"2024-09-21T03:44:54.669916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:25.687090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766389+0000","last_change":"2024-09-21T03:43:04.974265+0000","last_active":"2024-09-21T03:44:54.766389+0000","last_peered":"2024-09-21T03:44:54.766389+0000","last_clean":"2024-09-21T03:44:54.766389+0000","last_became_active":"2024-09-21T03:43:04.973383+0000","last_became_peered":"2024-09-21T03:43:04.973383+0000","last_unstale":"2024-09-21T03:44:54.766389+0000","last_undegraded":"2024-09-21T03:44:54.766389+0000","last_fullsized":"2024-09-21T03:44:54.766389+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:02.312378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768029+0000","last_change":"2024-09-21T03:43:04.980353+0000","last_active":"2024-09-21T03:44:54.768029+0000","last_peered":"2024-09-21T03:44:54.768029+0000","last_clean":"2024-09-21T03:44:54.768029+0000","last_became_active":"2024-09-21T03:43:04.979922+0000","last_became_peered":"2024-09-21T03:43:04.979922+0000","last_unstale":"2024-09-21T03:44:54.768029+0000","last_undegraded":"2024-09-21T03:44:54.768029+0000","last_fullsized":"2024-09-21T03:44:54.768029+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:17.636536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766686+0000","last_change":"2024-09-21T03:43:08.009225+0000","last_active":"2024-09-21T03:44:54.766686+0000","last_peered":"2024-09-21T03:44:54.766686+0000","last_clean":"2024-09-21T03:44:54.766686+0000","last_became_active":"2024-09-21T03:43:08.008830+0000","last_became_peered":"2024-09-21T03:43:08.008830+0000","last_unstale":"2024-09-21T03:44:54.766686+0000","last_undegraded":"2024-09-21T03:44:54.766686+0000","last_fullsized":"2024-09-21T03:44:54.766686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:38.047763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679485+0000","last_change":"2024-09-21T03:43:18.089302+0000","last_active":"2024-09-21T03:44:54.679485+0000","last_peered":"2024-09-21T03:44:54.679485+0000","last_clean":"2024-09-21T03:44:54.679485+0000","last_became_active":"2024-09-21T03:43:18.088867+0000","last_became_peered":"2024-09-21T03:43:18.088867+0000","last_unstale":"2024-09-21T03:44:54.679485+0000","last_undegraded":"2024-09-21T03:44:54.679485+0000","last_fullsized":"2024-09-21T03:44:54.679485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:08.960044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":127,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:12.652123+0000","last_change":"2024-09-21T03:43:54.464317+0000","last_active":"2024-09-21T03:45:12.652123+0000","last_peered":"2024-09-21T03:45:12.652123+0000","last_clean":"2024-09-21T03:45:12.652123+0000","last_became_active":"2024-09-21T03:43:54.463870+0000","last_became_peered":"2024-09-21T03:43:54.463870+0000","last_unstale":"2024-09-21T03:45:12.652123+0000","last_undegraded":"2024-09-21T03:45:12.652123+0000","last_fullsized":"2024-09-21T03:45:12.652123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:14.581726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.674905+0000","last_change":"2024-09-21T03:43:08.009398+0000","last_active":"2024-09-21T03:44:54.674905+0000","last_peered":"2024-09-21T03:44:54.674905+0000","last_clean":"2024-09-21T03:44:54.674905+0000","last_became_active":"2024-09-21T03:43:08.009000+0000","last_became_peered":"2024-09-21T03:43:08.009000+0000","last_unstale":"2024-09-21T03:44:54.674905+0000","last_undegraded":"2024-09-21T03:44:54.674905+0000","last_fullsized":"2024-09-21T03:44:54.674905+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:07.278613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679818+0000","last_change":"2024-09-21T03:43:18.081834+0000","last_active":"2024-09-21T03:44:54.679818+0000","last_peered":"2024-09-21T03:44:54.679818+0000","last_clean":"2024-09-21T03:44:54.679818+0000","last_became_active":"2024-09-21T03:43:18.081328+0000","last_became_peered":"2024-09-21T03:43:18.081328+0000","last_unstale":"2024-09-21T03:44:54.679818+0000","last_undegraded":"2024-09-21T03:44:54.679818+0000","last_fullsized":"2024-09-21T03:44:54.679818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:37.638826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678021+0000","last_change":"2024-09-21T03:43:08.005144+0000","last_active":"2024-09-21T03:44:54.678021+0000","last_peered":"2024-09-21T03:44:54.678021+0000","last_clean":"2024-09-21T03:44:54.678021+0000","last_became_active":"2024-09-21T03:43:08.003895+0000","last_became_peered":"2024-09-21T03:43:08.003895+0000","last_unstale":"2024-09-21T03:44:54.678021+0000","last_undegraded":"2024-09-21T03:44:54.678021+0000","last_fullsized":"2024-09-21T03:44:54.678021+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:25.822315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679102+0000","last_change":"2024-09-21T03:43:08.000218+0000","last_active":"2024-09-21T03:44:54.679102+0000","last_peered":"2024-09-21T03:44:54.679102+0000","last_clean":"2024-09-21T03:44:54.679102+0000","last_became_active":"2024-09-21T03:43:07.999840+0000","last_became_peered":"2024-09-21T03:43:07.999840+0000","last_unstale":"2024-09-21T03:44:54.679102+0000","last_undegraded":"2024-09-21T03:44:54.679102+0000","last_fullsized":"2024-09-21T03:44:54.679102+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:55.391706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679062+0000","last_change":"2024-09-21T03:43:08.012075+0000","last_active":"2024-09-21T03:44:54.679062+0000","last_peered":"2024-09-21T03:44:54.679062+0000","last_clean":"2024-09-21T03:44:54.679062+0000","last_became_active":"2024-09-21T03:43:08.011666+0000","last_became_peered":"2024-09-21T03:43:08.011666+0000","last_unstale":"2024-09-21T03:44:54.679062+0000","last_undegraded":"2024-09-21T03:44:54.679062+0000","last_fullsized":"2024-09-21T03:44:54.679062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:42.124948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679990+0000","last_change":"2024-09-21T03:43:08.021082+0000","last_active":"2024-09-21T03:44:54.679990+0000","last_peered":"2024-09-21T03:44:54.679990+0000","last_clean":"2024-09-21T03:44:54.679990+0000","last_became_active":"2024-09-21T03:43:08.020674+0000","last_became_peered":"2024-09-21T03:43:08.020674+0000","last_unstale":"2024-09-21T03:44:54.679990+0000","last_undegraded":"2024-09-21T03:44:54.679990+0000","last_fullsized":"2024-09-21T03:44:54.679990+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:13.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766965+0000","last_change":"2024-09-21T03:43:08.004199+0000","last_active":"2024-09-21T03:44:54.766965+0000","last_peered":"2024-09-21T03:44:54.766965+0000","last_clean":"2024-09-21T03:44:54.766965+0000","last_became_active":"2024-09-21T03:43:08.003806+0000","last_became_peered":"2024-09-21T03:43:08.003806+0000","last_unstale":"2024-09-21T03:44:54.766965+0000","last_undegraded":"2024-09-21T03:44:54.766965+0000","last_fullsized":"2024-09-21T03:44:54.766965+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:13.941130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678414+0000","last_change":"2024-09-21T03:43:08.007237+0000","last_active":"2024-09-21T03:44:54.678414+0000","last_peered":"2024-09-21T03:44:54.678414+0000","last_clean":"2024-09-21T03:44:54.678414+0000","last_became_active":"2024-09-21T03:43:08.006805+0000","last_became_peered":"2024-09-21T03:43:08.006805+0000","last_unstale":"2024-09-21T03:44:54.678414+0000","last_undegraded":"2024-09-21T03:44:54.678414+0000","last_fullsized":"2024-09-21T03:44:54.678414+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:26.274494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.680363+0000","last_change":"2024-09-21T03:43:08.007934+0000","last_active":"2024-09-21T03:44:54.680363+0000","last_peered":"2024-09-21T03:44:54.680363+0000","last_clean":"2024-09-21T03:44:54.680363+0000","last_became_active":"2024-09-21T03:43:08.007549+0000","last_became_peered":"2024-09-21T03:43:08.007549+0000","last_unstale":"2024-09-21T03:44:54.680363+0000","last_undegraded":"2024-09-21T03:44:54.680363+0000","last_fullsized":"2024-09-21T03:44:54.680363+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:58.158255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.672360+0000","last_change":"2024-09-21T03:43:54.493665+0000","last_active":"2024-09-21T03:44:54.672360+0000","last_peered":"2024-09-21T03:44:54.672360+0000","last_clean":"2024-09-21T03:44:54.672360+0000","last_became_active":"2024-09-21T03:43:54.493245+0000","last_became_peered":"2024-09-21T03:43:54.493245+0000","last_unstale":"2024-09-21T03:44:54.672360+0000","last_undegraded":"2024-09-21T03:44:54.672360+0000","last_fullsized":"2024-09-21T03:44:54.672360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:35.449852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766343+0000","last_change":"2024-09-21T03:43:54.463419+0000","last_active":"2024-09-21T03:44:54.766343+0000","last_peered":"2024-09-21T03:44:54.766343+0000","last_clean":"2024-09-21T03:44:54.766343+0000","last_became_active":"2024-09-21T03:43:54.462951+0000","last_became_peered":"2024-09-21T03:43:54.462951+0000","last_unstale":"2024-09-21T03:44:54.766343+0000","last_undegraded":"2024-09-21T03:44:54.766343+0000","last_fullsized":"2024-09-21T03:44:54.766343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:23.620275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671805+0000","last_change":"2024-09-21T03:43:08.011436+0000","last_active":"2024-09-21T03:44:54.671805+0000","last_peered":"2024-09-21T03:44:54.671805+0000","last_clean":"2024-09-21T03:44:54.671805+0000","last_became_active":"2024-09-21T03:43:08.011025+0000","last_became_peered":"2024-09-21T03:43:08.011025+0000","last_unstale":"2024-09-21T03:44:54.671805+0000","last_undegraded":"2024-09-21T03:44:54.671805+0000","last_fullsized":"2024-09-21T03:44:54.671805+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:31.411858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768311+0000","last_change":"2024-09-21T03:43:08.016378+0000","last_active":"2024-09-21T03:44:54.768311+0000","last_peered":"2024-09-21T03:44:54.768311+0000","last_clean":"2024-09-21T03:44:54.768311+0000","last_became_active":"2024-09-21T03:43:08.016013+0000","last_became_peered":"2024-09-21T03:43:08.016013+0000","last_unstale":"2024-09-21T03:44:54.768311+0000","last_undegraded":"2024-09-21T03:44:54.768311+0000","last_fullsized":"2024-09-21T03:44:54.768311+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:01.865669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766706+0000","last_change":"2024-09-21T03:43:08.007862+0000","last_active":"2024-09-21T03:44:54.766706+0000","last_peered":"2024-09-21T03:44:54.766706+0000","last_clean":"2024-09-21T03:44:54.766706+0000","last_became_active":"2024-09-21T03:43:08.007433+0000","last_became_peered":"2024-09-21T03:43:08.007433+0000","last_unstale":"2024-09-21T03:44:54.766706+0000","last_undegraded":"2024-09-21T03:44:54.766706+0000","last_fullsized":"2024-09-21T03:44:54.766706+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:22:03.364767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677476+0000","last_change":"2024-09-21T03:43:08.010542+0000","last_active":"2024-09-21T03:44:54.677476+0000","last_peered":"2024-09-21T03:44:54.677476+0000","last_clean":"2024-09-21T03:44:54.677476+0000","last_became_active":"2024-09-21T03:43:08.010195+0000","last_became_peered":"2024-09-21T03:43:08.010195+0000","last_unstale":"2024-09-21T03:44:54.677476+0000","last_undegraded":"2024-09-21T03:44:54.677476+0000","last_fullsized":"2024-09-21T03:44:54.677476+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:52.850657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.680060+0000","last_change":"2024-09-21T03:43:08.006168+0000","last_active":"2024-09-21T03:44:54.680060+0000","last_peered":"2024-09-21T03:44:54.680060+0000","last_clean":"2024-09-21T03:44:54.680060+0000","last_became_active":"2024-09-21T03:43:08.005283+0000","last_became_peered":"2024-09-21T03:43:08.005283+0000","last_unstale":"2024-09-21T03:44:54.680060+0000","last_undegraded":"2024-09-21T03:44:54.680060+0000","last_fullsized":"2024-09-21T03:44:54.680060+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:59.852412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670656+0000","last_change":"2024-09-21T03:43:08.005870+0000","last_active":"2024-09-21T03:44:54.670656+0000","last_peered":"2024-09-21T03:44:54.670656+0000","last_clean":"2024-09-21T03:44:54.670656+0000","last_became_active":"2024-09-21T03:43:08.005485+0000","last_became_peered":"2024-09-21T03:43:08.005485+0000","last_unstale":"2024-09-21T03:44:54.670656+0000","last_undegraded":"2024-09-21T03:44:54.670656+0000","last_fullsized":"2024-09-21T03:44:54.670656+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:41.387865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677705+0000","last_change":"2024-09-21T03:43:08.015176+0000","last_active":"2024-09-21T03:44:54.677705+0000","last_peered":"2024-09-21T03:44:54.677705+0000","last_clean":"2024-09-21T03:44:54.677705+0000","last_became_active":"2024-09-21T03:43:08.014451+0000","last_became_peered":"2024-09-21T03:43:08.014451+0000","last_unstale":"2024-09-21T03:44:54.677705+0000","last_undegraded":"2024-09-21T03:44:54.677705+0000","last_fullsized":"2024-09-21T03:44:54.677705+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:49.524800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767481+0000","last_change":"2024-09-21T03:43:08.016798+0000","last_active":"2024-09-21T03:44:54.767481+0000","last_peered":"2024-09-21T03:44:54.767481+0000","last_clean":"2024-09-21T03:44:54.767481+0000","last_became_active":"2024-09-21T03:43:08.014842+0000","last_became_peered":"2024-09-21T03:43:08.014842+0000","last_unstale":"2024-09-21T03:44:54.767481+0000","last_undegraded":"2024-09-21T03:44:54.767481+0000","last_fullsized":"2024-09-21T03:44:54.767481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:19.290618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679434+0000","last_change":"2024-09-21T03:43:08.021285+0000","last_active":"2024-09-21T03:44:54.679434+0000","last_peered":"2024-09-21T03:44:54.679434+0000","last_clean":"2024-09-21T03:44:54.679434+0000","last_became_active":"2024-09-21T03:43:08.020904+0000","last_became_peered":"2024-09-21T03:43:08.020904+0000","last_unstale":"2024-09-21T03:44:54.679434+0000","last_undegraded":"2024-09-21T03:44:54.679434+0000","last_fullsized":"2024-09-21T03:44:54.679434+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:56.589089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679378+0000","last_change":"2024-09-21T03:43:08.017599+0000","last_active":"2024-09-21T03:44:54.679378+0000","last_peered":"2024-09-21T03:44:54.679378+0000","last_clean":"2024-09-21T03:44:54.679378+0000","last_became_active":"2024-09-21T03:43:08.016138+0000","last_became_peered":"2024-09-21T03:43:08.016138+0000","last_unstale":"2024-09-21T03:44:54.679378+0000","last_undegraded":"2024-09-21T03:44:54.679378+0000","last_fullsized":"2024-09-21T03:44:54.679378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:09.902246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766674+0000","last_change":"2024-09-21T03:43:08.009598+0000","last_active":"2024-09-21T03:44:54.766674+0000","last_peered":"2024-09-21T03:44:54.766674+0000","last_clean":"2024-09-21T03:44:54.766674+0000","last_became_active":"2024-09-21T03:43:08.007488+0000","last_became_peered":"2024-09-21T03:43:08.007488+0000","last_unstale":"2024-09-21T03:44:54.766674+0000","last_undegraded":"2024-09-21T03:44:54.766674+0000","last_fullsized":"2024-09-21T03:44:54.766674+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:23.192761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768416+0000","last_change":"2024-09-21T03:43:08.016450+0000","last_active":"2024-09-21T03:44:54.768416+0000","last_peered":"2024-09-21T03:44:54.768416+0000","last_clean":"2024-09-21T03:44:54.768416+0000","last_became_active":"2024-09-21T03:43:08.016037+0000","last_became_peered":"2024-09-21T03:43:08.016037+0000","last_unstale":"2024-09-21T03:44:54.768416+0000","last_undegraded":"2024-09-21T03:44:54.768416+0000","last_fullsized":"2024-09-21T03:44:54.768416+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:39.109408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678614+0000","last_change":"2024-09-21T03:43:08.010298+0000","last_active":"2024-09-21T03:44:54.678614+0000","last_peered":"2024-09-21T03:44:54.678614+0000","last_clean":"2024-09-21T03:44:54.678614+0000","last_became_active":"2024-09-21T03:43:08.009847+0000","last_became_peered":"2024-09-21T03:43:08.009847+0000","last_unstale":"2024-09-21T03:44:54.678614+0000","last_undegraded":"2024-09-21T03:44:54.678614+0000","last_fullsized":"2024-09-21T03:44:54.678614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:37.463456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768895+0000","last_change":"2024-09-21T03:43:08.017834+0000","last_active":"2024-09-21T03:44:54.768895+0000","last_peered":"2024-09-21T03:44:54.768895+0000","last_clean":"2024-09-21T03:44:54.768895+0000","last_became_active":"2024-09-21T03:43:08.016910+0000","last_became_peered":"2024-09-21T03:43:08.016910+0000","last_unstale":"2024-09-21T03:44:54.768895+0000","last_undegraded":"2024-09-21T03:44:54.768895+0000","last_fullsized":"2024-09-21T03:44:54.768895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:50.528651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768591+0000","last_change":"2024-09-21T03:43:08.013945+0000","last_active":"2024-09-21T03:44:54.768591+0000","last_peered":"2024-09-21T03:44:54.768591+0000","last_clean":"2024-09-21T03:44:54.768591+0000","last_became_active":"2024-09-21T03:43:08.013598+0000","last_became_peered":"2024-09-21T03:43:08.013598+0000","last_unstale":"2024-09-21T03:44:54.768591+0000","last_undegraded":"2024-09-21T03:44:54.768591+0000","last_fullsized":"2024-09-21T03:44:54.768591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:58.182938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671440+0000","last_change":"2024-09-21T03:43:08.018824+0000","last_active":"2024-09-21T03:44:54.671440+0000","last_peered":"2024-09-21T03:44:54.671440+0000","last_clean":"2024-09-21T03:44:54.671440+0000","last_became_active":"2024-09-21T03:43:08.017196+0000","last_became_peered":"2024-09-21T03:43:08.017196+0000","last_unstale":"2024-09-21T03:44:54.671440+0000","last_undegraded":"2024-09-21T03:44:54.671440+0000","last_fullsized":"2024-09-21T03:44:54.671440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:02.357523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677945+0000","last_change":"2024-09-21T03:43:08.003514+0000","last_active":"2024-09-21T03:44:54.677945+0000","last_peered":"2024-09-21T03:44:54.677945+0000","last_clean":"2024-09-21T03:44:54.677945+0000","last_became_active":"2024-09-21T03:43:08.003131+0000","last_became_peered":"2024-09-21T03:43:08.003131+0000","last_unstale":"2024-09-21T03:44:54.677945+0000","last_undegraded":"2024-09-21T03:44:54.677945+0000","last_fullsized":"2024-09-21T03:44:54.677945+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:23.170302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767907+0000","last_change":"2024-09-21T03:43:08.015890+0000","last_active":"2024-09-21T03:44:54.767907+0000","last_peered":"2024-09-21T03:44:54.767907+0000","last_clean":"2024-09-21T03:44:54.767907+0000","last_became_active":"2024-09-21T03:43:08.015519+0000","last_became_peered":"2024-09-21T03:43:08.015519+0000","last_unstale":"2024-09-21T03:44:54.767907+0000","last_undegraded":"2024-09-21T03:44:54.767907+0000","last_fullsized":"2024-09-21T03:44:54.767907+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:10.875923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675017+0000","last_change":"2024-09-21T03:43:08.012261+0000","last_active":"2024-09-21T03:44:54.675017+0000","last_peered":"2024-09-21T03:44:54.675017+0000","last_clean":"2024-09-21T03:44:54.675017+0000","last_became_active":"2024-09-21T03:43:08.011783+0000","last_became_peered":"2024-09-21T03:43:08.011783+0000","last_unstale":"2024-09-21T03:44:54.675017+0000","last_undegraded":"2024-09-21T03:44:54.675017+0000","last_fullsized":"2024-09-21T03:44:54.675017+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:07.011340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670137+0000","last_change":"2024-09-21T03:43:54.494826+0000","last_active":"2024-09-21T03:44:54.670137+0000","last_peered":"2024-09-21T03:44:54.670137+0000","last_clean":"2024-09-21T03:44:54.670137+0000","last_became_active":"2024-09-21T03:43:54.494404+0000","last_became_peered":"2024-09-21T03:43:54.494404+0000","last_unstale":"2024-09-21T03:44:54.670137+0000","last_undegraded":"2024-09-21T03:44:54.670137+0000","last_fullsized":"2024-09-21T03:44:54.670137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:42.109053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679690+0000","last_change":"2024-09-21T03:43:08.016214+0000","last_active":"2024-09-21T03:44:54.679690+0000","last_peered":"2024-09-21T03:44:54.679690+0000","last_clean":"2024-09-21T03:44:54.679690+0000","last_became_active":"2024-09-21T03:43:08.014815+0000","last_became_peered":"2024-09-21T03:43:08.014815+0000","last_unstale":"2024-09-21T03:44:54.679690+0000","last_undegraded":"2024-09-21T03:44:54.679690+0000","last_fullsized":"2024-09-21T03:44:54.679690+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:04.742829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679723+0000","last_change":"2024-09-21T03:43:08.019269+0000","last_active":"2024-09-21T03:44:54.679723+0000","last_peered":"2024-09-21T03:44:54.679723+0000","last_clean":"2024-09-21T03:44:54.679723+0000","last_became_active":"2024-09-21T03:43:08.017962+0000","last_became_peered":"2024-09-21T03:43:08.017962+0000","last_unstale":"2024-09-21T03:44:54.679723+0000","last_undegraded":"2024-09-21T03:44:54.679723+0000","last_fullsized":"2024-09-21T03:44:54.679723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:05.950697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677702+0000","last_change":"2024-09-21T03:43:08.014790+0000","last_active":"2024-09-21T03:44:54.677702+0000","last_peered":"2024-09-21T03:44:54.677702+0000","last_clean":"2024-09-21T03:44:54.677702+0000","last_became_active":"2024-09-21T03:43:08.013891+0000","last_became_peered":"2024-09-21T03:43:08.013891+0000","last_unstale":"2024-09-21T03:44:54.677702+0000","last_undegraded":"2024-09-21T03:44:54.677702+0000","last_fullsized":"2024-09-21T03:44:54.677702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:46.045675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73880,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":707,"num_read_kb":430,"num_write":249,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":221640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808873,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1007616,"data_stored":501132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972398,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1703936,"data_stored":1158608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135922,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1024000,"data_stored":500559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332153,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1011712,"data_stored":500825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495679,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36932,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708220,"statfs":{"total":95995035648,"available":95957217280,"internally_reserved":0,"allocated":1617920,"data_stored":1097043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659204,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":999424,"data_stored":496063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855434,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1626112,"data_stored":1092625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018960,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1015808,"data_stored":500759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215188,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1011712,"data_stored":495771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444123,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1085440,"data_stored":564149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1073152,"data_stored":564160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1024000,"data_stored":497910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:45:16.341 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T03:45:16.341 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T03:45:16.341 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T03:45:16.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T03:45:16.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:16 smithi016 ceph-mon[30375]: pgmap v387: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T03:45:16.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:16 smithi016 ceph-mon[30375]: from='client.25120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:45:16.722 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:16.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:16 smithi134 ceph-mon[35031]: pgmap v387: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T03:45:16.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:16 smithi134 ceph-mon[35031]: from='client.25120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:45:16.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:16 smithi123 ceph-mon[34673]: pgmap v387: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T03:45:16.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:16 smithi123 ceph-mon[34673]: from='client.25120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:45:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.369+0000 7f5ff3000640 1 -- 172.21.15.16:0/1057294756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fec109c20 msgr2=0x7f5fec110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.369+0000 7f5ff3000640 1 --2- 172.21.15.16:0/1057294756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fec109c20 0x7f5fec110450 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009ca0 tx=0x7f5fe402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.369+0000 7f5ff3000640 1 -- 172.21.15.16:0/1057294756 shutdown_connections 2024-09-21T03:45:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.369+0000 7f5ff3000640 1 --2- 172.21.15.16:0/1057294756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fec109c20 0x7f5fec110450 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.369+0000 7f5ff3000640 1 --2- 172.21.15.16:0/1057294756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fec1092d0 0x7f5fec1096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.369+0000 7f5ff3000640 1 --2- 172.21.15.16:0/1057294756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fec103990 0x7f5fec103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.369+0000 7f5ff3000640 1 -- 172.21.15.16:0/1057294756 >> 172.21.15.16:0/1057294756 conn(0x7f5fec0ff440 msgr2=0x7f5fec101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.369+0000 7f5ff3000640 1 -- 172.21.15.16:0/1057294756 shutdown_connections 2024-09-21T03:45:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.369+0000 7f5ff3000640 1 -- 172.21.15.16:0/1057294756 wait complete. 2024-09-21T03:45:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.370+0000 7f5ff3000640 1 Processor -- start 2024-09-21T03:45:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.370+0000 7f5ff3000640 1 -- start start 2024-09-21T03:45:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.371+0000 7f5ff3000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fec103990 0x7f5fec19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.371+0000 7f5ff3000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fec1092d0 0x7f5fec19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.371+0000 7f5ff3000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fec109c20 0x7f5fec1a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.371+0000 7f5ff3000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1138e0 con 0x7f5fec103990 2024-09-21T03:45:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.371+0000 7f5ff3000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec113760 con 0x7f5fec1092d0 2024-09-21T03:45:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.371+0000 7f5ff3000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec113a60 con 0x7f5fec109c20 2024-09-21T03:45:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.371+0000 7f5ff1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fec103990 0x7f5fec19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.371+0000 7f5ff1200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fec1092d0 0x7f5fec19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.371+0000 7f5ff1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fec103990 0x7f5fec19c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46190/0 (socket says 172.21.15.16:46190) 2024-09-21T03:45:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.371+0000 7f5ff1c00640 1 -- 172.21.15.16:0/395189674 learned_addr learned my addr 172.21.15.16:0/395189674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:17.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.371+0000 7f5ff2600640 1 --2- 172.21.15.16:0/395189674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fec109c20 0x7f5fec1a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.371+0000 7f5ff1200640 1 -- 172.21.15.16:0/395189674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fec109c20 msgr2=0x7f5fec1a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.372+0000 7f5ff1200640 1 --2- 172.21.15.16:0/395189674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fec109c20 0x7f5fec1a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.372+0000 7f5ff1200640 1 -- 172.21.15.16:0/395189674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fec103990 msgr2=0x7f5fec19c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.372+0000 7f5ff1200640 1 --2- 172.21.15.16:0/395189674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fec103990 0x7f5fec19c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.372+0000 7f5ff1200640 1 -- 172.21.15.16:0/395189674 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fec1a2420 con 0x7f5fec1092d0 2024-09-21T03:45:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.372+0000 7f5ff1c00640 1 --2- 172.21.15.16:0/395189674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fec103990 0x7f5fec19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.372+0000 7f5ff1200640 1 --2- 172.21.15.16:0/395189674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fec1092d0 0x7f5fec19ca50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5fec06ca20 tx=0x7f5fd800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.372+0000 7f5fea000640 1 -- 172.21.15.16:0/395189674 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd800cd00 con 0x7f5fec1092d0 2024-09-21T03:45:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.372+0000 7f5ff3000640 1 -- 172.21.15.16:0/395189674 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fec1a2710 con 0x7f5fec1092d0 2024-09-21T03:45:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.372+0000 7f5fea000640 1 -- 172.21.15.16:0/395189674 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fd80035f0 con 0x7f5fec1092d0 2024-09-21T03:45:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.372+0000 7f5fea000640 1 -- 172.21.15.16:0/395189674 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8010690 con 0x7f5fec1092d0 2024-09-21T03:45:17.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.372+0000 7f5ff3000640 1 -- 172.21.15.16:0/395189674 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fec1a8e70 con 0x7f5fec1092d0 2024-09-21T03:45:17.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.373+0000 7f5ff3000640 1 -- 172.21.15.16:0/395189674 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fec103da0 con 0x7f5fec1092d0 2024-09-21T03:45:17.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.376+0000 7f5fea000640 1 -- 172.21.15.16:0/395189674 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f5fd80037a0 con 0x7f5fec1092d0 2024-09-21T03:45:17.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.378+0000 7f5fea000640 1 --2- 172.21.15.16:0/395189674 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5fd40b6620 0x7f5fd40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:17.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.378+0000 7f5fea000640 1 -- 172.21.15.16:0/395189674 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f5fd80e7c40 con 0x7f5fec1092d0 2024-09-21T03:45:17.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.379+0000 7f5ff1c00640 1 --2- 172.21.15.16:0/395189674 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5fd40b6620 0x7f5fd40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:17.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.379+0000 7f5ff1c00640 1 --2- 172.21.15.16:0/395189674 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5fd40b6620 0x7f5fd40b8b30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0011500 tx=0x7f5fe0005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:17.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.379+0000 7f5fea000640 1 -- 172.21.15.16:0/395189674 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5fd8056710 con 0x7f5fec1092d0 2024-09-21T03:45:17.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.622+0000 7f5ff3000640 1 -- 172.21.15.16:0/395189674 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5fec19d5e0 con 0x7f5fec1092d0 2024-09-21T03:45:17.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.622+0000 7f5fea000640 1 -- 172.21.15.16:0/395189674 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5fec19d5e0 con 0x7f5fec1092d0 2024-09-21T03:45:17.624 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:45:17.624 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T03:45:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.625+0000 7f5ff3000640 1 -- 172.21.15.16:0/395189674 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5fd40b6620 msgr2=0x7f5fd40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.625+0000 7f5ff3000640 1 --2- 172.21.15.16:0/395189674 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5fd40b6620 0x7f5fd40b8b30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0011500 tx=0x7f5fe0005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.625+0000 7f5ff3000640 1 -- 172.21.15.16:0/395189674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fec1092d0 msgr2=0x7f5fec19ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.625+0000 7f5ff3000640 1 --2- 172.21.15.16:0/395189674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fec1092d0 0x7f5fec19ca50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5fec06ca20 tx=0x7f5fd800ee50 comp rx=0 tx=0).stop 2024-09-21T03:45:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.625+0000 7f5ff3000640 1 -- 172.21.15.16:0/395189674 shutdown_connections 2024-09-21T03:45:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.625+0000 7f5ff3000640 1 --2- 172.21.15.16:0/395189674 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5fd40b6620 0x7f5fd40b8b30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.625+0000 7f5ff3000640 1 --2- 172.21.15.16:0/395189674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fec109c20 0x7f5fec1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.625+0000 7f5ff3000640 1 --2- 172.21.15.16:0/395189674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fec1092d0 0x7f5fec19ca50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.625+0000 7f5ff3000640 1 --2- 172.21.15.16:0/395189674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fec103990 0x7f5fec19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.625+0000 7f5ff3000640 1 -- 172.21.15.16:0/395189674 >> 172.21.15.16:0/395189674 conn(0x7f5fec0ff440 msgr2=0x7f5fec106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.626+0000 7f5ff3000640 1 -- 172.21.15.16:0/395189674 shutdown_connections 2024-09-21T03:45:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:17.626+0000 7f5ff3000640 1 -- 172.21.15.16:0/395189674 wait complete. 2024-09-21T03:45:18.377 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T03:45:18.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:45:18.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:18 smithi016 ceph-mon[30375]: pgmap v388: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:45:18.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:18 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/395189674' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:45:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:18.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:18 smithi134 ceph-mon[35031]: pgmap v388: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:45:18.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:18 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/395189674' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:45:18.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:18 smithi123 ceph-mon[34673]: pgmap v388: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:45:18.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:18 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/395189674' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:45:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.388+0000 7f7f5be00640 1 -- 172.21.15.16:0/2810736285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f5c109c40 msgr2=0x7f7f5c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.388+0000 7f7f5be00640 1 --2- 172.21.15.16:0/2810736285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f5c109c40 0x7f7f5c110470 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009ca0 tx=0x7f7f5402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.388+0000 7f7f5be00640 1 -- 172.21.15.16:0/2810736285 shutdown_connections 2024-09-21T03:45:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.388+0000 7f7f5be00640 1 --2- 172.21.15.16:0/2810736285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f5c109c40 0x7f7f5c110470 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.388+0000 7f7f5be00640 1 --2- 172.21.15.16:0/2810736285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f5c1092f0 0x7f7f5c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.389+0000 7f7f5be00640 1 --2- 172.21.15.16:0/2810736285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f5c1039b0 0x7f7f5c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.389+0000 7f7f5be00640 1 -- 172.21.15.16:0/2810736285 >> 172.21.15.16:0/2810736285 conn(0x7f7f5c0ff440 msgr2=0x7f7f5c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.389+0000 7f7f5be00640 1 -- 172.21.15.16:0/2810736285 shutdown_connections 2024-09-21T03:45:19.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.389+0000 7f7f5be00640 1 -- 172.21.15.16:0/2810736285 wait complete. 2024-09-21T03:45:19.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.390+0000 7f7f5be00640 1 Processor -- start 2024-09-21T03:45:19.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.390+0000 7f7f5be00640 1 -- start start 2024-09-21T03:45:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.390+0000 7f7f5be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f5c1039b0 0x7f7f5c19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.390+0000 7f7f5be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f5c1092f0 0x7f7f5c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.390+0000 7f7f5be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f5c109c40 0x7f7f5c1a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.390+0000 7f7f5be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c113950 con 0x7f7f5c1092f0 2024-09-21T03:45:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.390+0000 7f7f5be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1137d0 con 0x7f7f5c109c40 2024-09-21T03:45:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.390+0000 7f7f5be00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c113ad0 con 0x7f7f5c1039b0 2024-09-21T03:45:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.391+0000 7f7f53e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f5c1092f0 0x7f7f5c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.391+0000 7f7f5aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f5c1039b0 0x7f7f5c19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.391+0000 7f7f53e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f5c1092f0 0x7f7f5c19ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46212/0 (socket says 172.21.15.16:46212) 2024-09-21T03:45:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.391+0000 7f7f53e00640 1 -- 172.21.15.16:0/3552413038 learned_addr learned my addr 172.21.15.16:0/3552413038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.391+0000 7f7f5b400640 1 --2- 172.21.15.16:0/3552413038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f5c109c40 0x7f7f5c1a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.391+0000 7f7f53e00640 1 -- 172.21.15.16:0/3552413038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f5c1039b0 msgr2=0x7f7f5c19c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.391+0000 7f7f53e00640 1 --2- 172.21.15.16:0/3552413038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f5c1039b0 0x7f7f5c19c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.391+0000 7f7f53e00640 1 -- 172.21.15.16:0/3552413038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f5c109c40 msgr2=0x7f7f5c1a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.391+0000 7f7f53e00640 1 --2- 172.21.15.16:0/3552413038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f5c109c40 0x7f7f5c1a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.391+0000 7f7f53e00640 1 -- 172.21.15.16:0/3552413038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5c1a2450 con 0x7f7f5c1092f0 2024-09-21T03:45:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.392+0000 7f7f53e00640 1 --2- 172.21.15.16:0/3552413038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f5c1092f0 0x7f7f5c19ca80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400e9f0 tx=0x7f7f4400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.392+0000 7f7f60a00640 1 -- 172.21.15.16:0/3552413038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4400ccc0 con 0x7f7f5c1092f0 2024-09-21T03:45:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.392+0000 7f7f60a00640 1 -- 172.21.15.16:0/3552413038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f440108e0 con 0x7f7f5c1092f0 2024-09-21T03:45:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.392+0000 7f7f5be00640 1 -- 172.21.15.16:0/3552413038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5c1a2740 con 0x7f7f5c1092f0 2024-09-21T03:45:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.392+0000 7f7f60a00640 1 -- 172.21.15.16:0/3552413038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44018860 con 0x7f7f5c1092f0 2024-09-21T03:45:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.392+0000 7f7f5be00640 1 -- 172.21.15.16:0/3552413038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f5c1a8ea0 con 0x7f7f5c1092f0 2024-09-21T03:45:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.393+0000 7f7f5be00640 1 -- 172.21.15.16:0/3552413038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f5c103dc0 con 0x7f7f5c1092f0 2024-09-21T03:45:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.395+0000 7f7f60a00640 1 -- 172.21.15.16:0/3552413038 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f7f44010430 con 0x7f7f5c1092f0 2024-09-21T03:45:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.398+0000 7f7f60a00640 1 --2- 172.21.15.16:0/3552413038 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7f400b65a0 0x7f7f400b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.398+0000 7f7f60a00640 1 -- 172.21.15.16:0/3552413038 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f7f440e7b30 con 0x7f7f5c1092f0 2024-09-21T03:45:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.398+0000 7f7f5aa00640 1 --2- 172.21.15.16:0/3552413038 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7f400b65a0 0x7f7f400b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.399+0000 7f7f5aa00640 1 --2- 172.21.15.16:0/3552413038 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7f400b65a0 0x7f7f400b8ab0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00bbe0 tx=0x7f7f4c00b390 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.402+0000 7f7f60a00640 1 -- 172.21.15.16:0/3552413038 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7f440b2230 con 0x7f7f5c1092f0 2024-09-21T03:45:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.641+0000 7f7f5be00640 1 -- 172.21.15.16:0/3552413038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f5c1a9180 con 0x7f7f5c1092f0 2024-09-21T03:45:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.642+0000 7f7f60a00640 1 -- 172.21.15.16:0/3552413038 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11971 (secure 0 0 0) 0x7f7f440b7170 con 0x7f7f5c1092f0 2024-09-21T03:45:19.644 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:45:19.645 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"btime":"2024-09-21T03:44:08:542131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:44:07.645054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24668,24670,24682,24689,24731]},"id":1}]} 2024-09-21T03:45:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-09-21T03:45:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.645+0000 7f7f5be00640 1 -- 172.21.15.16:0/3552413038 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7f400b65a0 msgr2=0x7f7f400b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.645+0000 7f7f5be00640 1 --2- 172.21.15.16:0/3552413038 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7f400b65a0 0x7f7f400b8ab0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00bbe0 tx=0x7f7f4c00b390 comp rx=0 tx=0).stop 2024-09-21T03:45:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.645+0000 7f7f5be00640 1 -- 172.21.15.16:0/3552413038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f5c1092f0 msgr2=0x7f7f5c19ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.645+0000 7f7f5be00640 1 --2- 172.21.15.16:0/3552413038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f5c1092f0 0x7f7f5c19ca80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400e9f0 tx=0x7f7f4400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.645+0000 7f7f5be00640 1 -- 172.21.15.16:0/3552413038 shutdown_connections 2024-09-21T03:45:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.645+0000 7f7f5be00640 1 --2- 172.21.15.16:0/3552413038 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7f400b65a0 0x7f7f400b8ab0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.645+0000 7f7f5be00640 1 --2- 172.21.15.16:0/3552413038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f5c109c40 0x7f7f5c1a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.645+0000 7f7f5be00640 1 --2- 172.21.15.16:0/3552413038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f5c1092f0 0x7f7f5c19ca80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.645+0000 7f7f5be00640 1 --2- 172.21.15.16:0/3552413038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f5c1039b0 0x7f7f5c19c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.645+0000 7f7f5be00640 1 -- 172.21.15.16:0/3552413038 >> 172.21.15.16:0/3552413038 conn(0x7f7f5c0ff440 msgr2=0x7f7f5c108900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.645+0000 7f7f5be00640 1 -- 172.21.15.16:0/3552413038 shutdown_connections 2024-09-21T03:45:19.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:19.645+0000 7f7f5be00640 1 -- 172.21.15.16:0/3552413038 wait complete. 2024-09-21T03:45:20.456 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:45:20.774 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:20 smithi016 ceph-mon[30375]: pgmap v389: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:45:20.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:20 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3552413038' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:20.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:20 smithi134 ceph-mon[35031]: pgmap v389: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:45:20.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:20 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3552413038' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:20.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:20 smithi123 ceph-mon[34673]: pgmap v389: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:45:20.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:20 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3552413038' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.512+0000 7f045e800640 1 -- 172.21.15.16:0/3210317713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f045810be40 msgr2=0x7f0458112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.512+0000 7f045e800640 1 --2- 172.21.15.16:0/3210317713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f045810be40 0x7f0458112670 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0450009ca0 tx=0x7f045002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.512+0000 7f045e800640 1 -- 172.21.15.16:0/3210317713 shutdown_connections 2024-09-21T03:45:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.512+0000 7f045e800640 1 --2- 172.21.15.16:0/3210317713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f045810be40 0x7f0458112670 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.512+0000 7f045e800640 1 --2- 172.21.15.16:0/3210317713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045810b4f0 0x7f045810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.512+0000 7f045e800640 1 --2- 172.21.15.16:0/3210317713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0458105bb0 0x7f0458105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.512+0000 7f045e800640 1 -- 172.21.15.16:0/3210317713 >> 172.21.15.16:0/3210317713 conn(0x7f0458101680 msgr2=0x7f0458103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.512+0000 7f045e800640 1 -- 172.21.15.16:0/3210317713 shutdown_connections 2024-09-21T03:45:21.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.513+0000 7f045e800640 1 -- 172.21.15.16:0/3210317713 wait complete. 2024-09-21T03:45:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.514+0000 7f045e800640 1 Processor -- start 2024-09-21T03:45:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.514+0000 7f045e800640 1 -- start start 2024-09-21T03:45:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.514+0000 7f045e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0458105bb0 0x7f045819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.514+0000 7f045e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045810b4f0 0x7f045819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.514+0000 7f045e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045810be40 0x7f04581a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.514+0000 7f045e800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0458115b00 con 0x7f0458105bb0 2024-09-21T03:45:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.514+0000 7f045e800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0458115980 con 0x7f045810be40 2024-09-21T03:45:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.514+0000 7f045e800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0458115c80 con 0x7f045810b4f0 2024-09-21T03:45:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.514+0000 7f045d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0458105bb0 0x7f045819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.514+0000 7f045d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0458105bb0 0x7f045819c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46224/0 (socket says 172.21.15.16:46224) 2024-09-21T03:45:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.514+0000 7f045d400640 1 -- 172.21.15.16:0/2842697106 learned_addr learned my addr 172.21.15.16:0/2842697106 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.514+0000 7f045ca00640 1 --2- 172.21.15.16:0/2842697106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045810b4f0 0x7f045819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.514+0000 7f045de00640 1 --2- 172.21.15.16:0/2842697106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045810be40 0x7f04581a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.515+0000 7f045d400640 1 -- 172.21.15.16:0/2842697106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045810b4f0 msgr2=0x7f045819ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.515+0000 7f045d400640 1 --2- 172.21.15.16:0/2842697106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045810b4f0 0x7f045819ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.515+0000 7f045d400640 1 -- 172.21.15.16:0/2842697106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045810be40 msgr2=0x7f04581a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.515+0000 7f045d400640 1 --2- 172.21.15.16:0/2842697106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045810be40 0x7f04581a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.515+0000 7f045d400640 1 -- 172.21.15.16:0/2842697106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04581a2460 con 0x7f0458105bb0 2024-09-21T03:45:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.515+0000 7f045d400640 1 --2- 172.21.15.16:0/2842697106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0458105bb0 0x7f045819c4f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f044c00e9f0 tx=0x7f044c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.515+0000 7f0456000640 1 -- 172.21.15.16:0/2842697106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c00cd80 con 0x7f0458105bb0 2024-09-21T03:45:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.515+0000 7f0456000640 1 -- 172.21.15.16:0/2842697106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f044c0054d0 con 0x7f0458105bb0 2024-09-21T03:45:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.516+0000 7f045e800640 1 -- 172.21.15.16:0/2842697106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04581a2750 con 0x7f0458105bb0 2024-09-21T03:45:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.516+0000 7f0456000640 1 -- 172.21.15.16:0/2842697106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c010690 con 0x7f0458105bb0 2024-09-21T03:45:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.516+0000 7f045e800640 1 -- 172.21.15.16:0/2842697106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04581a8eb0 con 0x7f0458105bb0 2024-09-21T03:45:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.517+0000 7f045e800640 1 -- 172.21.15.16:0/2842697106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0458105fc0 con 0x7f0458105bb0 2024-09-21T03:45:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.519+0000 7f0456000640 1 -- 172.21.15.16:0/2842697106 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f044c004e30 con 0x7f0458105bb0 2024-09-21T03:45:21.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.521+0000 7f0456000640 1 --2- 172.21.15.16:0/2842697106 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04400b65a0 0x7f04400b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:21.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.521+0000 7f0456000640 1 -- 172.21.15.16:0/2842697106 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f044c0e7e60 con 0x7f0458105bb0 2024-09-21T03:45:21.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.521+0000 7f045ca00640 1 --2- 172.21.15.16:0/2842697106 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04400b65a0 0x7f04400b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:21.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.522+0000 7f045ca00640 1 --2- 172.21.15.16:0/2842697106 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04400b65a0 0x7f04400b8ab0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0458107e10 tx=0x7f0444005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:21.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.522+0000 7f0456000640 1 -- 172.21.15.16:0/2842697106 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f044c0ae180 con 0x7f0458105bb0 2024-09-21T03:45:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.763+0000 7f045e800640 1 -- 172.21.15.16:0/2842697106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f045819d620 con 0x7f0458105bb0 2024-09-21T03:45:21.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.764+0000 7f0456000640 1 -- 172.21.15.16:0/2842697106 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11971 (secure 0 0 0) 0x7f044c0b2560 con 0x7f0458105bb0 2024-09-21T03:45:21.767 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:45:21.767 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"btime":"2024-09-21T03:44:08:542131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:44:07.645054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24668,24670,24682,24689,24731]},"id":1}]} 2024-09-21T03:45:21.767 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-09-21T03:45:21.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.766+0000 7f045e800640 1 -- 172.21.15.16:0/2842697106 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04400b65a0 msgr2=0x7f04400b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:21.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.766+0000 7f045e800640 1 --2- 172.21.15.16:0/2842697106 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04400b65a0 0x7f04400b8ab0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0458107e10 tx=0x7f0444005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.766+0000 7f045e800640 1 -- 172.21.15.16:0/2842697106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0458105bb0 msgr2=0x7f045819c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.766+0000 7f045e800640 1 --2- 172.21.15.16:0/2842697106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0458105bb0 0x7f045819c4f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f044c00e9f0 tx=0x7f044c00edf0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.766+0000 7f045e800640 1 -- 172.21.15.16:0/2842697106 shutdown_connections 2024-09-21T03:45:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.766+0000 7f045e800640 1 --2- 172.21.15.16:0/2842697106 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04400b65a0 0x7f04400b8ab0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.766+0000 7f045e800640 1 --2- 172.21.15.16:0/2842697106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045810be40 0x7f04581a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.767+0000 7f045e800640 1 --2- 172.21.15.16:0/2842697106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045810b4f0 0x7f045819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.767+0000 7f045e800640 1 --2- 172.21.15.16:0/2842697106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0458105bb0 0x7f045819c4f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.767+0000 7f045e800640 1 -- 172.21.15.16:0/2842697106 >> 172.21.15.16:0/2842697106 conn(0x7f0458101680 msgr2=0x7f0458103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.767+0000 7f045e800640 1 -- 172.21.15.16:0/2842697106 shutdown_connections 2024-09-21T03:45:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:21.767+0000 7f045e800640 1 -- 172.21.15.16:0/2842697106 wait complete. 2024-09-21T03:45:22.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:45:22.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:22 smithi134 ceph-mon[35031]: pgmap v390: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:22.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:22 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2842697106' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:22.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:22 smithi123 ceph-mon[34673]: pgmap v390: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:22.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:22 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2842697106' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:22.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:22 smithi016 ceph-mon[30375]: pgmap v390: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:22.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:22 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2842697106' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.574+0000 7f9719000640 1 -- 172.21.15.16:0/1956659208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9714077940 msgr2=0x7f9714077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.574+0000 7f9719000640 1 --2- 172.21.15.16:0/1956659208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9714077940 0x7f9714077d50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9704009ca0 tx=0x7f970402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.574+0000 7f9719000640 1 -- 172.21.15.16:0/1956659208 shutdown_connections 2024-09-21T03:45:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.574+0000 7f9719000640 1 --2- 172.21.15.16:0/1956659208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9714114570 0x7f9714116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.574+0000 7f9719000640 1 --2- 172.21.15.16:0/1956659208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9714078290 0x7f9714114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.574+0000 7f9719000640 1 --2- 172.21.15.16:0/1956659208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9714077940 0x7f9714077d50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.574+0000 7f9719000640 1 -- 172.21.15.16:0/1956659208 >> 172.21.15.16:0/1956659208 conn(0x7f9714101680 msgr2=0x7f9714103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:23.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.574+0000 7f9719000640 1 -- 172.21.15.16:0/1956659208 shutdown_connections 2024-09-21T03:45:23.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.574+0000 7f9719000640 1 -- 172.21.15.16:0/1956659208 wait complete. 2024-09-21T03:45:23.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.575+0000 7f9719000640 1 Processor -- start 2024-09-21T03:45:23.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.575+0000 7f9719000640 1 -- start start 2024-09-21T03:45:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.575+0000 7f9719000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9714077940 0x7f97141a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.575+0000 7f9719000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9714078290 0x7f97141a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.575+0000 7f9719000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9714114570 0x7f97141a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.575+0000 7f9719000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9714119f20 con 0x7f9714114570 2024-09-21T03:45:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.575+0000 7f9719000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9714119da0 con 0x7f9714078290 2024-09-21T03:45:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.575+0000 7f9719000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f971411a0a0 con 0x7f9714077940 2024-09-21T03:45:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.576+0000 7f9713400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9714077940 0x7f97141a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.576+0000 7f9713400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9714077940 0x7f97141a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:54128/0 (socket says 172.21.15.16:54128) 2024-09-21T03:45:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.576+0000 7f9713400640 1 -- 172.21.15.16:0/42740443 learned_addr learned my addr 172.21.15.16:0/42740443 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.576+0000 7f970aa00640 1 --2- 172.21.15.16:0/42740443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9714078290 0x7f97141a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.576+0000 7f9713e00640 1 --2- 172.21.15.16:0/42740443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9714114570 0x7f97141a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.576+0000 7f9713400640 1 -- 172.21.15.16:0/42740443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9714078290 msgr2=0x7f97141a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.576+0000 7f9713400640 1 --2- 172.21.15.16:0/42740443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9714078290 0x7f97141a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.576+0000 7f9713400640 1 -- 172.21.15.16:0/42740443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9714114570 msgr2=0x7f97141a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.576+0000 7f9713400640 1 --2- 172.21.15.16:0/42740443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9714114570 0x7f97141a6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.576+0000 7f9713400640 1 -- 172.21.15.16:0/42740443 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97141a68a0 con 0x7f9714077940 2024-09-21T03:45:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.576+0000 7f970aa00640 1 --2- 172.21.15.16:0/42740443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9714078290 0x7f97141a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.577+0000 7f9713400640 1 --2- 172.21.15.16:0/42740443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9714077940 0x7f97141a0930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9704005e30 tx=0x7f9704031840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.577+0000 7f9710c00640 1 -- 172.21.15.16:0/42740443 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704003b70 con 0x7f9714077940 2024-09-21T03:45:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.577+0000 7f9719000640 1 -- 172.21.15.16:0/42740443 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97141a6b30 con 0x7f9714077940 2024-09-21T03:45:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.577+0000 7f9710c00640 1 -- 172.21.15.16:0/42740443 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9704003d10 con 0x7f9714077940 2024-09-21T03:45:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.577+0000 7f9710c00640 1 -- 172.21.15.16:0/42740443 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970402fa30 con 0x7f9714077940 2024-09-21T03:45:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.577+0000 7f9719000640 1 -- 172.21.15.16:0/42740443 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97141ad2f0 con 0x7f9714077940 2024-09-21T03:45:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.578+0000 7f9719000640 1 -- 172.21.15.16:0/42740443 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9714077d50 con 0x7f9714077940 2024-09-21T03:45:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.580+0000 7f9710c00640 1 -- 172.21.15.16:0/42740443 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f970402fbd0 con 0x7f9714077940 2024-09-21T03:45:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.582+0000 7f9710c00640 1 --2- 172.21.15.16:0/42740443 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f96f80b65a0 0x7f96f80b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.582+0000 7f9710c00640 1 -- 172.21.15.16:0/42740443 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f9704108ab0 con 0x7f9714077940 2024-09-21T03:45:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.582+0000 7f970aa00640 1 --2- 172.21.15.16:0/42740443 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f96f80b65a0 0x7f96f80b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.583+0000 7f970aa00640 1 --2- 172.21.15.16:0/42740443 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f96f80b65a0 0x7f96f80b8ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9714079a90 tx=0x7f96fc005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.584+0000 7f9710c00640 1 -- 172.21.15.16:0/42740443 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f97040cedd0 con 0x7f9714077940 2024-09-21T03:45:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.821+0000 7f9719000640 1 -- 172.21.15.16:0/42740443 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97141a1a60 con 0x7f9714077940 2024-09-21T03:45:23.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.823+0000 7f9710c00640 1 -- 172.21.15.16:0/42740443 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24902 (secure 0 0 0) 0x7f97040d31b0 con 0x7f9714077940 2024-09-21T03:45:23.826 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:45:23.826 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:33:53.220358+0000","modified":"2024-09-21T03:44:54.658912+0000","last_up_change":"2024-09-21T03:42:06.614247+0000","last_in_change":"2024-09-21T03:41:43.379404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:37:55.708915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:43:03.567522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:43:06.048083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:43:16.061356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6a7e4b9e-078f-4ae9-ab9e-37be995e4deb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6803","nonce":311974551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6805","nonce":311974551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6809","nonce":311974551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6807","nonce":311974551}]},"public_addr":"172.21.15.16:6803/311974551","cluster_addr":"172.21.15.16:6805/311974551","heartbeat_back_addr":"172.21.15.16:6809/311974551","heartbeat_front_addr":"172.21.15.16:6807/311974551","state":["exists","up"]},{"osd":1,"uuid":"3a497a64-e1a8-43ef-8509-0a79515da0aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6801","nonce":2152486571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6803","nonce":2152486571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6807","nonce":2152486571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6805","nonce":2152486571}]},"public_addr":"172.21.15.123:6801/2152486571","cluster_addr":"172.21.15.123:6803/2152486571","heartbeat_back_addr":"172.21.15.123:6807/2152486571","heartbeat_front_addr":"172.21.15.123:6805/2152486571","state":["exists","up"]},{"osd":2,"uuid":"1c39518e-6cce-4a0f-9ee8-55c60ad325f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6801","nonce":2812351279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6803","nonce":2812351279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6807","nonce":2812351279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6805","nonce":2812351279}]},"public_addr":"172.21.15.134:6801/2812351279","cluster_addr":"172.21.15.134:6803/2812351279","heartbeat_back_addr":"172.21.15.134:6807/2812351279","heartbeat_front_addr":"172.21.15.134:6805/2812351279","state":["exists","up"]},{"osd":3,"uuid":"0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6811","nonce":2366216026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6813","nonce":2366216026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6817","nonce":2366216026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6815","nonce":2366216026}]},"public_addr":"172.21.15.16:6811/2366216026","cluster_addr":"172.21.15.16:6813/2366216026","heartbeat_back_addr":"172.21.15.16:6817/2366216026","heartbeat_front_addr":"172.21.15.16:6815/2366216026","state":["exists","up"]},{"osd":4,"uuid":"ad5e6d0e-31e8-4779-9e74-f215032777ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6809","nonce":2354279116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6811","nonce":2354279116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6815","nonce":2354279116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6813","nonce":2354279116}]},"public_addr":"172.21.15.123:6809/2354279116","cluster_addr":"172.21.15.123:6811/2354279116","heartbeat_back_addr":"172.21.15.123:6815/2354279116","heartbeat_front_addr":"172.21.15.123:6813/2354279116","state":["exists","up"]},{"osd":5,"uuid":"ca09bf88-1772-4dc0-8711-cbf9a5b31977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6809","nonce":2678416182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6811","nonce":2678416182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6815","nonce":2678416182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6813","nonce":2678416182}]},"public_addr":"172.21.15.134:6809/2678416182","cluster_addr":"172.21.15.134:6811/2678416182","heartbeat_back_addr":"172.21.15.134:6815/2678416182","heartbeat_front_addr":"172.21.15.134:6813/2678416182","state":["exists","up"]},{"osd":6,"uuid":"0d41ed4e-8699-4d88-9254-12d3b84941e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6819","nonce":3377807744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6821","nonce":3377807744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6825","nonce":3377807744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6823","nonce":3377807744}]},"public_addr":"172.21.15.16:6819/3377807744","cluster_addr":"172.21.15.16:6821/3377807744","heartbeat_back_addr":"172.21.15.16:6825/3377807744","heartbeat_front_addr":"172.21.15.16:6823/3377807744","state":["exists","up"]},{"osd":7,"uuid":"1541a272-c524-4f12-9b94-763f69bb1c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6817","nonce":1463412739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6819","nonce":1463412739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6823","nonce":1463412739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6821","nonce":1463412739}]},"public_addr":"172.21.15.123:6817/1463412739","cluster_addr":"172.21.15.123:6819/1463412739","heartbeat_back_addr":"172.21.15.123:6823/1463412739","heartbeat_front_addr":"172.21.15.123:6821/1463412739","state":["exists","up"]},{"osd":8,"uuid":"cc00014c-eb99-49ab-984c-0bc26ef85130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6817","nonce":2644021450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6819","nonce":2644021450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6823","nonce":2644021450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6821","nonce":2644021450}]},"public_addr":"172.21.15.134:6817/2644021450","cluster_addr":"172.21.15.134:6819/2644021450","heartbeat_back_addr":"172.21.15.134:6823/2644021450","heartbeat_front_addr":"172.21.15.134:6821/2644021450","state":["exists","up"]},{"osd":9,"uuid":"3cbfff74-31ff-4ffd-844b-54ec8563ba32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6827","nonce":1653164371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6829","nonce":1653164371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6833","nonce":1653164371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6831","nonce":1653164371}]},"public_addr":"172.21.15.16:6827/1653164371","cluster_addr":"172.21.15.16:6829/1653164371","heartbeat_back_addr":"172.21.15.16:6833/1653164371","heartbeat_front_addr":"172.21.15.16:6831/1653164371","state":["exists","up"]},{"osd":10,"uuid":"0a1caaad-70cd-4c2c-88d3-e9a5cc885655","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6825","nonce":3682234496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6827","nonce":3682234496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6831","nonce":3682234496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6829","nonce":3682234496}]},"public_addr":"172.21.15.123:6825/3682234496","cluster_addr":"172.21.15.123:6827/3682234496","heartbeat_back_addr":"172.21.15.123:6831/3682234496","heartbeat_front_addr":"172.21.15.123:6829/3682234496","state":["exists","up"]},{"osd":11,"uuid":"3aff3258-5a38-41ff-aac3-4a082feffdae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6825","nonce":584758685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6827","nonce":584758685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6831","nonce":584758685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6829","nonce":584758685}]},"public_addr":"172.21.15.134:6825/584758685","cluster_addr":"172.21.15.134:6827/584758685","heartbeat_back_addr":"172.21.15.134:6831/584758685","heartbeat_front_addr":"172.21.15.134:6829/584758685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:02.177731+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:28.847562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:54.272384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:24.338596+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:52.430371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:17.832737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:50.397890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:16.842245+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:42.102933+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:13.074732+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:40.057905+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:04.898190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/557742783":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/3238676548":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6801/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/2357336233":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6801/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6800/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/3806844703":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/2351581621":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6800/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/355731724":"2024-09-22T03:34:22.549156+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:45:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.825+0000 7f9719000640 1 -- 172.21.15.16:0/42740443 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f96f80b65a0 msgr2=0x7f96f80b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.825+0000 7f9719000640 1 --2- 172.21.15.16:0/42740443 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f96f80b65a0 0x7f96f80b8ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9714079a90 tx=0x7f96fc005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.825+0000 7f9719000640 1 -- 172.21.15.16:0/42740443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9714077940 msgr2=0x7f97141a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.825+0000 7f9719000640 1 --2- 172.21.15.16:0/42740443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9714077940 0x7f97141a0930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9704005e30 tx=0x7f9704031840 comp rx=0 tx=0).stop 2024-09-21T03:45:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.825+0000 7f9719000640 1 -- 172.21.15.16:0/42740443 shutdown_connections 2024-09-21T03:45:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.825+0000 7f9719000640 1 --2- 172.21.15.16:0/42740443 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f96f80b65a0 0x7f96f80b8ab0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.825+0000 7f9719000640 1 --2- 172.21.15.16:0/42740443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9714114570 0x7f97141a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.825+0000 7f9719000640 1 --2- 172.21.15.16:0/42740443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9714078290 0x7f97141a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.825+0000 7f9719000640 1 --2- 172.21.15.16:0/42740443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9714077940 0x7f97141a0930 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.825+0000 7f9719000640 1 -- 172.21.15.16:0/42740443 >> 172.21.15.16:0/42740443 conn(0x7f9714101680 msgr2=0x7f9714116640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.826+0000 7f9719000640 1 -- 172.21.15.16:0/42740443 shutdown_connections 2024-09-21T03:45:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:23.826+0000 7f9719000640 1 -- 172.21.15.16:0/42740443 wait complete. 2024-09-21T03:45:24.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:45:24.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:24 smithi134 ceph-mon[35031]: pgmap v391: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:45:24.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:24 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/42740443' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:45:24.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:24 smithi123 ceph-mon[34673]: pgmap v391: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:45:24.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:24 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/42740443' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:45:24.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:24 smithi016 ceph-mon[30375]: pgmap v391: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:45:24.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:24 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/42740443' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:45:24.956 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.594+0000 7f4ba5e00640 1 -- 172.21.15.16:0/3708953918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba01092d0 msgr2=0x7f4ba01096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.594+0000 7f4ba5e00640 1 --2- 172.21.15.16:0/3708953918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba01092d0 0x7f4ba01096e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009900 tx=0x7f4b8802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.595+0000 7f4ba5e00640 1 -- 172.21.15.16:0/3708953918 shutdown_connections 2024-09-21T03:45:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.595+0000 7f4ba5e00640 1 --2- 172.21.15.16:0/3708953918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ba0109c20 0x7f4ba0110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.595+0000 7f4ba5e00640 1 --2- 172.21.15.16:0/3708953918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba01092d0 0x7f4ba01096e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.595+0000 7f4ba5e00640 1 --2- 172.21.15.16:0/3708953918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ba0103990 0x7f4ba0103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.595+0000 7f4ba5e00640 1 -- 172.21.15.16:0/3708953918 >> 172.21.15.16:0/3708953918 conn(0x7f4ba00ff440 msgr2=0x7f4ba0101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.595+0000 7f4ba5e00640 1 -- 172.21.15.16:0/3708953918 shutdown_connections 2024-09-21T03:45:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.595+0000 7f4ba5e00640 1 -- 172.21.15.16:0/3708953918 wait complete. 2024-09-21T03:45:25.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.596+0000 7f4ba5e00640 1 Processor -- start 2024-09-21T03:45:25.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.596+0000 7f4ba5e00640 1 -- start start 2024-09-21T03:45:25.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.596+0000 7f4ba5e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ba0103990 0x7f4ba0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:25.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.597+0000 7f4ba5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ba01092d0 0x7f4ba0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:25.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.597+0000 7f4ba5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba0109c20 0x7f4ba0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:25.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.597+0000 7f4ba5e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba01138e0 con 0x7f4ba0109c20 2024-09-21T03:45:25.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.597+0000 7f4ba5e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba0113760 con 0x7f4ba01092d0 2024-09-21T03:45:25.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.597+0000 7f4ba5e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba0113a60 con 0x7f4ba0103990 2024-09-21T03:45:25.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.597+0000 7f4b97e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ba01092d0 0x7f4ba0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:25.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.597+0000 7f4ba5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba0109c20 0x7f4ba0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:25.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.597+0000 7f4b97e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ba01092d0 0x7f4ba0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:58470/0 (socket says 172.21.15.16:58470) 2024-09-21T03:45:25.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.597+0000 7f4b97e00640 1 -- 172.21.15.16:0/3662984780 learned_addr learned my addr 172.21.15.16:0/3662984780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:25.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.597+0000 7f4ba4a00640 1 --2- 172.21.15.16:0/3662984780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ba0103990 0x7f4ba0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:25.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.597+0000 7f4b97e00640 1 -- 172.21.15.16:0/3662984780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ba0103990 msgr2=0x7f4ba0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:25.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.597+0000 7f4b97e00640 1 --2- 172.21.15.16:0/3662984780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ba0103990 0x7f4ba0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.597+0000 7f4b97e00640 1 -- 172.21.15.16:0/3662984780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba0109c20 msgr2=0x7f4ba0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:25.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.597+0000 7f4b97e00640 1 --2- 172.21.15.16:0/3662984780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba0109c20 0x7f4ba0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.597+0000 7f4b97e00640 1 -- 172.21.15.16:0/3662984780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba01a8590 con 0x7f4ba01092d0 2024-09-21T03:45:25.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.598+0000 7f4b97e00640 1 --2- 172.21.15.16:0/3662984780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ba01092d0 0x7f4ba0075c50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4b8802a910 tx=0x7f4b88031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.598+0000 7f4b9e000640 1 -- 172.21.15.16:0/3662984780 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8803f030 con 0x7f4ba01092d0 2024-09-21T03:45:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.598+0000 7f4ba5e00640 1 -- 172.21.15.16:0/3662984780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba01a8820 con 0x7f4ba01092d0 2024-09-21T03:45:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.598+0000 7f4b9e000640 1 -- 172.21.15.16:0/3662984780 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b88004490 con 0x7f4ba01092d0 2024-09-21T03:45:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.598+0000 7f4b9e000640 1 -- 172.21.15.16:0/3662984780 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88035070 con 0x7f4ba01092d0 2024-09-21T03:45:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.598+0000 7f4ba5e00640 1 -- 172.21.15.16:0/3662984780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba01a8d50 con 0x7f4ba01092d0 2024-09-21T03:45:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.599+0000 7f4ba5e00640 1 -- 172.21.15.16:0/3662984780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ba0103da0 con 0x7f4ba01092d0 2024-09-21T03:45:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.602+0000 7f4b9e000640 1 -- 172.21.15.16:0/3662984780 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f4b88042030 con 0x7f4ba01092d0 2024-09-21T03:45:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.603+0000 7f4b9e000640 1 --2- 172.21.15.16:0/3662984780 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4b840b65a0 0x7f4b840b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.603+0000 7f4b9e000640 1 -- 172.21.15.16:0/3662984780 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f4b88107f70 con 0x7f4ba01092d0 2024-09-21T03:45:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.604+0000 7f4ba4a00640 1 --2- 172.21.15.16:0/3662984780 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4b840b65a0 0x7f4b840b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:25.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.604+0000 7f4ba4a00640 1 --2- 172.21.15.16:0/3662984780 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4b840b65a0 0x7f4b840b8ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0077160 tx=0x7f4b90009500 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:25.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.605+0000 7f4b9e000640 1 -- 172.21.15.16:0/3662984780 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4b880d2670 con 0x7f4ba01092d0 2024-09-21T03:45:25.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.843+0000 7f4ba5e00640 1 -- 172.21.15.16:0/3662984780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ba0076a10 con 0x7f4ba01092d0 2024-09-21T03:45:25.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.844+0000 7f4b9e000640 1 -- 172.21.15.16:0/3662984780 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11971 (secure 0 0 0) 0x7f4b880d75b0 con 0x7f4ba01092d0 2024-09-21T03:45:25.847 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:45:25.847 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"btime":"2024-09-21T03:44:08:542131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:44:07.645054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24668,24670,24682,24689,24731]},"id":1}]} 2024-09-21T03:45:25.847 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-09-21T03:45:25.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.847+0000 7f4ba5e00640 1 -- 172.21.15.16:0/3662984780 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4b840b65a0 msgr2=0x7f4b840b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:25.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.847+0000 7f4ba5e00640 1 --2- 172.21.15.16:0/3662984780 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4b840b65a0 0x7f4b840b8ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0077160 tx=0x7f4b90009500 comp rx=0 tx=0).stop 2024-09-21T03:45:25.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.847+0000 7f4ba5e00640 1 -- 172.21.15.16:0/3662984780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ba01092d0 msgr2=0x7f4ba0075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:25.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.847+0000 7f4ba5e00640 1 --2- 172.21.15.16:0/3662984780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ba01092d0 0x7f4ba0075c50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4b8802a910 tx=0x7f4b88031a20 comp rx=0 tx=0).stop 2024-09-21T03:45:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.848+0000 7f4ba5e00640 1 -- 172.21.15.16:0/3662984780 shutdown_connections 2024-09-21T03:45:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.848+0000 7f4ba5e00640 1 --2- 172.21.15.16:0/3662984780 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4b840b65a0 0x7f4b840b8ab0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.848+0000 7f4ba5e00640 1 --2- 172.21.15.16:0/3662984780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba0109c20 0x7f4ba0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.848+0000 7f4ba5e00640 1 --2- 172.21.15.16:0/3662984780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ba01092d0 0x7f4ba0075c50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.848+0000 7f4ba5e00640 1 --2- 172.21.15.16:0/3662984780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ba0103990 0x7f4ba0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.848+0000 7f4ba5e00640 1 -- 172.21.15.16:0/3662984780 >> 172.21.15.16:0/3662984780 conn(0x7f4ba00ff440 msgr2=0x7f4ba0101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.848+0000 7f4ba5e00640 1 -- 172.21.15.16:0/3662984780 shutdown_connections 2024-09-21T03:45:25.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:25.848+0000 7f4ba5e00640 1 -- 172.21.15.16:0/3662984780 wait complete. 2024-09-21T03:45:26.623 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T03:43:09.332305+0000', 'modified': '2024-09-21T03:44:07.645054+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24731, 'mds_1': 24689, 'mds_2': 24682, 'mds_3': 24670, 'mds_4': 24668}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14700': {'gid': 14700, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6839/89613887', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 89613887}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 89613887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14712': {'gid': 14712, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6841/1546097467', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6840', 'nonce': 1546097467}, {'type': 'v1', 'addr': '172.21.15.16:6841', 'nonce': 1546097467}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24613': {'gid': 24613, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/2239407349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 2239407349}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 2239407349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24632': {'gid': 24632, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6833/3581513383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6832', 'nonce': 3581513383}, {'type': 'v1', 'addr': '172.21.15.123:6833', 'nonce': 3581513383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24658': {'gid': 24658, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/135285598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 135285598}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 135285598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24668': {'gid': 24668, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.123:6835/86959152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 86959152}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 86959152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24670': {'gid': 24670, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.134:6837/2624530453', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 2624530453}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 2624530453}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24682': {'gid': 24682, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.134:6839/606193011', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 606193011}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 606193011}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24689': {'gid': 24689, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.123:6837/2001424876', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 2001424876}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 2001424876}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24731': {'gid': 24731, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.123:6839/459522816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 459522816}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 459522816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24731, 'qdb_cluster': [24668, 24670, 24682, 24689, 24731]} 2024-09-21T03:45:26.624 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T03:45:26.624 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.e status 2024-09-21T03:45:26.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:26 smithi134 ceph-mon[35031]: pgmap v392: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:45:26.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:26 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3662984780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:26.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:26 smithi123 ceph-mon[34673]: pgmap v392: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:45:26.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:26 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3662984780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:26.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:26 smithi016 ceph-mon[30375]: pgmap v392: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:45:26.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:26 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3662984780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:27.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.658+0000 7ff848000640 1 -- 172.21.15.16:0/3796604741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840109df0 msgr2=0x7ff840110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.658+0000 7ff848000640 1 --2- 172.21.15.16:0/3796604741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840109df0 0x7ff840110620 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff838009ca0 tx=0x7ff83802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.658+0000 7ff848000640 1 -- 172.21.15.16:0/3796604741 shutdown_connections 2024-09-21T03:45:27.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.658+0000 7ff848000640 1 --2- 172.21.15.16:0/3796604741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840109df0 0x7ff840110620 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.658+0000 7ff848000640 1 --2- 172.21.15.16:0/3796604741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8401094a0 0x7ff8401098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.658+0000 7ff848000640 1 --2- 172.21.15.16:0/3796604741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff840103b60 0x7ff840103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.658+0000 7ff848000640 1 -- 172.21.15.16:0/3796604741 >> 172.21.15.16:0/3796604741 conn(0x7ff84007b850 msgr2=0x7ff84007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:27.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.658+0000 7ff848000640 1 -- 172.21.15.16:0/3796604741 shutdown_connections 2024-09-21T03:45:27.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.658+0000 7ff848000640 1 -- 172.21.15.16:0/3796604741 wait complete. 2024-09-21T03:45:27.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.659+0000 7ff848000640 1 Processor -- start 2024-09-21T03:45:27.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.659+0000 7ff848000640 1 -- start start 2024-09-21T03:45:27.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.660+0000 7ff848000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840103b60 0x7ff84019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.660+0000 7ff848000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8401094a0 0x7ff84019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.660+0000 7ff848000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff840109df0 0x7ff8401a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.660+0000 7ff848000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff840113b00 con 0x7ff840103b60 2024-09-21T03:45:27.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.660+0000 7ff848000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff840113980 con 0x7ff840109df0 2024-09-21T03:45:27.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.660+0000 7ff848000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff840113c80 con 0x7ff8401094a0 2024-09-21T03:45:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.660+0000 7ff845600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840103b60 0x7ff84019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.660+0000 7ff845600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840103b60 0x7ff84019c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58406/0 (socket says 172.21.15.16:58406) 2024-09-21T03:45:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.660+0000 7ff845600640 1 -- 172.21.15.16:0/2226647942 learned_addr learned my addr 172.21.15.16:0/2226647942 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.660+0000 7ff846000640 1 --2- 172.21.15.16:0/2226647942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff840109df0 0x7ff8401a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.660+0000 7ff844c00640 1 --2- 172.21.15.16:0/2226647942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8401094a0 0x7ff84019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.661+0000 7ff844c00640 1 -- 172.21.15.16:0/2226647942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff840109df0 msgr2=0x7ff8401a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.661+0000 7ff844c00640 1 --2- 172.21.15.16:0/2226647942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff840109df0 0x7ff8401a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.661+0000 7ff844c00640 1 -- 172.21.15.16:0/2226647942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840103b60 msgr2=0x7ff84019c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.661+0000 7ff844c00640 1 --2- 172.21.15.16:0/2226647942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840103b60 0x7ff84019c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.661+0000 7ff844c00640 1 -- 172.21.15.16:0/2226647942 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8401a25a0 con 0x7ff8401094a0 2024-09-21T03:45:27.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.661+0000 7ff845600640 1 --2- 172.21.15.16:0/2226647942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840103b60 0x7ff84019c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.661+0000 7ff844c00640 1 --2- 172.21.15.16:0/2226647942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8401094a0 0x7ff84019cbd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff84006c9f0 tx=0x7ff82c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.661+0000 7ff83e000640 1 -- 172.21.15.16:0/2226647942 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c00cd00 con 0x7ff8401094a0 2024-09-21T03:45:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.661+0000 7ff83e000640 1 -- 172.21.15.16:0/2226647942 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff82c0035f0 con 0x7ff8401094a0 2024-09-21T03:45:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.661+0000 7ff848000640 1 -- 172.21.15.16:0/2226647942 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8401a2890 con 0x7ff8401094a0 2024-09-21T03:45:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.661+0000 7ff83e000640 1 -- 172.21.15.16:0/2226647942 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c010690 con 0x7ff8401094a0 2024-09-21T03:45:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.661+0000 7ff848000640 1 -- 172.21.15.16:0/2226647942 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8401a8ff0 con 0x7ff8401094a0 2024-09-21T03:45:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.665+0000 7ff84edc7740 1 Processor -- start 2024-09-21T03:45:27.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.665+0000 7ff84edc7740 1 -- start start 2024-09-21T03:45:27.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.665+0000 7ff83e000640 1 -- 172.21.15.16:0/2226647942 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff82c0037a0 con 0x7ff8401094a0 2024-09-21T03:45:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.665+0000 7ff84edc7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ee4f60 0x55a9a828b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.665+0000 7ff84edc7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ebc0a0 0x55a9a828b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.665+0000 7ff84edc7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 0x55a9a828bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.665+0000 7ff84edc7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a9a8202d50 con 0x55a9a7ebc0a0 2024-09-21T03:45:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.665+0000 7ff84edc7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a9a81f2180 con 0x55a9a7ee4f60 2024-09-21T03:45:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.665+0000 7ff84edc7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a9a81f5f90 con 0x55a9a7ec4280 2024-09-21T03:45:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.666+0000 7ff845600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ebc0a0 0x55a9a828b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.666+0000 7ff845600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ebc0a0 0x55a9a828b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58420/0 (socket says 172.21.15.16:58420) 2024-09-21T03:45:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.666+0000 7ff845600640 1 -- 172.21.15.16:0/155202224 learned_addr learned my addr 172.21.15.16:0/155202224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.666+0000 7ff846000640 1 --2- 172.21.15.16:0/155202224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ee4f60 0x55a9a828b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.666+0000 7ff844c00640 1 --2- 172.21.15.16:0/155202224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 0x55a9a828bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.666+0000 7ff846000640 1 -- 172.21.15.16:0/155202224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 msgr2=0x55a9a828bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.666+0000 7ff846000640 1 --2- 172.21.15.16:0/155202224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 0x55a9a828bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.666+0000 7ff846000640 1 -- 172.21.15.16:0/155202224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ebc0a0 msgr2=0x55a9a828b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.666+0000 7ff846000640 1 --2- 172.21.15.16:0/155202224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ebc0a0 0x55a9a828b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.666+0000 7ff846000640 1 -- 172.21.15.16:0/155202224 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9a81bca70 con 0x55a9a7ee4f60 2024-09-21T03:45:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.667+0000 7ff846000640 1 --2- 172.21.15.16:0/155202224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ee4f60 0x55a9a828b490 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff838005e30 tx=0x7ff838009430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecfcce945ecd79a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.667+0000 7ff832600640 1 -- 172.21.15.16:0/155202224 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838031cf0 con 0x55a9a7ee4f60 2024-09-21T03:45:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.667+0000 7ff832600640 1 -- 172.21.15.16:0/155202224 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff838036030 con 0x55a9a7ee4f60 2024-09-21T03:45:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.667+0000 7ff832600640 1 -- 172.21.15.16:0/155202224 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838004240 con 0x55a9a7ee4f60 2024-09-21T03:45:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.667+0000 7ff83e000640 1 --2- 172.21.15.16:0/2226647942 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff8280b6590 0x7ff8280b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.668+0000 7ff83e000640 1 -- 172.21.15.16:0/2226647942 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7ff82c0e78e0 con 0x7ff8401094a0 2024-09-21T03:45:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.668+0000 7ff845600640 1 --2- 172.21.15.16:0/2226647942 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff8280b6590 0x7ff8280b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.668+0000 7ff84edc7740 1 -- 172.21.15.16:0/155202224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ee4f60 msgr2=0x55a9a828b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.668+0000 7ff84edc7740 1 --2- 172.21.15.16:0/155202224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ee4f60 0x55a9a828b490 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff838005e30 tx=0x7ff838009430 comp rx=0 tx=0).stop 2024-09-21T03:45:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.668+0000 7ff84edc7740 1 -- 172.21.15.16:0/155202224 shutdown_connections 2024-09-21T03:45:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.668+0000 7ff84edc7740 1 --2- 172.21.15.16:0/155202224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 0x55a9a828bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.668+0000 7ff84edc7740 1 --2- 172.21.15.16:0/155202224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ebc0a0 0x55a9a828b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.668+0000 7ff84edc7740 1 --2- 172.21.15.16:0/155202224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ee4f60 0x55a9a828b490 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.668+0000 7ff84edc7740 1 -- 172.21.15.16:0/155202224 >> 172.21.15.16:0/155202224 conn(0x55a9a7ed9350 msgr2=0x55a9a8289380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.668+0000 7ff84edc7740 1 -- 172.21.15.16:0/155202224 shutdown_connections 2024-09-21T03:45:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.669+0000 7ff845600640 1 --2- 172.21.15.16:0/2226647942 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff8280b6590 0x7ff8280b8aa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55a9a828a110 tx=0x7ff834005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.669+0000 7ff84edc7740 1 -- 172.21.15.16:0/155202224 wait complete. 2024-09-21T03:45:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.669+0000 7ff84edc7740 1 -- start start 2024-09-21T03:45:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.670+0000 7ff84edc7740 1 Processor -- start 2024-09-21T03:45:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.670+0000 7ff84edc7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ebc0a0 0x55a9a8297060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.670+0000 7ff84edc7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 0x55a9a82975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.670+0000 7ff84edc7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ee4f60 0x55a9a82a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.670+0000 7ff84edc7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a9a80535f0 con 0x55a9a7ebc0a0 2024-09-21T03:45:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.670+0000 7ff84edc7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a9a818d460 con 0x55a9a7ee4f60 2024-09-21T03:45:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.670+0000 7ff84edc7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a9a8299a90 con 0x55a9a7ec4280 2024-09-21T03:45:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.670+0000 7ff84edc7740 10 client.0 _ll_register_callbacks cb 0x55a9a819d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.671+0000 7ff844c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 0x55a9a82975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.671+0000 7ff844c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 0x55a9a82975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:54172/0 (socket says 172.21.15.16:54172) 2024-09-21T03:45:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.671+0000 7ff844c00640 1 -- 172.21.15.16:0/1487946559 learned_addr learned my addr 172.21.15.16:0/1487946559 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.671+0000 7ff845600640 1 --2- 172.21.15.16:0/1487946559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ebc0a0 0x55a9a8297060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.671+0000 7ff846000640 1 --2- 172.21.15.16:0/1487946559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ee4f60 0x55a9a82a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.671+0000 7ff844c00640 1 -- 172.21.15.16:0/1487946559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ee4f60 msgr2=0x55a9a82a2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.671+0000 7ff844c00640 1 --2- 172.21.15.16:0/1487946559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ee4f60 0x55a9a82a2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.671+0000 7ff844c00640 1 -- 172.21.15.16:0/1487946559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ebc0a0 msgr2=0x55a9a8297060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.671+0000 7ff844c00640 1 --2- 172.21.15.16:0/1487946559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ebc0a0 0x55a9a8297060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.671+0000 7ff844c00640 1 -- 172.21.15.16:0/1487946559 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9a7f50540 con 0x55a9a7ec4280 2024-09-21T03:45:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.671+0000 7ff844c00640 1 --2- 172.21.15.16:0/1487946559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 0x55a9a82975a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0ad360 tx=0x7ff82c003cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=166d7f3083bcee2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.671+0000 7ff826a00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:45:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.672+0000 7ff826a00640 1 -- 172.21.15.16:0/1487946559 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0b6240 con 0x55a9a7ec4280 2024-09-21T03:45:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.672+0000 7ff84edc7740 1 -- 172.21.15.16:0/1487946559 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a9a818fa70 con 0x55a9a7ec4280 2024-09-21T03:45:27.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.672+0000 7ff826a00640 1 -- 172.21.15.16:0/1487946559 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7ff82c0a8fb0 con 0x55a9a7ec4280 2024-09-21T03:45:27.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.672+0000 7ff826a00640 1 -- 172.21.15.16:0/1487946559 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff82c009460 con 0x55a9a7ec4280 2024-09-21T03:45:27.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.672+0000 7ff84edc7740 10 client.25162 fetch_fsmap learned FSMap version 29 2024-09-21T03:45:27.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.672+0000 7ff84edc7740 1 -- 172.21.15.16:0/1487946559 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a9a81aa6f0 con 0x55a9a7ec4280 2024-09-21T03:45:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.673+0000 7ff826a00640 1 -- 172.21.15.16:0/1487946559 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a9a818fa70 con 0x55a9a7ec4280 2024-09-21T03:45:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.673+0000 7ff826a00640 1 -- 172.21.15.16:0/1487946559 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff82c0a91b0 con 0x55a9a7ec4280 2024-09-21T03:45:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.673+0000 7ff84edc7740 10 client.25162 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:45:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.673+0000 7ff84edc7740 10 client.25162 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-21T03:45:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.673+0000 7ff84edc7740 20 client.25162 populate_metadata read hostname 'smithi016' 2024-09-21T03:45:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.673+0000 7ff84edc7740 1 --2- 172.21.15.16:0/1487946559 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x55a9a7ea38e0 0x55a9a82a9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.673+0000 7ff84edc7740 4 client.25162 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.673+0000 7ff84edc7740 1 -- 172.21.15.16:0/1487946559 --> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9a8204ca0 con 0x55a9a7ea38e0 2024-09-21T03:45:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.674+0000 7ff845600640 1 --2- 172.21.15.16:0/1487946559 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x55a9a7ea38e0 0x55a9a82a9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.674+0000 7ff845600640 1 --2- 172.21.15.16:0/1487946559 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x55a9a7ea38e0 0x55a9a82a9310 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7e09da8779ae4b32 server_cookie=fffb9eba40f92a1f in_seq=0 out_seq=0 2024-09-21T03:45:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.674+0000 7ff826a00640 10 client.25162 ms_handle_connect on v2:172.21.15.123:6834/86959152 2024-09-21T03:45:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.675+0000 7ff826a00640 1 -- 172.21.15.16:0/1487946559 <== mds.4 v2:172.21.15.123:6834/86959152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a9a8204ca0 con 0x55a9a7ea38e0 2024-09-21T03:45:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.675+0000 7ff826a00640 10 client.25162 handle_command_reply: tid=0 2024-09-21T03:45:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.675+0000 7ff84edc7740 1 client.25162 shutdown 2024-09-21T03:45:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.676+0000 7ff84edc7740 1 -- 172.21.15.16:0/1487946559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 msgr2=0x55a9a82975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.676+0000 7ff84edc7740 1 --2- 172.21.15.16:0/1487946559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 0x55a9a82975a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0ad360 tx=0x7ff82c003cf0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.676+0000 7ff84edc7740 1 -- 172.21.15.16:0/1487946559 shutdown_connections 2024-09-21T03:45:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.676+0000 7ff84edc7740 1 --2- 172.21.15.16:0/1487946559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ee4f60 0x55a9a82a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.676+0000 7ff84edc7740 1 --2- 172.21.15.16:0/1487946559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 0x55a9a82975a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.676+0000 7ff84edc7740 1 --2- 172.21.15.16:0/1487946559 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x55a9a7ea38e0 0x55a9a82a9310 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.676+0000 7ff84edc7740 1 --2- 172.21.15.16:0/1487946559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ebc0a0 0x55a9a8297060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.676+0000 7ff84edc7740 1 -- 172.21.15.16:0/1487946559 >> 172.21.15.16:0/1487946559 conn(0x55a9a7ed9350 msgr2=0x55a9a828e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.676+0000 7ff826a00640 0 client.25162 ms_handle_reset on v2:172.21.15.123:6834/86959152 2024-09-21T03:45:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.676+0000 7ff84edc7740 1 -- 172.21.15.16:0/1487946559 shutdown_connections 2024-09-21T03:45:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.676+0000 7ff84edc7740 1 -- 172.21.15.16:0/1487946559 wait complete. 2024-09-21T03:45:27.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.676+0000 7ff84edc7740 20 client.25162 trim_cache size 0 max 16384 2024-09-21T03:45:27.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.696+0000 7ff84edc7740 1 Processor -- start 2024-09-21T03:45:27.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.696+0000 7ff84edc7740 1 -- start start 2024-09-21T03:45:27.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.697+0000 7ff84edc7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ebc0a0 0x55a9a82a7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.697+0000 7ff84edc7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ec4280 0x55a9a8297040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.697+0000 7ff84edc7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ee4f60 0x55a9a8297580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.697+0000 7ff84edc7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a9a82a7650 con 0x55a9a7ee4f60 2024-09-21T03:45:27.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.697+0000 7ff84edc7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a9a82a7d20 con 0x55a9a7ec4280 2024-09-21T03:45:27.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.697+0000 7ff84edc7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a9a82a83f0 con 0x55a9a7ebc0a0 2024-09-21T03:45:27.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.697+0000 7ff845600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ebc0a0 0x55a9a82a7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.697+0000 7ff844c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ec4280 0x55a9a8297040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.697+0000 7ff845600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ebc0a0 0x55a9a82a7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:54174/0 (socket says 172.21.15.16:54174) 2024-09-21T03:45:27.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.697+0000 7ff846000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ee4f60 0x55a9a8297580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.697+0000 7ff845600640 1 -- 172.21.15.16:0/1263910274 learned_addr learned my addr 172.21.15.16:0/1263910274 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:27.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.698+0000 7ff845600640 1 -- 172.21.15.16:0/1263910274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ec4280 msgr2=0x55a9a8297040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:27.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.698+0000 7ff845600640 1 --2- 172.21.15.16:0/1263910274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ec4280 0x55a9a8297040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.698+0000 7ff845600640 1 -- 172.21.15.16:0/1263910274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ee4f60 msgr2=0x55a9a8297580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:27.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.698+0000 7ff845600640 1 --2- 172.21.15.16:0/1263910274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ee4f60 0x55a9a8297580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.698+0000 7ff845600640 1 -- 172.21.15.16:0/1263910274 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9a7ea3b10 con 0x55a9a7ebc0a0 2024-09-21T03:45:27.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.698+0000 7ff845600640 1 --2- 172.21.15.16:0/1263910274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ebc0a0 0x55a9a82a7ea0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff83400f870 tx=0x7ff83401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ea12780f1cd216b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:27.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.698+0000 7ff831c00640 1 -- 172.21.15.16:0/1263910274 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83402c040 con 0x55a9a7ebc0a0 2024-09-21T03:45:27.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.698+0000 7ff831c00640 1 -- 172.21.15.16:0/1263910274 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8340054d0 con 0x55a9a7ebc0a0 2024-09-21T03:45:27.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.698+0000 7ff831c00640 1 -- 172.21.15.16:0/1263910274 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834029790 con 0x55a9a7ebc0a0 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.699+0000 7ff84edc7740 1 -- 172.21.15.16:0/1263910274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ebc0a0 msgr2=0x55a9a82a7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.699+0000 7ff84edc7740 1 --2- 172.21.15.16:0/1263910274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ebc0a0 0x55a9a82a7ea0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff83400f870 tx=0x7ff83401b000 comp rx=0 tx=0).stop 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.699+0000 7ff84edc7740 1 -- 172.21.15.16:0/1263910274 shutdown_connections 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.699+0000 7ff84edc7740 1 --2- 172.21.15.16:0/1263910274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ee4f60 0x55a9a8297580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.699+0000 7ff84edc7740 1 --2- 172.21.15.16:0/1263910274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ec4280 0x55a9a8297040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.699+0000 7ff84edc7740 1 --2- 172.21.15.16:0/1263910274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ebc0a0 0x55a9a82a7ea0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.699+0000 7ff84edc7740 1 -- 172.21.15.16:0/1263910274 >> 172.21.15.16:0/1263910274 conn(0x55a9a7ed9350 msgr2=0x55a9a8284010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.699+0000 7ff84edc7740 1 -- 172.21.15.16:0/1263910274 shutdown_connections 2024-09-21T03:45:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.700+0000 7ff84edc7740 1 -- 172.21.15.16:0/1263910274 wait complete. 2024-09-21T03:45:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.700+0000 7ff84edc7740 1 -- start start 2024-09-21T03:45:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.700+0000 7ff84edc7740 1 Processor -- start 2024-09-21T03:45:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.701+0000 7ff84edc7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ebc0a0 0x55a9a82aad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.701+0000 7ff84edc7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 0x55a9a82ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.701+0000 7ff84edc7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ee4f60 0x55a9a82b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.701+0000 7ff84edc7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a9a82a9190 con 0x55a9a7ebc0a0 2024-09-21T03:45:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.701+0000 7ff84edc7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a9a82a8ac0 con 0x55a9a7ee4f60 2024-09-21T03:45:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.701+0000 7ff84edc7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a9a82a9860 con 0x55a9a7ec4280 2024-09-21T03:45:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.701+0000 7ff84edc7740 10 client.0 _ll_register_callbacks cb 0x7ff82c0ad430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.701+0000 7ff844c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 0x55a9a82ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.701+0000 7ff844c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 0x55a9a82ab2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:54182/0 (socket says 172.21.15.16:54182) 2024-09-21T03:45:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.701+0000 7ff844c00640 1 -- 172.21.15.16:0/1169448216 learned_addr learned my addr 172.21.15.16:0/1169448216 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.701+0000 7ff845600640 1 --2- 172.21.15.16:0/1169448216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ebc0a0 0x55a9a82aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.701+0000 7ff846000640 1 --2- 172.21.15.16:0/1169448216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ee4f60 0x55a9a82b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.702+0000 7ff844c00640 1 -- 172.21.15.16:0/1169448216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ee4f60 msgr2=0x55a9a82b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.702+0000 7ff844c00640 1 --2- 172.21.15.16:0/1169448216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ee4f60 0x55a9a82b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.702+0000 7ff844c00640 1 -- 172.21.15.16:0/1169448216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ebc0a0 msgr2=0x55a9a82aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.702+0000 7ff844c00640 1 --2- 172.21.15.16:0/1169448216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ebc0a0 0x55a9a82aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.702+0000 7ff844c00640 1 -- 172.21.15.16:0/1169448216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9a82b0930 con 0x55a9a7ec4280 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.702+0000 7ff844c00640 1 --2- 172.21.15.16:0/1169448216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 0x55a9a82ab2e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00cfd0 tx=0x7ff82c0e7b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9c17669b187445f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.702+0000 7ff826a00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.702+0000 7ff826a00640 1 -- 172.21.15.16:0/1169448216 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0a9500 con 0x55a9a7ec4280 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.702+0000 7ff826a00640 1 -- 172.21.15.16:0/1169448216 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7ff82c0ac330 con 0x55a9a7ec4280 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.702+0000 7ff84edc7740 1 -- 172.21.15.16:0/1169448216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8340054d0 con 0x55a9a7ec4280 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.702+0000 7ff826a00640 1 -- 172.21.15.16:0/1169448216 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff82c0ad170 con 0x55a9a7ec4280 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.702+0000 7ff84edc7740 10 client.25168 fetch_fsmap learned FSMap version 29 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.702+0000 7ff84edc7740 1 -- 172.21.15.16:0/1169448216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a9a82b1d20 con 0x55a9a7ec4280 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.703+0000 7ff826a00640 1 -- 172.21.15.16:0/1169448216 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8340054d0 con 0x55a9a7ec4280 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.703+0000 7ff826a00640 1 -- 172.21.15.16:0/1169448216 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff82c0ac530 con 0x55a9a7ec4280 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.703+0000 7ff84edc7740 10 client.25168 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:45:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.703+0000 7ff84edc7740 10 client.25168 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-21T03:45:27.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.703+0000 7ff84edc7740 20 client.25168 populate_metadata read hostname 'smithi016' 2024-09-21T03:45:27.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.703+0000 7ff84edc7740 1 --2- 172.21.15.16:0/1169448216 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x55a9a7e78c90 0x55a9a83178b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:27.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.703+0000 7ff84edc7740 4 client.25168 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:45:27.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.703+0000 7ff84edc7740 1 -- 172.21.15.16:0/1169448216 --> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] -- command(tid 0: {"prefix": "status"}) -- 0x55a9a8200750 con 0x55a9a7e78c90 2024-09-21T03:45:27.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.703+0000 7ff845600640 1 --2- 172.21.15.16:0/1169448216 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x55a9a7e78c90 0x55a9a83178b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:27.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.704+0000 7ff845600640 1 --2- 172.21.15.16:0/1169448216 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x55a9a7e78c90 0x55a9a83178b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=58cd5172bc08c5ee server_cookie=479e9b385d47d63f in_seq=0 out_seq=0 2024-09-21T03:45:27.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.704+0000 7ff826a00640 10 client.25168 ms_handle_connect on v2:172.21.15.123:6834/86959152 2024-09-21T03:45:27.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.704+0000 7ff826a00640 1 -- 172.21.15.16:0/1169448216 <== mds.4 v2:172.21.15.123:6834/86959152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a9a8200750 con 0x55a9a7e78c90 2024-09-21T03:45:27.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.704+0000 7ff826a00640 10 client.25168 handle_command_reply: tid=0 2024-09-21T03:45:27.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.704+0000 7ff84edc7740 1 client.25168 shutdown 2024-09-21T03:45:27.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.704+0000 7ff84edc7740 1 -- 172.21.15.16:0/1169448216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 msgr2=0x55a9a82ab2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.704+0000 7ff84edc7740 1 --2- 172.21.15.16:0/1169448216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 0x55a9a82ab2e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00cfd0 tx=0x7ff82c0e7b60 comp rx=0 tx=0).stop 2024-09-21T03:45:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.705+0000 7ff84edc7740 1 -- 172.21.15.16:0/1169448216 shutdown_connections 2024-09-21T03:45:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.705+0000 7ff84edc7740 1 --2- 172.21.15.16:0/1169448216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9a7ee4f60 0x55a9a82b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.705+0000 7ff84edc7740 1 --2- 172.21.15.16:0/1169448216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9a7ec4280 0x55a9a82ab2e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.705+0000 7ff84edc7740 1 --2- 172.21.15.16:0/1169448216 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x55a9a7e78c90 0x55a9a83178b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.705+0000 7ff84edc7740 1 --2- 172.21.15.16:0/1169448216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9a7ebc0a0 0x55a9a82aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.705+0000 7ff84edc7740 1 -- 172.21.15.16:0/1169448216 >> 172.21.15.16:0/1169448216 conn(0x55a9a7ed9350 msgr2=0x55a9a8292010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.705+0000 7ff826a00640 0 client.25168 ms_handle_reset on v2:172.21.15.123:6834/86959152 2024-09-21T03:45:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.705+0000 7ff84edc7740 1 -- 172.21.15.16:0/1169448216 shutdown_connections 2024-09-21T03:45:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.705+0000 7ff84edc7740 1 -- 172.21.15.16:0/1169448216 wait complete. 2024-09-21T03:45:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.705+0000 7ff84edc7740 20 client.25168 trim_cache size 0 max 16384 2024-09-21T03:45:27.712 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:45:27.712 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:45:27.712 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 4, 2024-09-21T03:45:27.712 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24668, 2024-09-21T03:45:27.712 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T03:45:27.712 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T03:45:27.712 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T03:45:27.712 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 82.172971466999996, 2024-09-21T03:45:27.712 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 29, 2024-09-21T03:45:27.712 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 86, 2024-09-21T03:45:27.712 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 84, 2024-09-21T03:45:27.712 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 175.45313280600001 2024-09-21T03:45:27.712 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:45:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.706+0000 7ff848000640 1 -- 172.21.15.16:0/2226647942 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff8280b6590 msgr2=0x7ff8280b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.706+0000 7ff848000640 1 --2- 172.21.15.16:0/2226647942 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff8280b6590 0x7ff8280b8aa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55a9a828a110 tx=0x7ff834005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.706+0000 7ff848000640 1 -- 172.21.15.16:0/2226647942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8401094a0 msgr2=0x7ff84019cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.706+0000 7ff848000640 1 --2- 172.21.15.16:0/2226647942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8401094a0 0x7ff84019cbd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff84006c9f0 tx=0x7ff82c00ee50 comp rx=0 tx=0).stop 2024-09-21T03:45:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.706+0000 7ff848000640 1 -- 172.21.15.16:0/2226647942 shutdown_connections 2024-09-21T03:45:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.706+0000 7ff848000640 1 --2- 172.21.15.16:0/2226647942 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff8280b6590 0x7ff8280b8aa0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.706+0000 7ff848000640 1 --2- 172.21.15.16:0/2226647942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff840109df0 0x7ff8401a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.706+0000 7ff848000640 1 --2- 172.21.15.16:0/2226647942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8401094a0 0x7ff84019cbd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.706+0000 7ff848000640 1 --2- 172.21.15.16:0/2226647942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840103b60 0x7ff84019c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.706+0000 7ff848000640 1 -- 172.21.15.16:0/2226647942 >> 172.21.15.16:0/2226647942 conn(0x7ff84007b850 msgr2=0x7ff8401072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.706+0000 7ff848000640 1 -- 172.21.15.16:0/2226647942 shutdown_connections 2024-09-21T03:45:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:27.707+0000 7ff848000640 1 -- 172.21.15.16:0/2226647942 wait complete. 2024-09-21T03:45:27.730 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:27 smithi016 ceph-mon[30375]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-21T03:45:27.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:27 smithi134 ceph-mon[35031]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-21T03:45:27.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:27 smithi123 ceph-mon[34673]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-21T03:45:28.589 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:28 smithi016 ceph-mon[30375]: pgmap v393: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:45:28.630 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T03:45:28.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:28 smithi134 ceph-mon[35031]: pgmap v393: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:45:28.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:28 smithi123 ceph-mon[34673]: pgmap v393: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:45:29.017 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:29.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.648+0000 7fc2d1200640 1 -- 172.21.15.16:0/3098895386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2cc109c40 msgr2=0x7fc2cc110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.648+0000 7fc2d1200640 1 --2- 172.21.15.16:0/3098895386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2cc109c40 0x7fc2cc110470 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009ca0 tx=0x7fc2c402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.648+0000 7fc2d1200640 1 -- 172.21.15.16:0/3098895386 shutdown_connections 2024-09-21T03:45:29.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.648+0000 7fc2d1200640 1 --2- 172.21.15.16:0/3098895386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2cc109c40 0x7fc2cc110470 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.648+0000 7fc2d1200640 1 --2- 172.21.15.16:0/3098895386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2cc1092f0 0x7fc2cc109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.648+0000 7fc2d1200640 1 --2- 172.21.15.16:0/3098895386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2cc1039b0 0x7fc2cc103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.648+0000 7fc2d1200640 1 -- 172.21.15.16:0/3098895386 >> 172.21.15.16:0/3098895386 conn(0x7fc2cc0ff440 msgr2=0x7fc2cc101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:29.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.648+0000 7fc2d1200640 1 -- 172.21.15.16:0/3098895386 shutdown_connections 2024-09-21T03:45:29.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.648+0000 7fc2d1200640 1 -- 172.21.15.16:0/3098895386 wait complete. 2024-09-21T03:45:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.649+0000 7fc2d1200640 1 Processor -- start 2024-09-21T03:45:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.650+0000 7fc2d1200640 1 -- start start 2024-09-21T03:45:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.650+0000 7fc2d1200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2cc1039b0 0x7fc2cc19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.650+0000 7fc2d1200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2cc1092f0 0x7fc2cc19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.650+0000 7fc2d1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2cc109c40 0x7fc2cc19fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.650+0000 7fc2d1200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc113950 con 0x7fc2cc109c40 2024-09-21T03:45:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.650+0000 7fc2d1200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1137d0 con 0x7fc2cc1039b0 2024-09-21T03:45:29.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.650+0000 7fc2d1200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc113ad0 con 0x7fc2cc1092f0 2024-09-21T03:45:29.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.650+0000 7fc2c3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2cc1092f0 0x7fc2cc19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.650+0000 7fc2c3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2cc1092f0 0x7fc2cc19a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:54202/0 (socket says 172.21.15.16:54202) 2024-09-21T03:45:29.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.650+0000 7fc2c3e00640 1 -- 172.21.15.16:0/1116622773 learned_addr learned my addr 172.21.15.16:0/1116622773 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:29.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.650+0000 7fc2cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2cc109c40 0x7fc2cc19fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.650+0000 7fc2cb400640 1 --2- 172.21.15.16:0/1116622773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2cc1039b0 0x7fc2cc19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.651+0000 7fc2c3e00640 1 -- 172.21.15.16:0/1116622773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2cc1039b0 msgr2=0x7fc2cc19a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.651+0000 7fc2c3e00640 1 --2- 172.21.15.16:0/1116622773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2cc1039b0 0x7fc2cc19a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.651+0000 7fc2c3e00640 1 -- 172.21.15.16:0/1116622773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2cc109c40 msgr2=0x7fc2cc19fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.651+0000 7fc2c3e00640 1 --2- 172.21.15.16:0/1116622773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2cc109c40 0x7fc2cc19fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.651+0000 7fc2c3e00640 1 -- 172.21.15.16:0/1116622773 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc1a0250 con 0x7fc2cc1092f0 2024-09-21T03:45:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.651+0000 7fc2c3e00640 1 --2- 172.21.15.16:0/1116622773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2cc1092f0 0x7fc2cc19a880 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400e9f0 tx=0x7fc2b400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.652+0000 7fc2c8c00640 1 -- 172.21.15.16:0/1116622773 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b400ccc0 con 0x7fc2cc1092f0 2024-09-21T03:45:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.652+0000 7fc2d1200640 1 -- 172.21.15.16:0/1116622773 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2cc1a0540 con 0x7fc2cc1092f0 2024-09-21T03:45:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.652+0000 7fc2c8c00640 1 -- 172.21.15.16:0/1116622773 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2b40108e0 con 0x7fc2cc1092f0 2024-09-21T03:45:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.652+0000 7fc2c8c00640 1 -- 172.21.15.16:0/1116622773 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4018860 con 0x7fc2cc1092f0 2024-09-21T03:45:29.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.652+0000 7fc2d1200640 1 -- 172.21.15.16:0/1116622773 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2cc1a6ca0 con 0x7fc2cc1092f0 2024-09-21T03:45:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.655+0000 7fc2d9479740 1 Processor -- start 2024-09-21T03:45:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.655+0000 7fc2d9479740 1 -- start start 2024-09-21T03:45:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.655+0000 7fc2d9479740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 0x56516f7a9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.655+0000 7fc2d9479740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 0x56516f7a9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.655+0000 7fc2d9479740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 0x56516f7aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.655+0000 7fc2d9479740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56516f720d50 con 0x56516f3e2280 2024-09-21T03:45:29.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.655+0000 7fc2d9479740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56516f710180 con 0x56516f3da0a0 2024-09-21T03:45:29.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.655+0000 7fc2d9479740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56516f713f90 con 0x56516f402f60 2024-09-21T03:45:29.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.656+0000 7fc2c8c00640 1 -- 172.21.15.16:0/1116622773 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc2b4010430 con 0x7fc2cc1092f0 2024-09-21T03:45:29.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.656+0000 7fc2cbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 0x56516f7a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.656+0000 7fc2cb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 0x56516f7a9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.656+0000 7fc2cbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 0x56516f7a9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:54214/0 (socket says 172.21.15.16:54214) 2024-09-21T03:45:29.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.656+0000 7fc2cbe00640 1 -- 172.21.15.16:0/1192202428 learned_addr learned my addr 172.21.15.16:0/1192202428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:29.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.656+0000 7fc2c3e00640 1 --2- 172.21.15.16:0/1192202428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 0x56516f7aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.656+0000 7fc2cbe00640 1 -- 172.21.15.16:0/1192202428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 msgr2=0x56516f7a9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:29.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.656+0000 7fc2cbe00640 1 --2- 172.21.15.16:0/1192202428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 0x56516f7a9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.656+0000 7fc2cbe00640 1 -- 172.21.15.16:0/1192202428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 msgr2=0x56516f7aa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:29.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.656+0000 7fc2cbe00640 1 --2- 172.21.15.16:0/1192202428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 0x56516f7aa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.656+0000 7fc2cbe00640 1 -- 172.21.15.16:0/1192202428 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56516f6daa70 con 0x56516f402f60 2024-09-21T03:45:29.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.656+0000 7fc2cb400640 1 --2- 172.21.15.16:0/1192202428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 0x56516f7a9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:29.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.657+0000 7fc2cbe00640 1 --2- 172.21.15.16:0/1192202428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 0x56516f7a9670 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009ca0 tx=0x7fc2c4009430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49fa52347a6df328 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:29.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.657+0000 7fc2c0c00640 1 -- 172.21.15.16:0/1192202428 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4031cf0 con 0x56516f402f60 2024-09-21T03:45:29.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.657+0000 7fc2c0c00640 1 -- 172.21.15.16:0/1192202428 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c4036030 con 0x56516f402f60 2024-09-21T03:45:29.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.657+0000 7fc2c0c00640 1 -- 172.21.15.16:0/1192202428 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c40042a0 con 0x56516f402f60 2024-09-21T03:45:29.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.658+0000 7fc2d9479740 1 -- 172.21.15.16:0/1192202428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 msgr2=0x56516f7a9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.658+0000 7fc2d9479740 1 --2- 172.21.15.16:0/1192202428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 0x56516f7a9670 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009ca0 tx=0x7fc2c4009430 comp rx=0 tx=0).stop 2024-09-21T03:45:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.658+0000 7fc2d9479740 1 -- 172.21.15.16:0/1192202428 shutdown_connections 2024-09-21T03:45:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.658+0000 7fc2d9479740 1 --2- 172.21.15.16:0/1192202428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 0x56516f7aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.658+0000 7fc2d9479740 1 --2- 172.21.15.16:0/1192202428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 0x56516f7a9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.658+0000 7fc2d9479740 1 --2- 172.21.15.16:0/1192202428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 0x56516f7a9670 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.658+0000 7fc2d9479740 1 -- 172.21.15.16:0/1192202428 >> 172.21.15.16:0/1192202428 conn(0x56516f3f7350 msgr2=0x56516f7a7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.658+0000 7fc2d9479740 1 -- 172.21.15.16:0/1192202428 shutdown_connections 2024-09-21T03:45:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.658+0000 7fc2c8c00640 1 --2- 172.21.15.16:0/1116622773 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc2b00b6430 0x7fc2b00b8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.658+0000 7fc2d9479740 1 -- 172.21.15.16:0/1192202428 wait complete. 2024-09-21T03:45:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.658+0000 7fc2c8c00640 1 -- 172.21.15.16:0/1116622773 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fc2b40e7970 con 0x7fc2cc1092f0 2024-09-21T03:45:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.659+0000 7fc2cb400640 1 --2- 172.21.15.16:0/1116622773 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc2b00b6430 0x7fc2b00b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.659+0000 7fc2d9479740 1 -- start start 2024-09-21T03:45:29.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.659+0000 7fc2d9479740 1 Processor -- start 2024-09-21T03:45:29.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.659+0000 7fc2cb400640 1 --2- 172.21.15.16:0/1116622773 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc2b00b6430 0x7fc2b00b8940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x56516f7a82f0 tx=0x7fc2bc005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:29.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.660+0000 7fc2d9479740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f3da0a0 0x56516f7b9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.660+0000 7fc2d9479740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3e2280 0x56516f7b9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.660+0000 7fc2d9479740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f402f60 0x56516f7c5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.660+0000 7fc2d9479740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56516f5715f0 con 0x56516f402f60 2024-09-21T03:45:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.660+0000 7fc2d9479740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56516f6ab460 con 0x56516f3e2280 2024-09-21T03:45:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.660+0000 7fc2d9479740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56516f7bbfd0 con 0x56516f3da0a0 2024-09-21T03:45:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.660+0000 7fc2d9479740 10 client.0 _ll_register_callbacks cb 0x56516f6bb210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.660+0000 7fc2c3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3e2280 0x56516f7b9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.660+0000 7fc2c3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3e2280 0x56516f7b9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:58558/0 (socket says 172.21.15.16:58558) 2024-09-21T03:45:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.660+0000 7fc2cb400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f3da0a0 0x56516f7b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.660+0000 7fc2c3e00640 1 -- 172.21.15.16:0/1248008616 learned_addr learned my addr 172.21.15.16:0/1248008616 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.660+0000 7fc2cbe00640 1 --2- 172.21.15.16:0/1248008616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f402f60 0x56516f7c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.661+0000 7fc2c3e00640 1 -- 172.21.15.16:0/1248008616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f3da0a0 msgr2=0x56516f7b9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.661+0000 7fc2c3e00640 1 --2- 172.21.15.16:0/1248008616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f3da0a0 0x56516f7b9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.661+0000 7fc2c3e00640 1 -- 172.21.15.16:0/1248008616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f402f60 msgr2=0x56516f7c5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.661+0000 7fc2c3e00640 1 --2- 172.21.15.16:0/1248008616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f402f60 0x56516f7c5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.661+0000 7fc2c3e00640 1 -- 172.21.15.16:0/1248008616 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56516f46e540 con 0x56516f3e2280 2024-09-21T03:45:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.661+0000 7fc2c3e00640 1 --2- 172.21.15.16:0/1248008616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3e2280 0x56516f7b9b70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc2b40ad4a0 tx=0x7fc2b40ad630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=628d14fbae867088 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.661+0000 7fc2b9600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:45:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.661+0000 7fc2b9600640 1 -- 172.21.15.16:0/1248008616 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4010720 con 0x56516f3e2280 2024-09-21T03:45:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.661+0000 7fc2d9479740 1 -- 172.21.15.16:0/1248008616 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56516f6ada70 con 0x56516f3e2280 2024-09-21T03:45:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.661+0000 7fc2b9600640 1 -- 172.21.15.16:0/1248008616 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fc2b40ac470 con 0x56516f3e2280 2024-09-21T03:45:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.661+0000 7fc2b9600640 1 -- 172.21.15.16:0/1248008616 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc2b40d4020 con 0x56516f3e2280 2024-09-21T03:45:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.661+0000 7fc2d9479740 10 client.25202 fetch_fsmap learned FSMap version 29 2024-09-21T03:45:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.661+0000 7fc2d9479740 1 -- 172.21.15.16:0/1248008616 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56516f6c86f0 con 0x56516f3e2280 2024-09-21T03:45:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.662+0000 7fc2b9600640 1 -- 172.21.15.16:0/1248008616 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56516f6ada70 con 0x56516f3e2280 2024-09-21T03:45:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.662+0000 7fc2b9600640 1 -- 172.21.15.16:0/1248008616 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc2b40ac670 con 0x56516f3e2280 2024-09-21T03:45:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.662+0000 7fc2d9479740 10 client.25202 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:45:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.662+0000 7fc2d9479740 10 client.25202 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.662+0000 7fc2d9479740 20 client.25202 populate_metadata read hostname 'smithi016' 2024-09-21T03:45:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.662+0000 7fc2d9479740 1 --2- 172.21.15.16:0/1248008616 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x56516f3c18e0 0x56516f7cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.662+0000 7fc2d9479740 4 client.25202 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.662+0000 7fc2d9479740 1 -- 172.21.15.16:0/1248008616 --> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56516f722ca0 con 0x56516f3c18e0 2024-09-21T03:45:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.663+0000 7fc2cb400640 1 --2- 172.21.15.16:0/1248008616 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x56516f3c18e0 0x56516f7cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.663+0000 7fc2cb400640 1 --2- 172.21.15.16:0/1248008616 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x56516f3c18e0 0x56516f7cb8a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b81609e77d731d6f server_cookie=276f994881b31435 in_seq=0 out_seq=0 2024-09-21T03:45:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.663+0000 7fc2b9600640 10 client.25202 ms_handle_connect on v2:172.21.15.134:6836/2624530453 2024-09-21T03:45:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.664+0000 7fc2b9600640 1 -- 172.21.15.16:0/1248008616 <== mds.3 v2:172.21.15.134:6836/2624530453 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56516f722ca0 con 0x56516f3c18e0 2024-09-21T03:45:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.664+0000 7fc2b9600640 10 client.25202 handle_command_reply: tid=0 2024-09-21T03:45:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.664+0000 7fc2d9479740 1 client.25202 shutdown 2024-09-21T03:45:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.664+0000 7fc2d9479740 1 -- 172.21.15.16:0/1248008616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3e2280 msgr2=0x56516f7b9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.664+0000 7fc2d9479740 1 --2- 172.21.15.16:0/1248008616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3e2280 0x56516f7b9b70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc2b40ad4a0 tx=0x7fc2b40ad630 comp rx=0 tx=0).stop 2024-09-21T03:45:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.664+0000 7fc2d9479740 1 -- 172.21.15.16:0/1248008616 shutdown_connections 2024-09-21T03:45:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.664+0000 7fc2d9479740 1 --2- 172.21.15.16:0/1248008616 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x56516f3c18e0 0x56516f7cb8a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.664+0000 7fc2d9479740 1 --2- 172.21.15.16:0/1248008616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f402f60 0x56516f7c5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.664+0000 7fc2d9479740 1 --2- 172.21.15.16:0/1248008616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3e2280 0x56516f7b9b70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.664+0000 7fc2d9479740 1 --2- 172.21.15.16:0/1248008616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f3da0a0 0x56516f7b9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.664+0000 7fc2b9600640 0 client.25202 ms_handle_reset on v2:172.21.15.134:6836/2624530453 2024-09-21T03:45:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.664+0000 7fc2d9479740 1 -- 172.21.15.16:0/1248008616 >> 172.21.15.16:0/1248008616 conn(0x56516f3f7350 msgr2=0x56516f7a5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.664+0000 7fc2d9479740 1 -- 172.21.15.16:0/1248008616 shutdown_connections 2024-09-21T03:45:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.665+0000 7fc2d9479740 1 -- 172.21.15.16:0/1248008616 wait complete. 2024-09-21T03:45:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.665+0000 7fc2d9479740 20 client.25202 trim_cache size 0 max 16384 2024-09-21T03:45:29.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.684+0000 7fc2d9479740 1 Processor -- start 2024-09-21T03:45:29.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.684+0000 7fc2d9479740 1 -- start start 2024-09-21T03:45:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.685+0000 7fc2d9479740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 0x56516f7cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.685+0000 7fc2d9479740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 0x56516f7c5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.685+0000 7fc2d9479740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 0x56516f7c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.685+0000 7fc2d9479740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56516f7c9be0 con 0x56516f3e2280 2024-09-21T03:45:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.685+0000 7fc2d9479740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56516f7ca2b0 con 0x56516f3da0a0 2024-09-21T03:45:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.685+0000 7fc2d9479740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56516f7ca980 con 0x56516f402f60 2024-09-21T03:45:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.685+0000 7fc2c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 0x56516f7c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.685+0000 7fc2c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 0x56516f7c5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58498/0 (socket says 172.21.15.16:58498) 2024-09-21T03:45:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.685+0000 7fc2c3e00640 1 -- 172.21.15.16:0/1174351094 learned_addr learned my addr 172.21.15.16:0/1174351094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.685+0000 7fc2cb400640 1 --2- 172.21.15.16:0/1174351094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 0x56516f7cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.685+0000 7fc2cbe00640 1 --2- 172.21.15.16:0/1174351094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 0x56516f7c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.685+0000 7fc2c3e00640 1 -- 172.21.15.16:0/1174351094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 msgr2=0x56516f7c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:29.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.685+0000 7fc2c3e00640 1 --2- 172.21.15.16:0/1174351094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 0x56516f7c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.685+0000 7fc2c3e00640 1 -- 172.21.15.16:0/1174351094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 msgr2=0x56516f7cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.685+0000 7fc2c3e00640 1 --2- 172.21.15.16:0/1174351094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 0x56516f7cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.685+0000 7fc2c3e00640 1 -- 172.21.15.16:0/1174351094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56516f7b8c50 con 0x56516f3e2280 2024-09-21T03:45:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.686+0000 7fc2cb400640 1 --2- 172.21.15.16:0/1174351094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 0x56516f7cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.686+0000 7fc2c3e00640 1 --2- 172.21.15.16:0/1174351094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 0x56516f7c5270 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc2b40ad730 tx=0x7fc2b40a9df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd4e4377c52bcbe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.686+0000 7fc2bbe00640 1 -- 172.21.15.16:0/1174351094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b40ad2b0 con 0x56516f3e2280 2024-09-21T03:45:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.686+0000 7fc2bbe00640 1 -- 172.21.15.16:0/1174351094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2b40b7110 con 0x56516f3e2280 2024-09-21T03:45:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.686+0000 7fc2bbe00640 1 -- 172.21.15.16:0/1174351094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b40ac780 con 0x56516f3e2280 2024-09-21T03:45:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.687+0000 7fc2d9479740 1 -- 172.21.15.16:0/1174351094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 msgr2=0x56516f7c5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.687+0000 7fc2d9479740 1 --2- 172.21.15.16:0/1174351094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 0x56516f7c5270 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc2b40ad730 tx=0x7fc2b40a9df0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.687+0000 7fc2d9479740 1 -- 172.21.15.16:0/1174351094 shutdown_connections 2024-09-21T03:45:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.687+0000 7fc2d9479740 1 --2- 172.21.15.16:0/1174351094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 0x56516f7c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.687+0000 7fc2d9479740 1 --2- 172.21.15.16:0/1174351094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 0x56516f7c5270 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.687+0000 7fc2d9479740 1 --2- 172.21.15.16:0/1174351094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 0x56516f7cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.687+0000 7fc2d9479740 1 -- 172.21.15.16:0/1174351094 >> 172.21.15.16:0/1174351094 conn(0x56516f3f7350 msgr2=0x56516f7b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.687+0000 7fc2d9479740 1 -- 172.21.15.16:0/1174351094 shutdown_connections 2024-09-21T03:45:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.687+0000 7fc2d9479740 1 -- 172.21.15.16:0/1174351094 wait complete. 2024-09-21T03:45:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.688+0000 7fc2d9479740 1 -- start start 2024-09-21T03:45:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.688+0000 7fc2d9479740 1 Processor -- start 2024-09-21T03:45:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.688+0000 7fc2d9479740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 0x56516f7cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.688+0000 7fc2d9479740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 0x56516f7ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.688+0000 7fc2d9479740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 0x56516f7ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.688+0000 7fc2d9479740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56516f7cb720 con 0x56516f3e2280 2024-09-21T03:45:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.688+0000 7fc2d9479740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56516f7cb050 con 0x56516f3da0a0 2024-09-21T03:45:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.688+0000 7fc2d9479740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56516f7cbdf0 con 0x56516f402f60 2024-09-21T03:45:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.688+0000 7fc2d9479740 10 client.0 _ll_register_callbacks cb 0x7fc2b40a9640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.689+0000 7fc2c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 0x56516f7ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.689+0000 7fc2c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 0x56516f7ce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58504/0 (socket says 172.21.15.16:58504) 2024-09-21T03:45:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.689+0000 7fc2c3e00640 1 -- 172.21.15.16:0/4287138502 learned_addr learned my addr 172.21.15.16:0/4287138502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.689+0000 7fc2cbe00640 1 --2- 172.21.15.16:0/4287138502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 0x56516f7ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.689+0000 7fc2cb400640 1 --2- 172.21.15.16:0/4287138502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 0x56516f7cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.689+0000 7fc2c3e00640 1 -- 172.21.15.16:0/4287138502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 msgr2=0x56516f7ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.689+0000 7fc2c3e00640 1 --2- 172.21.15.16:0/4287138502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 0x56516f7ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.689+0000 7fc2c3e00640 1 -- 172.21.15.16:0/4287138502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 msgr2=0x56516f7cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.689+0000 7fc2c3e00640 1 --2- 172.21.15.16:0/4287138502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 0x56516f7cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.689+0000 7fc2c3e00640 1 -- 172.21.15.16:0/4287138502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56516f7d3350 con 0x56516f3e2280 2024-09-21T03:45:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.689+0000 7fc2cb400640 1 --2- 172.21.15.16:0/4287138502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 0x56516f7cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.689+0000 7fc2cbe00640 1 --2- 172.21.15.16:0/4287138502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 0x56516f7ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.690+0000 7fc2c3e00640 1 --2- 172.21.15.16:0/4287138502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 0x56516f7ce0b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc2b40ad450 tx=0x7fc2b40b6460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623d3def3c66e119 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.690+0000 7fc2b9600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:45:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.690+0000 7fc2b9600640 1 -- 172.21.15.16:0/4287138502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b40ad2b0 con 0x56516f3e2280 2024-09-21T03:45:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.690+0000 7fc2b9600640 1 -- 172.21.15.16:0/4287138502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fc2b40ac4e0 con 0x56516f3e2280 2024-09-21T03:45:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.690+0000 7fc2d9479740 1 -- 172.21.15.16:0/4287138502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2b40b7110 con 0x56516f3e2280 2024-09-21T03:45:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.690+0000 7fc2b9600640 1 -- 172.21.15.16:0/4287138502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc2b40b21d0 con 0x56516f3e2280 2024-09-21T03:45:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.690+0000 7fc2d9479740 10 client.15195 fetch_fsmap learned FSMap version 29 2024-09-21T03:45:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.690+0000 7fc2d9479740 1 -- 172.21.15.16:0/4287138502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56516f7d3350 con 0x56516f3e2280 2024-09-21T03:45:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.690+0000 7fc2b9600640 1 -- 172.21.15.16:0/4287138502 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2b40b7110 con 0x56516f3e2280 2024-09-21T03:45:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.691+0000 7fc2b9600640 1 -- 172.21.15.16:0/4287138502 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc2b40cf150 con 0x56516f3e2280 2024-09-21T03:45:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.691+0000 7fc2d9479740 10 client.15195 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:45:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.691+0000 7fc2d9479740 10 client.15195 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:45:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.691+0000 7fc2d9479740 20 client.15195 populate_metadata read hostname 'smithi016' 2024-09-21T03:45:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.691+0000 7fc2d9479740 1 --2- 172.21.15.16:0/4287138502 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x56516f396c90 0x56516f83a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.691+0000 7fc2d9479740 4 client.15195 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:45:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.691+0000 7fc2d9479740 1 -- 172.21.15.16:0/4287138502 --> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] -- command(tid 0: {"prefix": "status"}) -- 0x56516f71e750 con 0x56516f396c90 2024-09-21T03:45:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.691+0000 7fc2cb400640 1 --2- 172.21.15.16:0/4287138502 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x56516f396c90 0x56516f83a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.692+0000 7fc2cb400640 1 --2- 172.21.15.16:0/4287138502 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x56516f396c90 0x56516f83a130 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=be2de19773b68c8f server_cookie=e9ed1b0b736475e9 in_seq=0 out_seq=0 2024-09-21T03:45:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.692+0000 7fc2b9600640 10 client.15195 ms_handle_connect on v2:172.21.15.134:6836/2624530453 2024-09-21T03:45:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.692+0000 7fc2b9600640 1 -- 172.21.15.16:0/4287138502 <== mds.3 v2:172.21.15.134:6836/2624530453 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56516f71e750 con 0x56516f396c90 2024-09-21T03:45:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.692+0000 7fc2b9600640 10 client.15195 handle_command_reply: tid=0 2024-09-21T03:45:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.692+0000 7fc2d9479740 1 client.15195 shutdown 2024-09-21T03:45:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.692+0000 7fc2d9479740 1 -- 172.21.15.16:0/4287138502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 msgr2=0x56516f7ce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.692+0000 7fc2d9479740 1 --2- 172.21.15.16:0/4287138502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 0x56516f7ce0b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc2b40ad450 tx=0x7fc2b40b6460 comp rx=0 tx=0).stop 2024-09-21T03:45:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.692+0000 7fc2d9479740 1 -- 172.21.15.16:0/4287138502 shutdown_connections 2024-09-21T03:45:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.692+0000 7fc2d9479740 1 --2- 172.21.15.16:0/4287138502 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x56516f396c90 0x56516f83a130 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.692+0000 7fc2d9479740 1 --2- 172.21.15.16:0/4287138502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56516f402f60 0x56516f7ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.692+0000 7fc2d9479740 1 --2- 172.21.15.16:0/4287138502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56516f3e2280 0x56516f7ce0b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.692+0000 7fc2d9479740 1 --2- 172.21.15.16:0/4287138502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56516f3da0a0 0x56516f7cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.692+0000 7fc2d9479740 1 -- 172.21.15.16:0/4287138502 >> 172.21.15.16:0/4287138502 conn(0x56516f3f7350 msgr2=0x56516f7ae810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.692+0000 7fc2b9600640 0 client.15195 ms_handle_reset on v2:172.21.15.134:6836/2624530453 2024-09-21T03:45:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.693+0000 7fc2d9479740 1 -- 172.21.15.16:0/4287138502 shutdown_connections 2024-09-21T03:45:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.693+0000 7fc2d9479740 1 -- 172.21.15.16:0/4287138502 wait complete. 2024-09-21T03:45:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.693+0000 7fc2d9479740 20 client.15195 trim_cache size 0 max 16384 2024-09-21T03:45:29.699 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:45:29.699 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:45:29.699 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 3, 2024-09-21T03:45:29.700 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24670, 2024-09-21T03:45:29.700 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T03:45:29.700 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T03:45:29.700 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T03:45:29.700 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 85.170292404999998, 2024-09-21T03:45:29.700 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 29, 2024-09-21T03:45:29.700 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 86, 2024-09-21T03:45:29.700 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 84, 2024-09-21T03:45:29.700 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 172.33769318200001 2024-09-21T03:45:29.700 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:45:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.694+0000 7fc2d1200640 1 -- 172.21.15.16:0/1116622773 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc2b00b6430 msgr2=0x7fc2b00b8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.694+0000 7fc2d1200640 1 --2- 172.21.15.16:0/1116622773 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc2b00b6430 0x7fc2b00b8940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x56516f7a82f0 tx=0x7fc2bc005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.694+0000 7fc2d1200640 1 -- 172.21.15.16:0/1116622773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2cc1092f0 msgr2=0x7fc2cc19a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.694+0000 7fc2d1200640 1 --2- 172.21.15.16:0/1116622773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2cc1092f0 0x7fc2cc19a880 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400e9f0 tx=0x7fc2b400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.694+0000 7fc2d1200640 1 -- 172.21.15.16:0/1116622773 shutdown_connections 2024-09-21T03:45:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.694+0000 7fc2d1200640 1 --2- 172.21.15.16:0/1116622773 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc2b00b6430 0x7fc2b00b8940 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.694+0000 7fc2d1200640 1 --2- 172.21.15.16:0/1116622773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2cc109c40 0x7fc2cc19fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.694+0000 7fc2d1200640 1 --2- 172.21.15.16:0/1116622773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2cc1092f0 0x7fc2cc19a880 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.694+0000 7fc2d1200640 1 --2- 172.21.15.16:0/1116622773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2cc1039b0 0x7fc2cc19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.694+0000 7fc2d1200640 1 -- 172.21.15.16:0/1116622773 >> 172.21.15.16:0/1116622773 conn(0x7fc2cc0ff440 msgr2=0x7fc2cc108930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.694+0000 7fc2d1200640 1 -- 172.21.15.16:0/1116622773 shutdown_connections 2024-09-21T03:45:29.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:29.694+0000 7fc2d1200640 1 -- 172.21.15.16:0/1116622773 wait complete. 2024-09-21T03:45:30.531 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T03:45:30.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:30 smithi134 ceph-mon[35031]: pgmap v394: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-21T03:45:30.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:30 smithi123 ceph-mon[34673]: pgmap v394: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-21T03:45:30.911 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:30.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:30 smithi016 ceph-mon[30375]: pgmap v394: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-21T03:45:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.556+0000 7f27dce00640 1 -- 172.21.15.16:0/978014912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8114430 msgr2=0x7f27d8116900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.556+0000 7f27dce00640 1 --2- 172.21.15.16:0/978014912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8114430 0x7f27d8116900 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009ca0 tx=0x7f27d002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.557+0000 7f27dce00640 1 -- 172.21.15.16:0/978014912 shutdown_connections 2024-09-21T03:45:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.557+0000 7f27dce00640 1 --2- 172.21.15.16:0/978014912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8114430 0x7f27d8116900 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.557+0000 7f27dce00640 1 --2- 172.21.15.16:0/978014912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27d8075dd0 0x7f27d80761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.557+0000 7f27dce00640 1 --2- 172.21.15.16:0/978014912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d8077380 0x7f27d80757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.557+0000 7f27dce00640 1 -- 172.21.15.16:0/978014912 >> 172.21.15.16:0/978014912 conn(0x7f27d807b850 msgr2=0x7f27d807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.557+0000 7f27dce00640 1 -- 172.21.15.16:0/978014912 shutdown_connections 2024-09-21T03:45:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.557+0000 7f27dce00640 1 -- 172.21.15.16:0/978014912 wait complete. 2024-09-21T03:45:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.558+0000 7f27dce00640 1 Processor -- start 2024-09-21T03:45:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.558+0000 7f27dce00640 1 -- start start 2024-09-21T03:45:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.558+0000 7f27dce00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d8075dd0 0x7f27d81a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.558+0000 7f27dce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8077380 0x7f27d81a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.558+0000 7f27dce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27d8114430 0x7f27d81a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.558+0000 7f27dce00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27d811a070 con 0x7f27d8077380 2024-09-21T03:45:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.558+0000 7f27dce00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27d8119ef0 con 0x7f27d8114430 2024-09-21T03:45:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.558+0000 7f27dce00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27d811a1f0 con 0x7f27d8075dd0 2024-09-21T03:45:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.559+0000 7f27d5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8077380 0x7f27d81a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.559+0000 7f27d5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8077380 0x7f27d81a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58510/0 (socket says 172.21.15.16:58510) 2024-09-21T03:45:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.559+0000 7f27d5400640 1 -- 172.21.15.16:0/294750402 learned_addr learned my addr 172.21.15.16:0/294750402 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.559+0000 7f27d5e00640 1 --2- 172.21.15.16:0/294750402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d8075dd0 0x7f27d81a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.559+0000 7f27d6800640 1 --2- 172.21.15.16:0/294750402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27d8114430 0x7f27d81a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.559+0000 7f27d5400640 1 -- 172.21.15.16:0/294750402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d8075dd0 msgr2=0x7f27d81a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.559+0000 7f27d5400640 1 --2- 172.21.15.16:0/294750402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d8075dd0 0x7f27d81a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.559+0000 7f27d5400640 1 -- 172.21.15.16:0/294750402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27d8114430 msgr2=0x7f27d81a6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.559+0000 7f27d5400640 1 --2- 172.21.15.16:0/294750402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27d8114430 0x7f27d81a6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.559+0000 7f27d5400640 1 -- 172.21.15.16:0/294750402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d81a6950 con 0x7f27d8077380 2024-09-21T03:45:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.560+0000 7f27d5400640 1 --2- 172.21.15.16:0/294750402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8077380 0x7f27d81a0f80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f27c400e9c0 tx=0x7f27c400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.560+0000 7f27cea00640 1 -- 172.21.15.16:0/294750402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c400cd80 con 0x7f27d8077380 2024-09-21T03:45:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.560+0000 7f27cea00640 1 -- 172.21.15.16:0/294750402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c40054d0 con 0x7f27d8077380 2024-09-21T03:45:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.560+0000 7f27dce00640 1 -- 172.21.15.16:0/294750402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d81a6c40 con 0x7f27d8077380 2024-09-21T03:45:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.560+0000 7f27cea00640 1 -- 172.21.15.16:0/294750402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c4010430 con 0x7f27d8077380 2024-09-21T03:45:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.560+0000 7f27dce00640 1 -- 172.21.15.16:0/294750402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d81ad3a0 con 0x7f27d8077380 2024-09-21T03:45:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.564+0000 7f27e39ce740 1 Processor -- start 2024-09-21T03:45:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.564+0000 7f27e39ce740 1 -- start start 2024-09-21T03:45:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.565+0000 7f27e39ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df78f60 0x56144e31f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.565+0000 7f27e39ce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df500a0 0x56144e31f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.565+0000 7f27e39ce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df58280 0x56144e31ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.565+0000 7f27e39ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56144e296d50 con 0x56144df78f60 2024-09-21T03:45:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.565+0000 7f27e39ce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56144e286180 con 0x56144df58280 2024-09-21T03:45:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.565+0000 7f27e39ce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56144e289f90 con 0x56144df500a0 2024-09-21T03:45:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.565+0000 7f27d6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df78f60 0x56144e31f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.565+0000 7f27d5e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df500a0 0x56144e31f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.565+0000 7f27d6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df78f60 0x56144e31f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58526/0 (socket says 172.21.15.16:58526) 2024-09-21T03:45:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.565+0000 7f27d6800640 1 -- 172.21.15.16:0/1910618964 learned_addr learned my addr 172.21.15.16:0/1910618964 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.565+0000 7f27cea00640 1 -- 172.21.15.16:0/294750402 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f27c4004fd0 con 0x7f27d8077380 2024-09-21T03:45:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.566+0000 7f27d5e00640 1 -- 172.21.15.16:0/1910618964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df58280 msgr2=0x56144e31ff10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:45:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.566+0000 7f27d5e00640 1 --2- 172.21.15.16:0/1910618964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df58280 0x56144e31ff10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.566+0000 7f27d5e00640 1 -- 172.21.15.16:0/1910618964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df78f60 msgr2=0x56144e31f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.566+0000 7f27d5e00640 1 --2- 172.21.15.16:0/1910618964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df78f60 0x56144e31f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.566+0000 7f27d5e00640 1 -- 172.21.15.16:0/1910618964 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56144e250a70 con 0x56144df500a0 2024-09-21T03:45:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.566+0000 7f27d5e00640 1 --2- 172.21.15.16:0/1910618964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df500a0 0x56144e31f9d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009c00 tx=0x7f27c8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44024d90d65f7bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.566+0000 7f27c3400640 1 -- 172.21.15.16:0/1910618964 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8004230 con 0x56144df500a0 2024-09-21T03:45:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.566+0000 7f27c3400640 1 -- 172.21.15.16:0/1910618964 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c80043d0 con 0x56144df500a0 2024-09-21T03:45:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.566+0000 7f27c3400640 1 -- 172.21.15.16:0/1910618964 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8004c10 con 0x56144df500a0 2024-09-21T03:45:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.567+0000 7f27cea00640 1 --2- 172.21.15.16:0/294750402 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f27bc0b6590 0x7f27bc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.567+0000 7f27e39ce740 1 -- 172.21.15.16:0/1910618964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df500a0 msgr2=0x56144e31f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.567+0000 7f27e39ce740 1 --2- 172.21.15.16:0/1910618964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df500a0 0x56144e31f9d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009c00 tx=0x7f27c8011810 comp rx=0 tx=0).stop 2024-09-21T03:45:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.567+0000 7f27cea00640 1 -- 172.21.15.16:0/294750402 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f27c40e7a90 con 0x7f27d8077380 2024-09-21T03:45:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.567+0000 7f27d6800640 1 --2- 172.21.15.16:0/294750402 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f27bc0b6590 0x7f27bc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.568+0000 7f27e39ce740 1 -- 172.21.15.16:0/1910618964 shutdown_connections 2024-09-21T03:45:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.568+0000 7f27e39ce740 1 --2- 172.21.15.16:0/1910618964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df58280 0x56144e31ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.568+0000 7f27e39ce740 1 --2- 172.21.15.16:0/1910618964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df500a0 0x56144e31f9d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.568+0000 7f27e39ce740 1 --2- 172.21.15.16:0/1910618964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df78f60 0x56144e31f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.568+0000 7f27e39ce740 1 -- 172.21.15.16:0/1910618964 >> 172.21.15.16:0/1910618964 conn(0x56144df6d350 msgr2=0x56144e31d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.568+0000 7f27e39ce740 1 -- 172.21.15.16:0/1910618964 shutdown_connections 2024-09-21T03:45:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.568+0000 7f27e39ce740 1 -- 172.21.15.16:0/1910618964 wait complete. 2024-09-21T03:45:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.568+0000 7f27d6800640 1 --2- 172.21.15.16:0/294750402 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f27bc0b6590 0x7f27bc0b8aa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f27d002f500 tx=0x7f27d002f310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.568+0000 7f27e39ce740 1 -- start start 2024-09-21T03:45:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.569+0000 7f27e39ce740 1 Processor -- start 2024-09-21T03:45:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.569+0000 7f27e39ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df500a0 0x56144e32b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.569+0000 7f27e39ce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df58280 0x56144e32b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.569+0000 7f27e39ce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df78f60 0x56144e336fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.569+0000 7f27e39ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56144e0e75f0 con 0x56144df500a0 2024-09-21T03:45:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.569+0000 7f27e39ce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56144e221460 con 0x56144df78f60 2024-09-21T03:45:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.569+0000 7f27e39ce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56144e32dc70 con 0x56144df58280 2024-09-21T03:45:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.569+0000 7f27e39ce740 10 client.0 _ll_register_callbacks cb 0x56144e231210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.569+0000 7f27d5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df500a0 0x56144e32b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.570+0000 7f27d5400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df58280 0x56144e32b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.570+0000 7f27d5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df500a0 0x56144e32b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58532/0 (socket says 172.21.15.16:58532) 2024-09-21T03:45:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.570+0000 7f27d5e00640 1 -- 172.21.15.16:0/2806086999 learned_addr learned my addr 172.21.15.16:0/2806086999 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.570+0000 7f27d6800640 1 --2- 172.21.15.16:0/2806086999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df78f60 0x56144e336fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.570+0000 7f27d5e00640 1 -- 172.21.15.16:0/2806086999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df58280 msgr2=0x56144e32b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.570+0000 7f27d5e00640 1 --2- 172.21.15.16:0/2806086999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df58280 0x56144e32b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.570+0000 7f27d5e00640 1 -- 172.21.15.16:0/2806086999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df78f60 msgr2=0x56144e336fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.570+0000 7f27d5e00640 1 --2- 172.21.15.16:0/2806086999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df78f60 0x56144e336fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.570+0000 7f27d5e00640 1 -- 172.21.15.16:0/2806086999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56144dfe4540 con 0x56144df500a0 2024-09-21T03:45:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.570+0000 7f27d5e00640 1 --2- 172.21.15.16:0/2806086999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df500a0 0x56144e32b0c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f27c8003820 tx=0x7f27c8003850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5985433e91efcdfd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.570+0000 7f27b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:45:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.571+0000 7f27b7e00640 1 -- 172.21.15.16:0/2806086999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8027020 con 0x56144df500a0 2024-09-21T03:45:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.571+0000 7f27e39ce740 1 -- 172.21.15.16:0/2806086999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56144e223a70 con 0x56144df500a0 2024-09-21T03:45:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.571+0000 7f27b7e00640 1 -- 172.21.15.16:0/2806086999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f27c8003c70 con 0x56144df500a0 2024-09-21T03:45:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.571+0000 7f27b7e00640 1 -- 172.21.15.16:0/2806086999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f27c8032050 con 0x56144df500a0 2024-09-21T03:45:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.571+0000 7f27e39ce740 10 client.15207 fetch_fsmap learned FSMap version 29 2024-09-21T03:45:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.571+0000 7f27e39ce740 1 -- 172.21.15.16:0/2806086999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56144e23e6f0 con 0x56144df500a0 2024-09-21T03:45:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.571+0000 7f27b7e00640 1 -- 172.21.15.16:0/2806086999 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56144e223a70 con 0x56144df500a0 2024-09-21T03:45:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.572+0000 7f27b7e00640 1 -- 172.21.15.16:0/2806086999 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f27c8018070 con 0x56144df500a0 2024-09-21T03:45:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.572+0000 7f27e39ce740 10 client.15207 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:45:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.572+0000 7f27e39ce740 10 client.15207 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:45:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.572+0000 7f27e39ce740 20 client.15207 populate_metadata read hostname 'smithi016' 2024-09-21T03:45:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.572+0000 7f27e39ce740 1 --2- 172.21.15.16:0/2806086999 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x56144df378e0 0x56144e33d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.572+0000 7f27e39ce740 4 client.15207 mds_command: new command op to 24682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.572+0000 7f27e39ce740 1 -- 172.21.15.16:0/2806086999 --> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56144e298ca0 con 0x56144df378e0 2024-09-21T03:45:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.572+0000 7f27d5400640 1 --2- 172.21.15.16:0/2806086999 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x56144df378e0 0x56144e33d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.573+0000 7f27d5400640 1 --2- 172.21.15.16:0/2806086999 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x56144df378e0 0x56144e33d3b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d96523c4188d1223 server_cookie=96d093ffd4092e3e in_seq=0 out_seq=0 2024-09-21T03:45:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.573+0000 7f27b7e00640 10 client.15207 ms_handle_connect on v2:172.21.15.134:6838/606193011 2024-09-21T03:45:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.574+0000 7f27b7e00640 1 -- 172.21.15.16:0/2806086999 <== mds.2 v2:172.21.15.134:6838/606193011 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56144e298ca0 con 0x56144df378e0 2024-09-21T03:45:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.574+0000 7f27b7e00640 10 client.15207 handle_command_reply: tid=0 2024-09-21T03:45:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.574+0000 7f27e39ce740 1 client.15207 shutdown 2024-09-21T03:45:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.574+0000 7f27e39ce740 1 -- 172.21.15.16:0/2806086999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df500a0 msgr2=0x56144e32b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.574+0000 7f27e39ce740 1 --2- 172.21.15.16:0/2806086999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df500a0 0x56144e32b0c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f27c8003820 tx=0x7f27c8003850 comp rx=0 tx=0).stop 2024-09-21T03:45:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.574+0000 7f27e39ce740 1 -- 172.21.15.16:0/2806086999 shutdown_connections 2024-09-21T03:45:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.574+0000 7f27e39ce740 1 --2- 172.21.15.16:0/2806086999 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x56144df378e0 0x56144e33d3b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.574+0000 7f27e39ce740 1 --2- 172.21.15.16:0/2806086999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df78f60 0x56144e336fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.574+0000 7f27e39ce740 1 --2- 172.21.15.16:0/2806086999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df58280 0x56144e32b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.574+0000 7f27e39ce740 1 --2- 172.21.15.16:0/2806086999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df500a0 0x56144e32b0c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.574+0000 7f27e39ce740 1 -- 172.21.15.16:0/2806086999 >> 172.21.15.16:0/2806086999 conn(0x56144df6d350 msgr2=0x56144e322670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.574+0000 7f27b7e00640 0 client.15207 ms_handle_reset on v2:172.21.15.134:6838/606193011 2024-09-21T03:45:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.574+0000 7f27e39ce740 1 -- 172.21.15.16:0/2806086999 shutdown_connections 2024-09-21T03:45:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.574+0000 7f27e39ce740 1 -- 172.21.15.16:0/2806086999 wait complete. 2024-09-21T03:45:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.574+0000 7f27e39ce740 20 client.15207 trim_cache size 0 max 16384 2024-09-21T03:45:31.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.593+0000 7f27e39ce740 1 Processor -- start 2024-09-21T03:45:31.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.594+0000 7f27e39ce740 1 -- start start 2024-09-21T03:45:31.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.594+0000 7f27e39ce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df500a0 0x56144e33bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.594+0000 7f27e39ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df58280 0x56144e329740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.594+0000 7f27e39ce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df78f60 0x56144e329c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.594+0000 7f27e39ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56144e33b6f0 con 0x56144df58280 2024-09-21T03:45:31.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.594+0000 7f27e39ce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56144e33bdc0 con 0x56144df500a0 2024-09-21T03:45:31.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.594+0000 7f27e39ce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56144e33c490 con 0x56144df78f60 2024-09-21T03:45:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.594+0000 7f27d5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df58280 0x56144e329740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.594+0000 7f27d6800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df78f60 0x56144e329c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.594+0000 7f27d6800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df78f60 0x56144e329c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:54276/0 (socket says 172.21.15.16:54276) 2024-09-21T03:45:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.594+0000 7f27d5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df58280 0x56144e329740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58538/0 (socket says 172.21.15.16:58538) 2024-09-21T03:45:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.594+0000 7f27d6800640 1 -- 172.21.15.16:0/1449616616 learned_addr learned my addr 172.21.15.16:0/1449616616 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.594+0000 7f27d5e00640 1 --2- 172.21.15.16:0/1449616616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df500a0 0x56144e33bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.595+0000 7f27d5400640 1 -- 172.21.15.16:0/1449616616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df78f60 msgr2=0x56144e329c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.595+0000 7f27d5400640 1 --2- 172.21.15.16:0/1449616616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df78f60 0x56144e329c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.595+0000 7f27d5400640 1 -- 172.21.15.16:0/1449616616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df500a0 msgr2=0x56144e33bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.595+0000 7f27d5400640 1 --2- 172.21.15.16:0/1449616616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df500a0 0x56144e33bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.595+0000 7f27d5400640 1 -- 172.21.15.16:0/1449616616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56144e32c180 con 0x56144df58280 2024-09-21T03:45:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.595+0000 7f27d5400640 1 --2- 172.21.15.16:0/1449616616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df58280 0x56144e329740 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f27c400c6a0 tx=0x7f27c40b2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cce3288d7a2efb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.595+0000 7f27c2a00640 1 -- 172.21.15.16:0/1449616616 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c40ad320 con 0x56144df58280 2024-09-21T03:45:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.595+0000 7f27c2a00640 1 -- 172.21.15.16:0/1449616616 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c40adf10 con 0x56144df58280 2024-09-21T03:45:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.595+0000 7f27c2a00640 1 -- 172.21.15.16:0/1449616616 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c40a9a80 con 0x56144df58280 2024-09-21T03:45:31.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.596+0000 7f27e39ce740 1 -- 172.21.15.16:0/1449616616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df58280 msgr2=0x56144e329740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.596+0000 7f27e39ce740 1 --2- 172.21.15.16:0/1449616616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df58280 0x56144e329740 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f27c400c6a0 tx=0x7f27c40b2040 comp rx=0 tx=0).stop 2024-09-21T03:45:31.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.596+0000 7f27e39ce740 1 -- 172.21.15.16:0/1449616616 shutdown_connections 2024-09-21T03:45:31.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.596+0000 7f27e39ce740 1 --2- 172.21.15.16:0/1449616616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df78f60 0x56144e329c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.596+0000 7f27e39ce740 1 --2- 172.21.15.16:0/1449616616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df58280 0x56144e329740 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.596+0000 7f27e39ce740 1 --2- 172.21.15.16:0/1449616616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df500a0 0x56144e33bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.596+0000 7f27e39ce740 1 -- 172.21.15.16:0/1449616616 >> 172.21.15.16:0/1449616616 conn(0x56144df6d350 msgr2=0x56144e326070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:31.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.596+0000 7f27e39ce740 1 -- 172.21.15.16:0/1449616616 shutdown_connections 2024-09-21T03:45:31.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.596+0000 7f27e39ce740 1 -- 172.21.15.16:0/1449616616 wait complete. 2024-09-21T03:45:31.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.597+0000 7f27e39ce740 1 -- start start 2024-09-21T03:45:31.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.597+0000 7f27e39ce740 1 Processor -- start 2024-09-21T03:45:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27e39ce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df500a0 0x56144e33eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27e39ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df58280 0x56144e33f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27e39ce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df78f60 0x56144e3443a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27e39ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56144e33d230 con 0x56144df58280 2024-09-21T03:45:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27e39ce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56144e33cb60 con 0x56144df500a0 2024-09-21T03:45:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27e39ce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56144e33d900 con 0x56144df78f60 2024-09-21T03:45:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27e39ce740 10 client.0 _ll_register_callbacks cb 0x7f27c8004450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27d5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df58280 0x56144e33f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27d5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df500a0 0x56144e33eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27d5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df58280 0x56144e33f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58550/0 (socket says 172.21.15.16:58550) 2024-09-21T03:45:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27d5400640 1 -- 172.21.15.16:0/1413735370 learned_addr learned my addr 172.21.15.16:0/1413735370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27d6800640 1 --2- 172.21.15.16:0/1413735370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df78f60 0x56144e3443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27d5400640 1 -- 172.21.15.16:0/1413735370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df78f60 msgr2=0x56144e3443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27d5400640 1 --2- 172.21.15.16:0/1413735370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df78f60 0x56144e3443a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27d5400640 1 -- 172.21.15.16:0/1413735370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df500a0 msgr2=0x56144e33eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27d5400640 1 --2- 172.21.15.16:0/1413735370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df500a0 0x56144e33eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27d5400640 1 -- 172.21.15.16:0/1413735370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56144e344a10 con 0x56144df58280 2024-09-21T03:45:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.598+0000 7f27d5e00640 1 --2- 172.21.15.16:0/1413735370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df500a0 0x56144e33eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.599+0000 7f27d5400640 1 --2- 172.21.15.16:0/1413735370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df58280 0x56144e33f360 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f27c400cfd0 tx=0x7f27c400efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3f24a270007e8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.599+0000 7f27b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:45:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.599+0000 7f27b7e00640 1 -- 172.21.15.16:0/1413735370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c40ad320 con 0x56144df58280 2024-09-21T03:45:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.599+0000 7f27b7e00640 1 -- 172.21.15.16:0/1413735370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f27c40a9a80 con 0x56144df58280 2024-09-21T03:45:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.599+0000 7f27e39ce740 1 -- 172.21.15.16:0/1413735370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27c40adf10 con 0x56144df58280 2024-09-21T03:45:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.599+0000 7f27b7e00640 1 -- 172.21.15.16:0/1413735370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f27c40b22f0 con 0x56144df58280 2024-09-21T03:45:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.599+0000 7f27e39ce740 10 client.15213 fetch_fsmap learned FSMap version 29 2024-09-21T03:45:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.599+0000 7f27e39ce740 1 -- 172.21.15.16:0/1413735370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56144e345dd0 con 0x56144df58280 2024-09-21T03:45:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.599+0000 7f27b7e00640 1 -- 172.21.15.16:0/1413735370 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c40adf10 con 0x56144df58280 2024-09-21T03:45:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.600+0000 7f27b7e00640 1 -- 172.21.15.16:0/1413735370 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f27c40acc00 con 0x56144df58280 2024-09-21T03:45:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.600+0000 7f27e39ce740 10 client.15213 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:45:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.600+0000 7f27e39ce740 10 client.15213 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:45:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.600+0000 7f27e39ce740 20 client.15213 populate_metadata read hostname 'smithi016' 2024-09-21T03:45:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.600+0000 7f27e39ce740 1 --2- 172.21.15.16:0/1413735370 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x56144df0cc90 0x56144e3460b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.600+0000 7f27e39ce740 4 client.15213 mds_command: new command op to 24682 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:45:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.600+0000 7f27e39ce740 1 -- 172.21.15.16:0/1413735370 --> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] -- command(tid 0: {"prefix": "status"}) -- 0x56144e294750 con 0x56144df0cc90 2024-09-21T03:45:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.600+0000 7f27d5e00640 1 --2- 172.21.15.16:0/1413735370 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x56144df0cc90 0x56144e3460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.601+0000 7f27d5e00640 1 --2- 172.21.15.16:0/1413735370 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x56144df0cc90 0x56144e3460b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63428e99c03772ee server_cookie=ab529d64cc6e67e in_seq=0 out_seq=0 2024-09-21T03:45:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.601+0000 7f27b7e00640 10 client.15213 ms_handle_connect on v2:172.21.15.134:6838/606193011 2024-09-21T03:45:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.601+0000 7f27b7e00640 1 -- 172.21.15.16:0/1413735370 <== mds.2 v2:172.21.15.134:6838/606193011 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56144e294750 con 0x56144df0cc90 2024-09-21T03:45:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.601+0000 7f27b7e00640 10 client.15213 handle_command_reply: tid=0 2024-09-21T03:45:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.601+0000 7f27e39ce740 1 client.15213 shutdown 2024-09-21T03:45:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.601+0000 7f27e39ce740 1 -- 172.21.15.16:0/1413735370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df58280 msgr2=0x56144e33f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.601+0000 7f27e39ce740 1 --2- 172.21.15.16:0/1413735370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df58280 0x56144e33f360 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f27c400cfd0 tx=0x7f27c400efd0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.601+0000 7f27e39ce740 1 -- 172.21.15.16:0/1413735370 shutdown_connections 2024-09-21T03:45:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.601+0000 7f27e39ce740 1 --2- 172.21.15.16:0/1413735370 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x56144df0cc90 0x56144e3460b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.601+0000 7f27e39ce740 1 --2- 172.21.15.16:0/1413735370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56144df78f60 0x56144e3443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.601+0000 7f27e39ce740 1 --2- 172.21.15.16:0/1413735370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56144df58280 0x56144e33f360 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.601+0000 7f27e39ce740 1 --2- 172.21.15.16:0/1413735370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56144df500a0 0x56144e33eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.601+0000 7f27e39ce740 1 -- 172.21.15.16:0/1413735370 >> 172.21.15.16:0/1413735370 conn(0x56144df6d350 msgr2=0x56144e32b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.601+0000 7f27b7e00640 0 client.15213 ms_handle_reset on v2:172.21.15.134:6838/606193011 2024-09-21T03:45:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.601+0000 7f27e39ce740 1 -- 172.21.15.16:0/1413735370 shutdown_connections 2024-09-21T03:45:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.602+0000 7f27e39ce740 1 -- 172.21.15.16:0/1413735370 wait complete. 2024-09-21T03:45:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.602+0000 7f27e39ce740 20 client.15213 trim_cache size 0 max 16384 2024-09-21T03:45:31.608 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:45:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:45:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-09-21T03:45:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24682, 2024-09-21T03:45:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T03:45:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T03:45:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T03:45:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 88.084798430999996, 2024-09-21T03:45:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 29, 2024-09-21T03:45:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 86, 2024-09-21T03:45:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 84, 2024-09-21T03:45:31.608 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 168.99805635800001 2024-09-21T03:45:31.609 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:45:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.602+0000 7f27dce00640 1 -- 172.21.15.16:0/294750402 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f27bc0b6590 msgr2=0x7f27bc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.602+0000 7f27dce00640 1 --2- 172.21.15.16:0/294750402 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f27bc0b6590 0x7f27bc0b8aa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f27d002f500 tx=0x7f27d002f310 comp rx=0 tx=0).stop 2024-09-21T03:45:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.602+0000 7f27dce00640 1 -- 172.21.15.16:0/294750402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8077380 msgr2=0x7f27d81a0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.602+0000 7f27dce00640 1 --2- 172.21.15.16:0/294750402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8077380 0x7f27d81a0f80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f27c400e9c0 tx=0x7f27c400ee80 comp rx=0 tx=0).stop 2024-09-21T03:45:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.602+0000 7f27dce00640 1 -- 172.21.15.16:0/294750402 shutdown_connections 2024-09-21T03:45:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.602+0000 7f27dce00640 1 --2- 172.21.15.16:0/294750402 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f27bc0b6590 0x7f27bc0b8aa0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.602+0000 7f27dce00640 1 --2- 172.21.15.16:0/294750402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27d8114430 0x7f27d81a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.602+0000 7f27dce00640 1 --2- 172.21.15.16:0/294750402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27d8077380 0x7f27d81a0f80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.602+0000 7f27dce00640 1 --2- 172.21.15.16:0/294750402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d8075dd0 0x7f27d81a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.602+0000 7f27dce00640 1 -- 172.21.15.16:0/294750402 >> 172.21.15.16:0/294750402 conn(0x7f27d807b850 msgr2=0x7f27d8113040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.603+0000 7f27dce00640 1 -- 172.21.15.16:0/294750402 shutdown_connections 2024-09-21T03:45:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:31.603+0000 7f27dce00640 1 -- 172.21.15.16:0/294750402 wait complete. 2024-09-21T03:45:32.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T03:45:32.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:32 smithi134 ceph-mon[35031]: pgmap v395: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T03:45:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:32.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:32 smithi123 ceph-mon[34673]: pgmap v395: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T03:45:32.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:32 smithi016 ceph-mon[30375]: pgmap v395: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T03:45:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.517+0000 7fdac7e00640 1 -- 172.21.15.16:0/3118557467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac8105bb0 msgr2=0x7fdac8105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.517+0000 7fdac7e00640 1 --2- 172.21.15.16:0/3118557467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac8105bb0 0x7fdac8105fc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009900 tx=0x7fdab802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.518+0000 7fdac7e00640 1 -- 172.21.15.16:0/3118557467 shutdown_connections 2024-09-21T03:45:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.518+0000 7fdac7e00640 1 --2- 172.21.15.16:0/3118557467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdac810be40 0x7fdac8112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.518+0000 7fdac7e00640 1 --2- 172.21.15.16:0/3118557467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdac810b4f0 0x7fdac810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.518+0000 7fdac7e00640 1 --2- 172.21.15.16:0/3118557467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac8105bb0 0x7fdac8105fc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.518+0000 7fdac7e00640 1 -- 172.21.15.16:0/3118557467 >> 172.21.15.16:0/3118557467 conn(0x7fdac8101680 msgr2=0x7fdac8103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.518+0000 7fdac7e00640 1 -- 172.21.15.16:0/3118557467 shutdown_connections 2024-09-21T03:45:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.518+0000 7fdac7e00640 1 -- 172.21.15.16:0/3118557467 wait complete. 2024-09-21T03:45:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.519+0000 7fdac7e00640 1 Processor -- start 2024-09-21T03:45:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.519+0000 7fdac7e00640 1 -- start start 2024-09-21T03:45:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.519+0000 7fdac7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdac8105bb0 0x7fdac819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.519+0000 7fdac7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac810b4f0 0x7fdac819caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.519+0000 7fdac7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdac810be40 0x7fdac81a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.519+0000 7fdac7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8115b00 con 0x7fdac810b4f0 2024-09-21T03:45:33.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.519+0000 7fdac7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8115980 con 0x7fdac8105bb0 2024-09-21T03:45:33.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.519+0000 7fdac7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8115c80 con 0x7fdac810be40 2024-09-21T03:45:33.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.520+0000 7fdabfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac810b4f0 0x7fdac819caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.520+0000 7fdac6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdac8105bb0 0x7fdac819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.520+0000 7fdac6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdac8105bb0 0x7fdac819c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34638/0 (socket says 172.21.15.16:34638) 2024-09-21T03:45:33.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.520+0000 7fdac6a00640 1 -- 172.21.15.16:0/3879172307 learned_addr learned my addr 172.21.15.16:0/3879172307 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:33.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.520+0000 7fdac7400640 1 --2- 172.21.15.16:0/3879172307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdac810be40 0x7fdac81a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.520+0000 7fdac6a00640 1 -- 172.21.15.16:0/3879172307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdac810be40 msgr2=0x7fdac81a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.520+0000 7fdac6a00640 1 --2- 172.21.15.16:0/3879172307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdac810be40 0x7fdac81a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.520+0000 7fdac6a00640 1 -- 172.21.15.16:0/3879172307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac810b4f0 msgr2=0x7fdac819caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.520+0000 7fdac6a00640 1 --2- 172.21.15.16:0/3879172307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac810b4f0 0x7fdac819caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.520+0000 7fdac6a00640 1 -- 172.21.15.16:0/3879172307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac81a24c0 con 0x7fdac8105bb0 2024-09-21T03:45:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.520+0000 7fdac7400640 1 --2- 172.21.15.16:0/3879172307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdac810be40 0x7fdac81a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.521+0000 7fdac6a00640 1 --2- 172.21.15.16:0/3879172307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdac8105bb0 0x7fdac819c550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdab8005940 tx=0x7fdab80040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.521+0000 7fdabf400640 1 -- 172.21.15.16:0/3879172307 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab803f030 con 0x7fdac8105bb0 2024-09-21T03:45:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.521+0000 7fdac7e00640 1 -- 172.21.15.16:0/3879172307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac81a2750 con 0x7fdac8105bb0 2024-09-21T03:45:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.521+0000 7fdabf400640 1 -- 172.21.15.16:0/3879172307 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab80044c0 con 0x7fdac8105bb0 2024-09-21T03:45:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.521+0000 7fdabf400640 1 -- 172.21.15.16:0/3879172307 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8035070 con 0x7fdac8105bb0 2024-09-21T03:45:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.521+0000 7fdac7e00640 1 -- 172.21.15.16:0/3879172307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac81a8f10 con 0x7fdac8105bb0 2024-09-21T03:45:33.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.524+0000 7fdad45dc740 1 Processor -- start 2024-09-21T03:45:33.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.525+0000 7fdad45dc740 1 -- start start 2024-09-21T03:45:33.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.525+0000 7fdabf400640 1 -- 172.21.15.16:0/3879172307 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fdab8042030 con 0x7fdac8105bb0 2024-09-21T03:45:33.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.525+0000 7fdad45dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfddf60 0x55fcce384670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.525+0000 7fdad45dc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfb50a0 0x55fcce384bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.525+0000 7fdad45dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfbd280 0x55fcce3850f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.525+0000 7fdad45dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fcce2fbd50 con 0x55fccdfbd280 2024-09-21T03:45:33.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.525+0000 7fdad45dc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fcce2eb180 con 0x55fccdfb50a0 2024-09-21T03:45:33.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.525+0000 7fdad45dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fcce2eef90 con 0x55fccdfddf60 2024-09-21T03:45:33.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.525+0000 7fdac6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfb50a0 0x55fcce384bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.525+0000 7fdabfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfbd280 0x55fcce3850f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.525+0000 7fdac6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfb50a0 0x55fcce384bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34654/0 (socket says 172.21.15.16:34654) 2024-09-21T03:45:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.525+0000 7fdac6a00640 1 -- 172.21.15.16:0/1915549643 learned_addr learned my addr 172.21.15.16:0/1915549643 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.525+0000 7fdac7400640 1 --2- 172.21.15.16:0/1915549643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfddf60 0x55fcce384670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.526+0000 7fdac6a00640 1 -- 172.21.15.16:0/1915549643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfddf60 msgr2=0x55fcce384670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.526+0000 7fdac6a00640 1 --2- 172.21.15.16:0/1915549643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfddf60 0x55fcce384670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.526+0000 7fdac6a00640 1 -- 172.21.15.16:0/1915549643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfbd280 msgr2=0x55fcce3850f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.526+0000 7fdac6a00640 1 --2- 172.21.15.16:0/1915549643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfbd280 0x55fcce3850f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.526+0000 7fdac6a00640 1 -- 172.21.15.16:0/1915549643 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcce2b5a70 con 0x55fccdfb50a0 2024-09-21T03:45:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.526+0000 7fdabfe00640 1 --2- 172.21.15.16:0/1915549643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfbd280 0x55fcce3850f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.526+0000 7fdac6a00640 1 --2- 172.21.15.16:0/1915549643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfb50a0 0x55fcce384bb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdab810dc20 tx=0x7fdab810dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcc02a726547ab66 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.526+0000 7fdab7e00640 1 -- 172.21.15.16:0/1915549643 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8116070 con 0x55fccdfb50a0 2024-09-21T03:45:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.527+0000 7fdab7e00640 1 -- 172.21.15.16:0/1915549643 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab810fc90 con 0x55fccdfb50a0 2024-09-21T03:45:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.527+0000 7fdab7e00640 1 -- 172.21.15.16:0/1915549643 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8111580 con 0x55fccdfb50a0 2024-09-21T03:45:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.527+0000 7fdabf400640 1 --2- 172.21.15.16:0/3879172307 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdaac0b6510 0x7fdaac0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.527+0000 7fdabf400640 1 -- 172.21.15.16:0/3879172307 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fdab8107a50 con 0x7fdac8105bb0 2024-09-21T03:45:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.527+0000 7fdac7400640 1 --2- 172.21.15.16:0/3879172307 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdaac0b6510 0x7fdaac0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.527+0000 7fdad45dc740 1 -- 172.21.15.16:0/1915549643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfb50a0 msgr2=0x55fcce384bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.527+0000 7fdad45dc740 1 --2- 172.21.15.16:0/1915549643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfb50a0 0x55fcce384bb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdab810dc20 tx=0x7fdab810dd10 comp rx=0 tx=0).stop 2024-09-21T03:45:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.528+0000 7fdad45dc740 1 -- 172.21.15.16:0/1915549643 shutdown_connections 2024-09-21T03:45:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.528+0000 7fdad45dc740 1 --2- 172.21.15.16:0/1915549643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfbd280 0x55fcce3850f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.528+0000 7fdad45dc740 1 --2- 172.21.15.16:0/1915549643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfb50a0 0x55fcce384bb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.528+0000 7fdad45dc740 1 --2- 172.21.15.16:0/1915549643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfddf60 0x55fcce384670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.528+0000 7fdad45dc740 1 -- 172.21.15.16:0/1915549643 >> 172.21.15.16:0/1915549643 conn(0x55fccdfd2350 msgr2=0x55fcce382560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.528+0000 7fdac7400640 1 --2- 172.21.15.16:0/3879172307 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdaac0b6510 0x7fdaac0b8a20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009fd0 tx=0x7fdac0009500 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.528+0000 7fdad45dc740 1 -- 172.21.15.16:0/1915549643 shutdown_connections 2024-09-21T03:45:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.528+0000 7fdad45dc740 1 -- 172.21.15.16:0/1915549643 wait complete. 2024-09-21T03:45:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.528+0000 7fdad45dc740 1 -- start start 2024-09-21T03:45:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.529+0000 7fdad45dc740 1 Processor -- start 2024-09-21T03:45:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.529+0000 7fdad45dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfb50a0 0x55fcce3901f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.529+0000 7fdad45dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfbd280 0x55fcce390730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.529+0000 7fdad45dc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfddf60 0x55fcce39be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.529+0000 7fdad45dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fcce14c5f0 con 0x55fccdfb50a0 2024-09-21T03:45:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.529+0000 7fdad45dc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fcce286460 con 0x55fccdfddf60 2024-09-21T03:45:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.529+0000 7fdad45dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fcce392c10 con 0x55fccdfbd280 2024-09-21T03:45:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.529+0000 7fdad45dc740 10 client.0 _ll_register_callbacks cb 0x55fcce296210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.529+0000 7fdac6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfb50a0 0x55fcce3901f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.530+0000 7fdabfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfbd280 0x55fcce390730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.530+0000 7fdac6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfb50a0 0x55fcce3901f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41046/0 (socket says 172.21.15.16:41046) 2024-09-21T03:45:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.530+0000 7fdabfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfbd280 0x55fcce390730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:49886/0 (socket says 172.21.15.16:49886) 2024-09-21T03:45:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.530+0000 7fdac6a00640 1 -- 172.21.15.16:0/3582830014 learned_addr learned my addr 172.21.15.16:0/3582830014 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.530+0000 7fdac7400640 1 --2- 172.21.15.16:0/3582830014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfddf60 0x55fcce39be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.530+0000 7fdabfe00640 1 -- 172.21.15.16:0/3582830014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfddf60 msgr2=0x55fcce39be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.530+0000 7fdabfe00640 1 --2- 172.21.15.16:0/3582830014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfddf60 0x55fcce39be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.530+0000 7fdabfe00640 1 -- 172.21.15.16:0/3582830014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfb50a0 msgr2=0x55fcce3901f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.530+0000 7fdabfe00640 1 --2- 172.21.15.16:0/3582830014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfb50a0 0x55fcce3901f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.530+0000 7fdabfe00640 1 -- 172.21.15.16:0/3582830014 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcce049540 con 0x55fccdfbd280 2024-09-21T03:45:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.530+0000 7fdac7400640 1 --2- 172.21.15.16:0/3582830014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfddf60 0x55fcce39be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.530+0000 7fdac6a00640 1 --2- 172.21.15.16:0/3582830014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfb50a0 0x55fcce3901f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.530+0000 7fdabfe00640 1 --2- 172.21.15.16:0/3582830014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfbd280 0x55fcce390730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdac806c630 tx=0x7fdab0012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee71e99bb77be02b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.530+0000 7fdab4c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:45:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.531+0000 7fdab4c00640 1 -- 172.21.15.16:0/3582830014 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0004030 con 0x55fccdfbd280 2024-09-21T03:45:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.531+0000 7fdad45dc740 1 -- 172.21.15.16:0/3582830014 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fcce288a70 con 0x55fccdfbd280 2024-09-21T03:45:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.531+0000 7fdab4c00640 1 -- 172.21.15.16:0/3582830014 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fdab00041d0 con 0x55fccdfbd280 2024-09-21T03:45:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.531+0000 7fdab4c00640 1 -- 172.21.15.16:0/3582830014 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdab0025050 con 0x55fccdfbd280 2024-09-21T03:45:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.531+0000 7fdad45dc740 10 client.25216 fetch_fsmap learned FSMap version 29 2024-09-21T03:45:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.531+0000 7fdad45dc740 1 -- 172.21.15.16:0/3582830014 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fcce2a36f0 con 0x55fccdfbd280 2024-09-21T03:45:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.531+0000 7fdab4c00640 1 -- 172.21.15.16:0/3582830014 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fcce288a70 con 0x55fccdfbd280 2024-09-21T03:45:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.532+0000 7fdab4c00640 1 -- 172.21.15.16:0/3582830014 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdab00038e0 con 0x55fccdfbd280 2024-09-21T03:45:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.532+0000 7fdad45dc740 10 client.25216 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:45:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.532+0000 7fdad45dc740 10 client.25216 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:45:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.532+0000 7fdad45dc740 20 client.25216 populate_metadata read hostname 'smithi016' 2024-09-21T03:45:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.532+0000 7fdad45dc740 1 --2- 172.21.15.16:0/3582830014 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x55fccdf9c8e0 0x55fcce3a2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.532+0000 7fdad45dc740 4 client.25216 mds_command: new command op to 24689 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.532+0000 7fdad45dc740 1 -- 172.21.15.16:0/3582830014 --> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcce2fdca0 con 0x55fccdf9c8e0 2024-09-21T03:45:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.532+0000 7fdac6a00640 1 --2- 172.21.15.16:0/3582830014 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x55fccdf9c8e0 0x55fcce3a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.533+0000 7fdac6a00640 1 --2- 172.21.15.16:0/3582830014 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x55fccdf9c8e0 0x55fcce3a2490 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d23ad8f8e5a9d server_cookie=f58afeff1aae00b6 in_seq=0 out_seq=0 2024-09-21T03:45:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.533+0000 7fdab4c00640 10 client.25216 ms_handle_connect on v2:172.21.15.123:6836/2001424876 2024-09-21T03:45:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.534+0000 7fdab4c00640 1 -- 172.21.15.16:0/3582830014 <== mds.1 v2:172.21.15.123:6836/2001424876 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fcce2fdca0 con 0x55fccdf9c8e0 2024-09-21T03:45:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.534+0000 7fdab4c00640 10 client.25216 handle_command_reply: tid=0 2024-09-21T03:45:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.534+0000 7fdad45dc740 1 client.25216 shutdown 2024-09-21T03:45:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.534+0000 7fdad45dc740 1 -- 172.21.15.16:0/3582830014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfbd280 msgr2=0x55fcce390730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.534+0000 7fdad45dc740 1 --2- 172.21.15.16:0/3582830014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfbd280 0x55fcce390730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdac806c630 tx=0x7fdab0012b60 comp rx=0 tx=0).stop 2024-09-21T03:45:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.534+0000 7fdad45dc740 1 -- 172.21.15.16:0/3582830014 shutdown_connections 2024-09-21T03:45:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.534+0000 7fdad45dc740 1 --2- 172.21.15.16:0/3582830014 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x55fccdf9c8e0 0x55fcce3a2490 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.534+0000 7fdad45dc740 1 --2- 172.21.15.16:0/3582830014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfddf60 0x55fcce39be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.534+0000 7fdad45dc740 1 --2- 172.21.15.16:0/3582830014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfbd280 0x55fcce390730 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.534+0000 7fdab4c00640 0 client.25216 ms_handle_reset on v2:172.21.15.123:6836/2001424876 2024-09-21T03:45:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.534+0000 7fdad45dc740 1 --2- 172.21.15.16:0/3582830014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfb50a0 0x55fcce3901f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.534+0000 7fdad45dc740 1 -- 172.21.15.16:0/3582830014 >> 172.21.15.16:0/3582830014 conn(0x55fccdfd2350 msgr2=0x55fcce387800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.534+0000 7fdad45dc740 1 -- 172.21.15.16:0/3582830014 shutdown_connections 2024-09-21T03:45:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.535+0000 7fdad45dc740 1 -- 172.21.15.16:0/3582830014 wait complete. 2024-09-21T03:45:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.535+0000 7fdad45dc740 20 client.25216 trim_cache size 0 max 16384 2024-09-21T03:45:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.563+0000 7fdad45dc740 1 Processor -- start 2024-09-21T03:45:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.563+0000 7fdad45dc740 1 -- start start 2024-09-21T03:45:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.563+0000 7fdad45dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfb50a0 0x55fcce3a2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.563+0000 7fdad45dc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfbd280 0x55fcce38e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.563+0000 7fdad45dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfddf60 0x55fcce38edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.563+0000 7fdad45dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fcce3a07d0 con 0x55fccdfb50a0 2024-09-21T03:45:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.563+0000 7fdad45dc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fcce3a0ea0 con 0x55fccdfbd280 2024-09-21T03:45:33.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.563+0000 7fdad45dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fcce3a1570 con 0x55fccdfddf60 2024-09-21T03:45:33.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.563+0000 7fdac6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfb50a0 0x55fcce3a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.564+0000 7fdac6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfb50a0 0x55fcce3a2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41052/0 (socket says 172.21.15.16:41052) 2024-09-21T03:45:33.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.564+0000 7fdac6a00640 1 -- 172.21.15.16:0/1641591424 learned_addr learned my addr 172.21.15.16:0/1641591424 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:33.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.564+0000 7fdac7400640 1 --2- 172.21.15.16:0/1641591424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfddf60 0x55fcce38edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.564+0000 7fdabfe00640 1 --2- 172.21.15.16:0/1641591424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfbd280 0x55fcce38e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.564+0000 7fdac6a00640 1 -- 172.21.15.16:0/1641591424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfddf60 msgr2=0x55fcce38edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.564+0000 7fdac6a00640 1 --2- 172.21.15.16:0/1641591424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfddf60 0x55fcce38edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.564+0000 7fdac6a00640 1 -- 172.21.15.16:0/1641591424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfbd280 msgr2=0x55fcce38e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.564+0000 7fdac6a00640 1 --2- 172.21.15.16:0/1641591424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfbd280 0x55fcce38e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.564+0000 7fdac6a00640 1 -- 172.21.15.16:0/1641591424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcce386960 con 0x55fccdfb50a0 2024-09-21T03:45:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.564+0000 7fdac6a00640 1 --2- 172.21.15.16:0/1641591424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfb50a0 0x55fcce3a2490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdab810dc20 tx=0x7fdab802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a4e2a199a3d396 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.565+0000 7fdab7400640 1 -- 172.21.15.16:0/1641591424 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8049070 con 0x55fccdfb50a0 2024-09-21T03:45:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.565+0000 7fdab7400640 1 -- 172.21.15.16:0/1641591424 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab810d300 con 0x55fccdfb50a0 2024-09-21T03:45:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.565+0000 7fdab7400640 1 -- 172.21.15.16:0/1641591424 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8116070 con 0x55fccdfb50a0 2024-09-21T03:45:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.566+0000 7fdad45dc740 1 -- 172.21.15.16:0/1641591424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfb50a0 msgr2=0x55fcce3a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.566+0000 7fdad45dc740 1 --2- 172.21.15.16:0/1641591424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfb50a0 0x55fcce3a2490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdab810dc20 tx=0x7fdab802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.566+0000 7fdad45dc740 1 -- 172.21.15.16:0/1641591424 shutdown_connections 2024-09-21T03:45:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.566+0000 7fdad45dc740 1 --2- 172.21.15.16:0/1641591424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfddf60 0x55fcce38edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.566+0000 7fdad45dc740 1 --2- 172.21.15.16:0/1641591424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfbd280 0x55fcce38e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.566+0000 7fdad45dc740 1 --2- 172.21.15.16:0/1641591424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfb50a0 0x55fcce3a2490 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.566+0000 7fdad45dc740 1 -- 172.21.15.16:0/1641591424 >> 172.21.15.16:0/1641591424 conn(0x55fccdfd2350 msgr2=0x55fcce39feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.566+0000 7fdad45dc740 1 -- 172.21.15.16:0/1641591424 shutdown_connections 2024-09-21T03:45:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.566+0000 7fdad45dc740 1 -- 172.21.15.16:0/1641591424 wait complete. 2024-09-21T03:45:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.566+0000 7fdad45dc740 1 -- start start 2024-09-21T03:45:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.567+0000 7fdad45dc740 1 Processor -- start 2024-09-21T03:45:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.567+0000 7fdad45dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfb50a0 0x55fcce3a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.567+0000 7fdad45dc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfbd280 0x55fcce3a4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.567+0000 7fdad45dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfddf60 0x55fcce3a9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.567+0000 7fdad45dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fcce3a2310 con 0x55fccdfddf60 2024-09-21T03:45:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.567+0000 7fdad45dc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fcce3a1c40 con 0x55fccdfbd280 2024-09-21T03:45:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.567+0000 7fdad45dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fcce3a29e0 con 0x55fccdfb50a0 2024-09-21T03:45:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.567+0000 7fdad45dc740 10 client.0 _ll_register_callbacks cb 0x7fdab0009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.567+0000 7fdac7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfddf60 0x55fcce3a9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.567+0000 7fdac7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfddf60 0x55fcce3a9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41060/0 (socket says 172.21.15.16:41060) 2024-09-21T03:45:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.568+0000 7fdac7400640 1 -- 172.21.15.16:0/714924706 learned_addr learned my addr 172.21.15.16:0/714924706 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.568+0000 7fdac6a00640 1 --2- 172.21.15.16:0/714924706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfb50a0 0x55fcce3a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.568+0000 7fdabfe00640 1 --2- 172.21.15.16:0/714924706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfbd280 0x55fcce3a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.568+0000 7fdac7400640 1 -- 172.21.15.16:0/714924706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfb50a0 msgr2=0x55fcce3a3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.568+0000 7fdac7400640 1 --2- 172.21.15.16:0/714924706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfb50a0 0x55fcce3a3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.568+0000 7fdac7400640 1 -- 172.21.15.16:0/714924706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfbd280 msgr2=0x55fcce3a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.568+0000 7fdac7400640 1 --2- 172.21.15.16:0/714924706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfbd280 0x55fcce3a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.568+0000 7fdac7400640 1 -- 172.21.15.16:0/714924706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcce3a9840 con 0x55fccdfddf60 2024-09-21T03:45:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.568+0000 7fdac7400640 1 --2- 172.21.15.16:0/714924706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfddf60 0x55fcce3a9200 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdac0003f00 tx=0x7fdac0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5105c7c660e7ac96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.568+0000 7fdab4c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:45:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.569+0000 7fdab4c00640 1 -- 172.21.15.16:0/714924706 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0004760 con 0x55fccdfddf60 2024-09-21T03:45:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.569+0000 7fdab4c00640 1 -- 172.21.15.16:0/714924706 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fdac0025040 con 0x55fccdfddf60 2024-09-21T03:45:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.569+0000 7fdad45dc740 1 -- 172.21.15.16:0/714924706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab810d300 con 0x55fccdfddf60 2024-09-21T03:45:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.569+0000 7fdab4c00640 1 -- 172.21.15.16:0/714924706 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdac0035050 con 0x55fccdfddf60 2024-09-21T03:45:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.569+0000 7fdad45dc740 10 client.15231 fetch_fsmap learned FSMap version 29 2024-09-21T03:45:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.569+0000 7fdad45dc740 1 -- 172.21.15.16:0/714924706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fcce3aacb0 con 0x55fccdfddf60 2024-09-21T03:45:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.569+0000 7fdab4c00640 1 -- 172.21.15.16:0/714924706 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab810d300 con 0x55fccdfddf60 2024-09-21T03:45:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.569+0000 7fdab4c00640 1 -- 172.21.15.16:0/714924706 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdac00056a0 con 0x55fccdfddf60 2024-09-21T03:45:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.569+0000 7fdad45dc740 10 client.15231 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:45:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.570+0000 7fdad45dc740 10 client.15231 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:45:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.570+0000 7fdad45dc740 20 client.15231 populate_metadata read hostname 'smithi016' 2024-09-21T03:45:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.570+0000 7fdad45dc740 1 --2- 172.21.15.16:0/714924706 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x55fccdf71c90 0x55fcce410910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.570+0000 7fdad45dc740 4 client.15231 mds_command: new command op to 24689 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:45:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.570+0000 7fdad45dc740 1 -- 172.21.15.16:0/714924706 --> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] -- command(tid 0: {"prefix": "status"}) -- 0x55fcce2f9750 con 0x55fccdf71c90 2024-09-21T03:45:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.570+0000 7fdac6a00640 1 --2- 172.21.15.16:0/714924706 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x55fccdf71c90 0x55fcce410910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.570+0000 7fdac6a00640 1 --2- 172.21.15.16:0/714924706 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x55fccdf71c90 0x55fcce410910 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=470a0451a91c4386 server_cookie=518f6e7d79018309 in_seq=0 out_seq=0 2024-09-21T03:45:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.570+0000 7fdab4c00640 10 client.15231 ms_handle_connect on v2:172.21.15.123:6836/2001424876 2024-09-21T03:45:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.571+0000 7fdab4c00640 1 -- 172.21.15.16:0/714924706 <== mds.1 v2:172.21.15.123:6836/2001424876 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fcce2f9750 con 0x55fccdf71c90 2024-09-21T03:45:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.571+0000 7fdab4c00640 10 client.15231 handle_command_reply: tid=0 2024-09-21T03:45:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.571+0000 7fdad45dc740 1 client.15231 shutdown 2024-09-21T03:45:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.571+0000 7fdad45dc740 1 -- 172.21.15.16:0/714924706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfddf60 msgr2=0x55fcce3a9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.571+0000 7fdad45dc740 1 --2- 172.21.15.16:0/714924706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfddf60 0x55fcce3a9200 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdac0003f00 tx=0x7fdac0004ab0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.571+0000 7fdad45dc740 1 -- 172.21.15.16:0/714924706 shutdown_connections 2024-09-21T03:45:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.571+0000 7fdad45dc740 1 --2- 172.21.15.16:0/714924706 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x55fccdf71c90 0x55fcce410910 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.571+0000 7fdad45dc740 1 --2- 172.21.15.16:0/714924706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fccdfddf60 0x55fcce3a9200 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.571+0000 7fdad45dc740 1 --2- 172.21.15.16:0/714924706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccdfbd280 0x55fcce3a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.571+0000 7fdad45dc740 1 --2- 172.21.15.16:0/714924706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fccdfb50a0 0x55fcce3a3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.571+0000 7fdad45dc740 1 -- 172.21.15.16:0/714924706 >> 172.21.15.16:0/714924706 conn(0x55fccdfd2350 msgr2=0x55fcce3901d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.571+0000 7fdab4c00640 0 client.15231 ms_handle_reset on v2:172.21.15.123:6836/2001424876 2024-09-21T03:45:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.571+0000 7fdad45dc740 1 -- 172.21.15.16:0/714924706 shutdown_connections 2024-09-21T03:45:33.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.571+0000 7fdad45dc740 1 -- 172.21.15.16:0/714924706 wait complete. 2024-09-21T03:45:33.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.571+0000 7fdad45dc740 20 client.15231 trim_cache size 0 max 16384 2024-09-21T03:45:33.580 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:45:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:45:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-09-21T03:45:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24689, 2024-09-21T03:45:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T03:45:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T03:45:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T03:45:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 91.069721577999999, 2024-09-21T03:45:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 29, 2024-09-21T03:45:33.581 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 86, 2024-09-21T03:45:33.581 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 84, 2024-09-21T03:45:33.581 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 173.67348698699999 2024-09-21T03:45:33.581 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:45:33.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.572+0000 7fdac7e00640 1 -- 172.21.15.16:0/3879172307 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdaac0b6510 msgr2=0x7fdaac0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.572+0000 7fdac7e00640 1 --2- 172.21.15.16:0/3879172307 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdaac0b6510 0x7fdaac0b8a20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009fd0 tx=0x7fdac0009500 comp rx=0 tx=0).stop 2024-09-21T03:45:33.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.572+0000 7fdac7e00640 1 -- 172.21.15.16:0/3879172307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdac8105bb0 msgr2=0x7fdac819c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:33.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.572+0000 7fdac7e00640 1 --2- 172.21.15.16:0/3879172307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdac8105bb0 0x7fdac819c550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdab8005940 tx=0x7fdab80040c0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.572+0000 7fdac7e00640 1 -- 172.21.15.16:0/3879172307 shutdown_connections 2024-09-21T03:45:33.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.572+0000 7fdac7e00640 1 --2- 172.21.15.16:0/3879172307 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdaac0b6510 0x7fdaac0b8a20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.572+0000 7fdac7e00640 1 --2- 172.21.15.16:0/3879172307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdac810be40 0x7fdac81a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.572+0000 7fdac7e00640 1 --2- 172.21.15.16:0/3879172307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac810b4f0 0x7fdac819caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.572+0000 7fdac7e00640 1 --2- 172.21.15.16:0/3879172307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdac8105bb0 0x7fdac819c550 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:33.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.572+0000 7fdac7e00640 1 -- 172.21.15.16:0/3879172307 >> 172.21.15.16:0/3879172307 conn(0x7fdac8101680 msgr2=0x7fdac8103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:33.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.572+0000 7fdac7e00640 1 -- 172.21.15.16:0/3879172307 shutdown_connections 2024-09-21T03:45:33.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:33.572+0000 7fdac7e00640 1 -- 172.21.15.16:0/3879172307 wait complete. 2024-09-21T03:45:34.418 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:45:34.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:34 smithi016 ceph-mon[30375]: pgmap v396: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-21T03:45:34.797 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:34.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:34 smithi134 ceph-mon[35031]: pgmap v396: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-21T03:45:34.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:34 smithi123 ceph-mon[34673]: pgmap v396: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-21T03:45:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.420+0000 7fb469c00640 1 -- 172.21.15.16:0/131434352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb46410be40 msgr2=0x7fb464112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.420+0000 7fb469c00640 1 --2- 172.21.15.16:0/131434352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb46410be40 0x7fb464112670 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009be0 tx=0x7fb45c02a6c0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.421+0000 7fb469c00640 1 -- 172.21.15.16:0/131434352 shutdown_connections 2024-09-21T03:45:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.421+0000 7fb469c00640 1 --2- 172.21.15.16:0/131434352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb46410be40 0x7fb464112670 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.421+0000 7fb469c00640 1 --2- 172.21.15.16:0/131434352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb46410b4f0 0x7fb46410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.421+0000 7fb469c00640 1 --2- 172.21.15.16:0/131434352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb464105bb0 0x7fb464105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.421+0000 7fb469c00640 1 -- 172.21.15.16:0/131434352 >> 172.21.15.16:0/131434352 conn(0x7fb464101680 msgr2=0x7fb464103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.421+0000 7fb469c00640 1 -- 172.21.15.16:0/131434352 shutdown_connections 2024-09-21T03:45:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.421+0000 7fb469c00640 1 -- 172.21.15.16:0/131434352 wait complete. 2024-09-21T03:45:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.422+0000 7fb469c00640 1 Processor -- start 2024-09-21T03:45:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.422+0000 7fb469c00640 1 -- start start 2024-09-21T03:45:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.422+0000 7fb469c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb464105bb0 0x7fb46419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.422+0000 7fb469c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb46410b4f0 0x7fb46419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.422+0000 7fb469c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb46410be40 0x7fb4641a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.422+0000 7fb469c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb464115b00 con 0x7fb46410be40 2024-09-21T03:45:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.422+0000 7fb469c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb464115980 con 0x7fb464105bb0 2024-09-21T03:45:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.422+0000 7fb469c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb464115c80 con 0x7fb46410b4f0 2024-09-21T03:45:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.423+0000 7fb463e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb464105bb0 0x7fb46419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.423+0000 7fb463e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb464105bb0 0x7fb46419c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34696/0 (socket says 172.21.15.16:34696) 2024-09-21T03:45:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.423+0000 7fb45b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb46410b4f0 0x7fb46419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.423+0000 7fb463e00640 1 -- 172.21.15.16:0/2961905582 learned_addr learned my addr 172.21.15.16:0/2961905582 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.423+0000 7fb469200640 1 --2- 172.21.15.16:0/2961905582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb46410be40 0x7fb4641a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.423+0000 7fb463e00640 1 -- 172.21.15.16:0/2961905582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb46410b4f0 msgr2=0x7fb46419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.423+0000 7fb463e00640 1 --2- 172.21.15.16:0/2961905582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb46410b4f0 0x7fb46419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.423+0000 7fb463e00640 1 -- 172.21.15.16:0/2961905582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb46410be40 msgr2=0x7fb4641a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.423+0000 7fb463e00640 1 --2- 172.21.15.16:0/2961905582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb46410be40 0x7fb4641a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.423+0000 7fb463e00640 1 -- 172.21.15.16:0/2961905582 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4641a2400 con 0x7fb464105bb0 2024-09-21T03:45:35.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.424+0000 7fb463e00640 1 --2- 172.21.15.16:0/2961905582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb464105bb0 0x7fb46419c490 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb45400e9f0 tx=0x7fb45400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.424+0000 7fb461600640 1 -- 172.21.15.16:0/2961905582 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45400cd80 con 0x7fb464105bb0 2024-09-21T03:45:35.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.424+0000 7fb469c00640 1 -- 172.21.15.16:0/2961905582 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4641a26f0 con 0x7fb464105bb0 2024-09-21T03:45:35.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.424+0000 7fb461600640 1 -- 172.21.15.16:0/2961905582 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4540108e0 con 0x7fb464105bb0 2024-09-21T03:45:35.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.424+0000 7fb461600640 1 -- 172.21.15.16:0/2961905582 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454018980 con 0x7fb464105bb0 2024-09-21T03:45:35.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.424+0000 7fb469c00640 1 -- 172.21.15.16:0/2961905582 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4641a8e50 con 0x7fb464105bb0 2024-09-21T03:45:35.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.427+0000 7fb471e2a740 1 Processor -- start 2024-09-21T03:45:35.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.427+0000 7fb471e2a740 1 -- start start 2024-09-21T03:45:35.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.428+0000 7fb471e2a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239e6f60 0x562923d8d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.428+0000 7fb471e2a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239be0a0 0x562923d8dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.428+0000 7fb471e2a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629239c6280 0x562923d8e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.428+0000 7fb471e2a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562923d04d50 con 0x5629239c6280 2024-09-21T03:45:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.428+0000 7fb471e2a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562923cf4180 con 0x5629239e6f60 2024-09-21T03:45:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.428+0000 7fb471e2a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562923cf7f90 con 0x5629239be0a0 2024-09-21T03:45:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.428+0000 7fb45b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629239c6280 0x562923d8e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.428+0000 7fb469200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239e6f60 0x562923d8d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.428+0000 7fb45b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629239c6280 0x562923d8e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41082/0 (socket says 172.21.15.16:41082) 2024-09-21T03:45:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.428+0000 7fb45b400640 1 -- 172.21.15.16:0/1978851250 learned_addr learned my addr 172.21.15.16:0/1978851250 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.428+0000 7fb461600640 1 -- 172.21.15.16:0/2961905582 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb454010430 con 0x7fb464105bb0 2024-09-21T03:45:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.428+0000 7fb463e00640 1 --2- 172.21.15.16:0/1978851250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239be0a0 0x562923d8dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.429+0000 7fb469200640 1 -- 172.21.15.16:0/1978851250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239be0a0 msgr2=0x562923d8dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.429+0000 7fb469200640 1 --2- 172.21.15.16:0/1978851250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239be0a0 0x562923d8dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.429+0000 7fb469200640 1 -- 172.21.15.16:0/1978851250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629239c6280 msgr2=0x562923d8e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.429+0000 7fb469200640 1 --2- 172.21.15.16:0/1978851250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629239c6280 0x562923d8e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.429+0000 7fb469200640 1 -- 172.21.15.16:0/1978851250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562923cbea70 con 0x5629239e6f60 2024-09-21T03:45:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.429+0000 7fb463e00640 1 --2- 172.21.15.16:0/1978851250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239be0a0 0x562923d8dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:45:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.429+0000 7fb469200640 1 --2- 172.21.15.16:0/1978851250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239e6f60 0x562923d8d670 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009b30 tx=0x7fb45c030430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50bbe05e7a20903a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.429+0000 7fb459600640 1 -- 172.21.15.16:0/1978851250 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c041030 con 0x5629239e6f60 2024-09-21T03:45:35.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.430+0000 7fb459600640 1 -- 172.21.15.16:0/1978851250 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb45c003e20 con 0x5629239e6f60 2024-09-21T03:45:35.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.430+0000 7fb459600640 1 -- 172.21.15.16:0/1978851250 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c036040 con 0x5629239e6f60 2024-09-21T03:45:35.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.430+0000 7fb461600640 1 --2- 172.21.15.16:0/2961905582 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb4480b6590 0x7fb4480b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.430+0000 7fb461600640 1 -- 172.21.15.16:0/2961905582 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb4540e7a00 con 0x7fb464105bb0 2024-09-21T03:45:35.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.430+0000 7fb463e00640 1 --2- 172.21.15.16:0/2961905582 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb4480b6590 0x7fb4480b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.430+0000 7fb471e2a740 1 -- 172.21.15.16:0/1978851250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239e6f60 msgr2=0x562923d8d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.430+0000 7fb471e2a740 1 --2- 172.21.15.16:0/1978851250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239e6f60 0x562923d8d670 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009b30 tx=0x7fb45c030430 comp rx=0 tx=0).stop 2024-09-21T03:45:35.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.431+0000 7fb471e2a740 1 -- 172.21.15.16:0/1978851250 shutdown_connections 2024-09-21T03:45:35.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.431+0000 7fb471e2a740 1 --2- 172.21.15.16:0/1978851250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629239c6280 0x562923d8e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.431+0000 7fb471e2a740 1 --2- 172.21.15.16:0/1978851250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239be0a0 0x562923d8dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.431+0000 7fb471e2a740 1 --2- 172.21.15.16:0/1978851250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239e6f60 0x562923d8d670 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.431+0000 7fb471e2a740 1 -- 172.21.15.16:0/1978851250 >> 172.21.15.16:0/1978851250 conn(0x5629239db350 msgr2=0x562923d8b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.431+0000 7fb471e2a740 1 -- 172.21.15.16:0/1978851250 shutdown_connections 2024-09-21T03:45:35.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.431+0000 7fb471e2a740 1 -- 172.21.15.16:0/1978851250 wait complete. 2024-09-21T03:45:35.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.431+0000 7fb463e00640 1 --2- 172.21.15.16:0/2961905582 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb4480b6590 0x7fb4480b8aa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb4540ad7c0 tx=0x7fb45400cf20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.432+0000 7fb471e2a740 1 -- start start 2024-09-21T03:45:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.432+0000 7fb471e2a740 1 Processor -- start 2024-09-21T03:45:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.432+0000 7fb471e2a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239be0a0 0x562923d992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.432+0000 7fb471e2a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239c6280 0x562923d99880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.432+0000 7fb471e2a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629239e6f60 0x562923da50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.432+0000 7fb471e2a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562923b555f0 con 0x5629239e6f60 2024-09-21T03:45:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.432+0000 7fb471e2a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562923c8f460 con 0x5629239be0a0 2024-09-21T03:45:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.432+0000 7fb471e2a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562923d9bdf0 con 0x5629239c6280 2024-09-21T03:45:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.432+0000 7fb471e2a740 10 client.0 _ll_register_callbacks cb 0x562923c9f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.433+0000 7fb469200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629239e6f60 0x562923da50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.433+0000 7fb469200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629239e6f60 0x562923da50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41094/0 (socket says 172.21.15.16:41094) 2024-09-21T03:45:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.433+0000 7fb45b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239c6280 0x562923d99880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.433+0000 7fb469200640 1 -- 172.21.15.16:0/2143649137 learned_addr learned my addr 172.21.15.16:0/2143649137 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.433+0000 7fb469200640 1 --2- 172.21.15.16:0/2143649137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239be0a0 0x562923d992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.433+0000 7fb45b400640 1 -- 172.21.15.16:0/2143649137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239be0a0 msgr2=0x562923d992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.433+0000 7fb45b400640 1 --2- 172.21.15.16:0/2143649137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239be0a0 0x562923d992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.433+0000 7fb45b400640 1 -- 172.21.15.16:0/2143649137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629239e6f60 msgr2=0x562923da50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.433+0000 7fb45b400640 1 --2- 172.21.15.16:0/2143649137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629239e6f60 0x562923da50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.433+0000 7fb45b400640 1 -- 172.21.15.16:0/2143649137 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562923a52540 con 0x5629239c6280 2024-09-21T03:45:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.434+0000 7fb45b400640 1 --2- 172.21.15.16:0/2143649137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239c6280 0x562923d99880 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb44c005fd0 tx=0x7fb44c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=679ef3e6cac326a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.434+0000 7fb452000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:45:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.434+0000 7fb452000640 1 -- 172.21.15.16:0/2143649137 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c004020 con 0x5629239c6280 2024-09-21T03:45:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.434+0000 7fb471e2a740 1 -- 172.21.15.16:0/2143649137 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562923c91a70 con 0x5629239c6280 2024-09-21T03:45:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.434+0000 7fb452000640 1 -- 172.21.15.16:0/2143649137 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb44c0041c0 con 0x5629239c6280 2024-09-21T03:45:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.434+0000 7fb452000640 1 -- 172.21.15.16:0/2143649137 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb44c0043c0 con 0x5629239c6280 2024-09-21T03:45:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.434+0000 7fb471e2a740 10 client.25231 fetch_fsmap learned FSMap version 29 2024-09-21T03:45:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.434+0000 7fb471e2a740 1 -- 172.21.15.16:0/2143649137 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562923cac6f0 con 0x5629239c6280 2024-09-21T03:45:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.435+0000 7fb452000640 1 -- 172.21.15.16:0/2143649137 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562923c91a70 con 0x5629239c6280 2024-09-21T03:45:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.435+0000 7fb452000640 1 -- 172.21.15.16:0/2143649137 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb44c003ad0 con 0x5629239c6280 2024-09-21T03:45:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.435+0000 7fb471e2a740 10 client.25231 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:45:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.435+0000 7fb471e2a740 10 client.25231 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:45:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.435+0000 7fb471e2a740 20 client.25231 populate_metadata read hostname 'smithi016' 2024-09-21T03:45:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.435+0000 7fb471e2a740 1 --2- 172.21.15.16:0/2143649137 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5629239a58e0 0x562923dab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.435+0000 7fb471e2a740 4 client.25231 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.435+0000 7fb471e2a740 1 -- 172.21.15.16:0/2143649137 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562923d06ca0 con 0x5629239a58e0 2024-09-21T03:45:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.436+0000 7fb469200640 1 --2- 172.21.15.16:0/2143649137 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5629239a58e0 0x562923dab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.436+0000 7fb469200640 1 --2- 172.21.15.16:0/2143649137 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5629239a58e0 0x562923dab760 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c18a3db2af8b841c server_cookie=de2f08494ad97624 in_seq=0 out_seq=0 2024-09-21T03:45:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.436+0000 7fb452000640 10 client.25231 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:45:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.437+0000 7fb452000640 1 -- 172.21.15.16:0/2143649137 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562923d06ca0 con 0x5629239a58e0 2024-09-21T03:45:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.437+0000 7fb452000640 10 client.25231 handle_command_reply: tid=0 2024-09-21T03:45:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.437+0000 7fb471e2a740 1 client.25231 shutdown 2024-09-21T03:45:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.437+0000 7fb471e2a740 1 -- 172.21.15.16:0/2143649137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239c6280 msgr2=0x562923d99880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.437+0000 7fb471e2a740 1 --2- 172.21.15.16:0/2143649137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239c6280 0x562923d99880 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb44c005fd0 tx=0x7fb44c012b60 comp rx=0 tx=0).stop 2024-09-21T03:45:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.437+0000 7fb471e2a740 1 -- 172.21.15.16:0/2143649137 shutdown_connections 2024-09-21T03:45:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.437+0000 7fb471e2a740 1 --2- 172.21.15.16:0/2143649137 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5629239a58e0 0x562923dab760 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.437+0000 7fb471e2a740 1 --2- 172.21.15.16:0/2143649137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629239e6f60 0x562923da50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.437+0000 7fb471e2a740 1 --2- 172.21.15.16:0/2143649137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239c6280 0x562923d99880 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.437+0000 7fb471e2a740 1 --2- 172.21.15.16:0/2143649137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239be0a0 0x562923d992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.437+0000 7fb471e2a740 1 -- 172.21.15.16:0/2143649137 >> 172.21.15.16:0/2143649137 conn(0x5629239db350 msgr2=0x562923d90850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.437+0000 7fb452000640 0 client.25231 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:45:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.438+0000 7fb471e2a740 1 -- 172.21.15.16:0/2143649137 shutdown_connections 2024-09-21T03:45:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.438+0000 7fb471e2a740 1 -- 172.21.15.16:0/2143649137 wait complete. 2024-09-21T03:45:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.438+0000 7fb471e2a740 20 client.25231 trim_cache size 0 max 16384 2024-09-21T03:45:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.457+0000 7fb471e2a740 1 Processor -- start 2024-09-21T03:45:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.457+0000 7fb471e2a740 1 -- start start 2024-09-21T03:45:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.458+0000 7fb471e2a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239be0a0 0x562923da5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.458+0000 7fb471e2a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239e6f60 0x562923da91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.458+0000 7fb471e2a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56292397ac90 0x562923d992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.458+0000 7fb471e2a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562923da9aa0 con 0x56292397ac90 2024-09-21T03:45:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.458+0000 7fb471e2a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562923daa170 con 0x5629239be0a0 2024-09-21T03:45:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.458+0000 7fb471e2a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562923daa840 con 0x5629239e6f60 2024-09-21T03:45:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.458+0000 7fb469200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56292397ac90 0x562923d992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.458+0000 7fb469200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239be0a0 0x562923da5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.458+0000 7fb469200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239be0a0 0x562923da5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34726/0 (socket says 172.21.15.16:34726) 2024-09-21T03:45:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.458+0000 7fb469200640 1 -- 172.21.15.16:0/2748643141 learned_addr learned my addr 172.21.15.16:0/2748643141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.458+0000 7fb45b400640 1 --2- 172.21.15.16:0/2748643141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239e6f60 0x562923da91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.458+0000 7fb469200640 1 -- 172.21.15.16:0/2748643141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239e6f60 msgr2=0x562923da91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.458+0000 7fb469200640 1 --2- 172.21.15.16:0/2748643141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239e6f60 0x562923da91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.458+0000 7fb469200640 1 -- 172.21.15.16:0/2748643141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56292397ac90 msgr2=0x562923d992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:35.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.458+0000 7fb469200640 1 --2- 172.21.15.16:0/2748643141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56292397ac90 0x562923d992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.458+0000 7fb469200640 1 -- 172.21.15.16:0/2748643141 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562923d9a160 con 0x5629239be0a0 2024-09-21T03:45:35.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.459+0000 7fb469200640 1 --2- 172.21.15.16:0/2748643141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239be0a0 0x562923da5050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb45c02abe0 tx=0x7fb45c030430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=629ece5bc5832d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.459+0000 7fb458c00640 1 -- 172.21.15.16:0/2748643141 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c036070 con 0x5629239be0a0 2024-09-21T03:45:35.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.459+0000 7fb458c00640 1 -- 172.21.15.16:0/2748643141 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb45c030be0 con 0x5629239be0a0 2024-09-21T03:45:35.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.459+0000 7fb458c00640 1 -- 172.21.15.16:0/2748643141 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c041040 con 0x5629239be0a0 2024-09-21T03:45:35.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.460+0000 7fb471e2a740 1 -- 172.21.15.16:0/2748643141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239be0a0 msgr2=0x562923da5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.460+0000 7fb471e2a740 1 --2- 172.21.15.16:0/2748643141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239be0a0 0x562923da5050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb45c02abe0 tx=0x7fb45c030430 comp rx=0 tx=0).stop 2024-09-21T03:45:35.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.460+0000 7fb471e2a740 1 -- 172.21.15.16:0/2748643141 shutdown_connections 2024-09-21T03:45:35.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.460+0000 7fb471e2a740 1 --2- 172.21.15.16:0/2748643141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56292397ac90 0x562923d992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.460+0000 7fb471e2a740 1 --2- 172.21.15.16:0/2748643141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239e6f60 0x562923da91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.460+0000 7fb471e2a740 1 --2- 172.21.15.16:0/2748643141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239be0a0 0x562923da5050 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.460+0000 7fb471e2a740 1 -- 172.21.15.16:0/2748643141 >> 172.21.15.16:0/2748643141 conn(0x5629239db350 msgr2=0x562923d862c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.460+0000 7fb471e2a740 1 -- 172.21.15.16:0/2748643141 shutdown_connections 2024-09-21T03:45:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.460+0000 7fb471e2a740 1 -- 172.21.15.16:0/2748643141 wait complete. 2024-09-21T03:45:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.461+0000 7fb471e2a740 1 -- start start 2024-09-21T03:45:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.461+0000 7fb471e2a740 1 Processor -- start 2024-09-21T03:45:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.461+0000 7fb471e2a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56292397ac90 0x562923dacc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.461+0000 7fb471e2a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239be0a0 0x562923dad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.461+0000 7fb471e2a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239e6f60 0x562923db2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.461+0000 7fb471e2a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562923dab5e0 con 0x56292397ac90 2024-09-21T03:45:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.461+0000 7fb471e2a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562923daaf10 con 0x5629239e6f60 2024-09-21T03:45:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.461+0000 7fb471e2a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562923dabcb0 con 0x5629239be0a0 2024-09-21T03:45:35.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.461+0000 7fb471e2a740 10 client.0 _ll_register_callbacks cb 0x7fb44c009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.462+0000 7fb469200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56292397ac90 0x562923dacc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.462+0000 7fb45b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239be0a0 0x562923dad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.462+0000 7fb45b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239be0a0 0x562923dad1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:49950/0 (socket says 172.21.15.16:49950) 2024-09-21T03:45:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.462+0000 7fb45b400640 1 -- 172.21.15.16:0/3480730011 learned_addr learned my addr 172.21.15.16:0/3480730011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.462+0000 7fb469200640 1 --2- 172.21.15.16:0/3480730011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239e6f60 0x562923db2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.462+0000 7fb45b400640 1 -- 172.21.15.16:0/3480730011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239e6f60 msgr2=0x562923db2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:35.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.462+0000 7fb45b400640 1 --2- 172.21.15.16:0/3480730011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239e6f60 0x562923db2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.462+0000 7fb45b400640 1 -- 172.21.15.16:0/3480730011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56292397ac90 msgr2=0x562923dacc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:35.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.462+0000 7fb45b400640 1 --2- 172.21.15.16:0/3480730011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56292397ac90 0x562923dacc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.462+0000 7fb45b400640 1 -- 172.21.15.16:0/3480730011 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562923db28a0 con 0x5629239be0a0 2024-09-21T03:45:35.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.462+0000 7fb469200640 1 --2- 172.21.15.16:0/3480730011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56292397ac90 0x562923dacc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.462+0000 7fb45b400640 1 --2- 172.21.15.16:0/3480730011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239be0a0 0x562923dad1f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009e90 tx=0x7fb44c005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ff046dcafba027c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.462+0000 7fb452000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:45:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.463+0000 7fb452000640 1 -- 172.21.15.16:0/3480730011 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c02e020 con 0x5629239be0a0 2024-09-21T03:45:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.463+0000 7fb452000640 1 -- 172.21.15.16:0/3480730011 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb44c00fbb0 con 0x5629239be0a0 2024-09-21T03:45:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.463+0000 7fb471e2a740 1 -- 172.21.15.16:0/3480730011 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562923d02750 con 0x5629239be0a0 2024-09-21T03:45:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.463+0000 7fb452000640 1 -- 172.21.15.16:0/3480730011 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb44c00fdb0 con 0x5629239be0a0 2024-09-21T03:45:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.463+0000 7fb471e2a740 10 client.25237 fetch_fsmap learned FSMap version 29 2024-09-21T03:45:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.463+0000 7fb471e2a740 1 -- 172.21.15.16:0/3480730011 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562923db3cf0 con 0x5629239be0a0 2024-09-21T03:45:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.463+0000 7fb452000640 1 -- 172.21.15.16:0/3480730011 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562923d02750 con 0x5629239be0a0 2024-09-21T03:45:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.463+0000 7fb452000640 1 -- 172.21.15.16:0/3480730011 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb44c01c860 con 0x5629239be0a0 2024-09-21T03:45:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.464+0000 7fb471e2a740 10 client.25237 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:45:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.464+0000 7fb471e2a740 10 client.25237 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:45:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.464+0000 7fb471e2a740 20 client.25237 populate_metadata read hostname 'smithi016' 2024-09-21T03:45:35.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.464+0000 7fb471e2a740 1 --2- 172.21.15.16:0/3480730011 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562923e18470 0x562923e198e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:35.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.464+0000 7fb471e2a740 4 client.25237 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:45:35.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.464+0000 7fb471e2a740 1 -- 172.21.15.16:0/3480730011 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "status"}) -- 0x562923cffb80 con 0x562923e18470 2024-09-21T03:45:35.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.464+0000 7fb469200640 1 --2- 172.21.15.16:0/3480730011 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562923e18470 0x562923e198e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:35.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.464+0000 7fb469200640 1 --2- 172.21.15.16:0/3480730011 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562923e18470 0x562923e198e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee0d2a028c51328a server_cookie=3772620b0f3cb45a in_seq=0 out_seq=0 2024-09-21T03:45:35.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.464+0000 7fb452000640 10 client.25237 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:45:35.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.465+0000 7fb452000640 1 -- 172.21.15.16:0/3480730011 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562923cffb80 con 0x562923e18470 2024-09-21T03:45:35.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.465+0000 7fb452000640 10 client.25237 handle_command_reply: tid=0 2024-09-21T03:45:35.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.465+0000 7fb471e2a740 1 client.25237 shutdown 2024-09-21T03:45:35.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.465+0000 7fb471e2a740 1 -- 172.21.15.16:0/3480730011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239be0a0 msgr2=0x562923dad1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.465+0000 7fb471e2a740 1 --2- 172.21.15.16:0/3480730011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239be0a0 0x562923dad1f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009e90 tx=0x7fb44c005df0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.465+0000 7fb471e2a740 1 -- 172.21.15.16:0/3480730011 shutdown_connections 2024-09-21T03:45:35.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.465+0000 7fb471e2a740 1 --2- 172.21.15.16:0/3480730011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629239e6f60 0x562923db2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.465+0000 7fb471e2a740 1 --2- 172.21.15.16:0/3480730011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629239be0a0 0x562923dad1f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.465+0000 7fb471e2a740 1 --2- 172.21.15.16:0/3480730011 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562923e18470 0x562923e198e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.465+0000 7fb471e2a740 1 --2- 172.21.15.16:0/3480730011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56292397ac90 0x562923dacc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.465+0000 7fb471e2a740 1 -- 172.21.15.16:0/3480730011 >> 172.21.15.16:0/3480730011 conn(0x5629239db350 msgr2=0x562923d94200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.465+0000 7fb452000640 0 client.25237 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:45:35.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.465+0000 7fb471e2a740 1 -- 172.21.15.16:0/3480730011 shutdown_connections 2024-09-21T03:45:35.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.465+0000 7fb471e2a740 1 -- 172.21.15.16:0/3480730011 wait complete. 2024-09-21T03:45:35.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.465+0000 7fb471e2a740 20 client.25237 trim_cache size 0 max 16384 2024-09-21T03:45:35.471 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:45:35.471 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:45:35.471 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-21T03:45:35.472 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24731, 2024-09-21T03:45:35.472 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T03:45:35.472 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T03:45:35.472 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T03:45:35.472 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 145.459709672, 2024-09-21T03:45:35.472 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 29, 2024-09-21T03:45:35.472 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 86, 2024-09-21T03:45:35.472 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 84, 2024-09-21T03:45:35.472 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 161.86227572499999 2024-09-21T03:45:35.472 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:45:35.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.466+0000 7fb469c00640 1 -- 172.21.15.16:0/2961905582 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb4480b6590 msgr2=0x7fb4480b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.466+0000 7fb469c00640 1 --2- 172.21.15.16:0/2961905582 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb4480b6590 0x7fb4480b8aa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb4540ad7c0 tx=0x7fb45400cf20 comp rx=0 tx=0).stop 2024-09-21T03:45:35.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.466+0000 7fb469c00640 1 -- 172.21.15.16:0/2961905582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb464105bb0 msgr2=0x7fb46419c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:35.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.466+0000 7fb469c00640 1 --2- 172.21.15.16:0/2961905582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb464105bb0 0x7fb46419c490 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb45400e9f0 tx=0x7fb45400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.466+0000 7fb469c00640 1 -- 172.21.15.16:0/2961905582 shutdown_connections 2024-09-21T03:45:35.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.466+0000 7fb469c00640 1 --2- 172.21.15.16:0/2961905582 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb4480b6590 0x7fb4480b8aa0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.466+0000 7fb469c00640 1 --2- 172.21.15.16:0/2961905582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb46410be40 0x7fb4641a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.466+0000 7fb469c00640 1 --2- 172.21.15.16:0/2961905582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb46410b4f0 0x7fb46419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.466+0000 7fb469c00640 1 --2- 172.21.15.16:0/2961905582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb464105bb0 0x7fb46419c490 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:35.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.466+0000 7fb469c00640 1 -- 172.21.15.16:0/2961905582 >> 172.21.15.16:0/2961905582 conn(0x7fb464101680 msgr2=0x7fb464108aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:35.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.466+0000 7fb469c00640 1 -- 172.21.15.16:0/2961905582 shutdown_connections 2024-09-21T03:45:35.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:35.466+0000 7fb469c00640 1 -- 172.21.15.16:0/2961905582 wait complete. 2024-09-21T03:45:36.556 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T03:45:36.557 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T03:45:36.569 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-21T03:45:36.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T03:45:36.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:36 smithi134 ceph-mon[35031]: pgmap v397: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T03:45:36.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:36 smithi123 ceph-mon[34673]: pgmap v397: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T03:45:36.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:36 smithi016 ceph-mon[30375]: pgmap v397: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T03:45:36.946 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:37.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.666+0000 7f0704800640 1 -- 172.21.15.16:0/3929896300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc103750 msgr2=0x7f06fc103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:37.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.666+0000 7f0704800640 1 --2- 172.21.15.16:0/3929896300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc103750 0x7f06fc103bc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f06f0009900 tx=0x7f06f002a480 comp rx=0 tx=0).stop 2024-09-21T03:45:37.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.666+0000 7f0704800640 1 -- 172.21.15.16:0/3929896300 shutdown_connections 2024-09-21T03:45:37.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.666+0000 7f0704800640 1 --2- 172.21.15.16:0/3929896300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06fc105180 0x7f06fc10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.666+0000 7f0704800640 1 --2- 172.21.15.16:0/3929896300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06fc1047e0 0x7f06fc104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.666+0000 7f0704800640 1 --2- 172.21.15.16:0/3929896300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc103750 0x7f06fc103bc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.666+0000 7f0704800640 1 -- 172.21.15.16:0/3929896300 >> 172.21.15.16:0/3929896300 conn(0x7f06fc0ff0e0 msgr2=0x7f06fc101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:37.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.666+0000 7f0704800640 1 -- 172.21.15.16:0/3929896300 shutdown_connections 2024-09-21T03:45:37.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.666+0000 7f0704800640 1 -- 172.21.15.16:0/3929896300 wait complete. 2024-09-21T03:45:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.667+0000 7f0704800640 1 Processor -- start 2024-09-21T03:45:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.668+0000 7f0704800640 1 -- start start 2024-09-21T03:45:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.668+0000 7f0704800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06fc103750 0x7f06fc076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.668+0000 7f0704800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc1047e0 0x7f06fc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.668+0000 7f0704800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06fc105180 0x7f06fc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.668+0000 7f0704800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc10f390 con 0x7f06fc1047e0 2024-09-21T03:45:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.668+0000 7f0704800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc10f210 con 0x7f06fc105180 2024-09-21T03:45:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.668+0000 7f0704800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc10f510 con 0x7f06fc103750 2024-09-21T03:45:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.668+0000 7f0701e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06fc103750 0x7f06fc076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.668+0000 7f0701400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc1047e0 0x7f06fc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.668+0000 7f0701e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06fc103750 0x7f06fc076e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:49970/0 (socket says 172.21.15.16:49970) 2024-09-21T03:45:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.668+0000 7f0701400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc1047e0 0x7f06fc075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41122/0 (socket says 172.21.15.16:41122) 2024-09-21T03:45:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.668+0000 7f0701e00640 1 -- 172.21.15.16:0/1729084041 learned_addr learned my addr 172.21.15.16:0/1729084041 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.669+0000 7f0702800640 1 --2- 172.21.15.16:0/1729084041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06fc105180 0x7f06fc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.669+0000 7f0702800640 1 -- 172.21.15.16:0/1729084041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06fc103750 msgr2=0x7f06fc076e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.669+0000 7f0702800640 1 --2- 172.21.15.16:0/1729084041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06fc103750 0x7f06fc076e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.669+0000 7f0702800640 1 -- 172.21.15.16:0/1729084041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc1047e0 msgr2=0x7f06fc075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.669+0000 7f0702800640 1 --2- 172.21.15.16:0/1729084041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc1047e0 0x7f06fc075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.669+0000 7f0702800640 1 -- 172.21.15.16:0/1729084041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06fc076350 con 0x7f06fc105180 2024-09-21T03:45:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.669+0000 7f0701400640 1 --2- 172.21.15.16:0/1729084041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc1047e0 0x7f06fc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.669+0000 7f0702800640 1 --2- 172.21.15.16:0/1729084041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06fc105180 0x7f06fc075c50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009fd0 tx=0x7f06f800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.670+0000 7f06f6a00640 1 -- 172.21.15.16:0/1729084041 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f800ce20 con 0x7f06fc105180 2024-09-21T03:45:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.670+0000 7f06f6a00640 1 -- 172.21.15.16:0/1729084041 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06f8005020 con 0x7f06fc105180 2024-09-21T03:45:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.670+0000 7f0704800640 1 -- 172.21.15.16:0/1729084041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06fc1a4030 con 0x7f06fc105180 2024-09-21T03:45:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.670+0000 7f06f6a00640 1 -- 172.21.15.16:0/1729084041 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8010690 con 0x7f06fc105180 2024-09-21T03:45:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.670+0000 7f0704800640 1 -- 172.21.15.16:0/1729084041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06fc1a4490 con 0x7f06fc105180 2024-09-21T03:45:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.671+0000 7f0704800640 1 -- 172.21.15.16:0/1729084041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06fc10a930 con 0x7f06fc105180 2024-09-21T03:45:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.673+0000 7f06f6a00640 1 -- 172.21.15.16:0/1729084041 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f06f80054d0 con 0x7f06fc105180 2024-09-21T03:45:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.675+0000 7f06f6a00640 1 --2- 172.21.15.16:0/1729084041 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f06e40b6510 0x7f06e40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.675+0000 7f06f6a00640 1 -- 172.21.15.16:0/1729084041 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f06f80e7d20 con 0x7f06fc105180 2024-09-21T03:45:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.675+0000 7f0701e00640 1 --2- 172.21.15.16:0/1729084041 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f06e40b6510 0x7f06e40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.676+0000 7f0701e00640 1 --2- 172.21.15.16:0/1729084041 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f06e40b6510 0x7f06e40b8a20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f06f00096c0 tx=0x7f06f00314a0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.677+0000 7f06f6a00640 1 -- 172.21.15.16:0/1729084041 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f06f80ae1a0 con 0x7f06fc105180 2024-09-21T03:45:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.895+0000 7f0704800640 1 -- 172.21.15.16:0/1729084041 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f06fc063800 con 0x7f06e40b6510 2024-09-21T03:45:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.896+0000 7f06f6a00640 1 -- 172.21.15.16:0/1729084041 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f06fc063800 con 0x7f06e40b6510 2024-09-21T03:45:37.899 INFO:teuthology.orchestra.run.smithi016.stdout:Backend: cephadm 2024-09-21T03:45:37.899 INFO:teuthology.orchestra.run.smithi016.stdout:Available: Yes 2024-09-21T03:45:37.899 INFO:teuthology.orchestra.run.smithi016.stdout:Paused: No 2024-09-21T03:45:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.899+0000 7f0704800640 1 -- 172.21.15.16:0/1729084041 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f06e40b6510 msgr2=0x7f06e40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.899+0000 7f0704800640 1 --2- 172.21.15.16:0/1729084041 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f06e40b6510 0x7f06e40b8a20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f06f00096c0 tx=0x7f06f00314a0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.899+0000 7f0704800640 1 -- 172.21.15.16:0/1729084041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06fc105180 msgr2=0x7f06fc075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.899+0000 7f0704800640 1 --2- 172.21.15.16:0/1729084041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06fc105180 0x7f06fc075c50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009fd0 tx=0x7f06f800ef10 comp rx=0 tx=0).stop 2024-09-21T03:45:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.899+0000 7f0704800640 1 -- 172.21.15.16:0/1729084041 shutdown_connections 2024-09-21T03:45:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.899+0000 7f0704800640 1 --2- 172.21.15.16:0/1729084041 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f06e40b6510 0x7f06e40b8a20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.899+0000 7f0704800640 1 --2- 172.21.15.16:0/1729084041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06fc105180 0x7f06fc075c50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.899+0000 7f0704800640 1 --2- 172.21.15.16:0/1729084041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc1047e0 0x7f06fc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.899+0000 7f0704800640 1 --2- 172.21.15.16:0/1729084041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06fc103750 0x7f06fc076e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.899+0000 7f0704800640 1 -- 172.21.15.16:0/1729084041 >> 172.21.15.16:0/1729084041 conn(0x7f06fc0ff0e0 msgr2=0x7f06fc107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.899+0000 7f0704800640 1 -- 172.21.15.16:0/1729084041 shutdown_connections 2024-09-21T03:45:37.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:37.899+0000 7f0704800640 1 -- 172.21.15.16:0/1729084041 wait complete. 2024-09-21T03:45:38.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:38 smithi016 ceph-mon[30375]: pgmap v398: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T03:45:38.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:38 smithi016 ceph-mon[30375]: from='client.25262 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:38.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T03:45:38.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:38 smithi134 ceph-mon[35031]: pgmap v398: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T03:45:38.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:38 smithi134 ceph-mon[35031]: from='client.25262 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:38.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:38 smithi123 ceph-mon[34673]: pgmap v398: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T03:45:38.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:38 smithi123 ceph-mon[34673]: from='client.25262 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:39.077 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.715+0000 7f93ef200640 1 -- 172.21.15.16:0/965036903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e81047e0 msgr2=0x7f93e8104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.715+0000 7f93ef200640 1 --2- 172.21.15.16:0/965036903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e81047e0 0x7f93e8104bf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009900 tx=0x7f93d002a480 comp rx=0 tx=0).stop 2024-09-21T03:45:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.715+0000 7f93ef200640 1 -- 172.21.15.16:0/965036903 shutdown_connections 2024-09-21T03:45:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.715+0000 7f93ef200640 1 --2- 172.21.15.16:0/965036903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93e8105180 0x7f93e810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.715+0000 7f93ef200640 1 --2- 172.21.15.16:0/965036903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e81047e0 0x7f93e8104bf0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.715+0000 7f93ef200640 1 --2- 172.21.15.16:0/965036903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93e8103750 0x7f93e8103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.715+0000 7f93ef200640 1 -- 172.21.15.16:0/965036903 >> 172.21.15.16:0/965036903 conn(0x7f93e80ff0c0 msgr2=0x7f93e8101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.715+0000 7f93ef200640 1 -- 172.21.15.16:0/965036903 shutdown_connections 2024-09-21T03:45:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.715+0000 7f93ef200640 1 -- 172.21.15.16:0/965036903 wait complete. 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.716+0000 7f93ef200640 1 Processor -- start 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.716+0000 7f93ef200640 1 -- start start 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.716+0000 7f93ef200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93e8103750 0x7f93e819c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.717+0000 7f93ef200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e81047e0 0x7f93e819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.717+0000 7f93ef200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93e8105180 0x7f93e819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:39.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.717+0000 7f93ef200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93e810f3a0 con 0x7f93e81047e0 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.717+0000 7f93ef200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93e810f220 con 0x7f93e8105180 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.717+0000 7f93ef200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f93e810f520 con 0x7f93e8103750 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.717+0000 7f93df400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e81047e0 0x7f93e819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.717+0000 7f93df400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e81047e0 0x7f93e819c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41144/0 (socket says 172.21.15.16:41144) 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.717+0000 7f93df400640 1 -- 172.21.15.16:0/999350439 learned_addr learned my addr 172.21.15.16:0/999350439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.717+0000 7f93e7e00640 1 --2- 172.21.15.16:0/999350439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93e8103750 0x7f93e819c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.717+0000 7f93ed200640 1 --2- 172.21.15.16:0/999350439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93e8105180 0x7f93e819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.717+0000 7f93df400640 1 -- 172.21.15.16:0/999350439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93e8103750 msgr2=0x7f93e819c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.717+0000 7f93df400640 1 --2- 172.21.15.16:0/999350439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93e8103750 0x7f93e819c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.717+0000 7f93df400640 1 -- 172.21.15.16:0/999350439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93e8105180 msgr2=0x7f93e819d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.717+0000 7f93df400640 1 --2- 172.21.15.16:0/999350439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93e8105180 0x7f93e819d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.717+0000 7f93df400640 1 -- 172.21.15.16:0/999350439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e819dc00 con 0x7f93e81047e0 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.717+0000 7f93ed200640 1 --2- 172.21.15.16:0/999350439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93e8105180 0x7f93e819d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.718+0000 7f93df400640 1 --2- 172.21.15.16:0/999350439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e81047e0 0x7f93e819c840 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f93d0006010 tx=0x7f93d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.718+0000 7f93e5600640 1 -- 172.21.15.16:0/999350439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d003f030 con 0x7f93e81047e0 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.718+0000 7f93e5600640 1 -- 172.21.15.16:0/999350439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d0031910 con 0x7f93e81047e0 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.718+0000 7f93e5600640 1 -- 172.21.15.16:0/999350439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0035070 con 0x7f93e81047e0 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.718+0000 7f93ef200640 1 -- 172.21.15.16:0/999350439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e819de90 con 0x7f93e81047e0 2024-09-21T03:45:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.718+0000 7f93ef200640 1 -- 172.21.15.16:0/999350439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e81a4420 con 0x7f93e81047e0 2024-09-21T03:45:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.720+0000 7f93ef200640 1 -- 172.21.15.16:0/999350439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e81a46d0 con 0x7f93e81047e0 2024-09-21T03:45:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.722+0000 7f93e5600640 1 -- 172.21.15.16:0/999350439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f93d0042030 con 0x7f93e81047e0 2024-09-21T03:45:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.723+0000 7f93e5600640 1 --2- 172.21.15.16:0/999350439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93cc0b6510 0x7f93cc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.724+0000 7f93e5600640 1 -- 172.21.15.16:0/999350439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f93d0107820 con 0x7f93e81047e0 2024-09-21T03:45:39.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.724+0000 7f93e7e00640 1 --2- 172.21.15.16:0/999350439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93cc0b6510 0x7f93cc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:39.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.724+0000 7f93e7e00640 1 --2- 172.21.15.16:0/999350439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93cc0b6510 0x7f93cc0b8a20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009bd0 tx=0x7f93d8005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:39.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.728+0000 7f93e5600640 1 -- 172.21.15.16:0/999350439 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f93d00d2080 con 0x7f93e81047e0 2024-09-21T03:45:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.936+0000 7f93ef200640 1 -- 172.21.15.16:0/999350439 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f93e8076d10 con 0x7f93cc0b6510 2024-09-21T03:45:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.948+0000 7f93e5600640 1 -- 172.21.15.16:0/999350439 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f93e8076d10 con 0x7f93cc0b6510 2024-09-21T03:45:39.951 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T03:45:39.951 INFO:teuthology.orchestra.run.smithi016.stdout:mds.a smithi016 running (2m) 26s ago 2m 84.2M - 19.3.0-5120-g18432218 ddce307e1f32 fc21c0a5e14e 2024-09-21T03:45:39.951 INFO:teuthology.orchestra.run.smithi016.stdout:mds.b smithi123 running (2m) 33s ago 2m 96.5M - 19.3.0-5120-g18432218 ddce307e1f32 f4db202a12dd 2024-09-21T03:45:39.952 INFO:teuthology.orchestra.run.smithi016.stdout:mds.c smithi134 running (3m) 33s ago 3m 86.6M - 19.3.0-5120-g18432218 ddce307e1f32 ca791862f453 2024-09-21T03:45:39.952 INFO:teuthology.orchestra.run.smithi016.stdout:mds.d smithi016 running (3m) 26s ago 3m 80.8M - 19.3.0-5120-g18432218 ddce307e1f32 e609d21e660b 2024-09-21T03:45:39.952 INFO:teuthology.orchestra.run.smithi016.stdout:mds.e smithi123 running (3m) 33s ago 3m 85.8M - 19.3.0-5120-g18432218 ddce307e1f32 28797bd39437 2024-09-21T03:45:39.952 INFO:teuthology.orchestra.run.smithi016.stdout:mds.f smithi134 running (3m) 33s ago 3m 87.0M - 19.3.0-5120-g18432218 ddce307e1f32 e9f1b37174f3 2024-09-21T03:45:39.952 INFO:teuthology.orchestra.run.smithi016.stdout:mds.g smithi016 running (2m) 26s ago 2m 84.2M - 19.3.0-5120-g18432218 ddce307e1f32 f21e48a25ecd 2024-09-21T03:45:39.952 INFO:teuthology.orchestra.run.smithi016.stdout:mds.h smithi123 running (3m) 33s ago 2m 88.9M - 19.3.0-5120-g18432218 ddce307e1f32 1059a271b1f2 2024-09-21T03:45:39.952 INFO:teuthology.orchestra.run.smithi016.stdout:mds.i smithi134 running (2m) 33s ago 2m 90.7M - 19.3.0-5120-g18432218 ddce307e1f32 5ec8b8f8c601 2024-09-21T03:45:39.952 INFO:teuthology.orchestra.run.smithi016.stdout:mds.j smithi016 running (3m) 26s ago 3m 80.8M - 19.3.0-5120-g18432218 ddce307e1f32 89ba4424df0b 2024-09-21T03:45:39.952 INFO:teuthology.orchestra.run.smithi016.stdout:mds.k smithi123 running (3m) 33s ago 3m 85.1M - 19.3.0-5120-g18432218 ddce307e1f32 e7bc5d6c90f6 2024-09-21T03:45:39.952 INFO:teuthology.orchestra.run.smithi016.stdout:mds.l smithi134 running (3m) 33s ago 3m 85.8M - 19.3.0-5120-g18432218 ddce307e1f32 d160eef18178 2024-09-21T03:45:39.953 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.x smithi016 *:9283,8765 running (11m) 26s ago 11m 593M - 19.3.0-5120-g18432218 ddce307e1f32 60b9b29bbfdb 2024-09-21T03:45:39.953 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.y smithi123 *:8443,8765 running (9m) 33s ago 9m 527M - 19.3.0-5120-g18432218 ddce307e1f32 40fbbada280e 2024-09-21T03:45:39.953 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.z smithi134 *:8443,8765 running (9m) 33s ago 9m 526M - 19.3.0-5120-g18432218 ddce307e1f32 3cf07047e786 2024-09-21T03:45:39.953 INFO:teuthology.orchestra.run.smithi016.stdout:mon.a smithi016 running (11m) 26s ago 11m 119M 2048M 19.3.0-5120-g18432218 ddce307e1f32 01d63f4758cf 2024-09-21T03:45:39.953 INFO:teuthology.orchestra.run.smithi016.stdout:mon.b smithi123 running (9m) 33s ago 9m 112M 2048M 19.3.0-5120-g18432218 ddce307e1f32 524ea8507232 2024-09-21T03:45:39.953 INFO:teuthology.orchestra.run.smithi016.stdout:mon.c smithi134 running (10m) 33s ago 10m 113M 2048M 19.3.0-5120-g18432218 ddce307e1f32 c60a9108c7f0 2024-09-21T03:45:39.953 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (8m) 26s ago 8m 112M 4273M 19.3.0-5120-g18432218 ddce307e1f32 dbb8feffe1b0 2024-09-21T03:45:39.953 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi123 running (8m) 33s ago 8m 121M 4272M 19.3.0-5120-g18432218 ddce307e1f32 c0f75a2d2bba 2024-09-21T03:45:39.953 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi134 running (7m) 33s ago 7m 116M 4272M 19.3.0-5120-g18432218 ddce307e1f32 a1bdb814fa5a 2024-09-21T03:45:39.953 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi016 running (7m) 26s ago 7m 111M 4273M 19.3.0-5120-g18432218 ddce307e1f32 2f0b90d7d69d 2024-09-21T03:45:39.954 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi123 running (6m) 33s ago 6m 110M 4272M 19.3.0-5120-g18432218 ddce307e1f32 a81f6c650f68 2024-09-21T03:45:39.954 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi134 running (6m) 33s ago 6m 122M 4272M 19.3.0-5120-g18432218 ddce307e1f32 1d1943ded8ac 2024-09-21T03:45:39.954 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 smithi016 running (5m) 26s ago 5m 116M 4273M 19.3.0-5120-g18432218 ddce307e1f32 28e4fbbbbf63 2024-09-21T03:45:39.954 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 smithi123 running (5m) 33s ago 5m 121M 4272M 19.3.0-5120-g18432218 ddce307e1f32 611c8a7fc0e7 2024-09-21T03:45:39.954 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 smithi134 running (5m) 33s ago 5m 114M 4272M 19.3.0-5120-g18432218 ddce307e1f32 861b3500a6ae 2024-09-21T03:45:39.954 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 smithi016 running (4m) 26s ago 4m 111M 4273M 19.3.0-5120-g18432218 ddce307e1f32 3d3215e10d76 2024-09-21T03:45:39.954 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 smithi123 running (4m) 33s ago 4m 112M 4272M 19.3.0-5120-g18432218 ddce307e1f32 13ba568f83f8 2024-09-21T03:45:39.954 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 smithi134 running (3m) 33s ago 3m 119M 4272M 19.3.0-5120-g18432218 ddce307e1f32 6390cfc5fa63 2024-09-21T03:45:39.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.950+0000 7f93ef200640 1 -- 172.21.15.16:0/999350439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93cc0b6510 msgr2=0x7f93cc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:39.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.950+0000 7f93ef200640 1 --2- 172.21.15.16:0/999350439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93cc0b6510 0x7f93cc0b8a20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009bd0 tx=0x7f93d8005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:39.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.950+0000 7f93ef200640 1 -- 172.21.15.16:0/999350439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e81047e0 msgr2=0x7f93e819c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:39.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.950+0000 7f93ef200640 1 --2- 172.21.15.16:0/999350439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e81047e0 0x7f93e819c840 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f93d0006010 tx=0x7f93d00042f0 comp rx=0 tx=0).stop 2024-09-21T03:45:39.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.951+0000 7f93ef200640 1 -- 172.21.15.16:0/999350439 shutdown_connections 2024-09-21T03:45:39.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.951+0000 7f93ef200640 1 --2- 172.21.15.16:0/999350439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f93cc0b6510 0x7f93cc0b8a20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:39.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.951+0000 7f93ef200640 1 --2- 172.21.15.16:0/999350439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93e8105180 0x7f93e819d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:39.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.951+0000 7f93ef200640 1 --2- 172.21.15.16:0/999350439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e81047e0 0x7f93e819c840 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:39.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.951+0000 7f93ef200640 1 --2- 172.21.15.16:0/999350439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93e8103750 0x7f93e819c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:39.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.951+0000 7f93ef200640 1 -- 172.21.15.16:0/999350439 >> 172.21.15.16:0/999350439 conn(0x7f93e80ff0c0 msgr2=0x7f93e80755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:39.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.951+0000 7f93ef200640 1 -- 172.21.15.16:0/999350439 shutdown_connections 2024-09-21T03:45:39.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:39.951+0000 7f93ef200640 1 -- 172.21.15.16:0/999350439 wait complete. 2024-09-21T03:45:40.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:40 smithi016 ceph-mon[30375]: pgmap v399: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-21T03:45:40.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:40 smithi016 ceph-mon[30375]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:40.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T03:45:40.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:40 smithi134 ceph-mon[35031]: pgmap v399: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-21T03:45:40.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:40 smithi134 ceph-mon[35031]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:40.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:40 smithi123 ceph-mon[34673]: pgmap v399: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-21T03:45:40.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:40 smithi123 ceph-mon[34673]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:41.210 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:41.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.889+0000 7efee5000640 1 -- 172.21.15.16:0/3405274562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee0106a00 msgr2=0x7efee0106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.889+0000 7efee5000640 1 --2- 172.21.15.16:0/3405274562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee0106a00 0x7efee0106e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efecc009900 tx=0x7efecc02a480 comp rx=0 tx=0).stop 2024-09-21T03:45:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.889+0000 7efee5000640 1 -- 172.21.15.16:0/3405274562 shutdown_connections 2024-09-21T03:45:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.889+0000 7efee5000640 1 --2- 172.21.15.16:0/3405274562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee01073a0 0x7efee010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.889+0000 7efee5000640 1 --2- 172.21.15.16:0/3405274562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee0106a00 0x7efee0106e10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.889+0000 7efee5000640 1 --2- 172.21.15.16:0/3405274562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee0105970 0x7efee0105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.889+0000 7efee5000640 1 -- 172.21.15.16:0/3405274562 >> 172.21.15.16:0/3405274562 conn(0x7efee01012c0 msgr2=0x7efee0103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.889+0000 7efee5000640 1 -- 172.21.15.16:0/3405274562 shutdown_connections 2024-09-21T03:45:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.889+0000 7efee5000640 1 -- 172.21.15.16:0/3405274562 wait complete. 2024-09-21T03:45:41.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.891+0000 7efee5000640 1 Processor -- start 2024-09-21T03:45:41.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.891+0000 7efee5000640 1 -- start start 2024-09-21T03:45:41.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.892+0000 7efee5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee0105970 0x7efee019c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:41.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.892+0000 7efee5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee0106a00 0x7efee019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:41.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.892+0000 7efee5000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee01073a0 0x7efee019d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:41.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.892+0000 7efee5000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efee01115f0 con 0x7efee0106a00 2024-09-21T03:45:41.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.892+0000 7efee5000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efee0111470 con 0x7efee0105970 2024-09-21T03:45:41.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.892+0000 7efee5000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efee0111770 con 0x7efee01073a0 2024-09-21T03:45:41.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.892+0000 7efedde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee0105970 0x7efee019c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:41.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.892+0000 7efede800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee01073a0 0x7efee019d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:41.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.892+0000 7efedde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee0105970 0x7efee019c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:45558/0 (socket says 172.21.15.16:45558) 2024-09-21T03:45:41.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.892+0000 7efede800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee01073a0 0x7efee019d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:44028/0 (socket says 172.21.15.16:44028) 2024-09-21T03:45:41.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.892+0000 7efedde00640 1 -- 172.21.15.16:0/1154740781 learned_addr learned my addr 172.21.15.16:0/1154740781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:41.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.892+0000 7efedd400640 1 --2- 172.21.15.16:0/1154740781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee0106a00 0x7efee019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:41.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.893+0000 7efedde00640 1 -- 172.21.15.16:0/1154740781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee01073a0 msgr2=0x7efee019d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:41.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.893+0000 7efedde00640 1 --2- 172.21.15.16:0/1154740781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee01073a0 0x7efee019d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:41.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.893+0000 7efedde00640 1 -- 172.21.15.16:0/1154740781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee0106a00 msgr2=0x7efee019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:41.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.893+0000 7efedde00640 1 --2- 172.21.15.16:0/1154740781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee0106a00 0x7efee019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:41.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.893+0000 7efedde00640 1 -- 172.21.15.16:0/1154740781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee019dbb0 con 0x7efee0105970 2024-09-21T03:45:41.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.893+0000 7efede800640 1 --2- 172.21.15.16:0/1154740781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee01073a0 0x7efee019d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:41.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.893+0000 7efedde00640 1 --2- 172.21.15.16:0/1154740781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee0105970 0x7efee019c180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efed000e9f0 tx=0x7efed000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:41.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.894+0000 7efed6a00640 1 -- 172.21.15.16:0/1154740781 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed000cd80 con 0x7efee0105970 2024-09-21T03:45:41.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.894+0000 7efed6a00640 1 -- 172.21.15.16:0/1154740781 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed00108e0 con 0x7efee0105970 2024-09-21T03:45:41.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.894+0000 7efee5000640 1 -- 172.21.15.16:0/1154740781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee019dea0 con 0x7efee0105970 2024-09-21T03:45:41.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.894+0000 7efed6a00640 1 -- 172.21.15.16:0/1154740781 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0018980 con 0x7efee0105970 2024-09-21T03:45:41.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.894+0000 7efee5000640 1 -- 172.21.15.16:0/1154740781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee01a45a0 con 0x7efee0105970 2024-09-21T03:45:41.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.895+0000 7efee5000640 1 -- 172.21.15.16:0/1154740781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee010cb50 con 0x7efee0105970 2024-09-21T03:45:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.898+0000 7efed6a00640 1 -- 172.21.15.16:0/1154740781 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7efed0010430 con 0x7efee0105970 2024-09-21T03:45:41.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.899+0000 7efed6a00640 1 --2- 172.21.15.16:0/1154740781 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efec40b6590 0x7efec40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:41.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.899+0000 7efed6a00640 1 -- 172.21.15.16:0/1154740781 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7efed00e7e30 con 0x7efee0105970 2024-09-21T03:45:41.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.899+0000 7efedd400640 1 --2- 172.21.15.16:0/1154740781 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efec40b6590 0x7efec40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:41.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.900+0000 7efedd400640 1 --2- 172.21.15.16:0/1154740781 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efec40b6590 0x7efec40b8aa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efecc004770 tx=0x7efecc0314a0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:41.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:41.901+0000 7efed6a00640 1 -- 172.21.15.16:0/1154740781 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efed00b2690 con 0x7efee0105970 2024-09-21T03:45:42.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:42.110+0000 7efee5000640 1 -- 172.21.15.16:0/1154740781 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7efee0063800 con 0x7efec40b6590 2024-09-21T03:45:42.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:42.116+0000 7efed6a00640 1 -- 172.21.15.16:0/1154740781 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7efee0063800 con 0x7efec40b6590 2024-09-21T03:45:42.119 INFO:teuthology.orchestra.run.smithi016.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T03:45:42.119 INFO:teuthology.orchestra.run.smithi016.stdout:mds.all 12/12 35s ago 3m smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-09-21T03:45:42.119 INFO:teuthology.orchestra.run.smithi016.stdout:mgr 3/3 35s ago 9m smithi016=x;smithi123=y;smithi134=z;count:3 2024-09-21T03:45:42.119 INFO:teuthology.orchestra.run.smithi016.stdout:mon 3/3 35s ago 10m smithi016:172.21.15.16=a;smithi123:172.21.15.123=b;smithi134:172.21.15.134=c;count:3 2024-09-21T03:45:42.119 INFO:teuthology.orchestra.run.smithi016.stdout:osd 12 35s ago - 2024-09-21T03:45:42.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:42.119+0000 7efee5000640 1 -- 172.21.15.16:0/1154740781 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efec40b6590 msgr2=0x7efec40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:42.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:42.119+0000 7efee5000640 1 --2- 172.21.15.16:0/1154740781 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efec40b6590 0x7efec40b8aa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efecc004770 tx=0x7efecc0314a0 comp rx=0 tx=0).stop 2024-09-21T03:45:42.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:42.119+0000 7efee5000640 1 -- 172.21.15.16:0/1154740781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee0105970 msgr2=0x7efee019c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:42.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:42.119+0000 7efee5000640 1 --2- 172.21.15.16:0/1154740781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee0105970 0x7efee019c180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efed000e9f0 tx=0x7efed000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:45:42.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:42.120+0000 7efee5000640 1 -- 172.21.15.16:0/1154740781 shutdown_connections 2024-09-21T03:45:42.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:42.120+0000 7efee5000640 1 --2- 172.21.15.16:0/1154740781 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efec40b6590 0x7efec40b8aa0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:42.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:42.120+0000 7efee5000640 1 --2- 172.21.15.16:0/1154740781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efee01073a0 0x7efee019d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:42.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:42.120+0000 7efee5000640 1 --2- 172.21.15.16:0/1154740781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee0106a00 0x7efee019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:42.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:42.120+0000 7efee5000640 1 --2- 172.21.15.16:0/1154740781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee0105970 0x7efee019c180 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:42.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:42.120+0000 7efee5000640 1 -- 172.21.15.16:0/1154740781 >> 172.21.15.16:0/1154740781 conn(0x7efee01012c0 msgr2=0x7efee0109d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:42.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:42.120+0000 7efee5000640 1 -- 172.21.15.16:0/1154740781 shutdown_connections 2024-09-21T03:45:42.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:42.120+0000 7efee5000640 1 -- 172.21.15.16:0/1154740781 wait complete. 2024-09-21T03:45:42.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:42 smithi134 ceph-mon[35031]: pgmap v400: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:42.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:42 smithi123 ceph-mon[34673]: pgmap v400: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:42.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:42 smithi016 ceph-mon[30375]: pgmap v400: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:43.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T03:45:43.426 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:43.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:43 smithi134 ceph-mon[35031]: from='client.25274 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:43.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:43 smithi016 ceph-mon[30375]: from='client.25274 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:43.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:43 smithi123 ceph-mon[34673]: from='client.25274 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:44.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.056+0000 7faf29c00640 1 -- 172.21.15.16:0/42518055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf24106820 msgr2=0x7faf24106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.056+0000 7faf29c00640 1 --2- 172.21.15.16:0/42518055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf24106820 0x7faf24106c30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009900 tx=0x7faf0c02a480 comp rx=0 tx=0).stop 2024-09-21T03:45:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.056+0000 7faf29c00640 1 -- 172.21.15.16:0/42518055 shutdown_connections 2024-09-21T03:45:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.056+0000 7faf29c00640 1 --2- 172.21.15.16:0/42518055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf241071c0 0x7faf2410db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.056+0000 7faf29c00640 1 --2- 172.21.15.16:0/42518055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf24106820 0x7faf24106c30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.056+0000 7faf29c00640 1 --2- 172.21.15.16:0/42518055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf24105790 0x7faf24105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.056+0000 7faf29c00640 1 -- 172.21.15.16:0/42518055 >> 172.21.15.16:0/42518055 conn(0x7faf24101100 msgr2=0x7faf241035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.056+0000 7faf29c00640 1 -- 172.21.15.16:0/42518055 shutdown_connections 2024-09-21T03:45:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.056+0000 7faf29c00640 1 -- 172.21.15.16:0/42518055 wait complete. 2024-09-21T03:45:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.058+0000 7faf29c00640 1 Processor -- start 2024-09-21T03:45:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.058+0000 7faf29c00640 1 -- start start 2024-09-21T03:45:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.058+0000 7faf29c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf24105790 0x7faf2419bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.058+0000 7faf29c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf24106820 0x7faf2419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.058+0000 7faf29c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf241071c0 0x7faf2419d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.058+0000 7faf29c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf241113c0 con 0x7faf241071c0 2024-09-21T03:45:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.058+0000 7faf29c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faf24111240 con 0x7faf24106820 2024-09-21T03:45:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.058+0000 7faf29c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faf24111540 con 0x7faf24105790 2024-09-21T03:45:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.059+0000 7faf1be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf24106820 0x7faf2419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.059+0000 7faf23e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf24105790 0x7faf2419bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.059+0000 7faf1be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf24106820 0x7faf2419c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:45582/0 (socket says 172.21.15.16:45582) 2024-09-21T03:45:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.059+0000 7faf1be00640 1 -- 172.21.15.16:0/2952395517 learned_addr learned my addr 172.21.15.16:0/2952395517 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.059+0000 7faf29200640 1 --2- 172.21.15.16:0/2952395517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf241071c0 0x7faf2419d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.059+0000 7faf1be00640 1 -- 172.21.15.16:0/2952395517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf24105790 msgr2=0x7faf2419bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.059+0000 7faf1be00640 1 --2- 172.21.15.16:0/2952395517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf24105790 0x7faf2419bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.059+0000 7faf1be00640 1 -- 172.21.15.16:0/2952395517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf241071c0 msgr2=0x7faf2419d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.059+0000 7faf1be00640 1 --2- 172.21.15.16:0/2952395517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf241071c0 0x7faf2419d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.059+0000 7faf1be00640 1 -- 172.21.15.16:0/2952395517 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2419da20 con 0x7faf24106820 2024-09-21T03:45:44.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.059+0000 7faf23e00640 1 --2- 172.21.15.16:0/2952395517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf24105790 0x7faf2419bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:44.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.060+0000 7faf1be00640 1 --2- 172.21.15.16:0/2952395517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf24106820 0x7faf2419c660 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faf0c006010 tx=0x7faf0c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:44.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.060+0000 7faf21600640 1 -- 172.21.15.16:0/2952395517 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c03f030 con 0x7faf24106820 2024-09-21T03:45:44.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.060+0000 7faf29c00640 1 -- 172.21.15.16:0/2952395517 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2419dcb0 con 0x7faf24106820 2024-09-21T03:45:44.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.060+0000 7faf21600640 1 -- 172.21.15.16:0/2952395517 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf0c031910 con 0x7faf24106820 2024-09-21T03:45:44.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.060+0000 7faf21600640 1 -- 172.21.15.16:0/2952395517 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c035070 con 0x7faf24106820 2024-09-21T03:45:44.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.060+0000 7faf29c00640 1 -- 172.21.15.16:0/2952395517 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf241a4240 con 0x7faf24106820 2024-09-21T03:45:44.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.061+0000 7faf29c00640 1 -- 172.21.15.16:0/2952395517 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf2410c950 con 0x7faf24106820 2024-09-21T03:45:44.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.064+0000 7faf21600640 1 -- 172.21.15.16:0/2952395517 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7faf0c042030 con 0x7faf24106820 2024-09-21T03:45:44.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.065+0000 7faf21600640 1 --2- 172.21.15.16:0/2952395517 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faf080b6510 0x7faf080b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:44.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.065+0000 7faf21600640 1 -- 172.21.15.16:0/2952395517 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7faf0c108240 con 0x7faf24106820 2024-09-21T03:45:44.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.065+0000 7faf23e00640 1 --2- 172.21.15.16:0/2952395517 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faf080b6510 0x7faf080b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:44.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.066+0000 7faf23e00640 1 --2- 172.21.15.16:0/2952395517 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faf080b6510 0x7faf080b8a20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faf14009fc0 tx=0x7faf14005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:44.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.067+0000 7faf21600640 1 -- 172.21.15.16:0/2952395517 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faf0c0d2aa0 con 0x7faf24106820 2024-09-21T03:45:44.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.270+0000 7faf29c00640 1 -- 172.21.15.16:0/2952395517 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7faf24063800 con 0x7faf080b6510 2024-09-21T03:45:44.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.271+0000 7faf21600640 1 -- 172.21.15.16:0/2952395517 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7faf24063800 con 0x7faf080b6510 2024-09-21T03:45:44.274 INFO:teuthology.orchestra.run.smithi016.stdout:HOST ADDR LABELS STATUS 2024-09-21T03:45:44.274 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 172.21.15.16 2024-09-21T03:45:44.275 INFO:teuthology.orchestra.run.smithi016.stdout:smithi123 172.21.15.123 2024-09-21T03:45:44.275 INFO:teuthology.orchestra.run.smithi016.stdout:smithi134 172.21.15.134 2024-09-21T03:45:44.275 INFO:teuthology.orchestra.run.smithi016.stdout:3 hosts in cluster 2024-09-21T03:45:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.274+0000 7faf29c00640 1 -- 172.21.15.16:0/2952395517 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faf080b6510 msgr2=0x7faf080b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.274+0000 7faf29c00640 1 --2- 172.21.15.16:0/2952395517 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faf080b6510 0x7faf080b8a20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faf14009fc0 tx=0x7faf14005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.274+0000 7faf29c00640 1 -- 172.21.15.16:0/2952395517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf24106820 msgr2=0x7faf2419c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.274+0000 7faf29c00640 1 --2- 172.21.15.16:0/2952395517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf24106820 0x7faf2419c660 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faf0c006010 tx=0x7faf0c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:45:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.274+0000 7faf29c00640 1 -- 172.21.15.16:0/2952395517 shutdown_connections 2024-09-21T03:45:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.274+0000 7faf29c00640 1 --2- 172.21.15.16:0/2952395517 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7faf080b6510 0x7faf080b8a20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.274+0000 7faf29c00640 1 --2- 172.21.15.16:0/2952395517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf241071c0 0x7faf2419d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.274+0000 7faf29c00640 1 --2- 172.21.15.16:0/2952395517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf24106820 0x7faf2419c660 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.274+0000 7faf29c00640 1 --2- 172.21.15.16:0/2952395517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf24105790 0x7faf2419bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.274+0000 7faf29c00640 1 -- 172.21.15.16:0/2952395517 >> 172.21.15.16:0/2952395517 conn(0x7faf24101100 msgr2=0x7faf24109b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.274+0000 7faf29c00640 1 -- 172.21.15.16:0/2952395517 shutdown_connections 2024-09-21T03:45:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:44.274+0000 7faf29c00640 1 -- 172.21.15.16:0/2952395517 wait complete. 2024-09-21T03:45:44.654 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:44 smithi016 ceph-mon[30375]: pgmap v401: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:45:44.807 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:44 smithi123 ceph-mon[34673]: pgmap v401: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:45:44.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:44 smithi134 ceph-mon[35031]: pgmap v401: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:45:44.995 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T03:45:45.376 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:45.843 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:45 smithi016 ceph-mon[30375]: from='client.25280 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:45.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:45 smithi134 ceph-mon[35031]: from='client.25280 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:45.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:45 smithi123 ceph-mon[34673]: from='client.25280 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.024+0000 7f8c18800640 1 -- 172.21.15.16:0/1350760177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c10103750 msgr2=0x7f8c10103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.024+0000 7f8c18800640 1 --2- 172.21.15.16:0/1350760177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c10103750 0x7f8c10103bc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009900 tx=0x7f8c0402a480 comp rx=0 tx=0).stop 2024-09-21T03:45:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.024+0000 7f8c18800640 1 -- 172.21.15.16:0/1350760177 shutdown_connections 2024-09-21T03:45:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.024+0000 7f8c18800640 1 --2- 172.21.15.16:0/1350760177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c10105180 0x7f8c1010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.024+0000 7f8c18800640 1 --2- 172.21.15.16:0/1350760177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c101047e0 0x7f8c10104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.024+0000 7f8c18800640 1 --2- 172.21.15.16:0/1350760177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c10103750 0x7f8c10103bc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.024+0000 7f8c18800640 1 -- 172.21.15.16:0/1350760177 >> 172.21.15.16:0/1350760177 conn(0x7f8c100ff0c0 msgr2=0x7f8c10101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.024+0000 7f8c18800640 1 -- 172.21.15.16:0/1350760177 shutdown_connections 2024-09-21T03:45:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.024+0000 7f8c18800640 1 -- 172.21.15.16:0/1350760177 wait complete. 2024-09-21T03:45:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.025+0000 7f8c18800640 1 Processor -- start 2024-09-21T03:45:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.025+0000 7f8c18800640 1 -- start start 2024-09-21T03:45:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.026+0000 7f8c18800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c10103750 0x7f8c1019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:46.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.026+0000 7f8c18800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c101047e0 0x7f8c1019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:46.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.026+0000 7f8c18800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c10105180 0x7f8c1019d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:46.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.026+0000 7f8c18800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1010f390 con 0x7f8c10103750 2024-09-21T03:45:46.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.026+0000 7f8c18800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1010f210 con 0x7f8c10105180 2024-09-21T03:45:46.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.026+0000 7f8c18800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1010f510 con 0x7f8c101047e0 2024-09-21T03:45:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.026+0000 7f8c15e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c10103750 0x7f8c1019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.026+0000 7f8c15e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c10103750 0x7f8c1019c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36024/0 (socket says 172.21.15.16:36024) 2024-09-21T03:45:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.026+0000 7f8c15e00640 1 -- 172.21.15.16:0/3041320636 learned_addr learned my addr 172.21.15.16:0/3041320636 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.026+0000 7f8c16800640 1 --2- 172.21.15.16:0/3041320636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c10105180 0x7f8c1019d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.026+0000 7f8c15400640 1 --2- 172.21.15.16:0/3041320636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c101047e0 0x7f8c1019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.027+0000 7f8c15e00640 1 -- 172.21.15.16:0/3041320636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c101047e0 msgr2=0x7f8c1019c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.027+0000 7f8c15e00640 1 --2- 172.21.15.16:0/3041320636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c101047e0 0x7f8c1019c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.027+0000 7f8c15e00640 1 -- 172.21.15.16:0/3041320636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c10105180 msgr2=0x7f8c1019d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.027+0000 7f8c15e00640 1 --2- 172.21.15.16:0/3041320636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c10105180 0x7f8c1019d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.027+0000 7f8c15e00640 1 -- 172.21.15.16:0/3041320636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1019dc00 con 0x7f8c10103750 2024-09-21T03:45:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.027+0000 7f8c16800640 1 --2- 172.21.15.16:0/3041320636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c10105180 0x7f8c1019d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.027+0000 7f8c15e00640 1 --2- 172.21.15.16:0/3041320636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c10103750 0x7f8c1019c1d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8c04006010 tx=0x7f8c040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.027+0000 7f8c0aa00640 1 -- 172.21.15.16:0/3041320636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0403f030 con 0x7f8c10103750 2024-09-21T03:45:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.027+0000 7f8c18800640 1 -- 172.21.15.16:0/3041320636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1019de90 con 0x7f8c10103750 2024-09-21T03:45:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.027+0000 7f8c0aa00640 1 -- 172.21.15.16:0/3041320636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c04031910 con 0x7f8c10103750 2024-09-21T03:45:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.027+0000 7f8c0aa00640 1 -- 172.21.15.16:0/3041320636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04035070 con 0x7f8c10103750 2024-09-21T03:45:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.028+0000 7f8c18800640 1 -- 172.21.15.16:0/3041320636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c101a4420 con 0x7f8c10103750 2024-09-21T03:45:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.029+0000 7f8c18800640 1 -- 172.21.15.16:0/3041320636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c101a4740 con 0x7f8c10103750 2024-09-21T03:45:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.031+0000 7f8c0aa00640 1 -- 172.21.15.16:0/3041320636 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8c04042030 con 0x7f8c10103750 2024-09-21T03:45:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.033+0000 7f8c0aa00640 1 --2- 172.21.15.16:0/3041320636 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8bf80b6510 0x7f8bf80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:46.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.033+0000 7f8c0aa00640 1 -- 172.21.15.16:0/3041320636 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f8c041082c0 con 0x7f8c10103750 2024-09-21T03:45:46.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.033+0000 7f8c15400640 1 --2- 172.21.15.16:0/3041320636 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8bf80b6510 0x7f8bf80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:46.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.034+0000 7f8c15400640 1 --2- 172.21.15.16:0/3041320636 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8bf80b6510 0x7f8bf80b8a20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009bd0 tx=0x7f8c00005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:46.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.035+0000 7f8c0aa00640 1 -- 172.21.15.16:0/3041320636 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8c040d2b20 con 0x7f8c10103750 2024-09-21T03:45:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.241+0000 7f8c18800640 1 -- 172.21.15.16:0/3041320636 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f8c10076d10 con 0x7f8bf80b6510 2024-09-21T03:45:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.243+0000 7f8c0aa00640 1 -- 172.21.15.16:0/3041320636 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f8c10076d10 con 0x7f8bf80b6510 2024-09-21T03:45:46.246 INFO:teuthology.orchestra.run.smithi016.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T03:45:46.246 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002C480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-09-21T03:45:46.246 INFO:teuthology.orchestra.run.smithi016.stdout:smithi123 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400YK400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-09-21T03:45:46.246 INFO:teuthology.orchestra.run.smithi016.stdout:smithi134 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62420002400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T03:45:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.246+0000 7f8c18800640 1 -- 172.21.15.16:0/3041320636 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8bf80b6510 msgr2=0x7f8bf80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.246+0000 7f8c18800640 1 --2- 172.21.15.16:0/3041320636 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8bf80b6510 0x7f8bf80b8a20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009bd0 tx=0x7f8c00005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.246+0000 7f8c18800640 1 -- 172.21.15.16:0/3041320636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c10103750 msgr2=0x7f8c1019c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.246+0000 7f8c18800640 1 --2- 172.21.15.16:0/3041320636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c10103750 0x7f8c1019c1d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8c04006010 tx=0x7f8c040042f0 comp rx=0 tx=0).stop 2024-09-21T03:45:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.246+0000 7f8c18800640 1 -- 172.21.15.16:0/3041320636 shutdown_connections 2024-09-21T03:45:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.246+0000 7f8c18800640 1 --2- 172.21.15.16:0/3041320636 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8bf80b6510 0x7f8bf80b8a20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.246+0000 7f8c18800640 1 --2- 172.21.15.16:0/3041320636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c10105180 0x7f8c1019d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.246+0000 7f8c18800640 1 --2- 172.21.15.16:0/3041320636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c101047e0 0x7f8c1019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.246+0000 7f8c18800640 1 --2- 172.21.15.16:0/3041320636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c10103750 0x7f8c1019c1d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.246+0000 7f8c18800640 1 -- 172.21.15.16:0/3041320636 >> 172.21.15.16:0/3041320636 conn(0x7f8c100ff0c0 msgr2=0x7f8c100755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.246+0000 7f8c18800640 1 -- 172.21.15.16:0/3041320636 shutdown_connections 2024-09-21T03:45:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:46.246+0000 7f8c18800640 1 -- 172.21.15.16:0/3041320636 wait complete. 2024-09-21T03:45:46.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:46 smithi134 ceph-mon[35031]: pgmap v402: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:45:46.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:46 smithi123 ceph-mon[34673]: pgmap v402: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:45:46.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:46 smithi016 ceph-mon[30375]: pgmap v402: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:45:47.187 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T03:45:47.199 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-21T03:45:47.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T03:45:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:47.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:47 smithi016 ceph-mon[30375]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:47.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:47 smithi134 ceph-mon[35031]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:47.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:47 smithi123 ceph-mon[34673]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:45:48.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.239+0000 7f4c9e400640 1 -- 172.21.15.16:0/1245192927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c981092b0 msgr2=0x7f4c9810b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:48.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.239+0000 7f4c9e400640 1 --2- 172.21.15.16:0/1245192927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c981092b0 0x7f4c9810b7c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009ca0 tx=0x7f4c9002a8b0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.239+0000 7f4c9e400640 1 -- 172.21.15.16:0/1245192927 shutdown_connections 2024-09-21T03:45:48.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.239+0000 7f4c9e400640 1 --2- 172.21.15.16:0/1245192927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c981092b0 0x7f4c9810b7c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.239+0000 7f4c9e400640 1 --2- 172.21.15.16:0/1245192927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c98106710 0x7f4c98108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.239+0000 7f4c9e400640 1 --2- 172.21.15.16:0/1245192927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c9806c420 0x7f4c981061d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.239+0000 7f4c9e400640 1 -- 172.21.15.16:0/1245192927 >> 172.21.15.16:0/1245192927 conn(0x7f4c980fde10 msgr2=0x7f4c981002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:48.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.239+0000 7f4c9e400640 1 -- 172.21.15.16:0/1245192927 shutdown_connections 2024-09-21T03:45:48.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.239+0000 7f4c9e400640 1 -- 172.21.15.16:0/1245192927 wait complete. 2024-09-21T03:45:48.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.240+0000 7f4c9e400640 1 Processor -- start 2024-09-21T03:45:48.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.241+0000 7f4c9e400640 1 -- start start 2024-09-21T03:45:48.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.241+0000 7f4c9e400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c9806c420 0x7f4c98199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:48.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.241+0000 7f4c9e400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c98106710 0x7f4c9819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:48.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.241+0000 7f4c9e400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c981092b0 0x7f4c9819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:48.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.241+0000 7f4c9e400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9810f200 con 0x7f4c981092b0 2024-09-21T03:45:48.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.241+0000 7f4c9e400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9810f080 con 0x7f4c9806c420 2024-09-21T03:45:48.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.241+0000 7f4c9e400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9810f380 con 0x7f4c98106710 2024-09-21T03:45:48.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.241+0000 7f4c9da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c981092b0 0x7f4c9819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:48.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.241+0000 7f4c9d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c9806c420 0x7f4c98199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:48.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.241+0000 7f4c8fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c98106710 0x7f4c9819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:48.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.241+0000 7f4c9da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c981092b0 0x7f4c9819afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36038/0 (socket says 172.21.15.16:36038) 2024-09-21T03:45:48.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.241+0000 7f4c8fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c98106710 0x7f4c9819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:44098/0 (socket says 172.21.15.16:44098) 2024-09-21T03:45:48.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.241+0000 7f4c9da00640 1 -- 172.21.15.16:0/621913087 learned_addr learned my addr 172.21.15.16:0/621913087 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:48.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.241+0000 7f4c9d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c9806c420 0x7f4c98199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:45622/0 (socket says 172.21.15.16:45622) 2024-09-21T03:45:48.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.242+0000 7f4c8fe00640 1 -- 172.21.15.16:0/621913087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c9806c420 msgr2=0x7f4c98199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:48.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.242+0000 7f4c8fe00640 1 --2- 172.21.15.16:0/621913087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c9806c420 0x7f4c98199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.242+0000 7f4c8fe00640 1 -- 172.21.15.16:0/621913087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c981092b0 msgr2=0x7f4c9819afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:48.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.242+0000 7f4c8fe00640 1 --2- 172.21.15.16:0/621913087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c981092b0 0x7f4c9819afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.242+0000 7f4c8fe00640 1 -- 172.21.15.16:0/621913087 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9819b7c0 con 0x7f4c98106710 2024-09-21T03:45:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.242+0000 7f4c9d000640 1 --2- 172.21.15.16:0/621913087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c9806c420 0x7f4c98199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.242+0000 7f4c9da00640 1 --2- 172.21.15.16:0/621913087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c981092b0 0x7f4c9819afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.242+0000 7f4c8fe00640 1 --2- 172.21.15.16:0/621913087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c98106710 0x7f4c9819a400 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000d6d0 tx=0x7f4c8000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.243+0000 7f4c96000640 1 -- 172.21.15.16:0/621913087 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80017070 con 0x7f4c98106710 2024-09-21T03:45:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.243+0000 7f4c9e400640 1 -- 172.21.15.16:0/621913087 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9819bab0 con 0x7f4c98106710 2024-09-21T03:45:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.243+0000 7f4c96000640 1 -- 172.21.15.16:0/621913087 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c80009820 con 0x7f4c98106710 2024-09-21T03:45:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.243+0000 7f4c96000640 1 -- 172.21.15.16:0/621913087 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80004c40 con 0x7f4c98106710 2024-09-21T03:45:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.243+0000 7f4c9e400640 1 -- 172.21.15.16:0/621913087 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c981a21b0 con 0x7f4c98106710 2024-09-21T03:45:48.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.244+0000 7f4c9e400640 1 -- 172.21.15.16:0/621913087 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c98106b60 con 0x7f4c98106710 2024-09-21T03:45:48.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.248+0000 7f4c96000640 1 -- 172.21.15.16:0/621913087 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4c80005220 con 0x7f4c98106710 2024-09-21T03:45:48.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.248+0000 7f4c96000640 1 --2- 172.21.15.16:0/621913087 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4c7c0b6590 0x7f4c7c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:48.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.248+0000 7f4c96000640 1 -- 172.21.15.16:0/621913087 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f4c800e7550 con 0x7f4c98106710 2024-09-21T03:45:48.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.249+0000 7f4c9d000640 1 --2- 172.21.15.16:0/621913087 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4c7c0b6590 0x7f4c7c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:48.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.249+0000 7f4c9d000640 1 --2- 172.21.15.16:0/621913087 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4c7c0b6590 0x7f4c7c0b8aa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800bfd0 tx=0x7f4c8800b390 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:48.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.250+0000 7f4c96000640 1 -- 172.21.15.16:0/621913087 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c800b1e30 con 0x7f4c98106710 2024-09-21T03:45:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.490+0000 7f4c9e400640 1 -- 172.21.15.16:0/621913087 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f4c9802e640 con 0x7f4c98106710 2024-09-21T03:45:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.491+0000 7f4c96000640 1 -- 172.21.15.16:0/621913087 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3128 (secure 0 0 0) 0x7f4c800b6020 con 0x7f4c98106710 2024-09-21T03:45:48.494 INFO:teuthology.orchestra.run.smithi016.stdout:e29 2024-09-21T03:45:48.494 INFO:teuthology.orchestra.run.smithi016.stdout:btime 2024-09-21T03:44:08:542131+0000 2024-09-21T03:45:48.494 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:45:48.494 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:45:48.494 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-09-21T03:45:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:45:48.494 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-09-21T03:45:48.494 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-09-21T03:45:48.495 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 29 2024-09-21T03:45:48.495 INFO:teuthology.orchestra.run.smithi016.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T03:45:48.495 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-21T03:43:09.332305+0000 2024-09-21T03:45:48.495 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-09-21T03:44:07.645054+0000 2024-09-21T03:45:48.495 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-09-21T03:45:48.495 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-09-21T03:45:48.495 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 300 2024-09-21T03:45:48.495 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-09-21T03:45:48.495 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-09-21T03:45:48.495 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-09-21T03:45:48.495 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-09-21T03:45:48.496 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-09-21T03:45:48.496 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 0 2024-09-21T03:45:48.496 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:45:48.496 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 5 2024-09-21T03:45:48.496 INFO:teuthology.orchestra.run.smithi016.stdout:in 0,1,2,3,4 2024-09-21T03:45:48.496 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24731,1=24689,2=24682,3=24670,4=24668} 2024-09-21T03:45:48.496 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-09-21T03:45:48.496 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-09-21T03:45:48.496 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-09-21T03:45:48.496 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3,4] 2024-09-21T03:45:48.496 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-09-21T03:45:48.496 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data disabled 2024-09-21T03:45:48.497 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-09-21T03:45:48.497 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-09-21T03:45:48.497 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 0 2024-09-21T03:45:48.497 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 24731 members: 24668,24670,24682,24689,24731 2024-09-21T03:45:48.497 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b{0:24731} state up:active seq 6 addr [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.497 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.f{0:24658} state up:standby-replay seq 1 addr [v2:172.21.15.134:6834/135285598,v1:172.21.15.134:6835/135285598] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.497 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.h{1:24689} state up:active seq 22 addr [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.497 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.k{1:24632} state up:standby-replay seq 1 addr [v2:172.21.15.123:6832/3581513383,v1:172.21.15.123:6833/3581513383] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.497 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.i{2:24682} state up:active seq 22 addr [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.497 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.l{2:24613} state up:standby-replay seq 1 addr [v2:172.21.15.134:6832/2239407349,v1:172.21.15.134:6833/2239407349] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.497 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c{3:24670} state up:active seq 23 addr [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.497 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a{3:14712} state up:standby-replay seq 1 addr [v2:172.21.15.16:6840/1546097467,v1:172.21.15.16:6841/1546097467] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.497 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.e{4:24668} state up:active seq 25 addr [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.498 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.g{4:14700} state up:standby-replay seq 1 addr [v2:172.21.15.16:6838/89613887,v1:172.21.15.16:6839/89613887] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.498 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:45:48.498 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:45:48.498 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-09-21T03:45:48.498 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:45:48.498 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.j{-1:14640} state up:standby seq 1 addr [v2:172.21.15.16:6834/2280740148,v1:172.21.15.16:6835/2280740148] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.498 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d{-1:14652} state up:standby seq 1 addr [v2:172.21.15.16:6836/1667623391,v1:172.21.15.16:6837/1667623391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:45:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-09-21T03:45:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.493+0000 7f4c9e400640 1 -- 172.21.15.16:0/621913087 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4c7c0b6590 msgr2=0x7f4c7c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.493+0000 7f4c9e400640 1 --2- 172.21.15.16:0/621913087 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4c7c0b6590 0x7f4c7c0b8aa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800bfd0 tx=0x7f4c8800b390 comp rx=0 tx=0).stop 2024-09-21T03:45:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.493+0000 7f4c9e400640 1 -- 172.21.15.16:0/621913087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c98106710 msgr2=0x7f4c9819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.493+0000 7f4c9e400640 1 --2- 172.21.15.16:0/621913087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c98106710 0x7f4c9819a400 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000d6d0 tx=0x7f4c8000db90 comp rx=0 tx=0).stop 2024-09-21T03:45:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.493+0000 7f4c9e400640 1 -- 172.21.15.16:0/621913087 shutdown_connections 2024-09-21T03:45:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.493+0000 7f4c9e400640 1 --2- 172.21.15.16:0/621913087 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4c7c0b6590 0x7f4c7c0b8aa0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.493+0000 7f4c9e400640 1 --2- 172.21.15.16:0/621913087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c981092b0 0x7f4c9819afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.493+0000 7f4c9e400640 1 --2- 172.21.15.16:0/621913087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c98106710 0x7f4c9819a400 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.493+0000 7f4c9e400640 1 --2- 172.21.15.16:0/621913087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c9806c420 0x7f4c98199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.493+0000 7f4c9e400640 1 -- 172.21.15.16:0/621913087 >> 172.21.15.16:0/621913087 conn(0x7f4c980fde10 msgr2=0x7f4c98104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.494+0000 7f4c9e400640 1 -- 172.21.15.16:0/621913087 shutdown_connections 2024-09-21T03:45:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:48.494+0000 7f4c9e400640 1 -- 172.21.15.16:0/621913087 wait complete. 2024-09-21T03:45:48.842 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:48 smithi016 ceph-mon[30375]: pgmap v403: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:48.842 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:48 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/621913087' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:45:48.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:48 smithi134 ceph-mon[35031]: pgmap v403: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:48.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:48 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/621913087' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:45:48.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:48 smithi123 ceph-mon[34673]: pgmap v403: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:48.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:48 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/621913087' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:45:49.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T03:45:49.613 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:50.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.266+0000 7f353e000640 1 -- 172.21.15.16:0/1823923264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35381047e0 msgr2=0x7f3538104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:50.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.266+0000 7f353e000640 1 --2- 172.21.15.16:0/1823923264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35381047e0 0x7f3538104bf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3520009900 tx=0x7f352002a480 comp rx=0 tx=0).stop 2024-09-21T03:45:50.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.266+0000 7f353e000640 1 -- 172.21.15.16:0/1823923264 shutdown_connections 2024-09-21T03:45:50.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.266+0000 7f353e000640 1 --2- 172.21.15.16:0/1823923264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3538105180 0x7f353810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.266+0000 7f353e000640 1 --2- 172.21.15.16:0/1823923264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35381047e0 0x7f3538104bf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.266+0000 7f353e000640 1 --2- 172.21.15.16:0/1823923264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3538103750 0x7f3538103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.266+0000 7f353e000640 1 -- 172.21.15.16:0/1823923264 >> 172.21.15.16:0/1823923264 conn(0x7f35380ff0c0 msgr2=0x7f3538101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.266+0000 7f353e000640 1 -- 172.21.15.16:0/1823923264 shutdown_connections 2024-09-21T03:45:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.266+0000 7f353e000640 1 -- 172.21.15.16:0/1823923264 wait complete. 2024-09-21T03:45:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.267+0000 7f353e000640 1 Processor -- start 2024-09-21T03:45:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.268+0000 7f353e000640 1 -- start start 2024-09-21T03:45:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.268+0000 7f353e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3538103750 0x7f3538076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.268+0000 7f353e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35381047e0 0x7f3538075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.268+0000 7f353e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3538105180 0x7f3538075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.268+0000 7f353e000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f353810f3f0 con 0x7f35381047e0 2024-09-21T03:45:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.268+0000 7f353e000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f353810f270 con 0x7f3538105180 2024-09-21T03:45:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.268+0000 7f353e000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f353810f570 con 0x7f3538103750 2024-09-21T03:45:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.268+0000 7f3537400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3538103750 0x7f3538076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.268+0000 7f352ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35381047e0 0x7f3538075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.268+0000 7f3537400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3538103750 0x7f3538076e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:44114/0 (socket says 172.21.15.16:44114) 2024-09-21T03:45:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.268+0000 7f3537400640 1 -- 172.21.15.16:0/43174561 learned_addr learned my addr 172.21.15.16:0/43174561 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.269+0000 7f3537e00640 1 --2- 172.21.15.16:0/43174561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3538105180 0x7f3538075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.269+0000 7f3537400640 1 -- 172.21.15.16:0/43174561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3538105180 msgr2=0x7f3538075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.269+0000 7f3537400640 1 --2- 172.21.15.16:0/43174561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3538105180 0x7f3538075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.269+0000 7f3537400640 1 -- 172.21.15.16:0/43174561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35381047e0 msgr2=0x7f3538075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.269+0000 7f3537400640 1 --2- 172.21.15.16:0/43174561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35381047e0 0x7f3538075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.269+0000 7f3537400640 1 -- 172.21.15.16:0/43174561 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3538076350 con 0x7f3538103750 2024-09-21T03:45:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.269+0000 7f3537e00640 1 --2- 172.21.15.16:0/43174561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3538105180 0x7f3538075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.269+0000 7f3537400640 1 --2- 172.21.15.16:0/43174561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3538103750 0x7f3538076e40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f352800e9f0 tx=0x7f352800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.270+0000 7f3534c00640 1 -- 172.21.15.16:0/43174561 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352800cd80 con 0x7f3538103750 2024-09-21T03:45:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.270+0000 7f353e000640 1 -- 172.21.15.16:0/43174561 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35381a4040 con 0x7f3538103750 2024-09-21T03:45:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.270+0000 7f3534c00640 1 -- 172.21.15.16:0/43174561 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35280054d0 con 0x7f3538103750 2024-09-21T03:45:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.270+0000 7f3534c00640 1 -- 172.21.15.16:0/43174561 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528010690 con 0x7f3538103750 2024-09-21T03:45:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.270+0000 7f353e000640 1 -- 172.21.15.16:0/43174561 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35381a44a0 con 0x7f3538103750 2024-09-21T03:45:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.271+0000 7f353e000640 1 -- 172.21.15.16:0/43174561 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f353810a930 con 0x7f3538103750 2024-09-21T03:45:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.274+0000 7f3534c00640 1 -- 172.21.15.16:0/43174561 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3528004e30 con 0x7f3538103750 2024-09-21T03:45:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.275+0000 7f3534c00640 1 --2- 172.21.15.16:0/43174561 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f351c0b6590 0x7f351c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.275+0000 7f3534c00640 1 -- 172.21.15.16:0/43174561 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f35280e7c40 con 0x7f3538103750 2024-09-21T03:45:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.275+0000 7f352ea00640 1 --2- 172.21.15.16:0/43174561 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f351c0b6590 0x7f351c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:50.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.276+0000 7f352ea00640 1 --2- 172.21.15.16:0/43174561 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f351c0b6590 0x7f351c0b8aa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3520004770 tx=0x7f35200314a0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.277+0000 7f3534c00640 1 -- 172.21.15.16:0/43174561 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f35280ae0c0 con 0x7f3538103750 2024-09-21T03:45:50.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.478+0000 7f353e000640 1 -- 172.21.15.16:0/43174561 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f353802e640 con 0x7f3538103750 2024-09-21T03:45:50.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.480+0000 7f3534c00640 1 -- 172.21.15.16:0/43174561 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v86) ==== 56+0+5682 (secure 0 0 0) 0x7f35280b24a0 con 0x7f3538103750 2024-09-21T03:45:50.482 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 86 2024-09-21T03:45:50.482 INFO:teuthology.orchestra.run.smithi016.stdout:fsid 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T03:45:50.482 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-21T03:33:53.220358+0000 2024-09-21T03:45:50.482 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-09-21T03:44:54.658912+0000 2024-09-21T03:45:50.482 INFO:teuthology.orchestra.run.smithi016.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T03:45:50.483 INFO:teuthology.orchestra.run.smithi016.stdout:crush_version 27 2024-09-21T03:45:50.483 INFO:teuthology.orchestra.run.smithi016.stdout:full_ratio 0.95 2024-09-21T03:45:50.483 INFO:teuthology.orchestra.run.smithi016.stdout:backfillfull_ratio 0.9 2024-09-21T03:45:50.483 INFO:teuthology.orchestra.run.smithi016.stdout:nearfull_ratio 0.85 2024-09-21T03:45:50.483 INFO:teuthology.orchestra.run.smithi016.stdout:require_min_compat_client luminous 2024-09-21T03:45:50.483 INFO:teuthology.orchestra.run.smithi016.stdout:min_compat_client luminous 2024-09-21T03:45:50.483 INFO:teuthology.orchestra.run.smithi016.stdout:require_osd_release squid 2024-09-21T03:45:50.483 INFO:teuthology.orchestra.run.smithi016.stdout:stretch_mode_enabled false 2024-09-21T03:45:50.483 INFO:teuthology.orchestra.run.smithi016.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T03:45:50.483 INFO:teuthology.orchestra.run.smithi016.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-21T03:45:50.483 INFO:teuthology.orchestra.run.smithi016.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-21T03:45:50.483 INFO:teuthology.orchestra.run.smithi016.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-21T03:45:50.483 INFO:teuthology.orchestra.run.smithi016.stdout:max_osd 12 2024-09-21T03:45:50.483 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 up in weight 1 up_from 8 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6802/311974551,v1:172.21.15.16:6803/311974551] [v2:172.21.15.16:6804/311974551,v1:172.21.15.16:6805/311974551] exists,up 6a7e4b9e-078f-4ae9-ab9e-37be995e4deb 2024-09-21T03:45:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 up in weight 1 up_from 13 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6800/2152486571,v1:172.21.15.123:6801/2152486571] [v2:172.21.15.123:6802/2152486571,v1:172.21.15.123:6803/2152486571] exists,up 3a497a64-e1a8-43ef-8509-0a79515da0aa 2024-09-21T03:45:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 up in weight 1 up_from 17 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6800/2812351279,v1:172.21.15.134:6801/2812351279] [v2:172.21.15.134:6802/2812351279,v1:172.21.15.134:6803/2812351279] exists,up 1c39518e-6cce-4a0f-9ee8-55c60ad325f9 2024-09-21T03:45:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 up in weight 1 up_from 24 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6810/2366216026,v1:172.21.15.16:6811/2366216026] [v2:172.21.15.16:6812/2366216026,v1:172.21.15.16:6813/2366216026] exists,up 0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf 2024-09-21T03:45:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 up in weight 1 up_from 30 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6808/2354279116,v1:172.21.15.123:6809/2354279116] [v2:172.21.15.123:6810/2354279116,v1:172.21.15.123:6811/2354279116] exists,up ad5e6d0e-31e8-4779-9e74-f215032777ba 2024-09-21T03:45:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 up in weight 1 up_from 35 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6808/2678416182,v1:172.21.15.134:6809/2678416182] [v2:172.21.15.134:6810/2678416182,v1:172.21.15.134:6811/2678416182] exists,up ca09bf88-1772-4dc0-8711-cbf9a5b31977 2024-09-21T03:45:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 up in weight 1 up_from 41 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6818/3377807744,v1:172.21.15.16:6819/3377807744] [v2:172.21.15.16:6820/3377807744,v1:172.21.15.16:6821/3377807744] exists,up 0d41ed4e-8699-4d88-9254-12d3b84941e3 2024-09-21T03:45:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 up in weight 1 up_from 46 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6816/1463412739,v1:172.21.15.123:6817/1463412739] [v2:172.21.15.123:6818/1463412739,v1:172.21.15.123:6819/1463412739] exists,up 1541a272-c524-4f12-9b94-763f69bb1c8d 2024-09-21T03:45:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 up in weight 1 up_from 51 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6816/2644021450,v1:172.21.15.134:6817/2644021450] [v2:172.21.15.134:6818/2644021450,v1:172.21.15.134:6819/2644021450] exists,up cc00014c-eb99-49ab-984c-0bc26ef85130 2024-09-21T03:45:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 up in weight 1 up_from 57 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6826/1653164371,v1:172.21.15.16:6827/1653164371] [v2:172.21.15.16:6828/1653164371,v1:172.21.15.16:6829/1653164371] exists,up 3cbfff74-31ff-4ffd-844b-54ec8563ba32 2024-09-21T03:45:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 up in weight 1 up_from 62 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6824/3682234496,v1:172.21.15.123:6825/3682234496] [v2:172.21.15.123:6826/3682234496,v1:172.21.15.123:6827/3682234496] exists,up 0a1caaad-70cd-4c2c-88d3-e9a5cc885655 2024-09-21T03:45:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 up in weight 1 up_from 67 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6824/584758685,v1:172.21.15.134:6825/584758685] [v2:172.21.15.134:6826/584758685,v1:172.21.15.134:6827/584758685] exists,up 3aff3258-5a38-41ff-aac3-4a082feffdae 2024-09-21T03:45:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.b [2,1] 2024-09-21T03:45:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.10 [8,11] 2024-09-21T03:45:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.1b [2,4] 2024-09-21T03:45:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.1d [8,11] 2024-09-21T03:45:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.13 [4,9] 2024-09-21T03:45:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.18 [3,9] 2024-09-21T03:45:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.27 [4,8] 2024-09-21T03:45:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.28 [5,6] 2024-09-21T03:45:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.3c [4,9] 2024-09-21T03:45:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.c [9,10] 2024-09-21T03:45:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.11 [5,8] 2024-09-21T03:45:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.18 [9,8] 2024-09-21T03:45:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.19 [9,8] 2024-09-21T03:45:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/557742783 expires 2024-09-22T03:34:53.217208+0000 2024-09-21T03:45:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3238676548 expires 2024-09-22T03:34:53.217208+0000 2024-09-21T03:45:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/500178737 expires 2024-09-22T03:34:53.217208+0000 2024-09-21T03:45:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2357336233 expires 2024-09-22T03:34:22.549156+0000 2024-09-21T03:45:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/428542610 expires 2024-09-22T03:34:22.549156+0000 2024-09-21T03:45:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/428542610 expires 2024-09-22T03:34:22.549156+0000 2024-09-21T03:45:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3806844703 expires 2024-09-22T03:34:22.549156+0000 2024-09-21T03:45:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2351581621 expires 2024-09-22T03:34:53.217208+0000 2024-09-21T03:45:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/500178737 expires 2024-09-22T03:34:53.217208+0000 2024-09-21T03:45:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/355731724 expires 2024-09-22T03:34:22.549156+0000 2024-09-21T03:45:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.484+0000 7f353e000640 1 -- 172.21.15.16:0/43174561 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f351c0b6590 msgr2=0x7f351c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.484+0000 7f353e000640 1 --2- 172.21.15.16:0/43174561 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f351c0b6590 0x7f351c0b8aa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3520004770 tx=0x7f35200314a0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.484+0000 7f353e000640 1 -- 172.21.15.16:0/43174561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3538103750 msgr2=0x7f3538076e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.484+0000 7f353e000640 1 --2- 172.21.15.16:0/43174561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3538103750 0x7f3538076e40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f352800e9f0 tx=0x7f352800edf0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.484+0000 7f353e000640 1 -- 172.21.15.16:0/43174561 shutdown_connections 2024-09-21T03:45:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.484+0000 7f353e000640 1 --2- 172.21.15.16:0/43174561 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f351c0b6590 0x7f351c0b8aa0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.484+0000 7f353e000640 1 --2- 172.21.15.16:0/43174561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3538105180 0x7f3538075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.484+0000 7f353e000640 1 --2- 172.21.15.16:0/43174561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35381047e0 0x7f3538075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.484+0000 7f353e000640 1 --2- 172.21.15.16:0/43174561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3538103750 0x7f3538076e40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.484+0000 7f353e000640 1 -- 172.21.15.16:0/43174561 >> 172.21.15.16:0/43174561 conn(0x7f35380ff0c0 msgr2=0x7f3538107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.484+0000 7f353e000640 1 -- 172.21.15.16:0/43174561 shutdown_connections 2024-09-21T03:45:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:50.484+0000 7f353e000640 1 -- 172.21.15.16:0/43174561 wait complete. 2024-09-21T03:45:50.842 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:50 smithi016 ceph-mon[30375]: pgmap v404: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:45:50.842 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:50 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/43174561' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:45:50.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:50 smithi134 ceph-mon[35031]: pgmap v404: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:45:50.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:50 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/43174561' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:45:50.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:50 smithi123 ceph-mon[34673]: pgmap v404: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:45:50.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:50 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/43174561' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:45:51.266 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T03:45:51.283 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:45:51.664 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:52.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.301+0000 7f35dca00640 1 -- 172.21.15.16:0/3211433834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d8109c20 msgr2=0x7f35d8110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:52.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.301+0000 7f35dca00640 1 --2- 172.21.15.16:0/3211433834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d8109c20 0x7f35d8110450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009ca0 tx=0x7f35d002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.301+0000 7f35dca00640 1 -- 172.21.15.16:0/3211433834 shutdown_connections 2024-09-21T03:45:52.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.301+0000 7f35dca00640 1 --2- 172.21.15.16:0/3211433834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d8109c20 0x7f35d8110450 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.301+0000 7f35dca00640 1 --2- 172.21.15.16:0/3211433834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35d81092d0 0x7f35d81096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.301+0000 7f35dca00640 1 --2- 172.21.15.16:0/3211433834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d8103990 0x7f35d8103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.301+0000 7f35dca00640 1 -- 172.21.15.16:0/3211433834 >> 172.21.15.16:0/3211433834 conn(0x7f35d80ff440 msgr2=0x7f35d8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:52.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.301+0000 7f35dca00640 1 -- 172.21.15.16:0/3211433834 shutdown_connections 2024-09-21T03:45:52.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.301+0000 7f35dca00640 1 -- 172.21.15.16:0/3211433834 wait complete. 2024-09-21T03:45:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.302+0000 7f35dca00640 1 Processor -- start 2024-09-21T03:45:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.302+0000 7f35dca00640 1 -- start start 2024-09-21T03:45:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.303+0000 7f35dca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d8103990 0x7f35d819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.303+0000 7f35dca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d81092d0 0x7f35d819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.303+0000 7f35dca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35d8109c20 0x7f35d81a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.303+0000 7f35dca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35d81138e0 con 0x7f35d8103990 2024-09-21T03:45:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.303+0000 7f35dca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35d8113760 con 0x7f35d8109c20 2024-09-21T03:45:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.303+0000 7f35dca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f35d8113a60 con 0x7f35d81092d0 2024-09-21T03:45:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.303+0000 7f35d7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d8103990 0x7f35d819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.303+0000 7f35d7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d8103990 0x7f35d819c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32808/0 (socket says 172.21.15.16:32808) 2024-09-21T03:45:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.303+0000 7f35d7400640 1 -- 172.21.15.16:0/1505933589 learned_addr learned my addr 172.21.15.16:0/1505933589 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:52.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.303+0000 7f35d7e00640 1 --2- 172.21.15.16:0/1505933589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35d8109c20 0x7f35d81a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:52.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.303+0000 7f35cea00640 1 --2- 172.21.15.16:0/1505933589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d81092d0 0x7f35d819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:52.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.303+0000 7f35d7400640 1 -- 172.21.15.16:0/1505933589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d81092d0 msgr2=0x7f35d819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:52.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.303+0000 7f35d7400640 1 --2- 172.21.15.16:0/1505933589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d81092d0 0x7f35d819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.303+0000 7f35d7400640 1 -- 172.21.15.16:0/1505933589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35d8109c20 msgr2=0x7f35d81a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:52.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.303+0000 7f35d7400640 1 --2- 172.21.15.16:0/1505933589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35d8109c20 0x7f35d81a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.303+0000 7f35d7400640 1 -- 172.21.15.16:0/1505933589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d81a2420 con 0x7f35d8103990 2024-09-21T03:45:52.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.304+0000 7f35d7400640 1 --2- 172.21.15.16:0/1505933589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d8103990 0x7f35d819c4b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f35c800e9f0 tx=0x7f35c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:52.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.304+0000 7f35d4c00640 1 -- 172.21.15.16:0/1505933589 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c800cd80 con 0x7f35d8103990 2024-09-21T03:45:52.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.304+0000 7f35d4c00640 1 -- 172.21.15.16:0/1505933589 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35c80054d0 con 0x7f35d8103990 2024-09-21T03:45:52.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.304+0000 7f35d4c00640 1 -- 172.21.15.16:0/1505933589 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8010690 con 0x7f35d8103990 2024-09-21T03:45:52.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.304+0000 7f35dca00640 1 -- 172.21.15.16:0/1505933589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d81a2710 con 0x7f35d8103990 2024-09-21T03:45:52.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.304+0000 7f35dca00640 1 -- 172.21.15.16:0/1505933589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d81a8e70 con 0x7f35d8103990 2024-09-21T03:45:52.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.305+0000 7f35dca00640 1 -- 172.21.15.16:0/1505933589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d8103da0 con 0x7f35d8103990 2024-09-21T03:45:52.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.308+0000 7f35d4c00640 1 -- 172.21.15.16:0/1505933589 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f35c8004e30 con 0x7f35d8103990 2024-09-21T03:45:52.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.309+0000 7f35d4c00640 1 --2- 172.21.15.16:0/1505933589 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f35bc0b6590 0x7f35bc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:52.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.309+0000 7f35d4c00640 1 -- 172.21.15.16:0/1505933589 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f35c80e7c20 con 0x7f35d8103990 2024-09-21T03:45:52.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.310+0000 7f35cea00640 1 --2- 172.21.15.16:0/1505933589 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f35bc0b6590 0x7f35bc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:52.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.310+0000 7f35cea00640 1 --2- 172.21.15.16:0/1505933589 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f35bc0b6590 0x7f35bc0b8aa0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f35c00095f0 tx=0x7f35c0005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:52.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.311+0000 7f35d4c00640 1 -- 172.21.15.16:0/1505933589 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f35c80ae0a0 con 0x7f35d8103990 2024-09-21T03:45:52.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.549+0000 7f35dca00640 1 -- 172.21.15.16:0/1505933589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35d819d5e0 con 0x7f35d8103990 2024-09-21T03:45:52.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.550+0000 7f35d4c00640 1 -- 172.21.15.16:0/1505933589 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11971 (secure 0 0 0) 0x7f35c80b2480 con 0x7f35d8103990 2024-09-21T03:45:52.552 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:45:52.552 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"btime":"2024-09-21T03:44:08:542131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:44:07.645054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24668,24670,24682,24689,24731]},"id":1}]} 2024-09-21T03:45:52.552 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-09-21T03:45:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.552+0000 7f35dca00640 1 -- 172.21.15.16:0/1505933589 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f35bc0b6590 msgr2=0x7f35bc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.552+0000 7f35dca00640 1 --2- 172.21.15.16:0/1505933589 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f35bc0b6590 0x7f35bc0b8aa0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f35c00095f0 tx=0x7f35c0005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.552+0000 7f35dca00640 1 -- 172.21.15.16:0/1505933589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d8103990 msgr2=0x7f35d819c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.552+0000 7f35dca00640 1 --2- 172.21.15.16:0/1505933589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d8103990 0x7f35d819c4b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f35c800e9f0 tx=0x7f35c800edf0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.552+0000 7f35dca00640 1 -- 172.21.15.16:0/1505933589 shutdown_connections 2024-09-21T03:45:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.552+0000 7f35dca00640 1 --2- 172.21.15.16:0/1505933589 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f35bc0b6590 0x7f35bc0b8aa0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.552+0000 7f35dca00640 1 --2- 172.21.15.16:0/1505933589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35d8109c20 0x7f35d81a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.552+0000 7f35dca00640 1 --2- 172.21.15.16:0/1505933589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35d81092d0 0x7f35d819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.552+0000 7f35dca00640 1 --2- 172.21.15.16:0/1505933589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d8103990 0x7f35d819c4b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.552+0000 7f35dca00640 1 -- 172.21.15.16:0/1505933589 >> 172.21.15.16:0/1505933589 conn(0x7f35d80ff440 msgr2=0x7f35d8106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.553+0000 7f35dca00640 1 -- 172.21.15.16:0/1505933589 shutdown_connections 2024-09-21T03:45:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:52.553+0000 7f35dca00640 1 -- 172.21.15.16:0/1505933589 wait complete. 2024-09-21T03:45:52.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:52 smithi134 ceph-mon[35031]: pgmap v405: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:52.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:52 smithi123 ceph-mon[34673]: pgmap v405: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:52.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:52 smithi016 ceph-mon[30375]: pgmap v405: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:53.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:45:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:53.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:53 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1505933589' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:53.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:53 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1505933589' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:53.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:53 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1505933589' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:54.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.459+0000 7fd2aea00640 1 -- 172.21.15.16:0/1607148818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a8109c40 msgr2=0x7fd2a8110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.459+0000 7fd2aea00640 1 --2- 172.21.15.16:0/1607148818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a8109c40 0x7fd2a8110470 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009ca0 tx=0x7fd2a002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.459+0000 7fd2aea00640 1 -- 172.21.15.16:0/1607148818 shutdown_connections 2024-09-21T03:45:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.459+0000 7fd2aea00640 1 --2- 172.21.15.16:0/1607148818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a8109c40 0x7fd2a8110470 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.459+0000 7fd2aea00640 1 --2- 172.21.15.16:0/1607148818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a81092f0 0x7fd2a8109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.459+0000 7fd2aea00640 1 --2- 172.21.15.16:0/1607148818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a81039b0 0x7fd2a8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.459+0000 7fd2aea00640 1 -- 172.21.15.16:0/1607148818 >> 172.21.15.16:0/1607148818 conn(0x7fd2a80ff400 msgr2=0x7fd2a8101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.459+0000 7fd2aea00640 1 -- 172.21.15.16:0/1607148818 shutdown_connections 2024-09-21T03:45:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.460+0000 7fd2aea00640 1 -- 172.21.15.16:0/1607148818 wait complete. 2024-09-21T03:45:54.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.460+0000 7fd2aea00640 1 Processor -- start 2024-09-21T03:45:54.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.461+0000 7fd2aea00640 1 -- start start 2024-09-21T03:45:54.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.461+0000 7fd2aea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a81039b0 0x7fd2a819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:54.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.461+0000 7fd2aea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a81092f0 0x7fd2a819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:54.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.461+0000 7fd2aea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a8109c40 0x7fd2a81a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:54.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.461+0000 7fd2aea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8113900 con 0x7fd2a81092f0 2024-09-21T03:45:54.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.461+0000 7fd2aea00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8113780 con 0x7fd2a8109c40 2024-09-21T03:45:54.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.461+0000 7fd2aea00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8113a80 con 0x7fd2a81039b0 2024-09-21T03:45:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.461+0000 7fd2acc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a81092f0 0x7fd2a819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.461+0000 7fd2ad600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a81039b0 0x7fd2a819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.462+0000 7fd2acc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a81092f0 0x7fd2a819ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32826/0 (socket says 172.21.15.16:32826) 2024-09-21T03:45:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.462+0000 7fd2ad600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a81039b0 0x7fd2a819c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34096/0 (socket says 172.21.15.16:34096) 2024-09-21T03:45:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.462+0000 7fd2acc00640 1 -- 172.21.15.16:0/2519945046 learned_addr learned my addr 172.21.15.16:0/2519945046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.462+0000 7fd2ae000640 1 --2- 172.21.15.16:0/2519945046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a8109c40 0x7fd2a81a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.462+0000 7fd2ae000640 1 -- 172.21.15.16:0/2519945046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a81039b0 msgr2=0x7fd2a819c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.462+0000 7fd2ae000640 1 --2- 172.21.15.16:0/2519945046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a81039b0 0x7fd2a819c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.462+0000 7fd2ae000640 1 -- 172.21.15.16:0/2519945046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a81092f0 msgr2=0x7fd2a819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.462+0000 7fd2ae000640 1 --2- 172.21.15.16:0/2519945046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a81092f0 0x7fd2a819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.462+0000 7fd2ae000640 1 -- 172.21.15.16:0/2519945046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a81a2400 con 0x7fd2a8109c40 2024-09-21T03:45:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.462+0000 7fd2acc00640 1 --2- 172.21.15.16:0/2519945046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a81092f0 0x7fd2a819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.463+0000 7fd2ae000640 1 --2- 172.21.15.16:0/2519945046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a8109c40 0x7fd2a81a1cd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd2a002ade0 tx=0x7fd2a00316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.463+0000 7fd2a6000640 1 -- 172.21.15.16:0/2519945046 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0003b70 con 0x7fd2a8109c40 2024-09-21T03:45:54.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.463+0000 7fd2aea00640 1 -- 172.21.15.16:0/2519945046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a81a2690 con 0x7fd2a8109c40 2024-09-21T03:45:54.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.463+0000 7fd2a6000640 1 -- 172.21.15.16:0/2519945046 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2a0003d10 con 0x7fd2a8109c40 2024-09-21T03:45:54.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.463+0000 7fd2a6000640 1 -- 172.21.15.16:0/2519945046 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a002f9e0 con 0x7fd2a8109c40 2024-09-21T03:45:54.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.463+0000 7fd2aea00640 1 -- 172.21.15.16:0/2519945046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a81a8e50 con 0x7fd2a8109c40 2024-09-21T03:45:54.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.464+0000 7fd2aea00640 1 -- 172.21.15.16:0/2519945046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a8103dc0 con 0x7fd2a8109c40 2024-09-21T03:45:54.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.467+0000 7fd2a6000640 1 -- 172.21.15.16:0/2519945046 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd2a002fb80 con 0x7fd2a8109c40 2024-09-21T03:45:54.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.468+0000 7fd2a6000640 1 --2- 172.21.15.16:0/2519945046 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd2900b6510 0x7fd2900b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:54.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.468+0000 7fd2a6000640 1 -- 172.21.15.16:0/2519945046 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fd2a01087b0 con 0x7fd2a8109c40 2024-09-21T03:45:54.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.468+0000 7fd2ad600640 1 --2- 172.21.15.16:0/2519945046 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd2900b6510 0x7fd2900b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.469+0000 7fd2ad600640 1 --2- 172.21.15.16:0/2519945046 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd2900b6510 0x7fd2900b8a20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00bbe0 tx=0x7fd29c00b390 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:54.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.473+0000 7fd2a6000640 1 -- 172.21.15.16:0/2519945046 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd2a00cec30 con 0x7fd2a8109c40 2024-09-21T03:45:54.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.719+0000 7fd2aea00640 1 -- 172.21.15.16:0/2519945046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2a81a9080 con 0x7fd2a8109c40 2024-09-21T03:45:54.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.720+0000 7fd2a6000640 1 -- 172.21.15.16:0/2519945046 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11971 (secure 0 0 0) 0x7fd2a00d3010 con 0x7fd2a8109c40 2024-09-21T03:45:54.722 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:45:54.722 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"btime":"2024-09-21T03:44:08:542131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:44:07.645054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24668,24670,24682,24689,24731]},"id":1}]} 2024-09-21T03:45:54.722 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-09-21T03:45:54.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.722+0000 7fd2aea00640 1 -- 172.21.15.16:0/2519945046 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd2900b6510 msgr2=0x7fd2900b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:54.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.722+0000 7fd2aea00640 1 --2- 172.21.15.16:0/2519945046 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd2900b6510 0x7fd2900b8a20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00bbe0 tx=0x7fd29c00b390 comp rx=0 tx=0).stop 2024-09-21T03:45:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.722+0000 7fd2aea00640 1 -- 172.21.15.16:0/2519945046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a8109c40 msgr2=0x7fd2a81a1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.722+0000 7fd2aea00640 1 --2- 172.21.15.16:0/2519945046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a8109c40 0x7fd2a81a1cd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd2a002ade0 tx=0x7fd2a00316f0 comp rx=0 tx=0).stop 2024-09-21T03:45:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.722+0000 7fd2aea00640 1 -- 172.21.15.16:0/2519945046 shutdown_connections 2024-09-21T03:45:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.722+0000 7fd2aea00640 1 --2- 172.21.15.16:0/2519945046 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd2900b6510 0x7fd2900b8a20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.722+0000 7fd2aea00640 1 --2- 172.21.15.16:0/2519945046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a8109c40 0x7fd2a81a1cd0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.722+0000 7fd2aea00640 1 --2- 172.21.15.16:0/2519945046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a81092f0 0x7fd2a819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.722+0000 7fd2aea00640 1 --2- 172.21.15.16:0/2519945046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a81039b0 0x7fd2a819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.722+0000 7fd2aea00640 1 -- 172.21.15.16:0/2519945046 >> 172.21.15.16:0/2519945046 conn(0x7fd2a80ff400 msgr2=0x7fd2a8108960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.723+0000 7fd2aea00640 1 -- 172.21.15.16:0/2519945046 shutdown_connections 2024-09-21T03:45:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:54.723+0000 7fd2aea00640 1 -- 172.21.15.16:0/2519945046 wait complete. 2024-09-21T03:45:54.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:54 smithi134 ceph-mon[35031]: pgmap v406: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:45:54.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:54 smithi123 ceph-mon[34673]: pgmap v406: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:45:54.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:54 smithi016 ceph-mon[30375]: pgmap v406: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:45:55.586 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:45:55.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:55 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2519945046' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:55.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:55 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2519945046' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:55.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:55 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2519945046' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:45:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:56.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.640+0000 7fe2dbe00640 1 -- 172.21.15.16:0/2171924235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc109c40 msgr2=0x7fe2dc110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.640+0000 7fe2dbe00640 1 --2- 172.21.15.16:0/2171924235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc109c40 0x7fe2dc110470 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4009900 tx=0x7fe2d402a410 comp rx=0 tx=0).stop 2024-09-21T03:45:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.640+0000 7fe2dbe00640 1 -- 172.21.15.16:0/2171924235 shutdown_connections 2024-09-21T03:45:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.640+0000 7fe2dbe00640 1 --2- 172.21.15.16:0/2171924235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc109c40 0x7fe2dc110470 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.640+0000 7fe2dbe00640 1 --2- 172.21.15.16:0/2171924235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2dc1092f0 0x7fe2dc109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.640+0000 7fe2dbe00640 1 --2- 172.21.15.16:0/2171924235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2dc1039b0 0x7fe2dc103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.640+0000 7fe2dbe00640 1 -- 172.21.15.16:0/2171924235 >> 172.21.15.16:0/2171924235 conn(0x7fe2dc0ff440 msgr2=0x7fe2dc101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.641+0000 7fe2dbe00640 1 -- 172.21.15.16:0/2171924235 shutdown_connections 2024-09-21T03:45:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.641+0000 7fe2dbe00640 1 -- 172.21.15.16:0/2171924235 wait complete. 2024-09-21T03:45:56.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.642+0000 7fe2dbe00640 1 Processor -- start 2024-09-21T03:45:56.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.642+0000 7fe2dbe00640 1 -- start start 2024-09-21T03:45:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.643+0000 7fe2dbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2dc1039b0 0x7fe2dc19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.643+0000 7fe2dbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2dc1092f0 0x7fe2dc19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.643+0000 7fe2dbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc109c40 0x7fe2dc19fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.643+0000 7fe2dbe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc113950 con 0x7fe2dc109c40 2024-09-21T03:45:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.643+0000 7fe2dbe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1137d0 con 0x7fe2dc1039b0 2024-09-21T03:45:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.643+0000 7fe2dbe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc113ad0 con 0x7fe2dc1092f0 2024-09-21T03:45:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.643+0000 7fe2daa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2dc1039b0 0x7fe2dc19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.643+0000 7fe2db400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc109c40 0x7fe2dc19fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.643+0000 7fe2daa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2dc1039b0 0x7fe2dc19a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:46364/0 (socket says 172.21.15.16:46364) 2024-09-21T03:45:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.643+0000 7fe2daa00640 1 -- 172.21.15.16:0/77945569 learned_addr learned my addr 172.21.15.16:0/77945569 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.643+0000 7fe2d2000640 1 --2- 172.21.15.16:0/77945569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2dc1092f0 0x7fe2dc19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:56.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.644+0000 7fe2daa00640 1 -- 172.21.15.16:0/77945569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2dc1092f0 msgr2=0x7fe2dc19a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:56.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.644+0000 7fe2daa00640 1 --2- 172.21.15.16:0/77945569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2dc1092f0 0x7fe2dc19a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:56.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.644+0000 7fe2daa00640 1 -- 172.21.15.16:0/77945569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc109c40 msgr2=0x7fe2dc19fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:56.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.644+0000 7fe2daa00640 1 --2- 172.21.15.16:0/77945569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc109c40 0x7fe2dc19fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:56.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.644+0000 7fe2daa00640 1 -- 172.21.15.16:0/77945569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2dc1a0210 con 0x7fe2dc1039b0 2024-09-21T03:45:56.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.644+0000 7fe2db400640 1 --2- 172.21.15.16:0/77945569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc109c40 0x7fe2dc19fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:56.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.644+0000 7fe2daa00640 1 --2- 172.21.15.16:0/77945569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2dc1039b0 0x7fe2dc19a2a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00ebc0 tx=0x7fe2cc00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.644+0000 7fe2d3e00640 1 -- 172.21.15.16:0/77945569 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc010630 con 0x7fe2dc1039b0 2024-09-21T03:45:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.644+0000 7fe2dbe00640 1 -- 172.21.15.16:0/77945569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2dc1a0500 con 0x7fe2dc1039b0 2024-09-21T03:45:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.644+0000 7fe2d3e00640 1 -- 172.21.15.16:0/77945569 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2cc010c80 con 0x7fe2dc1039b0 2024-09-21T03:45:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.644+0000 7fe2d3e00640 1 -- 172.21.15.16:0/77945569 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc018b60 con 0x7fe2dc1039b0 2024-09-21T03:45:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.645+0000 7fe2dbe00640 1 -- 172.21.15.16:0/77945569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2dc1a6c60 con 0x7fe2dc1039b0 2024-09-21T03:45:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.645+0000 7fe2dbe00640 1 -- 172.21.15.16:0/77945569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2dc103dc0 con 0x7fe2dc1039b0 2024-09-21T03:45:56.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.648+0000 7fe2d3e00640 1 -- 172.21.15.16:0/77945569 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe2cc018d00 con 0x7fe2dc1039b0 2024-09-21T03:45:56.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.649+0000 7fe2d3e00640 1 --2- 172.21.15.16:0/77945569 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe2c00b6510 0x7fe2c00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:56.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.650+0000 7fe2d3e00640 1 -- 172.21.15.16:0/77945569 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fe2cc0e8160 con 0x7fe2dc1039b0 2024-09-21T03:45:56.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.650+0000 7fe2d2000640 1 --2- 172.21.15.16:0/77945569 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe2c00b6510 0x7fe2c00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:56.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.650+0000 7fe2d2000640 1 --2- 172.21.15.16:0/77945569 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe2c00b6510 0x7fe2c00b8a20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc102cb0 tx=0x7fe2c4005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:56.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.651+0000 7fe2d3e00640 1 -- 172.21.15.16:0/77945569 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe2cc0b29c0 con 0x7fe2dc1039b0 2024-09-21T03:45:56.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:56 smithi134 ceph-mon[35031]: pgmap v407: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:45:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.853+0000 7fe2dbe00640 1 -- 172.21.15.16:0/77945569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2dc10ba90 con 0x7fe2dc1039b0 2024-09-21T03:45:56.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:56 smithi016 ceph-mon[30375]: pgmap v407: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:45:56.856 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:45:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.854+0000 7fe2d3e00640 1 -- 172.21.15.16:0/77945569 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24902 (secure 0 0 0) 0x7fe2cc0b7900 con 0x7fe2dc1039b0 2024-09-21T03:45:56.857 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:33:53.220358+0000","modified":"2024-09-21T03:44:54.658912+0000","last_up_change":"2024-09-21T03:42:06.614247+0000","last_in_change":"2024-09-21T03:41:43.379404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:37:55.708915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:43:03.567522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:43:06.048083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:43:16.061356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6a7e4b9e-078f-4ae9-ab9e-37be995e4deb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6803","nonce":311974551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6805","nonce":311974551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6809","nonce":311974551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6807","nonce":311974551}]},"public_addr":"172.21.15.16:6803/311974551","cluster_addr":"172.21.15.16:6805/311974551","heartbeat_back_addr":"172.21.15.16:6809/311974551","heartbeat_front_addr":"172.21.15.16:6807/311974551","state":["exists","up"]},{"osd":1,"uuid":"3a497a64-e1a8-43ef-8509-0a79515da0aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6801","nonce":2152486571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6803","nonce":2152486571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6807","nonce":2152486571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6805","nonce":2152486571}]},"public_addr":"172.21.15.123:6801/2152486571","cluster_addr":"172.21.15.123:6803/2152486571","heartbeat_back_addr":"172.21.15.123:6807/2152486571","heartbeat_front_addr":"172.21.15.123:6805/2152486571","state":["exists","up"]},{"osd":2,"uuid":"1c39518e-6cce-4a0f-9ee8-55c60ad325f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6801","nonce":2812351279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6803","nonce":2812351279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6807","nonce":2812351279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6805","nonce":2812351279}]},"public_addr":"172.21.15.134:6801/2812351279","cluster_addr":"172.21.15.134:6803/2812351279","heartbeat_back_addr":"172.21.15.134:6807/2812351279","heartbeat_front_addr":"172.21.15.134:6805/2812351279","state":["exists","up"]},{"osd":3,"uuid":"0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6811","nonce":2366216026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6813","nonce":2366216026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6817","nonce":2366216026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6815","nonce":2366216026}]},"public_addr":"172.21.15.16:6811/2366216026","cluster_addr":"172.21.15.16:6813/2366216026","heartbeat_back_addr":"172.21.15.16:6817/2366216026","heartbeat_front_addr":"172.21.15.16:6815/2366216026","state":["exists","up"]},{"osd":4,"uuid":"ad5e6d0e-31e8-4779-9e74-f215032777ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6809","nonce":2354279116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6811","nonce":2354279116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6815","nonce":2354279116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6813","nonce":2354279116}]},"public_addr":"172.21.15.123:6809/2354279116","cluster_addr":"172.21.15.123:6811/2354279116","heartbeat_back_addr":"172.21.15.123:6815/2354279116","heartbeat_front_addr":"172.21.15.123:6813/2354279116","state":["exists","up"]},{"osd":5,"uuid":"ca09bf88-1772-4dc0-8711-cbf9a5b31977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6809","nonce":2678416182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6811","nonce":2678416182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6815","nonce":2678416182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6813","nonce":2678416182}]},"public_addr":"172.21.15.134:6809/2678416182","cluster_addr":"172.21.15.134:6811/2678416182","heartbeat_back_addr":"172.21.15.134:6815/2678416182","heartbeat_front_addr":"172.21.15.134:6813/2678416182","state":["exists","up"]},{"osd":6,"uuid":"0d41ed4e-8699-4d88-9254-12d3b84941e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6819","nonce":3377807744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6821","nonce":3377807744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6825","nonce":3377807744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6823","nonce":3377807744}]},"public_addr":"172.21.15.16:6819/3377807744","cluster_addr":"172.21.15.16:6821/3377807744","heartbeat_back_addr":"172.21.15.16:6825/3377807744","heartbeat_front_addr":"172.21.15.16:6823/3377807744","state":["exists","up"]},{"osd":7,"uuid":"1541a272-c524-4f12-9b94-763f69bb1c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6817","nonce":1463412739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6819","nonce":1463412739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6823","nonce":1463412739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6821","nonce":1463412739}]},"public_addr":"172.21.15.123:6817/1463412739","cluster_addr":"172.21.15.123:6819/1463412739","heartbeat_back_addr":"172.21.15.123:6823/1463412739","heartbeat_front_addr":"172.21.15.123:6821/1463412739","state":["exists","up"]},{"osd":8,"uuid":"cc00014c-eb99-49ab-984c-0bc26ef85130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6817","nonce":2644021450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6819","nonce":2644021450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6823","nonce":2644021450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6821","nonce":2644021450}]},"public_addr":"172.21.15.134:6817/2644021450","cluster_addr":"172.21.15.134:6819/2644021450","heartbeat_back_addr":"172.21.15.134:6823/2644021450","heartbeat_front_addr":"172.21.15.134:6821/2644021450","state":["exists","up"]},{"osd":9,"uuid":"3cbfff74-31ff-4ffd-844b-54ec8563ba32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6827","nonce":1653164371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6829","nonce":1653164371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6833","nonce":1653164371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6831","nonce":1653164371}]},"public_addr":"172.21.15.16:6827/1653164371","cluster_addr":"172.21.15.16:6829/1653164371","heartbeat_back_addr":"172.21.15.16:6833/1653164371","heartbeat_front_addr":"172.21.15.16:6831/1653164371","state":["exists","up"]},{"osd":10,"uuid":"0a1caaad-70cd-4c2c-88d3-e9a5cc885655","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6825","nonce":3682234496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6827","nonce":3682234496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6831","nonce":3682234496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6829","nonce":3682234496}]},"public_addr":"172.21.15.123:6825/3682234496","cluster_addr":"172.21.15.123:6827/3682234496","heartbeat_back_addr":"172.21.15.123:6831/3682234496","heartbeat_front_addr":"172.21.15.123:6829/3682234496","state":["exists","up"]},{"osd":11,"uuid":"3aff3258-5a38-41ff-aac3-4a082feffdae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6825","nonce":584758685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6827","nonce":584758685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6831","nonce":584758685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6829","nonce":584758685}]},"public_addr":"172.21.15.134:6825/584758685","cluster_addr":"172.21.15.134:6827/584758685","heartbeat_back_addr":"172.21.15.134:6831/584758685","heartbeat_front_addr":"172.21.15.134:6829/584758685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:02.177731+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:28.847562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:54.272384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:24.338596+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:52.430371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:17.832737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:50.397890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:16.842245+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:42.102933+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:13.074732+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:40.057905+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:04.898190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/557742783":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/3238676548":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6801/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/2357336233":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6801/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6800/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/3806844703":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/2351581621":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6800/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/355731724":"2024-09-22T03:34:22.549156+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:45:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.857+0000 7fe2dbe00640 1 -- 172.21.15.16:0/77945569 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe2c00b6510 msgr2=0x7fe2c00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.857+0000 7fe2dbe00640 1 --2- 172.21.15.16:0/77945569 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe2c00b6510 0x7fe2c00b8a20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc102cb0 tx=0x7fe2c4005f90 comp rx=0 tx=0).stop 2024-09-21T03:45:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.857+0000 7fe2dbe00640 1 -- 172.21.15.16:0/77945569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2dc1039b0 msgr2=0x7fe2dc19a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.857+0000 7fe2dbe00640 1 --2- 172.21.15.16:0/77945569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2dc1039b0 0x7fe2dc19a2a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00ebc0 tx=0x7fe2cc00c6a0 comp rx=0 tx=0).stop 2024-09-21T03:45:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.857+0000 7fe2dbe00640 1 -- 172.21.15.16:0/77945569 shutdown_connections 2024-09-21T03:45:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.857+0000 7fe2dbe00640 1 --2- 172.21.15.16:0/77945569 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe2c00b6510 0x7fe2c00b8a20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.857+0000 7fe2dbe00640 1 --2- 172.21.15.16:0/77945569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc109c40 0x7fe2dc19fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.857+0000 7fe2dbe00640 1 --2- 172.21.15.16:0/77945569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2dc1092f0 0x7fe2dc19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.857+0000 7fe2dbe00640 1 --2- 172.21.15.16:0/77945569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2dc1039b0 0x7fe2dc19a2a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.857+0000 7fe2dbe00640 1 -- 172.21.15.16:0/77945569 >> 172.21.15.16:0/77945569 conn(0x7fe2dc0ff440 msgr2=0x7fe2dc106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.858+0000 7fe2dbe00640 1 -- 172.21.15.16:0/77945569 shutdown_connections 2024-09-21T03:45:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:56.858+0000 7fe2dbe00640 1 -- 172.21.15.16:0/77945569 wait complete. 2024-09-21T03:45:56.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:56 smithi123 ceph-mon[34673]: pgmap v407: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:45:57.796 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:57 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/77945569' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:45:57.799 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T03:43:09.332305+0000', 'modified': '2024-09-21T03:44:07.645054+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24731, 'mds_1': 24689, 'mds_2': 24682, 'mds_3': 24670, 'mds_4': 24668}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14700': {'gid': 14700, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6839/89613887', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 89613887}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 89613887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14712': {'gid': 14712, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6841/1546097467', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6840', 'nonce': 1546097467}, {'type': 'v1', 'addr': '172.21.15.16:6841', 'nonce': 1546097467}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24613': {'gid': 24613, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/2239407349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 2239407349}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 2239407349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24632': {'gid': 24632, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6833/3581513383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6832', 'nonce': 3581513383}, {'type': 'v1', 'addr': '172.21.15.123:6833', 'nonce': 3581513383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24658': {'gid': 24658, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/135285598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 135285598}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 135285598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24668': {'gid': 24668, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.123:6835/86959152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 86959152}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 86959152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24670': {'gid': 24670, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.134:6837/2624530453', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 2624530453}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 2624530453}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24682': {'gid': 24682, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.134:6839/606193011', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 606193011}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 606193011}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24689': {'gid': 24689, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.123:6837/2001424876', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 2001424876}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 2001424876}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24731': {'gid': 24731, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.123:6839/459522816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 459522816}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 459522816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24731, 'qdb_cluster': [24668, 24670, 24682, 24689, 24731]} 2024-09-21T03:45:57.799 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T03:45:57.799 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.e status 2024-09-21T03:45:57.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:57 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/77945569' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:45:57.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:57 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/77945569' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:45:58.189 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:58.651 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:45:58 smithi016 ceph-mon[30375]: pgmap v408: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.831+0000 7f3dbc600640 1 -- 172.21.15.16:0/2788018630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db410c010 msgr2=0x7f3db4112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.831+0000 7f3dbc600640 1 --2- 172.21.15.16:0/2788018630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db410c010 0x7f3db4112840 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009ca0 tx=0x7f3dac02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.831+0000 7f3dbc600640 1 -- 172.21.15.16:0/2788018630 shutdown_connections 2024-09-21T03:45:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.831+0000 7f3dbc600640 1 --2- 172.21.15.16:0/2788018630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db410c010 0x7f3db4112840 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.831+0000 7f3dbc600640 1 --2- 172.21.15.16:0/2788018630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3db410b6c0 0x7f3db410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.831+0000 7f3dbc600640 1 --2- 172.21.15.16:0/2788018630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db4105d80 0x7f3db4106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.831+0000 7f3dbc600640 1 -- 172.21.15.16:0/2788018630 >> 172.21.15.16:0/2788018630 conn(0x7f3db4101810 msgr2=0x7f3db4103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.831+0000 7f3dbc600640 1 -- 172.21.15.16:0/2788018630 shutdown_connections 2024-09-21T03:45:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.831+0000 7f3dbc600640 1 -- 172.21.15.16:0/2788018630 wait complete. 2024-09-21T03:45:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.832+0000 7f3dbc600640 1 Processor -- start 2024-09-21T03:45:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.833+0000 7f3dbc600640 1 -- start start 2024-09-21T03:45:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.833+0000 7f3dbc600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db4105d80 0x7f3db419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.833+0000 7f3dbc600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db410b6c0 0x7f3db419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.833+0000 7f3dbc600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3db410c010 0x7f3db41a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.833+0000 7f3dbc600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3db4115cd0 con 0x7f3db4105d80 2024-09-21T03:45:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.833+0000 7f3dbc600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3db4115b50 con 0x7f3db410b6c0 2024-09-21T03:45:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.833+0000 7f3dbc600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3db4115e50 con 0x7f3db410c010 2024-09-21T03:45:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.833+0000 7f3db9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db4105d80 0x7f3db419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.833+0000 7f3db9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db4105d80 0x7f3db419c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32866/0 (socket says 172.21.15.16:32866) 2024-09-21T03:45:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.833+0000 7f3db9c00640 1 -- 172.21.15.16:0/2495510341 learned_addr learned my addr 172.21.15.16:0/2495510341 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.834+0000 7f3dba600640 1 --2- 172.21.15.16:0/2495510341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3db410c010 0x7f3db41a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.834+0000 7f3db9200640 1 --2- 172.21.15.16:0/2495510341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db410b6c0 0x7f3db419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.834+0000 7f3db9c00640 1 -- 172.21.15.16:0/2495510341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3db410c010 msgr2=0x7f3db41a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.834+0000 7f3db9c00640 1 --2- 172.21.15.16:0/2495510341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3db410c010 0x7f3db41a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.834+0000 7f3db9c00640 1 -- 172.21.15.16:0/2495510341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db410b6c0 msgr2=0x7f3db419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.834+0000 7f3db9c00640 1 --2- 172.21.15.16:0/2495510341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db410b6c0 0x7f3db419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.834+0000 7f3db9c00640 1 -- 172.21.15.16:0/2495510341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db41a2650 con 0x7f3db4105d80 2024-09-21T03:45:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.834+0000 7f3db9c00640 1 --2- 172.21.15.16:0/2495510341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db4105d80 0x7f3db419c6e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3da800e9f0 tx=0x7f3da800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.835+0000 7f3db2000640 1 -- 172.21.15.16:0/2495510341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da800cd80 con 0x7f3db4105d80 2024-09-21T03:45:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.835+0000 7f3dbc600640 1 -- 172.21.15.16:0/2495510341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db41a2940 con 0x7f3db4105d80 2024-09-21T03:45:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.835+0000 7f3db2000640 1 -- 172.21.15.16:0/2495510341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da80108e0 con 0x7f3db4105d80 2024-09-21T03:45:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.835+0000 7f3dbc600640 1 -- 172.21.15.16:0/2495510341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db41a90a0 con 0x7f3db4105d80 2024-09-21T03:45:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.835+0000 7f3db2000640 1 -- 172.21.15.16:0/2495510341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8026060 con 0x7f3db4105d80 2024-09-21T03:45:58.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.838+0000 7f3dc3235740 1 Processor -- start 2024-09-21T03:45:58.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.838+0000 7f3dc3235740 1 -- start start 2024-09-21T03:45:58.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.838+0000 7f3dc3235740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e5622f60 0x5603e59c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.838+0000 7f3dc3235740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e55fa0a0 0x5603e59c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.838+0000 7f3dc3235740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5602280 0x5603e59c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.839+0000 7f3dc3235740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5603e5940d50 con 0x5603e5602280 2024-09-21T03:45:58.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.839+0000 7f3dc3235740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5603e5930180 con 0x5603e55fa0a0 2024-09-21T03:45:58.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.839+0000 7f3dc3235740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5603e5933f90 con 0x5603e5622f60 2024-09-21T03:45:58.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.839+0000 7f3db2000640 1 -- 172.21.15.16:0/2495510341 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3da8010430 con 0x7f3db4105d80 2024-09-21T03:45:58.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.839+0000 7f3db9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5602280 0x5603e59c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.839+0000 7f3db9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5602280 0x5603e59c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32874/0 (socket says 172.21.15.16:32874) 2024-09-21T03:45:58.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.839+0000 7f3db9200640 1 -- 172.21.15.16:0/4143048534 learned_addr learned my addr 172.21.15.16:0/4143048534 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:58.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.839+0000 7f3dba600640 1 --2- 172.21.15.16:0/4143048534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e5622f60 0x5603e59c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.839+0000 7f3db9c00640 1 --2- 172.21.15.16:0/4143048534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e55fa0a0 0x5603e59c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.839+0000 7f3db9200640 1 -- 172.21.15.16:0/4143048534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e5622f60 msgr2=0x5603e59c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.839+0000 7f3db9200640 1 --2- 172.21.15.16:0/4143048534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e5622f60 0x5603e59c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.839+0000 7f3db9200640 1 -- 172.21.15.16:0/4143048534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e55fa0a0 msgr2=0x5603e59c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.839+0000 7f3db9200640 1 --2- 172.21.15.16:0/4143048534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e55fa0a0 0x5603e59c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.839+0000 7f3db9200640 1 -- 172.21.15.16:0/4143048534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603e58faa70 con 0x5603e5602280 2024-09-21T03:45:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.839+0000 7f3db9c00640 1 --2- 172.21.15.16:0/4143048534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e55fa0a0 0x5603e59c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:45:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.840+0000 7f3db9200640 1 --2- 172.21.15.16:0/4143048534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5602280 0x5603e59c9f10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009c00 tx=0x7f3da0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d6c165630b3b2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.840+0000 7f3da6a00640 1 -- 172.21.15.16:0/4143048534 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0004050 con 0x5603e5602280 2024-09-21T03:45:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.840+0000 7f3da6a00640 1 -- 172.21.15.16:0/4143048534 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da00054d0 con 0x5603e5602280 2024-09-21T03:45:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.840+0000 7f3da6a00640 1 -- 172.21.15.16:0/4143048534 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0004ba0 con 0x5603e5602280 2024-09-21T03:45:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.840+0000 7f3db2000640 1 --2- 172.21.15.16:0/2495510341 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3d9c0b6590 0x7f3d9c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.841+0000 7f3db2000640 1 -- 172.21.15.16:0/2495510341 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f3da80e7a50 con 0x7f3db4105d80 2024-09-21T03:45:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.841+0000 7f3dba600640 1 --2- 172.21.15.16:0/2495510341 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3d9c0b6590 0x7f3d9c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.841+0000 7f3dc3235740 1 -- 172.21.15.16:0/4143048534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5602280 msgr2=0x5603e59c9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.841+0000 7f3dc3235740 1 --2- 172.21.15.16:0/4143048534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5602280 0x5603e59c9f10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009c00 tx=0x7f3da0011810 comp rx=0 tx=0).stop 2024-09-21T03:45:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.841+0000 7f3dc3235740 1 -- 172.21.15.16:0/4143048534 shutdown_connections 2024-09-21T03:45:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.841+0000 7f3dc3235740 1 --2- 172.21.15.16:0/4143048534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5602280 0x5603e59c9f10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.841+0000 7f3dc3235740 1 --2- 172.21.15.16:0/4143048534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e55fa0a0 0x5603e59c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.841+0000 7f3dc3235740 1 --2- 172.21.15.16:0/4143048534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e5622f60 0x5603e59c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.841+0000 7f3dc3235740 1 -- 172.21.15.16:0/4143048534 >> 172.21.15.16:0/4143048534 conn(0x5603e5617350 msgr2=0x5603e59c7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.842+0000 7f3dc3235740 1 -- 172.21.15.16:0/4143048534 shutdown_connections 2024-09-21T03:45:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.842+0000 7f3dba600640 1 --2- 172.21.15.16:0/2495510341 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3d9c0b6590 0x7f3d9c0b8aa0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3dac02f500 tx=0x7f3dac02f310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.842+0000 7f3dc3235740 1 -- 172.21.15.16:0/4143048534 wait complete. 2024-09-21T03:45:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.842+0000 7f3dc3235740 1 -- start start 2024-09-21T03:45:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.842+0000 7f3dc3235740 1 Processor -- start 2024-09-21T03:45:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.843+0000 7f3dc3235740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 0x5603e59d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.843+0000 7f3dc3235740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e5602280 0x5603e59d5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.843+0000 7f3dc3235740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5622f60 0x5603e59e0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.843+0000 7f3dc3235740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5603e57915f0 con 0x5603e5622f60 2024-09-21T03:45:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.843+0000 7f3dc3235740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5603e58cb460 con 0x5603e5602280 2024-09-21T03:45:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.843+0000 7f3dc3235740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5603e59d7cc0 con 0x5603e55fa0a0 2024-09-21T03:45:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.843+0000 7f3dc3235740 10 client.0 _ll_register_callbacks cb 0x5603e58db210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.843+0000 7f3db9c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 0x5603e59d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.843+0000 7f3dba600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5622f60 0x5603e59e0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.843+0000 7f3db9c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 0x5603e59d4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34158/0 (socket says 172.21.15.16:34158) 2024-09-21T03:45:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.843+0000 7f3db9c00640 1 -- 172.21.15.16:0/2439356984 learned_addr learned my addr 172.21.15.16:0/2439356984 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.844+0000 7f3db9200640 1 --2- 172.21.15.16:0/2439356984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e5602280 0x5603e59d5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.844+0000 7f3db9c00640 1 -- 172.21.15.16:0/2439356984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e5602280 msgr2=0x5603e59d5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.844+0000 7f3db9c00640 1 --2- 172.21.15.16:0/2439356984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e5602280 0x5603e59d5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.844+0000 7f3db9c00640 1 -- 172.21.15.16:0/2439356984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5622f60 msgr2=0x5603e59e0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.844+0000 7f3db9c00640 1 --2- 172.21.15.16:0/2439356984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5622f60 0x5603e59e0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.844+0000 7f3db9c00640 1 -- 172.21.15.16:0/2439356984 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603e568e540 con 0x5603e55fa0a0 2024-09-21T03:45:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.844+0000 7f3db9c00640 1 --2- 172.21.15.16:0/2439356984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 0x5603e59d4f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3da80ad810 tx=0x7f3da80aa000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3d7260414a8d569 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.844+0000 7f3d9b400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:45:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.844+0000 7f3d9b400640 1 -- 172.21.15.16:0/2439356984 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da80ad650 con 0x5603e55fa0a0 2024-09-21T03:45:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.844+0000 7f3d9b400640 1 -- 172.21.15.16:0/2439356984 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f3da80ac550 con 0x5603e55fa0a0 2024-09-21T03:45:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.845+0000 7f3dc3235740 1 -- 172.21.15.16:0/2439356984 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5603e58cda70 con 0x5603e55fa0a0 2024-09-21T03:45:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.845+0000 7f3d9b400640 1 -- 172.21.15.16:0/2439356984 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3da80acff0 con 0x5603e55fa0a0 2024-09-21T03:45:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.845+0000 7f3dc3235740 10 client.25309 fetch_fsmap learned FSMap version 29 2024-09-21T03:45:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.845+0000 7f3dc3235740 1 -- 172.21.15.16:0/2439356984 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5603e58e86f0 con 0x5603e55fa0a0 2024-09-21T03:45:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.845+0000 7f3d9b400640 1 -- 172.21.15.16:0/2439356984 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5603e58cda70 con 0x5603e55fa0a0 2024-09-21T03:45:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.846+0000 7f3d9b400640 1 -- 172.21.15.16:0/2439356984 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3da80ac750 con 0x5603e55fa0a0 2024-09-21T03:45:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.846+0000 7f3dc3235740 10 client.25309 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:45:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.846+0000 7f3dc3235740 10 client.25309 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-21T03:45:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.846+0000 7f3dc3235740 20 client.25309 populate_metadata read hostname 'smithi016' 2024-09-21T03:45:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.846+0000 7f3dc3235740 1 --2- 172.21.15.16:0/2439356984 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x5603e55e18e0 0x5603e59e74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.846+0000 7f3dc3235740 4 client.25309 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:45:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.846+0000 7f3dc3235740 1 -- 172.21.15.16:0/2439356984 --> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603e5942ca0 con 0x5603e55e18e0 2024-09-21T03:45:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.846+0000 7f3db9200640 1 --2- 172.21.15.16:0/2439356984 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x5603e55e18e0 0x5603e59e74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.847+0000 7f3db9200640 1 --2- 172.21.15.16:0/2439356984 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x5603e55e18e0 0x5603e59e74a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=800cb02c01d3bf9c server_cookie=3250460d1f0d9daa in_seq=0 out_seq=0 2024-09-21T03:45:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.847+0000 7f3d9b400640 10 client.25309 ms_handle_connect on v2:172.21.15.123:6834/86959152 2024-09-21T03:45:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.848+0000 7f3d9b400640 1 -- 172.21.15.16:0/2439356984 <== mds.4 v2:172.21.15.123:6834/86959152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5603e5942ca0 con 0x5603e55e18e0 2024-09-21T03:45:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.848+0000 7f3d9b400640 10 client.25309 handle_command_reply: tid=0 2024-09-21T03:45:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.848+0000 7f3dc3235740 1 client.25309 shutdown 2024-09-21T03:45:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.848+0000 7f3dc3235740 1 -- 172.21.15.16:0/2439356984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 msgr2=0x5603e59d4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.848+0000 7f3dc3235740 1 --2- 172.21.15.16:0/2439356984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 0x5603e59d4f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3da80ad810 tx=0x7f3da80aa000 comp rx=0 tx=0).stop 2024-09-21T03:45:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.848+0000 7f3dc3235740 1 -- 172.21.15.16:0/2439356984 shutdown_connections 2024-09-21T03:45:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.848+0000 7f3dc3235740 1 --2- 172.21.15.16:0/2439356984 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x5603e55e18e0 0x5603e59e74a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.848+0000 7f3dc3235740 1 --2- 172.21.15.16:0/2439356984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5622f60 0x5603e59e0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.848+0000 7f3dc3235740 1 --2- 172.21.15.16:0/2439356984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e5602280 0x5603e59d5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.848+0000 7f3dc3235740 1 --2- 172.21.15.16:0/2439356984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 0x5603e59d4f30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.848+0000 7f3d9b400640 0 client.25309 ms_handle_reset on v2:172.21.15.123:6834/86959152 2024-09-21T03:45:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.848+0000 7f3dc3235740 1 -- 172.21.15.16:0/2439356984 >> 172.21.15.16:0/2439356984 conn(0x5603e5617350 msgr2=0x5603e59cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.848+0000 7f3dc3235740 1 -- 172.21.15.16:0/2439356984 shutdown_connections 2024-09-21T03:45:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.848+0000 7f3dc3235740 1 -- 172.21.15.16:0/2439356984 wait complete. 2024-09-21T03:45:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.848+0000 7f3dc3235740 20 client.25309 trim_cache size 0 max 16384 2024-09-21T03:45:58.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:45:58 smithi134 ceph-mon[35031]: pgmap v408: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.867+0000 7f3dc3235740 1 Processor -- start 2024-09-21T03:45:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.867+0000 7f3dc3235740 1 -- start start 2024-09-21T03:45:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.868+0000 7f3dc3235740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 0x5603e59c2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.868+0000 7f3dc3235740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e5602280 0x5603e59ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.868+0000 7f3dc3235740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5622f60 0x5603e59ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.868+0000 7f3dc3235740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5603e59e57e0 con 0x5603e5622f60 2024-09-21T03:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.868+0000 7f3dc3235740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5603e59e5eb0 con 0x5603e5602280 2024-09-21T03:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.868+0000 7f3dc3235740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5603e59e6580 con 0x5603e55fa0a0 2024-09-21T03:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.868+0000 7f3db9c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 0x5603e59c2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.868+0000 7f3dba600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5622f60 0x5603e59ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.868+0000 7f3db9c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 0x5603e59c2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34174/0 (socket says 172.21.15.16:34174) 2024-09-21T03:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.868+0000 7f3db9c00640 1 -- 172.21.15.16:0/570264293 learned_addr learned my addr 172.21.15.16:0/570264293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.868+0000 7f3db9200640 1 --2- 172.21.15.16:0/570264293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e5602280 0x5603e59ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.869+0000 7f3db9c00640 1 -- 172.21.15.16:0/570264293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e5602280 msgr2=0x5603e59ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.869+0000 7f3db9c00640 1 --2- 172.21.15.16:0/570264293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e5602280 0x5603e59ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.869+0000 7f3db9c00640 1 -- 172.21.15.16:0/570264293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5622f60 msgr2=0x5603e59ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.869+0000 7f3db9c00640 1 --2- 172.21.15.16:0/570264293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5622f60 0x5603e59ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.869+0000 7f3db9c00640 1 -- 172.21.15.16:0/570264293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603e59cb780 con 0x5603e55fa0a0 2024-09-21T03:45:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.869+0000 7f3db9c00640 1 --2- 172.21.15.16:0/570264293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 0x5603e59c2010 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3da80b3530 tx=0x7f3da8018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da16ed94c6375c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.869+0000 7f3da6000640 1 -- 172.21.15.16:0/570264293 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da80ad1d0 con 0x5603e55fa0a0 2024-09-21T03:45:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.869+0000 7f3da6000640 1 -- 172.21.15.16:0/570264293 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da8010720 con 0x5603e55fa0a0 2024-09-21T03:45:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.869+0000 7f3da6000640 1 -- 172.21.15.16:0/570264293 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da80b6a30 con 0x5603e55fa0a0 2024-09-21T03:45:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.870+0000 7f3dc3235740 1 -- 172.21.15.16:0/570264293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 msgr2=0x5603e59c2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.870+0000 7f3dc3235740 1 --2- 172.21.15.16:0/570264293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 0x5603e59c2010 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3da80b3530 tx=0x7f3da8018420 comp rx=0 tx=0).stop 2024-09-21T03:45:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.870+0000 7f3dc3235740 1 -- 172.21.15.16:0/570264293 shutdown_connections 2024-09-21T03:45:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.870+0000 7f3dc3235740 1 --2- 172.21.15.16:0/570264293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5622f60 0x5603e59ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.870+0000 7f3dc3235740 1 --2- 172.21.15.16:0/570264293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e5602280 0x5603e59ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.870+0000 7f3dc3235740 1 --2- 172.21.15.16:0/570264293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 0x5603e59c2010 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.870+0000 7f3dc3235740 1 -- 172.21.15.16:0/570264293 >> 172.21.15.16:0/570264293 conn(0x5603e5617350 msgr2=0x5603e59e1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.870+0000 7f3dc3235740 1 -- 172.21.15.16:0/570264293 shutdown_connections 2024-09-21T03:45:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.870+0000 7f3dc3235740 1 -- 172.21.15.16:0/570264293 wait complete. 2024-09-21T03:45:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.871+0000 7f3dc3235740 1 -- start start 2024-09-21T03:45:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.871+0000 7f3dc3235740 1 Processor -- start 2024-09-21T03:45:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.871+0000 7f3dc3235740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 0x5603e59ef980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.871+0000 7f3dc3235740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5602280 0x5603e59eff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.871+0000 7f3dc3235740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e5622f60 0x5603e59e8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.872+0000 7f3dc3235740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5603e59e7320 con 0x5603e5602280 2024-09-21T03:45:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.872+0000 7f3dc3235740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5603e59e6c50 con 0x5603e5622f60 2024-09-21T03:45:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.872+0000 7f3dc3235740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5603e59e79f0 con 0x5603e55fa0a0 2024-09-21T03:45:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.872+0000 7f3dc3235740 10 client.0 _ll_register_callbacks cb 0x7f3da80a90e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:45:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.872+0000 7f3db9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5602280 0x5603e59eff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.872+0000 7f3dba600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e5622f60 0x5603e59e8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.872+0000 7f3db9c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 0x5603e59ef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.872+0000 7f3db9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5602280 0x5603e59eff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32888/0 (socket says 172.21.15.16:32888) 2024-09-21T03:45:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.872+0000 7f3db9200640 1 -- 172.21.15.16:0/3822113755 learned_addr learned my addr 172.21.15.16:0/3822113755 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:45:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.872+0000 7f3dba600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e5622f60 0x5603e59e8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:46442/0 (socket says 172.21.15.16:46442) 2024-09-21T03:45:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.872+0000 7f3db9c00640 1 -- 172.21.15.16:0/3822113755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e5622f60 msgr2=0x5603e59e8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.872+0000 7f3db9c00640 1 --2- 172.21.15.16:0/3822113755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e5622f60 0x5603e59e8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.872+0000 7f3db9c00640 1 -- 172.21.15.16:0/3822113755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5602280 msgr2=0x5603e59eff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:45:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.872+0000 7f3db9c00640 1 --2- 172.21.15.16:0/3822113755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5602280 0x5603e59eff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.872+0000 7f3db9c00640 1 -- 172.21.15.16:0/3822113755 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603e59e9570 con 0x5603e55fa0a0 2024-09-21T03:45:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.872+0000 7f3db9200640 1 --2- 172.21.15.16:0/3822113755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5602280 0x5603e59eff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.873+0000 7f3db9c00640 1 --2- 172.21.15.16:0/3822113755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 0x5603e59ef980 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3da80ad810 tx=0x7f3da80aed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be9ce7b963f3012 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.873+0000 7f3d9b400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:45:58.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.873+0000 7f3d9b400640 1 -- 172.21.15.16:0/3822113755 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da80ad1d0 con 0x5603e55fa0a0 2024-09-21T03:45:58.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.873+0000 7f3d9b400640 1 -- 172.21.15.16:0/3822113755 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f3da80ac550 con 0x5603e55fa0a0 2024-09-21T03:45:58.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.873+0000 7f3dc3235740 1 -- 172.21.15.16:0/3822113755 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5603e593e750 con 0x5603e55fa0a0 2024-09-21T03:45:58.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.873+0000 7f3d9b400640 1 -- 172.21.15.16:0/3822113755 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3da80ad390 con 0x5603e55fa0a0 2024-09-21T03:45:58.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.873+0000 7f3dc3235740 10 client.25315 fetch_fsmap learned FSMap version 29 2024-09-21T03:45:58.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.873+0000 7f3dc3235740 1 -- 172.21.15.16:0/3822113755 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5603e59e9570 con 0x5603e55fa0a0 2024-09-21T03:45:58.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.874+0000 7f3d9b400640 1 -- 172.21.15.16:0/3822113755 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5603e593e750 con 0x5603e55fa0a0 2024-09-21T03:45:58.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.874+0000 7f3d9b400640 1 -- 172.21.15.16:0/3822113755 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3da80cf2f0 con 0x5603e55fa0a0 2024-09-21T03:45:58.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.874+0000 7f3dc3235740 10 client.25315 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:45:58.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.874+0000 7f3dc3235740 10 client.25315 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-21T03:45:58.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.874+0000 7f3dc3235740 20 client.25315 populate_metadata read hostname 'smithi016' 2024-09-21T03:45:58.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.874+0000 7f3dc3235740 1 --2- 172.21.15.16:0/3822113755 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x5603e55b6c90 0x5603e5a55590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:58.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.874+0000 7f3dc3235740 4 client.25315 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:45:58.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.874+0000 7f3dc3235740 1 -- 172.21.15.16:0/3822113755 --> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] -- command(tid 0: {"prefix": "status"}) -- 0x5603e593bb80 con 0x5603e55b6c90 2024-09-21T03:45:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.875+0000 7f3db9200640 1 --2- 172.21.15.16:0/3822113755 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x5603e55b6c90 0x5603e5a55590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.875+0000 7f3db9200640 1 --2- 172.21.15.16:0/3822113755 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x5603e55b6c90 0x5603e5a55590 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bcd5467638b7d0a4 server_cookie=f23311d624e9c35 in_seq=0 out_seq=0 2024-09-21T03:45:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.875+0000 7f3d9b400640 10 client.25315 ms_handle_connect on v2:172.21.15.123:6834/86959152 2024-09-21T03:45:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.876+0000 7f3d9b400640 1 -- 172.21.15.16:0/3822113755 <== mds.4 v2:172.21.15.123:6834/86959152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5603e593bb80 con 0x5603e55b6c90 2024-09-21T03:45:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.876+0000 7f3d9b400640 10 client.25315 handle_command_reply: tid=0 2024-09-21T03:45:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.876+0000 7f3dc3235740 1 client.25315 shutdown 2024-09-21T03:45:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.876+0000 7f3dc3235740 1 -- 172.21.15.16:0/3822113755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 msgr2=0x5603e59ef980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.876+0000 7f3dc3235740 1 --2- 172.21.15.16:0/3822113755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 0x5603e59ef980 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3da80ad810 tx=0x7f3da80aed30 comp rx=0 tx=0).stop 2024-09-21T03:45:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.876+0000 7f3dc3235740 1 -- 172.21.15.16:0/3822113755 shutdown_connections 2024-09-21T03:45:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.876+0000 7f3dc3235740 1 --2- 172.21.15.16:0/3822113755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603e5622f60 0x5603e59e8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.876+0000 7f3dc3235740 1 --2- 172.21.15.16:0/3822113755 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x5603e55b6c90 0x5603e5a55590 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.876+0000 7f3dc3235740 1 --2- 172.21.15.16:0/3822113755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603e5602280 0x5603e59eff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.876+0000 7f3dc3235740 1 --2- 172.21.15.16:0/3822113755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603e55fa0a0 0x5603e59ef980 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.876+0000 7f3d9b400640 0 client.25315 ms_handle_reset on v2:172.21.15.123:6834/86959152 2024-09-21T03:45:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.876+0000 7f3dc3235740 1 -- 172.21.15.16:0/3822113755 >> 172.21.15.16:0/3822113755 conn(0x5603e5617350 msgr2=0x5603e59ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.876+0000 7f3dc3235740 1 -- 172.21.15.16:0/3822113755 shutdown_connections 2024-09-21T03:45:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.876+0000 7f3dc3235740 1 -- 172.21.15.16:0/3822113755 wait complete. 2024-09-21T03:45:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.876+0000 7f3dc3235740 20 client.25315 trim_cache size 0 max 16384 2024-09-21T03:45:58.881 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:45:58.881 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:45:58.881 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 4, 2024-09-21T03:45:58.881 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24668, 2024-09-21T03:45:58.881 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T03:45:58.881 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T03:45:58.881 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T03:45:58.882 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 113.343467956, 2024-09-21T03:45:58.882 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 29, 2024-09-21T03:45:58.882 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 86, 2024-09-21T03:45:58.882 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 84, 2024-09-21T03:45:58.882 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 206.62362927000001 2024-09-21T03:45:58.882 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:45:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.877+0000 7f3dbc600640 1 -- 172.21.15.16:0/2495510341 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3d9c0b6590 msgr2=0x7f3d9c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.877+0000 7f3dbc600640 1 --2- 172.21.15.16:0/2495510341 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3d9c0b6590 0x7f3d9c0b8aa0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3dac02f500 tx=0x7f3dac02f310 comp rx=0 tx=0).stop 2024-09-21T03:45:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.877+0000 7f3dbc600640 1 -- 172.21.15.16:0/2495510341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db4105d80 msgr2=0x7f3db419c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.877+0000 7f3dbc600640 1 --2- 172.21.15.16:0/2495510341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db4105d80 0x7f3db419c6e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3da800e9f0 tx=0x7f3da800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.877+0000 7f3dbc600640 1 -- 172.21.15.16:0/2495510341 shutdown_connections 2024-09-21T03:45:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.877+0000 7f3dbc600640 1 --2- 172.21.15.16:0/2495510341 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3d9c0b6590 0x7f3d9c0b8aa0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.877+0000 7f3dbc600640 1 --2- 172.21.15.16:0/2495510341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3db410c010 0x7f3db41a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.877+0000 7f3dbc600640 1 --2- 172.21.15.16:0/2495510341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db410b6c0 0x7f3db419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.877+0000 7f3dbc600640 1 --2- 172.21.15.16:0/2495510341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db4105d80 0x7f3db419c6e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.877+0000 7f3dbc600640 1 -- 172.21.15.16:0/2495510341 >> 172.21.15.16:0/2495510341 conn(0x7f3db4101810 msgr2=0x7f3db410acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.877+0000 7f3dbc600640 1 -- 172.21.15.16:0/2495510341 shutdown_connections 2024-09-21T03:45:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:45:58.877+0000 7f3dbc600640 1 -- 172.21.15.16:0/2495510341 wait complete. 2024-09-21T03:45:58.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:45:58 smithi123 ceph-mon[34673]: pgmap v408: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:45:59.846 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T03:46:00.225 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:00.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:00 smithi016 ceph-mon[30375]: pgmap v409: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:46:00.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:00 smithi134 ceph-mon[35031]: pgmap v409: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:46:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.892+0000 7f2d2d600640 1 -- 172.21.15.16:0/4166236019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d28078290 msgr2=0x7f2d28114200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.892+0000 7f2d2d600640 1 --2- 172.21.15.16:0/4166236019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d28078290 0x7f2d28114200 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009900 tx=0x7f2d1402a410 comp rx=0 tx=0).stop 2024-09-21T03:46:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.892+0000 7f2d2d600640 1 -- 172.21.15.16:0/4166236019 shutdown_connections 2024-09-21T03:46:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.892+0000 7f2d2d600640 1 --2- 172.21.15.16:0/4166236019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d28114740 0x7f2d28116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.892+0000 7f2d2d600640 1 --2- 172.21.15.16:0/4166236019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d28078290 0x7f2d28114200 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.892+0000 7f2d2d600640 1 --2- 172.21.15.16:0/4166236019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d28077940 0x7f2d28077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.892+0000 7f2d2d600640 1 -- 172.21.15.16:0/4166236019 >> 172.21.15.16:0/4166236019 conn(0x7f2d28101810 msgr2=0x7f2d28103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.892+0000 7f2d2d600640 1 -- 172.21.15.16:0/4166236019 shutdown_connections 2024-09-21T03:46:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.892+0000 7f2d2d600640 1 -- 172.21.15.16:0/4166236019 wait complete. 2024-09-21T03:46:00.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.893+0000 7f2d2d600640 1 Processor -- start 2024-09-21T03:46:00.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.893+0000 7f2d2d600640 1 -- start start 2024-09-21T03:46:00.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.894+0000 7f2d2d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d28077940 0x7f2d281a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.894+0000 7f2d2d600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d28078290 0x7f2d281a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.894+0000 7f2d2d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d28114740 0x7f2d281a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.894+0000 7f2d2d600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2811a0c0 con 0x7f2d28077940 2024-09-21T03:46:00.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.894+0000 7f2d2d600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28119f40 con 0x7f2d28114740 2024-09-21T03:46:00.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.894+0000 7f2d2d600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2811a240 con 0x7f2d28078290 2024-09-21T03:46:00.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.894+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d28077940 0x7f2d281a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.894+0000 7f2d26000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d28078290 0x7f2d281a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.894+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d28077940 0x7f2d281a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32904/0 (socket says 172.21.15.16:32904) 2024-09-21T03:46:00.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.894+0000 7f2d26a00640 1 -- 172.21.15.16:0/403319598 learned_addr learned my addr 172.21.15.16:0/403319598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:00.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.894+0000 7f2d26000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d28078290 0x7f2d281a1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34204/0 (socket says 172.21.15.16:34204) 2024-09-21T03:46:00.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.894+0000 7f2d27400640 1 --2- 172.21.15.16:0/403319598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d28114740 0x7f2d281a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.895+0000 7f2d26a00640 1 -- 172.21.15.16:0/403319598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d28078290 msgr2=0x7f2d281a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.895+0000 7f2d26a00640 1 --2- 172.21.15.16:0/403319598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d28078290 0x7f2d281a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.895+0000 7f2d26a00640 1 -- 172.21.15.16:0/403319598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d28114740 msgr2=0x7f2d281a6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.895+0000 7f2d26a00640 1 --2- 172.21.15.16:0/403319598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d28114740 0x7f2d281a6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.895+0000 7f2d26a00640 1 -- 172.21.15.16:0/403319598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d281a6a40 con 0x7f2d28077940 2024-09-21T03:46:00.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.895+0000 7f2d26000640 1 --2- 172.21.15.16:0/403319598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d28078290 0x7f2d281a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:00.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.895+0000 7f2d27400640 1 --2- 172.21.15.16:0/403319598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d28114740 0x7f2d281a6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:00.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.895+0000 7f2d26a00640 1 --2- 172.21.15.16:0/403319598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d28077940 0x7f2d281a0ad0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800d950 tx=0x7f2d1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.895+0000 7f2d1f400640 1 -- 172.21.15.16:0/403319598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18017030 con 0x7f2d28077940 2024-09-21T03:46:00.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.895+0000 7f2d1f400640 1 -- 172.21.15.16:0/403319598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d18004020 con 0x7f2d28077940 2024-09-21T03:46:00.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.895+0000 7f2d1f400640 1 -- 172.21.15.16:0/403319598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d180047a0 con 0x7f2d28077940 2024-09-21T03:46:00.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.895+0000 7f2d2d600640 1 -- 172.21.15.16:0/403319598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d281a6d30 con 0x7f2d28077940 2024-09-21T03:46:00.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.895+0000 7f2d2d600640 1 -- 172.21.15.16:0/403319598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d281ad490 con 0x7f2d28077940 2024-09-21T03:46:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.899+0000 7f2d34237740 1 Processor -- start 2024-09-21T03:46:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.899+0000 7f2d34237740 1 -- start start 2024-09-21T03:46:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.899+0000 7f2d1f400640 1 -- 172.21.15.16:0/403319598 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2d18009a30 con 0x7f2d28077940 2024-09-21T03:46:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.899+0000 7f2d34237740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608341bff60 0x560834566490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.899+0000 7f2d34237740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608341970a0 0x5608345669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.899+0000 7f2d34237740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083419f280 0x560834566f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.899+0000 7f2d34237740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5608344ddd50 con 0x56083419f280 2024-09-21T03:46:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.899+0000 7f2d34237740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5608344cd180 con 0x5608341970a0 2024-09-21T03:46:00.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.899+0000 7f2d34237740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5608344d0f90 con 0x5608341bff60 2024-09-21T03:46:00.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.900+0000 7f2d26000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083419f280 0x560834566f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.900+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608341970a0 0x5608345669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.900+0000 7f2d27400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608341bff60 0x560834566490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.900+0000 7f2d27400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608341bff60 0x560834566490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34216/0 (socket says 172.21.15.16:34216) 2024-09-21T03:46:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.900+0000 7f2d26000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083419f280 0x560834566f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32912/0 (socket says 172.21.15.16:32912) 2024-09-21T03:46:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.900+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608341970a0 0x5608345669d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:46480/0 (socket says 172.21.15.16:46480) 2024-09-21T03:46:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.900+0000 7f2d27400640 1 -- 172.21.15.16:0/2888491349 learned_addr learned my addr 172.21.15.16:0/2888491349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.900+0000 7f2d27400640 1 -- 172.21.15.16:0/2888491349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608341970a0 msgr2=0x5608345669d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.900+0000 7f2d27400640 1 --2- 172.21.15.16:0/2888491349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608341970a0 0x5608345669d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.900+0000 7f2d27400640 1 -- 172.21.15.16:0/2888491349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083419f280 msgr2=0x560834566f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.900+0000 7f2d27400640 1 --2- 172.21.15.16:0/2888491349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083419f280 0x560834566f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.900+0000 7f2d27400640 1 -- 172.21.15.16:0/2888491349 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560834497a70 con 0x5608341bff60 2024-09-21T03:46:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.901+0000 7f2d27400640 1 --2- 172.21.15.16:0/2888491349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608341bff60 0x560834566490 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d20010710 tx=0x7f2d20010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac5be7b4da24d41 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.901+0000 7f2d13400640 1 -- 172.21.15.16:0/2888491349 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000e8d0 con 0x5608341bff60 2024-09-21T03:46:00.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.901+0000 7f2d13400640 1 -- 172.21.15.16:0/2888491349 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d20012450 con 0x5608341bff60 2024-09-21T03:46:00.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.901+0000 7f2d13400640 1 -- 172.21.15.16:0/2888491349 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2001c670 con 0x5608341bff60 2024-09-21T03:46:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.901+0000 7f2d1f400640 1 --2- 172.21.15.16:0/403319598 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2d0c0b6510 0x7f2d0c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.902+0000 7f2d1f400640 1 -- 172.21.15.16:0/403319598 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f2d180e7130 con 0x7f2d28077940 2024-09-21T03:46:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.902+0000 7f2d26a00640 1 --2- 172.21.15.16:0/403319598 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2d0c0b6510 0x7f2d0c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.902+0000 7f2d34237740 1 -- 172.21.15.16:0/2888491349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608341bff60 msgr2=0x560834566490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.902+0000 7f2d34237740 1 --2- 172.21.15.16:0/2888491349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608341bff60 0x560834566490 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d20010710 tx=0x7f2d20010bd0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.902+0000 7f2d34237740 1 -- 172.21.15.16:0/2888491349 shutdown_connections 2024-09-21T03:46:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.902+0000 7f2d34237740 1 --2- 172.21.15.16:0/2888491349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083419f280 0x560834566f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.902+0000 7f2d34237740 1 --2- 172.21.15.16:0/2888491349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608341970a0 0x5608345669d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.902+0000 7f2d34237740 1 --2- 172.21.15.16:0/2888491349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608341bff60 0x560834566490 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.902+0000 7f2d34237740 1 -- 172.21.15.16:0/2888491349 >> 172.21.15.16:0/2888491349 conn(0x5608341b4350 msgr2=0x560834564380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.902+0000 7f2d26a00640 1 --2- 172.21.15.16:0/403319598 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2d0c0b6510 0x7f2d0c0b8a20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2d180a9480 tx=0x7f2d180eab70 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.902+0000 7f2d34237740 1 -- 172.21.15.16:0/2888491349 shutdown_connections 2024-09-21T03:46:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.902+0000 7f2d34237740 1 -- 172.21.15.16:0/2888491349 wait complete. 2024-09-21T03:46:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.903+0000 7f2d34237740 1 -- start start 2024-09-21T03:46:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.903+0000 7f2d34237740 1 Processor -- start 2024-09-21T03:46:00.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:00 smithi123 ceph-mon[34673]: pgmap v409: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:46:00.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.904+0000 7f2d34237740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341970a0 0x560834572080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.904+0000 7f2d34237740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56083419f280 0x560834572600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.904+0000 7f2d34237740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608341bff60 0x56083457df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.904+0000 7f2d34237740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56083432e5f0 con 0x5608341970a0 2024-09-21T03:46:00.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.904+0000 7f2d34237740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560834468460 con 0x56083419f280 2024-09-21T03:46:00.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.904+0000 7f2d34237740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560834574c70 con 0x5608341bff60 2024-09-21T03:46:00.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.904+0000 7f2d34237740 10 client.0 _ll_register_callbacks cb 0x560834478210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.904+0000 7f2d27400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341970a0 0x560834572080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.904+0000 7f2d27400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341970a0 0x560834572080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32928/0 (socket says 172.21.15.16:32928) 2024-09-21T03:46:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.904+0000 7f2d27400640 1 -- 172.21.15.16:0/1938046221 learned_addr learned my addr 172.21.15.16:0/1938046221 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.904+0000 7f2d26000640 1 --2- 172.21.15.16:0/1938046221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56083419f280 0x560834572600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.905+0000 7f2d27400640 1 --2- 172.21.15.16:0/1938046221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608341bff60 0x56083457df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.905+0000 7f2d27400640 1 -- 172.21.15.16:0/1938046221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608341bff60 msgr2=0x56083457df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.905+0000 7f2d27400640 1 --2- 172.21.15.16:0/1938046221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608341bff60 0x56083457df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.905+0000 7f2d27400640 1 -- 172.21.15.16:0/1938046221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56083419f280 msgr2=0x560834572600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.905+0000 7f2d27400640 1 --2- 172.21.15.16:0/1938046221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56083419f280 0x560834572600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.905+0000 7f2d27400640 1 -- 172.21.15.16:0/1938046221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56083422b540 con 0x5608341970a0 2024-09-21T03:46:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.905+0000 7f2d26000640 1 --2- 172.21.15.16:0/1938046221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56083419f280 0x560834572600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.905+0000 7f2d27400640 1 --2- 172.21.15.16:0/1938046221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341970a0 0x560834572080 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2d20006010 tx=0x7f2d20005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f2d0d1210da875 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.905+0000 7f2d07e00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:46:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.905+0000 7f2d07e00640 1 -- 172.21.15.16:0/1938046221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000ece0 con 0x5608341970a0 2024-09-21T03:46:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.905+0000 7f2d07e00640 1 -- 172.21.15.16:0/1938046221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f2d2001c410 con 0x5608341970a0 2024-09-21T03:46:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.905+0000 7f2d34237740 1 -- 172.21.15.16:0/1938046221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56083446aa70 con 0x5608341970a0 2024-09-21T03:46:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.906+0000 7f2d07e00640 1 -- 172.21.15.16:0/1938046221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2d2000ee80 con 0x5608341970a0 2024-09-21T03:46:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.906+0000 7f2d34237740 10 client.15339 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.906+0000 7f2d34237740 1 -- 172.21.15.16:0/1938046221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5608344856f0 con 0x5608341970a0 2024-09-21T03:46:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.906+0000 7f2d07e00640 1 -- 172.21.15.16:0/1938046221 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56083446aa70 con 0x5608341970a0 2024-09-21T03:46:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.906+0000 7f2d07e00640 1 -- 172.21.15.16:0/1938046221 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2d20029070 con 0x5608341970a0 2024-09-21T03:46:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.906+0000 7f2d34237740 10 client.15339 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.907+0000 7f2d34237740 10 client.15339 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:46:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.907+0000 7f2d34237740 20 client.15339 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.907+0000 7f2d34237740 1 --2- 172.21.15.16:0/1938046221 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x56083417e8e0 0x560834584360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.907+0000 7f2d34237740 4 client.15339 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.907+0000 7f2d34237740 1 -- 172.21.15.16:0/1938046221 --> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608344dfca0 con 0x56083417e8e0 2024-09-21T03:46:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.907+0000 7f2d26000640 1 --2- 172.21.15.16:0/1938046221 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x56083417e8e0 0x560834584360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.908+0000 7f2d26000640 1 --2- 172.21.15.16:0/1938046221 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x56083417e8e0 0x560834584360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=46d646601ff4c702 server_cookie=f7a2b81f594527c1 in_seq=0 out_seq=0 2024-09-21T03:46:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.908+0000 7f2d07e00640 10 client.15339 ms_handle_connect on v2:172.21.15.134:6836/2624530453 2024-09-21T03:46:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.909+0000 7f2d07e00640 1 -- 172.21.15.16:0/1938046221 <== mds.3 v2:172.21.15.134:6836/2624530453 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5608344dfca0 con 0x56083417e8e0 2024-09-21T03:46:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.909+0000 7f2d07e00640 10 client.15339 handle_command_reply: tid=0 2024-09-21T03:46:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.909+0000 7f2d34237740 1 client.15339 shutdown 2024-09-21T03:46:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.909+0000 7f2d34237740 1 -- 172.21.15.16:0/1938046221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341970a0 msgr2=0x560834572080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.909+0000 7f2d34237740 1 --2- 172.21.15.16:0/1938046221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341970a0 0x560834572080 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2d20006010 tx=0x7f2d20005bd0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.909+0000 7f2d34237740 1 -- 172.21.15.16:0/1938046221 shutdown_connections 2024-09-21T03:46:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.909+0000 7f2d34237740 1 --2- 172.21.15.16:0/1938046221 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x56083417e8e0 0x560834584360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.909+0000 7f2d34237740 1 --2- 172.21.15.16:0/1938046221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608341bff60 0x56083457df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.909+0000 7f2d34237740 1 --2- 172.21.15.16:0/1938046221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56083419f280 0x560834572600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.909+0000 7f2d34237740 1 --2- 172.21.15.16:0/1938046221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341970a0 0x560834572080 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.909+0000 7f2d07e00640 0 client.15339 ms_handle_reset on v2:172.21.15.134:6836/2624530453 2024-09-21T03:46:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.909+0000 7f2d34237740 1 -- 172.21.15.16:0/1938046221 >> 172.21.15.16:0/1938046221 conn(0x5608341b4350 msgr2=0x560834569620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.909+0000 7f2d34237740 1 -- 172.21.15.16:0/1938046221 shutdown_connections 2024-09-21T03:46:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.909+0000 7f2d34237740 1 -- 172.21.15.16:0/1938046221 wait complete. 2024-09-21T03:46:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.909+0000 7f2d34237740 20 client.15339 trim_cache size 0 max 16384 2024-09-21T03:46:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.929+0000 7f2d34237740 1 Processor -- start 2024-09-21T03:46:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.929+0000 7f2d34237740 1 -- start start 2024-09-21T03:46:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.929+0000 7f2d34237740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608341970a0 0x560834570700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.929+0000 7f2d34237740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56083419f280 0x560834570c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.929+0000 7f2d34237740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341bff60 0x56083456b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.929+0000 7f2d34237740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5608345826a0 con 0x5608341bff60 2024-09-21T03:46:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.929+0000 7f2d34237740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560834582d70 con 0x56083419f280 2024-09-21T03:46:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.929+0000 7f2d34237740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560834583440 con 0x5608341970a0 2024-09-21T03:46:00.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.930+0000 7f2d27400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341bff60 0x56083456b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.930+0000 7f2d27400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608341970a0 0x560834570700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.930+0000 7f2d26000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56083419f280 0x560834570c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.930+0000 7f2d27400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341bff60 0x56083456b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32934/0 (socket says 172.21.15.16:32934) 2024-09-21T03:46:00.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.930+0000 7f2d27400640 1 -- 172.21.15.16:0/3584358650 learned_addr learned my addr 172.21.15.16:0/3584358650 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:00.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.930+0000 7f2d26000640 1 -- 172.21.15.16:0/3584358650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608341970a0 msgr2=0x560834570700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.930+0000 7f2d26000640 1 --2- 172.21.15.16:0/3584358650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608341970a0 0x560834570700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.930+0000 7f2d26000640 1 -- 172.21.15.16:0/3584358650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341bff60 msgr2=0x56083456b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.930+0000 7f2d26000640 1 --2- 172.21.15.16:0/3584358650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341bff60 0x56083456b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.930+0000 7f2d26000640 1 -- 172.21.15.16:0/3584358650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56083455e610 con 0x56083419f280 2024-09-21T03:46:00.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.930+0000 7f2d26000640 1 --2- 172.21.15.16:0/3584358650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56083419f280 0x560834570c40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009230 tx=0x7f2d1402ab90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62d3ff648cf9ee2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.931+0000 7f2d12a00640 1 -- 172.21.15.16:0/3584358650 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14035040 con 0x56083419f280 2024-09-21T03:46:00.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.931+0000 7f2d12a00640 1 -- 172.21.15.16:0/3584358650 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d14004770 con 0x56083419f280 2024-09-21T03:46:00.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.931+0000 7f2d12a00640 1 -- 172.21.15.16:0/3584358650 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14003830 con 0x56083419f280 2024-09-21T03:46:00.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.932+0000 7f2d34237740 1 -- 172.21.15.16:0/3584358650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56083419f280 msgr2=0x560834570c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.932+0000 7f2d34237740 1 --2- 172.21.15.16:0/3584358650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56083419f280 0x560834570c40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009230 tx=0x7f2d1402ab90 comp rx=0 tx=0).stop 2024-09-21T03:46:00.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.932+0000 7f2d34237740 1 -- 172.21.15.16:0/3584358650 shutdown_connections 2024-09-21T03:46:00.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.932+0000 7f2d34237740 1 --2- 172.21.15.16:0/3584358650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341bff60 0x56083456b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.932+0000 7f2d34237740 1 --2- 172.21.15.16:0/3584358650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56083419f280 0x560834570c40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.932+0000 7f2d34237740 1 --2- 172.21.15.16:0/3584358650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608341970a0 0x560834570700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.932+0000 7f2d34237740 1 -- 172.21.15.16:0/3584358650 >> 172.21.15.16:0/3584358650 conn(0x5608341b4350 msgr2=0x560834572060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:00.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.932+0000 7f2d34237740 1 -- 172.21.15.16:0/3584358650 shutdown_connections 2024-09-21T03:46:00.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.932+0000 7f2d34237740 1 -- 172.21.15.16:0/3584358650 wait complete. 2024-09-21T03:46:00.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.932+0000 7f2d34237740 1 -- start start 2024-09-21T03:46:00.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.933+0000 7f2d34237740 1 Processor -- start 2024-09-21T03:46:00.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.933+0000 7f2d34237740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341970a0 0x56083458cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.933+0000 7f2d34237740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56083419f280 0x560834585fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.933+0000 7f2d34237740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608341bff60 0x5608345864f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.933+0000 7f2d34237740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5608345841e0 con 0x5608341970a0 2024-09-21T03:46:00.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.933+0000 7f2d34237740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560834583b10 con 0x5608341bff60 2024-09-21T03:46:00.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.933+0000 7f2d34237740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5608345848b0 con 0x56083419f280 2024-09-21T03:46:00.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.933+0000 7f2d34237740 10 client.0 _ll_register_callbacks cb 0x7f2d180eaa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:00.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.934+0000 7f2d26000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56083419f280 0x560834585fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.934+0000 7f2d26000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56083419f280 0x560834585fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34246/0 (socket says 172.21.15.16:34246) 2024-09-21T03:46:00.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.934+0000 7f2d26000640 1 -- 172.21.15.16:0/3330665026 learned_addr learned my addr 172.21.15.16:0/3330665026 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.934+0000 7f2d27400640 1 --2- 172.21.15.16:0/3330665026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341970a0 0x56083458cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.934+0000 7f2d27400640 1 --2- 172.21.15.16:0/3330665026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608341bff60 0x5608345864f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.934+0000 7f2d26000640 1 -- 172.21.15.16:0/3330665026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608341bff60 msgr2=0x5608345864f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.934+0000 7f2d26000640 1 --2- 172.21.15.16:0/3330665026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608341bff60 0x5608345864f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.934+0000 7f2d26000640 1 -- 172.21.15.16:0/3330665026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341970a0 msgr2=0x56083458cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.934+0000 7f2d26000640 1 --2- 172.21.15.16:0/3330665026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341970a0 0x56083458cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.934+0000 7f2d26000640 1 -- 172.21.15.16:0/3330665026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560834586a30 con 0x56083419f280 2024-09-21T03:46:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.934+0000 7f2d27400640 1 --2- 172.21.15.16:0/3330665026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608341bff60 0x5608345864f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.934+0000 7f2d26000640 1 --2- 172.21.15.16:0/3330665026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56083419f280 0x560834585fb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d14006010 tx=0x7f2d14005410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d32ae92b7fbc122b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.934+0000 7f2d07e00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:46:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.935+0000 7f2d07e00640 1 -- 172.21.15.16:0/3330665026 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14035030 con 0x56083419f280 2024-09-21T03:46:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.935+0000 7f2d34237740 1 -- 172.21.15.16:0/3330665026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d14004770 con 0x56083419f280 2024-09-21T03:46:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.935+0000 7f2d07e00640 1 -- 172.21.15.16:0/3330665026 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f2d14004d30 con 0x56083419f280 2024-09-21T03:46:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.935+0000 7f2d07e00640 1 -- 172.21.15.16:0/3330665026 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2d14060050 con 0x56083419f280 2024-09-21T03:46:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.935+0000 7f2d34237740 10 client.25333 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.935+0000 7f2d34237740 1 -- 172.21.15.16:0/3330665026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56083458c2f0 con 0x56083419f280 2024-09-21T03:46:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.935+0000 7f2d07e00640 1 -- 172.21.15.16:0/3330665026 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d14004770 con 0x56083419f280 2024-09-21T03:46:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.936+0000 7f2d07e00640 1 -- 172.21.15.16:0/3330665026 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2d140315c0 con 0x56083419f280 2024-09-21T03:46:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.936+0000 7f2d34237740 10 client.25333 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.936+0000 7f2d34237740 10 client.25333 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:46:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.936+0000 7f2d34237740 20 client.25333 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.936+0000 7f2d34237740 1 --2- 172.21.15.16:0/3330665026 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x560834153c90 0x5608345f2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.936+0000 7f2d34237740 4 client.25333 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:46:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.936+0000 7f2d34237740 1 -- 172.21.15.16:0/3330665026 --> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] -- command(tid 0: {"prefix": "status"}) -- 0x5608344db750 con 0x560834153c90 2024-09-21T03:46:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.936+0000 7f2d27400640 1 --2- 172.21.15.16:0/3330665026 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x560834153c90 0x5608345f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.936+0000 7f2d27400640 1 --2- 172.21.15.16:0/3330665026 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x560834153c90 0x5608345f2900 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=716d0d75976ac10e server_cookie=ba37567e25448845 in_seq=0 out_seq=0 2024-09-21T03:46:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.936+0000 7f2d07e00640 10 client.25333 ms_handle_connect on v2:172.21.15.134:6836/2624530453 2024-09-21T03:46:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.937+0000 7f2d07e00640 1 -- 172.21.15.16:0/3330665026 <== mds.3 v2:172.21.15.134:6836/2624530453 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5608344db750 con 0x560834153c90 2024-09-21T03:46:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.937+0000 7f2d07e00640 10 client.25333 handle_command_reply: tid=0 2024-09-21T03:46:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.937+0000 7f2d34237740 1 client.25333 shutdown 2024-09-21T03:46:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.937+0000 7f2d34237740 1 -- 172.21.15.16:0/3330665026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56083419f280 msgr2=0x560834585fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.937+0000 7f2d34237740 1 --2- 172.21.15.16:0/3330665026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56083419f280 0x560834585fb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d14006010 tx=0x7f2d14005410 comp rx=0 tx=0).stop 2024-09-21T03:46:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.937+0000 7f2d34237740 1 -- 172.21.15.16:0/3330665026 shutdown_connections 2024-09-21T03:46:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.937+0000 7f2d34237740 1 --2- 172.21.15.16:0/3330665026 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x560834153c90 0x5608345f2900 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.937+0000 7f2d34237740 1 --2- 172.21.15.16:0/3330665026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608341bff60 0x5608345864f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.937+0000 7f2d34237740 1 --2- 172.21.15.16:0/3330665026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56083419f280 0x560834585fb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.937+0000 7f2d07e00640 0 client.25333 ms_handle_reset on v2:172.21.15.134:6836/2624530453 2024-09-21T03:46:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.937+0000 7f2d34237740 1 --2- 172.21.15.16:0/3330665026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608341970a0 0x56083458cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.937+0000 7f2d34237740 1 -- 172.21.15.16:0/3330665026 >> 172.21.15.16:0/3330665026 conn(0x5608341b4350 msgr2=0x56083456b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.937+0000 7f2d34237740 1 -- 172.21.15.16:0/3330665026 shutdown_connections 2024-09-21T03:46:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.937+0000 7f2d34237740 1 -- 172.21.15.16:0/3330665026 wait complete. 2024-09-21T03:46:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.938+0000 7f2d34237740 20 client.25333 trim_cache size 0 max 16384 2024-09-21T03:46:00.942 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:46:00.942 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:46:00.942 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 3, 2024-09-21T03:46:00.942 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24670, 2024-09-21T03:46:00.942 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T03:46:00.942 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T03:46:00.942 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T03:46:00.942 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 116.414022954, 2024-09-21T03:46:00.942 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 29, 2024-09-21T03:46:00.943 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 86, 2024-09-21T03:46:00.943 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 84, 2024-09-21T03:46:00.943 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 203.58142326699999 2024-09-21T03:46:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:46:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.938+0000 7f2d2d600640 1 -- 172.21.15.16:0/403319598 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2d0c0b6510 msgr2=0x7f2d0c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.938+0000 7f2d2d600640 1 --2- 172.21.15.16:0/403319598 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2d0c0b6510 0x7f2d0c0b8a20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2d180a9480 tx=0x7f2d180eab70 comp rx=0 tx=0).stop 2024-09-21T03:46:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.938+0000 7f2d2d600640 1 -- 172.21.15.16:0/403319598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d28077940 msgr2=0x7f2d281a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.938+0000 7f2d2d600640 1 --2- 172.21.15.16:0/403319598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d28077940 0x7f2d281a0ad0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800d950 tx=0x7f2d1800de10 comp rx=0 tx=0).stop 2024-09-21T03:46:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.938+0000 7f2d2d600640 1 -- 172.21.15.16:0/403319598 shutdown_connections 2024-09-21T03:46:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.938+0000 7f2d2d600640 1 --2- 172.21.15.16:0/403319598 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2d0c0b6510 0x7f2d0c0b8a20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.938+0000 7f2d2d600640 1 --2- 172.21.15.16:0/403319598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d28114740 0x7f2d281a6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.938+0000 7f2d2d600640 1 --2- 172.21.15.16:0/403319598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d28078290 0x7f2d281a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.938+0000 7f2d2d600640 1 --2- 172.21.15.16:0/403319598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d28077940 0x7f2d281a0ad0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.938+0000 7f2d2d600640 1 -- 172.21.15.16:0/403319598 >> 172.21.15.16:0/403319598 conn(0x7f2d28101810 msgr2=0x7f2d281167b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.938+0000 7f2d2d600640 1 -- 172.21.15.16:0/403319598 shutdown_connections 2024-09-21T03:46:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:00.939+0000 7f2d2d600640 1 -- 172.21.15.16:0/403319598 wait complete. 2024-09-21T03:46:01.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T03:46:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:02.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:02 smithi016 ceph-mon[30375]: pgmap v410: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:46:02.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.845+0000 7ff734400640 1 -- 172.21.15.16:0/1056028688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c103b60 msgr2=0x7ff72c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:02.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.845+0000 7ff734400640 1 --2- 172.21.15.16:0/1056028688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c103b60 0x7ff72c103f70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff720009900 tx=0x7ff72002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.845+0000 7ff734400640 1 -- 172.21.15.16:0/1056028688 shutdown_connections 2024-09-21T03:46:02.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.845+0000 7ff734400640 1 --2- 172.21.15.16:0/1056028688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff72c109df0 0x7ff72c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.845+0000 7ff734400640 1 --2- 172.21.15.16:0/1056028688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff72c1094a0 0x7ff72c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.845+0000 7ff734400640 1 --2- 172.21.15.16:0/1056028688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c103b60 0x7ff72c103f70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.845+0000 7ff734400640 1 -- 172.21.15.16:0/1056028688 >> 172.21.15.16:0/1056028688 conn(0x7ff72c07b850 msgr2=0x7ff72c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:02.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.845+0000 7ff734400640 1 -- 172.21.15.16:0/1056028688 shutdown_connections 2024-09-21T03:46:02.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.845+0000 7ff734400640 1 -- 172.21.15.16:0/1056028688 wait complete. 2024-09-21T03:46:02.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.846+0000 7ff734400640 1 Processor -- start 2024-09-21T03:46:02.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.847+0000 7ff734400640 1 -- start start 2024-09-21T03:46:02.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.847+0000 7ff734400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff72c103b60 0x7ff72c19c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.847+0000 7ff734400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c1094a0 0x7ff72c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.847+0000 7ff734400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff72c109df0 0x7ff72c1a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.847+0000 7ff734400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c113ab0 con 0x7ff72c1094a0 2024-09-21T03:46:02.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.847+0000 7ff734400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c113930 con 0x7ff72c103b60 2024-09-21T03:46:02.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.847+0000 7ff734400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c113c30 con 0x7ff72c109df0 2024-09-21T03:46:02.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.847+0000 7ff731000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c1094a0 0x7ff72c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.847+0000 7ff731a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff72c103b60 0x7ff72c19c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.847+0000 7ff731000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c1094a0 0x7ff72c19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53056/0 (socket says 172.21.15.16:53056) 2024-09-21T03:46:02.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.847+0000 7ff731000640 1 -- 172.21.15.16:0/618116444 learned_addr learned my addr 172.21.15.16:0/618116444 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:02.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.847+0000 7ff732400640 1 --2- 172.21.15.16:0/618116444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff72c109df0 0x7ff72c1a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.848+0000 7ff731000640 1 -- 172.21.15.16:0/618116444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff72c109df0 msgr2=0x7ff72c1a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:02.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.848+0000 7ff731000640 1 --2- 172.21.15.16:0/618116444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff72c109df0 0x7ff72c1a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.848+0000 7ff731000640 1 -- 172.21.15.16:0/618116444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff72c103b60 msgr2=0x7ff72c19c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:02.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.848+0000 7ff731000640 1 --2- 172.21.15.16:0/618116444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff72c103b60 0x7ff72c19c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.848+0000 7ff731000640 1 -- 172.21.15.16:0/618116444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72c1a26d0 con 0x7ff72c1094a0 2024-09-21T03:46:02.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.848+0000 7ff731000640 1 --2- 172.21.15.16:0/618116444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c1094a0 0x7ff72c19cd00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff71800e9c0 tx=0x7ff71800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:02.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.849+0000 7ff72a000640 1 -- 172.21.15.16:0/618116444 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71800ccc0 con 0x7ff72c1094a0 2024-09-21T03:46:02.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.849+0000 7ff734400640 1 -- 172.21.15.16:0/618116444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72c1a29c0 con 0x7ff72c1094a0 2024-09-21T03:46:02.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.849+0000 7ff72a000640 1 -- 172.21.15.16:0/618116444 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7180054d0 con 0x7ff72c1094a0 2024-09-21T03:46:02.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.849+0000 7ff72a000640 1 -- 172.21.15.16:0/618116444 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff718010430 con 0x7ff72c1094a0 2024-09-21T03:46:02.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.849+0000 7ff734400640 1 -- 172.21.15.16:0/618116444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff72c1a9120 con 0x7ff72c1094a0 2024-09-21T03:46:02.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:02 smithi134 ceph-mon[35031]: pgmap v410: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:46:02.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.852+0000 7ff73b0a8740 1 Processor -- start 2024-09-21T03:46:02.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.852+0000 7ff73b0a8740 1 -- start start 2024-09-21T03:46:02.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.853+0000 7ff72a000640 1 -- 172.21.15.16:0/618116444 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff718004f40 con 0x7ff72c1094a0 2024-09-21T03:46:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.853+0000 7ff73b0a8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 0x558780a8b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.853+0000 7ff73b0a8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 0x558780a8b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.853+0000 7ff73b0a8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780a8bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.853+0000 7ff73b0a8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558780a02d50 con 0x5587806c4280 2024-09-21T03:46:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.853+0000 7ff73b0a8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5587809f2180 con 0x5587806e4f60 2024-09-21T03:46:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.853+0000 7ff73b0a8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5587809f5f90 con 0x5587806bc0a0 2024-09-21T03:46:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.853+0000 7ff732400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 0x558780a8b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.853+0000 7ff731000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780a8bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.853+0000 7ff732400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 0x558780a8b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:60968/0 (socket says 172.21.15.16:60968) 2024-09-21T03:46:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.853+0000 7ff732400640 1 -- 172.21.15.16:0/3668909253 learned_addr learned my addr 172.21.15.16:0/3668909253 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.853+0000 7ff731a00640 1 --2- 172.21.15.16:0/3668909253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 0x558780a8b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.854+0000 7ff732400640 1 -- 172.21.15.16:0/3668909253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 msgr2=0x558780a8b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.854+0000 7ff732400640 1 --2- 172.21.15.16:0/3668909253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 0x558780a8b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.854+0000 7ff732400640 1 -- 172.21.15.16:0/3668909253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 msgr2=0x558780a8bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.854+0000 7ff732400640 1 --2- 172.21.15.16:0/3668909253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780a8bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.854+0000 7ff732400640 1 -- 172.21.15.16:0/3668909253 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587809bca70 con 0x5587806e4f60 2024-09-21T03:46:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.854+0000 7ff731a00640 1 --2- 172.21.15.16:0/3668909253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 0x558780a8b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.854+0000 7ff732400640 1 --2- 172.21.15.16:0/3668909253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 0x558780a8b490 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff724009f60 tx=0x7ff724009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abe25def05833196 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.854+0000 7ff71ea00640 1 -- 172.21.15.16:0/3668909253 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72401e070 con 0x5587806e4f60 2024-09-21T03:46:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.855+0000 7ff71ea00640 1 -- 172.21.15.16:0/3668909253 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff724003c30 con 0x5587806e4f60 2024-09-21T03:46:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.855+0000 7ff71ea00640 1 -- 172.21.15.16:0/3668909253 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72400f840 con 0x5587806e4f60 2024-09-21T03:46:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.855+0000 7ff72a000640 1 --2- 172.21.15.16:0/618116444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff7140b6510 0x7ff7140b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.855+0000 7ff72a000640 1 -- 172.21.15.16:0/618116444 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7ff7180e7b20 con 0x7ff72c1094a0 2024-09-21T03:46:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.855+0000 7ff731a00640 1 --2- 172.21.15.16:0/618116444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff7140b6510 0x7ff7140b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.855+0000 7ff73b0a8740 1 -- 172.21.15.16:0/3668909253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 msgr2=0x558780a8b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.855+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/3668909253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 0x558780a8b490 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff724009f60 tx=0x7ff724009f90 comp rx=0 tx=0).stop 2024-09-21T03:46:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.856+0000 7ff73b0a8740 1 -- 172.21.15.16:0/3668909253 shutdown_connections 2024-09-21T03:46:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.856+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/3668909253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780a8bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.856+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/3668909253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 0x558780a8b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.856+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/3668909253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 0x558780a8b490 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.856+0000 7ff73b0a8740 1 -- 172.21.15.16:0/3668909253 >> 172.21.15.16:0/3668909253 conn(0x5587806d9350 msgr2=0x558780a89380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.856+0000 7ff73b0a8740 1 -- 172.21.15.16:0/3668909253 shutdown_connections 2024-09-21T03:46:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.856+0000 7ff73b0a8740 1 -- 172.21.15.16:0/3668909253 wait complete. 2024-09-21T03:46:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.856+0000 7ff731a00640 1 --2- 172.21.15.16:0/618116444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff7140b6510 0x7ff7140b8a20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff720031440 tx=0x7ff720009560 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.856+0000 7ff73b0a8740 1 -- start start 2024-09-21T03:46:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.857+0000 7ff73b0a8740 1 Processor -- start 2024-09-21T03:46:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.857+0000 7ff73b0a8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 0x558780a97090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.857+0000 7ff73b0a8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780a97610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.857+0000 7ff73b0a8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 0x558780aa2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.857+0000 7ff73b0a8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5587808535f0 con 0x5587806c4280 2024-09-21T03:46:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.857+0000 7ff73b0a8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55878098d460 con 0x5587806e4f60 2024-09-21T03:46:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.857+0000 7ff73b0a8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558780a99c70 con 0x5587806bc0a0 2024-09-21T03:46:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.857+0000 7ff73b0a8740 10 client.0 _ll_register_callbacks cb 0x55878099d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.858+0000 7ff731000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780a97610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.858+0000 7ff731a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 0x558780a97090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.858+0000 7ff731a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 0x558780a97090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34584/0 (socket says 172.21.15.16:34584) 2024-09-21T03:46:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.858+0000 7ff731a00640 1 -- 172.21.15.16:0/4044407167 learned_addr learned my addr 172.21.15.16:0/4044407167 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.858+0000 7ff731000640 1 --2- 172.21.15.16:0/4044407167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780a97610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53088/0 (socket says 172.21.15.16:53088) 2024-09-21T03:46:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.858+0000 7ff732400640 1 --2- 172.21.15.16:0/4044407167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 0x558780aa2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.858+0000 7ff731a00640 1 -- 172.21.15.16:0/4044407167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 msgr2=0x558780aa2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.858+0000 7ff731a00640 1 --2- 172.21.15.16:0/4044407167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 0x558780aa2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.858+0000 7ff731a00640 1 -- 172.21.15.16:0/4044407167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 msgr2=0x558780a97610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.858+0000 7ff731a00640 1 --2- 172.21.15.16:0/4044407167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780a97610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.858+0000 7ff731a00640 1 -- 172.21.15.16:0/4044407167 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558780750540 con 0x5587806bc0a0 2024-09-21T03:46:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.858+0000 7ff731000640 1 --2- 172.21.15.16:0/4044407167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780a97610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.858+0000 7ff732400640 1 --2- 172.21.15.16:0/4044407167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 0x558780aa2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.859+0000 7ff731a00640 1 --2- 172.21.15.16:0/4044407167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 0x558780a97090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff720009900 tx=0x7ff720009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14cff1f76cedef0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.859+0000 7ff713400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:46:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.859+0000 7ff713400640 1 -- 172.21.15.16:0/4044407167 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72004d070 con 0x5587806bc0a0 2024-09-21T03:46:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.859+0000 7ff73b0a8740 1 -- 172.21.15.16:0/4044407167 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55878098fa70 con 0x5587806bc0a0 2024-09-21T03:46:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.859+0000 7ff713400640 1 -- 172.21.15.16:0/4044407167 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7ff720005230 con 0x5587806bc0a0 2024-09-21T03:46:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.859+0000 7ff713400640 1 -- 172.21.15.16:0/4044407167 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff720059050 con 0x5587806bc0a0 2024-09-21T03:46:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.859+0000 7ff73b0a8740 10 client.25345 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.860+0000 7ff73b0a8740 1 -- 172.21.15.16:0/4044407167 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5587809aa6f0 con 0x5587806bc0a0 2024-09-21T03:46:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.860+0000 7ff713400640 1 -- 172.21.15.16:0/4044407167 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55878098fa70 con 0x5587806bc0a0 2024-09-21T03:46:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.860+0000 7ff713400640 1 -- 172.21.15.16:0/4044407167 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff720003710 con 0x5587806bc0a0 2024-09-21T03:46:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.860+0000 7ff73b0a8740 10 client.25345 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.860+0000 7ff73b0a8740 10 client.25345 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:46:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.860+0000 7ff73b0a8740 20 client.25345 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.860+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/4044407167 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x5587806a38e0 0x558780aa9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.860+0000 7ff73b0a8740 4 client.25345 mds_command: new command op to 24682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.860+0000 7ff73b0a8740 1 -- 172.21.15.16:0/4044407167 --> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558780a04ca0 con 0x5587806a38e0 2024-09-21T03:46:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.861+0000 7ff731000640 1 --2- 172.21.15.16:0/4044407167 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x5587806a38e0 0x558780aa9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.861+0000 7ff731000640 1 --2- 172.21.15.16:0/4044407167 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x5587806a38e0 0x558780aa9580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aff5c564a1103e3b server_cookie=2c1951a9b41c0807 in_seq=0 out_seq=0 2024-09-21T03:46:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.861+0000 7ff713400640 10 client.25345 ms_handle_connect on v2:172.21.15.134:6838/606193011 2024-09-21T03:46:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.862+0000 7ff713400640 1 -- 172.21.15.16:0/4044407167 <== mds.2 v2:172.21.15.134:6838/606193011 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558780a04ca0 con 0x5587806a38e0 2024-09-21T03:46:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.862+0000 7ff713400640 10 client.25345 handle_command_reply: tid=0 2024-09-21T03:46:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.862+0000 7ff73b0a8740 1 client.25345 shutdown 2024-09-21T03:46:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.862+0000 7ff73b0a8740 1 -- 172.21.15.16:0/4044407167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 msgr2=0x558780a97090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.862+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/4044407167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 0x558780a97090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff720009900 tx=0x7ff720009260 comp rx=0 tx=0).stop 2024-09-21T03:46:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.862+0000 7ff73b0a8740 1 -- 172.21.15.16:0/4044407167 shutdown_connections 2024-09-21T03:46:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.862+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/4044407167 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x5587806a38e0 0x558780aa9580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.862+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/4044407167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 0x558780aa2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.862+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/4044407167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780a97610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.862+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/4044407167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 0x558780a97090 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.862+0000 7ff713400640 0 client.25345 ms_handle_reset on v2:172.21.15.134:6838/606193011 2024-09-21T03:46:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.862+0000 7ff73b0a8740 1 -- 172.21.15.16:0/4044407167 >> 172.21.15.16:0/4044407167 conn(0x5587806d9350 msgr2=0x558780a8e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.863+0000 7ff73b0a8740 1 -- 172.21.15.16:0/4044407167 shutdown_connections 2024-09-21T03:46:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.863+0000 7ff73b0a8740 1 -- 172.21.15.16:0/4044407167 wait complete. 2024-09-21T03:46:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.863+0000 7ff73b0a8740 20 client.25345 trim_cache size 0 max 16384 2024-09-21T03:46:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.882+0000 7ff73b0a8740 1 Processor -- start 2024-09-21T03:46:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.882+0000 7ff73b0a8740 1 -- start start 2024-09-21T03:46:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.882+0000 7ff73b0a8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 0x558780a92040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.882+0000 7ff73b0a8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780a92580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.882+0000 7ff73b0a8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 0x558780a95710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.882+0000 7ff73b0a8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558780aa78c0 con 0x5587806c4280 2024-09-21T03:46:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.882+0000 7ff73b0a8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558780aa7f90 con 0x5587806e4f60 2024-09-21T03:46:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.882+0000 7ff73b0a8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558780aa8660 con 0x5587806bc0a0 2024-09-21T03:46:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.883+0000 7ff731000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780a92580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.883+0000 7ff731a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 0x558780a92040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.883+0000 7ff731a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 0x558780a92040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34592/0 (socket says 172.21.15.16:34592) 2024-09-21T03:46:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.883+0000 7ff731a00640 1 -- 172.21.15.16:0/996634243 learned_addr learned my addr 172.21.15.16:0/996634243 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.883+0000 7ff732400640 1 --2- 172.21.15.16:0/996634243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 0x558780a95710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.883+0000 7ff731000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780a92580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53092/0 (socket says 172.21.15.16:53092) 2024-09-21T03:46:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.883+0000 7ff731a00640 1 -- 172.21.15.16:0/996634243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 msgr2=0x558780a95710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.883+0000 7ff731a00640 1 --2- 172.21.15.16:0/996634243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 0x558780a95710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.883+0000 7ff731a00640 1 -- 172.21.15.16:0/996634243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 msgr2=0x558780a92580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.883+0000 7ff731a00640 1 --2- 172.21.15.16:0/996634243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780a92580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.883+0000 7ff731a00640 1 -- 172.21.15.16:0/996634243 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558780a83630 con 0x5587806bc0a0 2024-09-21T03:46:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.883+0000 7ff731a00640 1 --2- 172.21.15.16:0/996634243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 0x558780a92040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff720031a90 tx=0x7ff720003f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0a7a5a6ba3dce34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.884+0000 7ff71e000640 1 -- 172.21.15.16:0/996634243 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72004d070 con 0x5587806bc0a0 2024-09-21T03:46:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.884+0000 7ff71e000640 1 -- 172.21.15.16:0/996634243 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7200037f0 con 0x5587806bc0a0 2024-09-21T03:46:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.884+0000 7ff71e000640 1 -- 172.21.15.16:0/996634243 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720035040 con 0x5587806bc0a0 2024-09-21T03:46:02.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.885+0000 7ff73b0a8740 1 -- 172.21.15.16:0/996634243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 msgr2=0x558780a92040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:02.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.885+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/996634243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 0x558780a92040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff720031a90 tx=0x7ff720003f00 comp rx=0 tx=0).stop 2024-09-21T03:46:02.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.885+0000 7ff73b0a8740 1 -- 172.21.15.16:0/996634243 shutdown_connections 2024-09-21T03:46:02.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.885+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/996634243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806e4f60 0x558780a95710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.885+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/996634243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780a92580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.885+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/996634243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806bc0a0 0x558780a92040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.885+0000 7ff73b0a8740 1 -- 172.21.15.16:0/996634243 >> 172.21.15.16:0/996634243 conn(0x5587806d9350 msgr2=0x558780a84010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.885+0000 7ff73b0a8740 1 -- 172.21.15.16:0/996634243 shutdown_connections 2024-09-21T03:46:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.885+0000 7ff73b0a8740 1 -- 172.21.15.16:0/996634243 wait complete. 2024-09-21T03:46:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.885+0000 7ff73b0a8740 1 -- start start 2024-09-21T03:46:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.886+0000 7ff73b0a8740 1 Processor -- start 2024-09-21T03:46:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.886+0000 7ff73b0a8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806bc0a0 0x558780ab1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.886+0000 7ff73b0a8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780ab20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.886+0000 7ff73b0a8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806e4f60 0x558780a90940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.886+0000 7ff73b0a8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558780aa9400 con 0x5587806c4280 2024-09-21T03:46:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.886+0000 7ff73b0a8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558780aa8d30 con 0x5587806bc0a0 2024-09-21T03:46:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.886+0000 7ff73b0a8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558780aa9ad0 con 0x5587806e4f60 2024-09-21T03:46:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.886+0000 7ff73b0a8740 10 client.0 _ll_register_callbacks cb 0x7ff720032db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.886+0000 7ff731000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780ab20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.887+0000 7ff731000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780ab20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53108/0 (socket says 172.21.15.16:53108) 2024-09-21T03:46:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.887+0000 7ff731000640 1 -- 172.21.15.16:0/1533611562 learned_addr learned my addr 172.21.15.16:0/1533611562 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.887+0000 7ff731a00640 1 --2- 172.21.15.16:0/1533611562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806bc0a0 0x558780ab1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.887+0000 7ff732400640 1 --2- 172.21.15.16:0/1533611562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806e4f60 0x558780a90940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.887+0000 7ff731000640 1 -- 172.21.15.16:0/1533611562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806e4f60 msgr2=0x558780a90940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.887+0000 7ff731000640 1 --2- 172.21.15.16:0/1533611562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806e4f60 0x558780a90940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.887+0000 7ff731000640 1 -- 172.21.15.16:0/1533611562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806bc0a0 msgr2=0x558780ab1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.887+0000 7ff731000640 1 --2- 172.21.15.16:0/1533611562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806bc0a0 0x558780ab1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.887+0000 7ff731000640 1 -- 172.21.15.16:0/1533611562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558780a910d0 con 0x5587806c4280 2024-09-21T03:46:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.887+0000 7ff731a00640 1 --2- 172.21.15.16:0/1533611562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806bc0a0 0x558780ab1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.887+0000 7ff731000640 1 --2- 172.21.15.16:0/1533611562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780ab20b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff7180ad890 tx=0x7ff7180e7eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177f4a886defbeb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.887+0000 7ff713400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:46:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.888+0000 7ff713400640 1 -- 172.21.15.16:0/1533611562 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7180ed070 con 0x5587806c4280 2024-09-21T03:46:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.888+0000 7ff73b0a8740 1 -- 172.21.15.16:0/1533611562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7200037f0 con 0x5587806c4280 2024-09-21T03:46:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.888+0000 7ff713400640 1 -- 172.21.15.16:0/1533611562 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7ff7180acd60 con 0x5587806c4280 2024-09-21T03:46:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.888+0000 7ff713400640 1 -- 172.21.15.16:0/1533611562 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7180ad6d0 con 0x5587806c4280 2024-09-21T03:46:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.888+0000 7ff73b0a8740 10 client.15363 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.888+0000 7ff73b0a8740 1 -- 172.21.15.16:0/1533611562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558780aac390 con 0x5587806c4280 2024-09-21T03:46:02.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.888+0000 7ff713400640 1 -- 172.21.15.16:0/1533611562 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7200037f0 con 0x5587806c4280 2024-09-21T03:46:02.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.888+0000 7ff713400640 1 -- 172.21.15.16:0/1533611562 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff7180c3ab0 con 0x5587806c4280 2024-09-21T03:46:02.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.888+0000 7ff73b0a8740 10 client.15363 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:02.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.888+0000 7ff73b0a8740 10 client.15363 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:46:02.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.888+0000 7ff73b0a8740 20 client.15363 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:02.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.889+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/1533611562 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x558780678c90 0x558780b17f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:02.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.889+0000 7ff73b0a8740 4 client.15363 mds_command: new command op to 24682 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:46:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.889+0000 7ff73b0a8740 1 -- 172.21.15.16:0/1533611562 --> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] -- command(tid 0: {"prefix": "status"}) -- 0x558780a00750 con 0x558780678c90 2024-09-21T03:46:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.889+0000 7ff731a00640 1 --2- 172.21.15.16:0/1533611562 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x558780678c90 0x558780b17f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.889+0000 7ff731a00640 1 --2- 172.21.15.16:0/1533611562 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x558780678c90 0x558780b17f60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29469a750f733275 server_cookie=d6888c7f636147e7 in_seq=0 out_seq=0 2024-09-21T03:46:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.889+0000 7ff713400640 10 client.15363 ms_handle_connect on v2:172.21.15.134:6838/606193011 2024-09-21T03:46:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.890+0000 7ff713400640 1 -- 172.21.15.16:0/1533611562 <== mds.2 v2:172.21.15.134:6838/606193011 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x558780a00750 con 0x558780678c90 2024-09-21T03:46:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.890+0000 7ff713400640 10 client.15363 handle_command_reply: tid=0 2024-09-21T03:46:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.890+0000 7ff73b0a8740 1 client.15363 shutdown 2024-09-21T03:46:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.890+0000 7ff73b0a8740 1 -- 172.21.15.16:0/1533611562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 msgr2=0x558780ab20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.890+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/1533611562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780ab20b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff7180ad890 tx=0x7ff7180e7eb0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.890+0000 7ff73b0a8740 1 -- 172.21.15.16:0/1533611562 shutdown_connections 2024-09-21T03:46:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.890+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/1533611562 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x558780678c90 0x558780b17f60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.890+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/1533611562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587806e4f60 0x558780a90940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.890+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/1533611562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587806c4280 0x558780ab20b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.890+0000 7ff73b0a8740 1 --2- 172.21.15.16:0/1533611562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587806bc0a0 0x558780ab1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.890+0000 7ff73b0a8740 1 -- 172.21.15.16:0/1533611562 >> 172.21.15.16:0/1533611562 conn(0x5587806d9350 msgr2=0x558780a97070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:02.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.890+0000 7ff713400640 0 client.15363 ms_handle_reset on v2:172.21.15.134:6838/606193011 2024-09-21T03:46:02.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.890+0000 7ff73b0a8740 1 -- 172.21.15.16:0/1533611562 shutdown_connections 2024-09-21T03:46:02.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.890+0000 7ff73b0a8740 1 -- 172.21.15.16:0/1533611562 wait complete. 2024-09-21T03:46:02.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.890+0000 7ff73b0a8740 20 client.15363 trim_cache size 0 max 16384 2024-09-21T03:46:02.895 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:46:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:46:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-09-21T03:46:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24682, 2024-09-21T03:46:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T03:46:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T03:46:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T03:46:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 119.37256742, 2024-09-21T03:46:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 29, 2024-09-21T03:46:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 86, 2024-09-21T03:46:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 84, 2024-09-21T03:46:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 200.28582557199999 2024-09-21T03:46:02.896 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:46:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.891+0000 7ff734400640 1 -- 172.21.15.16:0/618116444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff7140b6510 msgr2=0x7ff7140b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.891+0000 7ff734400640 1 --2- 172.21.15.16:0/618116444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff7140b6510 0x7ff7140b8a20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff720031440 tx=0x7ff720009560 comp rx=0 tx=0).stop 2024-09-21T03:46:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.891+0000 7ff734400640 1 -- 172.21.15.16:0/618116444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c1094a0 msgr2=0x7ff72c19cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.891+0000 7ff734400640 1 --2- 172.21.15.16:0/618116444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c1094a0 0x7ff72c19cd00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff71800e9c0 tx=0x7ff71800edc0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.891+0000 7ff734400640 1 -- 172.21.15.16:0/618116444 shutdown_connections 2024-09-21T03:46:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.891+0000 7ff734400640 1 --2- 172.21.15.16:0/618116444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff7140b6510 0x7ff7140b8a20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.891+0000 7ff734400640 1 --2- 172.21.15.16:0/618116444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff72c109df0 0x7ff72c1a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.891+0000 7ff734400640 1 --2- 172.21.15.16:0/618116444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c1094a0 0x7ff72c19cd00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.891+0000 7ff734400640 1 --2- 172.21.15.16:0/618116444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff72c103b60 0x7ff72c19c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.891+0000 7ff734400640 1 -- 172.21.15.16:0/618116444 >> 172.21.15.16:0/618116444 conn(0x7ff72c07b850 msgr2=0x7ff72c107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.891+0000 7ff734400640 1 -- 172.21.15.16:0/618116444 shutdown_connections 2024-09-21T03:46:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:02.891+0000 7ff734400640 1 -- 172.21.15.16:0/618116444 wait complete. 2024-09-21T03:46:02.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:02 smithi123 ceph-mon[34673]: pgmap v410: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:46:03.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T03:46:04.106 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.778+0000 7fdcb8e00640 1 -- 172.21.15.16:0/589013201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcb410dc30 msgr2=0x7fdcb4110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.778+0000 7fdcb8e00640 1 --2- 172.21.15.16:0/589013201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcb410dc30 0x7fdcb4110120 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009be0 tx=0x7fdcac02a6c0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.778+0000 7fdcb8e00640 1 -- 172.21.15.16:0/589013201 shutdown_connections 2024-09-21T03:46:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.778+0000 7fdcb8e00640 1 --2- 172.21.15.16:0/589013201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcb410dc30 0x7fdcb4110120 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.778+0000 7fdcb8e00640 1 --2- 172.21.15.16:0/589013201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcb4105120 0x7fdcb410d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.778+0000 7fdcb8e00640 1 --2- 172.21.15.16:0/589013201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcb41047d0 0x7fdcb4104be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.778+0000 7fdcb8e00640 1 -- 172.21.15.16:0/589013201 >> 172.21.15.16:0/589013201 conn(0x7fdcb40fe2d0 msgr2=0x7fdcb4100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.779+0000 7fdcb8e00640 1 -- 172.21.15.16:0/589013201 shutdown_connections 2024-09-21T03:46:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.779+0000 7fdcb8e00640 1 -- 172.21.15.16:0/589013201 wait complete. 2024-09-21T03:46:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.780+0000 7fdcb8e00640 1 Processor -- start 2024-09-21T03:46:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.780+0000 7fdcb8e00640 1 -- start start 2024-09-21T03:46:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.780+0000 7fdcb8e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcb41047d0 0x7fdcb4102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.780+0000 7fdcb8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcb4105120 0x7fdcb4102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.780+0000 7fdcb8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcb410dc30 0x7fdcb4102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.780+0000 7fdcb8e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb4113890 con 0x7fdcb410dc30 2024-09-21T03:46:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.780+0000 7fdcb8e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb4113710 con 0x7fdcb4105120 2024-09-21T03:46:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.780+0000 7fdcb8e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb4113a10 con 0x7fdcb41047d0 2024-09-21T03:46:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.780+0000 7fdcb3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcb41047d0 0x7fdcb4102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.780+0000 7fdcb3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcb41047d0 0x7fdcb4102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34634/0 (socket says 172.21.15.16:34634) 2024-09-21T03:46:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.781+0000 7fdcb3400640 1 -- 172.21.15.16:0/1494355880 learned_addr learned my addr 172.21.15.16:0/1494355880 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.781+0000 7fdcb3e00640 1 --2- 172.21.15.16:0/1494355880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcb410dc30 0x7fdcb4102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.781+0000 7fdcb2a00640 1 --2- 172.21.15.16:0/1494355880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcb4105120 0x7fdcb4102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.781+0000 7fdcb3400640 1 -- 172.21.15.16:0/1494355880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcb4105120 msgr2=0x7fdcb4102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.781+0000 7fdcb3400640 1 --2- 172.21.15.16:0/1494355880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcb4105120 0x7fdcb4102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.781+0000 7fdcb3400640 1 -- 172.21.15.16:0/1494355880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcb410dc30 msgr2=0x7fdcb4102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.781+0000 7fdcb3400640 1 --2- 172.21.15.16:0/1494355880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcb410dc30 0x7fdcb4102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.781+0000 7fdcb3400640 1 -- 172.21.15.16:0/1494355880 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb41a8500 con 0x7fdcb41047d0 2024-09-21T03:46:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.781+0000 7fdcb3400640 1 --2- 172.21.15.16:0/1494355880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcb41047d0 0x7fdcb4102510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdca400ea60 tx=0x7fdca400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.782+0000 7fdcabe00640 1 -- 172.21.15.16:0/1494355880 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca400cde0 con 0x7fdcb41047d0 2024-09-21T03:46:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.782+0000 7fdcb8e00640 1 -- 172.21.15.16:0/1494355880 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb41a87f0 con 0x7fdcb41047d0 2024-09-21T03:46:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.782+0000 7fdcabe00640 1 -- 172.21.15.16:0/1494355880 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdca40035f0 con 0x7fdcb41047d0 2024-09-21T03:46:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.782+0000 7fdcabe00640 1 -- 172.21.15.16:0/1494355880 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4010690 con 0x7fdcb41047d0 2024-09-21T03:46:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.782+0000 7fdcb8e00640 1 -- 172.21.15.16:0/1494355880 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb41a8d80 con 0x7fdcb41047d0 2024-09-21T03:46:04.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.785+0000 7fdcc10c2740 1 Processor -- start 2024-09-21T03:46:04.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.785+0000 7fdcc10c2740 1 -- start start 2024-09-21T03:46:04.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.786+0000 7fdcc10c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68c2f60 0x5627b6c69670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.786+0000 7fdcc10c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b689a0a0 0x5627b6c69bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.786+0000 7fdcc10c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b68a2280 0x5627b6c6a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.786+0000 7fdcc10c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5627b6be0d50 con 0x5627b689a0a0 2024-09-21T03:46:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.786+0000 7fdcc10c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5627b6bd0180 con 0x5627b68a2280 2024-09-21T03:46:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.786+0000 7fdcc10c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5627b6bd3f90 con 0x5627b68c2f60 2024-09-21T03:46:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.786+0000 7fdcabe00640 1 -- 172.21.15.16:0/1494355880 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fdca40037a0 con 0x7fdcb41047d0 2024-09-21T03:46:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.786+0000 7fdcb3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68c2f60 0x5627b6c69670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.786+0000 7fdcb3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68c2f60 0x5627b6c69670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34642/0 (socket says 172.21.15.16:34642) 2024-09-21T03:46:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.786+0000 7fdcb3e00640 1 -- 172.21.15.16:0/3420448883 learned_addr learned my addr 172.21.15.16:0/3420448883 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.786+0000 7fdcb2a00640 1 --2- 172.21.15.16:0/3420448883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b68a2280 0x5627b6c6a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.786+0000 7fdcb3400640 1 --2- 172.21.15.16:0/3420448883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b689a0a0 0x5627b6c69bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.786+0000 7fdcb3e00640 1 -- 172.21.15.16:0/3420448883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b68a2280 msgr2=0x5627b6c6a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.786+0000 7fdcb3e00640 1 --2- 172.21.15.16:0/3420448883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b68a2280 0x5627b6c6a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.786+0000 7fdcb3e00640 1 -- 172.21.15.16:0/3420448883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b689a0a0 msgr2=0x5627b6c69bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.786+0000 7fdcb3e00640 1 --2- 172.21.15.16:0/3420448883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b689a0a0 0x5627b6c69bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.787+0000 7fdcb3e00640 1 -- 172.21.15.16:0/3420448883 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b6b9aa70 con 0x5627b68c2f60 2024-09-21T03:46:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.787+0000 7fdcb2a00640 1 --2- 172.21.15.16:0/3420448883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b68a2280 0x5627b6c6a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.787+0000 7fdcb3e00640 1 --2- 172.21.15.16:0/3420448883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68c2f60 0x5627b6c69670 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009be0 tx=0x7fdcac030430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=251e456ab84639a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.787+0000 7fdca8c00640 1 -- 172.21.15.16:0/3420448883 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac041030 con 0x5627b68c2f60 2024-09-21T03:46:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.787+0000 7fdca8c00640 1 -- 172.21.15.16:0/3420448883 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcac003e20 con 0x5627b68c2f60 2024-09-21T03:46:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.787+0000 7fdca8c00640 1 -- 172.21.15.16:0/3420448883 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac036040 con 0x5627b68c2f60 2024-09-21T03:46:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.788+0000 7fdcabe00640 1 --2- 172.21.15.16:0/1494355880 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdc980b6510 0x7fdc980b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.788+0000 7fdcabe00640 1 -- 172.21.15.16:0/1494355880 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fdca40e79e0 con 0x7fdcb41047d0 2024-09-21T03:46:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.788+0000 7fdcb3400640 1 --2- 172.21.15.16:0/1494355880 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdc980b6510 0x7fdc980b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.788+0000 7fdcc10c2740 1 -- 172.21.15.16:0/3420448883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68c2f60 msgr2=0x5627b6c69670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.788+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/3420448883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68c2f60 0x5627b6c69670 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009be0 tx=0x7fdcac030430 comp rx=0 tx=0).stop 2024-09-21T03:46:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.788+0000 7fdcc10c2740 1 -- 172.21.15.16:0/3420448883 shutdown_connections 2024-09-21T03:46:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.788+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/3420448883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b68a2280 0x5627b6c6a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.788+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/3420448883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b689a0a0 0x5627b6c69bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.788+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/3420448883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68c2f60 0x5627b6c69670 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.788+0000 7fdcc10c2740 1 -- 172.21.15.16:0/3420448883 >> 172.21.15.16:0/3420448883 conn(0x5627b68b7350 msgr2=0x5627b6c67560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.789+0000 7fdcc10c2740 1 -- 172.21.15.16:0/3420448883 shutdown_connections 2024-09-21T03:46:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.789+0000 7fdcc10c2740 1 -- 172.21.15.16:0/3420448883 wait complete. 2024-09-21T03:46:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.789+0000 7fdcb3400640 1 --2- 172.21.15.16:0/1494355880 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdc980b6510 0x7fdc980b8a20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdca40a9d30 tx=0x7fdca400ef90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.789+0000 7fdcc10c2740 1 -- start start 2024-09-21T03:46:04.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.790+0000 7fdcc10c2740 1 Processor -- start 2024-09-21T03:46:04.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.790+0000 7fdcc10c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b689a0a0 0x5627b6c752a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.790+0000 7fdcc10c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b68a2280 0x5627b6c757e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.790+0000 7fdcc10c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68c2f60 0x5627b6c81110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.790+0000 7fdcc10c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5627b6a315f0 con 0x5627b68a2280 2024-09-21T03:46:04.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.790+0000 7fdcc10c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5627b6b6b460 con 0x5627b689a0a0 2024-09-21T03:46:04.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.790+0000 7fdcc10c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5627b6c77df0 con 0x5627b68c2f60 2024-09-21T03:46:04.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.790+0000 7fdcc10c2740 10 client.0 _ll_register_callbacks cb 0x5627b6b7b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:04.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.790+0000 7fdcb2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b68a2280 0x5627b6c757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:04.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.790+0000 7fdcb3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b689a0a0 0x5627b6c752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:04.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.790+0000 7fdcb3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b689a0a0 0x5627b6c752a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:32830/0 (socket says 172.21.15.16:32830) 2024-09-21T03:46:04.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.790+0000 7fdcb3e00640 1 -- 172.21.15.16:0/85717985 learned_addr learned my addr 172.21.15.16:0/85717985 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.791+0000 7fdcb3e00640 1 --2- 172.21.15.16:0/85717985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68c2f60 0x5627b6c81110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.791+0000 7fdcb3e00640 1 -- 172.21.15.16:0/85717985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68c2f60 msgr2=0x5627b6c81110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.791+0000 7fdcb3e00640 1 --2- 172.21.15.16:0/85717985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68c2f60 0x5627b6c81110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.791+0000 7fdcb3e00640 1 -- 172.21.15.16:0/85717985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b68a2280 msgr2=0x5627b6c757e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.791+0000 7fdcb3e00640 1 --2- 172.21.15.16:0/85717985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b68a2280 0x5627b6c757e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.791+0000 7fdcb3e00640 1 -- 172.21.15.16:0/85717985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b692e540 con 0x5627b689a0a0 2024-09-21T03:46:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.791+0000 7fdcb3e00640 1 --2- 172.21.15.16:0/85717985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b689a0a0 0x5627b6c752a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009420 tx=0x7fdcac030d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=475514a42829332c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.791+0000 7fdc9d600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:46:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.792+0000 7fdc9d600640 1 -- 172.21.15.16:0/85717985 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac036050 con 0x5627b689a0a0 2024-09-21T03:46:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.792+0000 7fdcc10c2740 1 -- 172.21.15.16:0/85717985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5627b6b6da70 con 0x5627b689a0a0 2024-09-21T03:46:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.792+0000 7fdc9d600640 1 -- 172.21.15.16:0/85717985 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fdcac030720 con 0x5627b689a0a0 2024-09-21T03:46:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.792+0000 7fdc9d600640 1 -- 172.21.15.16:0/85717985 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdcac059050 con 0x5627b689a0a0 2024-09-21T03:46:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.792+0000 7fdcc10c2740 10 client.25379 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:04.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.792+0000 7fdcc10c2740 1 -- 172.21.15.16:0/85717985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5627b6b886f0 con 0x5627b689a0a0 2024-09-21T03:46:04.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.792+0000 7fdc9d600640 1 -- 172.21.15.16:0/85717985 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5627b6b6da70 con 0x5627b689a0a0 2024-09-21T03:46:04.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.792+0000 7fdc9d600640 1 -- 172.21.15.16:0/85717985 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdcac004530 con 0x5627b689a0a0 2024-09-21T03:46:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.793+0000 7fdcc10c2740 10 client.25379 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.793+0000 7fdcc10c2740 10 client.25379 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:46:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.793+0000 7fdcc10c2740 20 client.25379 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.793+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/85717985 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x5627b68818e0 0x5627b6c872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.793+0000 7fdcc10c2740 4 client.25379 mds_command: new command op to 24689 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.793+0000 7fdcc10c2740 1 -- 172.21.15.16:0/85717985 --> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627b6be2ca0 con 0x5627b68818e0 2024-09-21T03:46:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.793+0000 7fdcb2a00640 1 --2- 172.21.15.16:0/85717985 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x5627b68818e0 0x5627b6c872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.794+0000 7fdcb2a00640 1 --2- 172.21.15.16:0/85717985 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x5627b68818e0 0x5627b6c872e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2075f373bdeb6f1 server_cookie=cb97701fd82a2a93 in_seq=0 out_seq=0 2024-09-21T03:46:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.794+0000 7fdc9d600640 10 client.25379 ms_handle_connect on v2:172.21.15.123:6836/2001424876 2024-09-21T03:46:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.794+0000 7fdc9d600640 1 -- 172.21.15.16:0/85717985 <== mds.1 v2:172.21.15.123:6836/2001424876 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5627b6be2ca0 con 0x5627b68818e0 2024-09-21T03:46:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.794+0000 7fdc9d600640 10 client.25379 handle_command_reply: tid=0 2024-09-21T03:46:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.794+0000 7fdcc10c2740 1 client.25379 shutdown 2024-09-21T03:46:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.795+0000 7fdcc10c2740 1 -- 172.21.15.16:0/85717985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b689a0a0 msgr2=0x5627b6c752a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.795+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/85717985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b689a0a0 0x5627b6c752a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009420 tx=0x7fdcac030d70 comp rx=0 tx=0).stop 2024-09-21T03:46:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.795+0000 7fdcc10c2740 1 -- 172.21.15.16:0/85717985 shutdown_connections 2024-09-21T03:46:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.795+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/85717985 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x5627b68818e0 0x5627b6c872e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.795+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/85717985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68c2f60 0x5627b6c81110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.795+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/85717985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b68a2280 0x5627b6c757e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.795+0000 7fdc9d600640 0 client.25379 ms_handle_reset on v2:172.21.15.123:6836/2001424876 2024-09-21T03:46:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.795+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/85717985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b689a0a0 0x5627b6c752a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.795+0000 7fdcc10c2740 1 -- 172.21.15.16:0/85717985 >> 172.21.15.16:0/85717985 conn(0x5627b68b7350 msgr2=0x5627b6c6c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.795+0000 7fdcc10c2740 1 -- 172.21.15.16:0/85717985 shutdown_connections 2024-09-21T03:46:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.795+0000 7fdcc10c2740 1 -- 172.21.15.16:0/85717985 wait complete. 2024-09-21T03:46:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.795+0000 7fdcc10c2740 20 client.25379 trim_cache size 0 max 16384 2024-09-21T03:46:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.814+0000 7fdcc10c2740 1 Processor -- start 2024-09-21T03:46:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.814+0000 7fdcc10c2740 1 -- start start 2024-09-21T03:46:04.817 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:04 smithi016 ceph-mon[30375]: pgmap v411: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:46:04.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.815+0000 7fdcc10c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b689a0a0 0x5627b6c857a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.815+0000 7fdcc10c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68a2280 0x5627b6c872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.815+0000 7fdcc10c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b68c2f60 0x5627b6c84fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.815+0000 7fdcc10c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5627b6c85620 con 0x5627b689a0a0 2024-09-21T03:46:04.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.815+0000 7fdcc10c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5627b6c85cf0 con 0x5627b68c2f60 2024-09-21T03:46:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.815+0000 7fdcc10c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5627b6c863c0 con 0x5627b68a2280 2024-09-21T03:46:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.815+0000 7fdcb3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b689a0a0 0x5627b6c857a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.815+0000 7fdcb3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b689a0a0 0x5627b6c857a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53144/0 (socket says 172.21.15.16:53144) 2024-09-21T03:46:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.815+0000 7fdcb3e00640 1 -- 172.21.15.16:0/1316224612 learned_addr learned my addr 172.21.15.16:0/1316224612 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.815+0000 7fdcb2a00640 1 --2- 172.21.15.16:0/1316224612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68a2280 0x5627b6c872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.816+0000 7fdcb3e00640 1 -- 172.21.15.16:0/1316224612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68a2280 msgr2=0x5627b6c872e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.816+0000 7fdcb3e00640 1 --2- 172.21.15.16:0/1316224612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68a2280 0x5627b6c872e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.816+0000 7fdcb3e00640 1 -- 172.21.15.16:0/1316224612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b68c2f60 msgr2=0x5627b6c84fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.816+0000 7fdcb3e00640 1 --2- 172.21.15.16:0/1316224612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b68c2f60 0x5627b6c84fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.816+0000 7fdcb3e00640 1 -- 172.21.15.16:0/1316224612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b6c76360 con 0x5627b689a0a0 2024-09-21T03:46:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.816+0000 7fdcb2a00640 1 --2- 172.21.15.16:0/1316224612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68a2280 0x5627b6c872e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.816+0000 7fdcb3e00640 1 --2- 172.21.15.16:0/1316224612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b689a0a0 0x5627b6c857a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdcac030d40 tx=0x7fdcac03d760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10f2054ebeea0f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.817+0000 7fdc9fe00640 1 -- 172.21.15.16:0/1316224612 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac036040 con 0x5627b689a0a0 2024-09-21T03:46:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.817+0000 7fdc9fe00640 1 -- 172.21.15.16:0/1316224612 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcac030720 con 0x5627b689a0a0 2024-09-21T03:46:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.817+0000 7fdc9fe00640 1 -- 172.21.15.16:0/1316224612 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac041040 con 0x5627b689a0a0 2024-09-21T03:46:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.818+0000 7fdcc10c2740 1 -- 172.21.15.16:0/1316224612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b689a0a0 msgr2=0x5627b6c857a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.818+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/1316224612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b689a0a0 0x5627b6c857a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdcac030d40 tx=0x7fdcac03d760 comp rx=0 tx=0).stop 2024-09-21T03:46:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.818+0000 7fdcc10c2740 1 -- 172.21.15.16:0/1316224612 shutdown_connections 2024-09-21T03:46:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.818+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/1316224612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b68c2f60 0x5627b6c84fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.818+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/1316224612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68a2280 0x5627b6c872e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.818+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/1316224612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b689a0a0 0x5627b6c857a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.818+0000 7fdcc10c2740 1 -- 172.21.15.16:0/1316224612 >> 172.21.15.16:0/1316224612 conn(0x5627b68b7350 msgr2=0x5627b6c75280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.818+0000 7fdcc10c2740 1 -- 172.21.15.16:0/1316224612 shutdown_connections 2024-09-21T03:46:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.818+0000 7fdcc10c2740 1 -- 172.21.15.16:0/1316224612 wait complete. 2024-09-21T03:46:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.818+0000 7fdcc10c2740 1 -- start start 2024-09-21T03:46:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.819+0000 7fdcc10c2740 1 Processor -- start 2024-09-21T03:46:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.819+0000 7fdcc10c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b689a0a0 0x5627b6c8fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.819+0000 7fdcc10c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b68a2280 0x5627b6c88f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.819+0000 7fdcc10c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68c2f60 0x5627b6c894d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.819+0000 7fdcc10c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5627b6c87160 con 0x5627b68a2280 2024-09-21T03:46:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.819+0000 7fdcc10c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5627b6c86a90 con 0x5627b689a0a0 2024-09-21T03:46:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.819+0000 7fdcc10c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5627b6c87830 con 0x5627b68c2f60 2024-09-21T03:46:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.819+0000 7fdcc10c2740 10 client.0 _ll_register_callbacks cb 0x7fdcac004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.820+0000 7fdcb2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b68a2280 0x5627b6c88f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.820+0000 7fdcb2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b68a2280 0x5627b6c88f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53152/0 (socket says 172.21.15.16:53152) 2024-09-21T03:46:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.820+0000 7fdcb2a00640 1 -- 172.21.15.16:0/1641701075 learned_addr learned my addr 172.21.15.16:0/1641701075 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.820+0000 7fdcb3e00640 1 --2- 172.21.15.16:0/1641701075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b689a0a0 0x5627b6c8fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.820+0000 7fdcb3e00640 1 --2- 172.21.15.16:0/1641701075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68c2f60 0x5627b6c894d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.820+0000 7fdcb2a00640 1 -- 172.21.15.16:0/1641701075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68c2f60 msgr2=0x5627b6c894d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.820+0000 7fdcb2a00640 1 --2- 172.21.15.16:0/1641701075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68c2f60 0x5627b6c894d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.820+0000 7fdcb2a00640 1 -- 172.21.15.16:0/1641701075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b689a0a0 msgr2=0x5627b6c8fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.820+0000 7fdcb2a00640 1 --2- 172.21.15.16:0/1641701075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b689a0a0 0x5627b6c8fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.820+0000 7fdcb2a00640 1 -- 172.21.15.16:0/1641701075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b6c89a10 con 0x5627b68a2280 2024-09-21T03:46:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.821+0000 7fdcb2a00640 1 --2- 172.21.15.16:0/1641701075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b68a2280 0x5627b6c88f90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdcb406b4a0 tx=0x7fdca0010a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfc5074281e014f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.821+0000 7fdc9d600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:46:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.821+0000 7fdc9d600640 1 -- 172.21.15.16:0/1641701075 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca0005220 con 0x5627b68a2280 2024-09-21T03:46:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.821+0000 7fdcc10c2740 1 -- 172.21.15.16:0/1641701075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcac030720 con 0x5627b68a2280 2024-09-21T03:46:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.821+0000 7fdc9d600640 1 -- 172.21.15.16:0/1641701075 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fdca00053c0 con 0x5627b68a2280 2024-09-21T03:46:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.821+0000 7fdc9d600640 1 -- 172.21.15.16:0/1641701075 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdca00055c0 con 0x5627b68a2280 2024-09-21T03:46:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.821+0000 7fdcc10c2740 10 client.15381 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.821+0000 7fdcc10c2740 1 -- 172.21.15.16:0/1641701075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5627b6c8f460 con 0x5627b68a2280 2024-09-21T03:46:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.822+0000 7fdc9d600640 1 -- 172.21.15.16:0/1641701075 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcac030720 con 0x5627b68a2280 2024-09-21T03:46:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.822+0000 7fdc9d600640 1 -- 172.21.15.16:0/1641701075 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdca00047a0 con 0x5627b68a2280 2024-09-21T03:46:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.822+0000 7fdcc10c2740 10 client.15381 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.822+0000 7fdcc10c2740 10 client.15381 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:46:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.822+0000 7fdcc10c2740 20 client.15381 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.822+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/1641701075 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x5627b6856c90 0x5627b6cf5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.822+0000 7fdcc10c2740 4 client.15381 mds_command: new command op to 24689 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:46:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.822+0000 7fdcc10c2740 1 -- 172.21.15.16:0/1641701075 --> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] -- command(tid 0: {"prefix": "status"}) -- 0x5627b6bde750 con 0x5627b6856c90 2024-09-21T03:46:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.823+0000 7fdcb3e00640 1 --2- 172.21.15.16:0/1641701075 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x5627b6856c90 0x5627b6cf5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.823+0000 7fdcb3e00640 1 --2- 172.21.15.16:0/1641701075 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x5627b6856c90 0x5627b6cf5df0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2ea4a125ba449af server_cookie=b6ffa4e33748dd2a in_seq=0 out_seq=0 2024-09-21T03:46:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.823+0000 7fdc9d600640 10 client.15381 ms_handle_connect on v2:172.21.15.123:6836/2001424876 2024-09-21T03:46:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.823+0000 7fdc9d600640 1 -- 172.21.15.16:0/1641701075 <== mds.1 v2:172.21.15.123:6836/2001424876 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5627b6bde750 con 0x5627b6856c90 2024-09-21T03:46:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.823+0000 7fdc9d600640 10 client.15381 handle_command_reply: tid=0 2024-09-21T03:46:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.823+0000 7fdcc10c2740 1 client.15381 shutdown 2024-09-21T03:46:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.824+0000 7fdcc10c2740 1 -- 172.21.15.16:0/1641701075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b68a2280 msgr2=0x5627b6c88f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.824+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/1641701075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b68a2280 0x5627b6c88f90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdcb406b4a0 tx=0x7fdca0010a60 comp rx=0 tx=0).stop 2024-09-21T03:46:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.824+0000 7fdcc10c2740 1 -- 172.21.15.16:0/1641701075 shutdown_connections 2024-09-21T03:46:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.824+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/1641701075 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x5627b6856c90 0x5627b6cf5df0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.824+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/1641701075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627b68c2f60 0x5627b6c894d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.824+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/1641701075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627b68a2280 0x5627b6c88f90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.824+0000 7fdcc10c2740 1 --2- 172.21.15.16:0/1641701075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627b689a0a0 0x5627b6c8fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.824+0000 7fdcc10c2740 1 -- 172.21.15.16:0/1641701075 >> 172.21.15.16:0/1641701075 conn(0x5627b68b7350 msgr2=0x5627b6c66df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.824+0000 7fdc9d600640 0 client.15381 ms_handle_reset on v2:172.21.15.123:6836/2001424876 2024-09-21T03:46:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.824+0000 7fdcc10c2740 1 -- 172.21.15.16:0/1641701075 shutdown_connections 2024-09-21T03:46:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.824+0000 7fdcc10c2740 1 -- 172.21.15.16:0/1641701075 wait complete. 2024-09-21T03:46:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.824+0000 7fdcc10c2740 20 client.15381 trim_cache size 0 max 16384 2024-09-21T03:46:04.828 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:46:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:46:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-09-21T03:46:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24689, 2024-09-21T03:46:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T03:46:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T03:46:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T03:46:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 122.321379455, 2024-09-21T03:46:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 29, 2024-09-21T03:46:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 86, 2024-09-21T03:46:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 84, 2024-09-21T03:46:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 204.925147635 2024-09-21T03:46:04.829 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:46:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.825+0000 7fdcb8e00640 1 -- 172.21.15.16:0/1494355880 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdc980b6510 msgr2=0x7fdc980b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.825+0000 7fdcb8e00640 1 --2- 172.21.15.16:0/1494355880 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdc980b6510 0x7fdc980b8a20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdca40a9d30 tx=0x7fdca400ef90 comp rx=0 tx=0).stop 2024-09-21T03:46:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.825+0000 7fdcb8e00640 1 -- 172.21.15.16:0/1494355880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcb41047d0 msgr2=0x7fdcb4102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.825+0000 7fdcb8e00640 1 --2- 172.21.15.16:0/1494355880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcb41047d0 0x7fdcb4102510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdca400ea60 tx=0x7fdca400ef20 comp rx=0 tx=0).stop 2024-09-21T03:46:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.825+0000 7fdcb8e00640 1 -- 172.21.15.16:0/1494355880 shutdown_connections 2024-09-21T03:46:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.825+0000 7fdcb8e00640 1 --2- 172.21.15.16:0/1494355880 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdc980b6510 0x7fdc980b8a20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.825+0000 7fdcb8e00640 1 --2- 172.21.15.16:0/1494355880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcb410dc30 0x7fdcb4102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.825+0000 7fdcb8e00640 1 --2- 172.21.15.16:0/1494355880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcb4105120 0x7fdcb4102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.825+0000 7fdcb8e00640 1 --2- 172.21.15.16:0/1494355880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcb41047d0 0x7fdcb4102510 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.825+0000 7fdcb8e00640 1 -- 172.21.15.16:0/1494355880 >> 172.21.15.16:0/1494355880 conn(0x7fdcb40fe2d0 msgr2=0x7fdcb4107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.825+0000 7fdcb8e00640 1 -- 172.21.15.16:0/1494355880 shutdown_connections 2024-09-21T03:46:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:04.825+0000 7fdcb8e00640 1 -- 172.21.15.16:0/1494355880 wait complete. 2024-09-21T03:46:04.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:04 smithi134 ceph-mon[35031]: pgmap v411: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:46:04.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:04 smithi123 ceph-mon[34673]: pgmap v411: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:46:05.774 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:46:06.158 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.806+0000 7f766fe00640 1 -- 172.21.15.16:0/2736897988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7670077380 msgr2=0x7f76700757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.806+0000 7f766fe00640 1 --2- 172.21.15.16:0/2736897988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7670077380 0x7f76700757b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7660009900 tx=0x7f766002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.806+0000 7f766fe00640 1 -- 172.21.15.16:0/2736897988 shutdown_connections 2024-09-21T03:46:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.806+0000 7f766fe00640 1 --2- 172.21.15.16:0/2736897988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7670114260 0x7f7670116730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.806+0000 7f766fe00640 1 --2- 172.21.15.16:0/2736897988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7670075dd0 0x7f76700761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.806+0000 7f766fe00640 1 --2- 172.21.15.16:0/2736897988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7670077380 0x7f76700757b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.806+0000 7f766fe00640 1 -- 172.21.15.16:0/2736897988 >> 172.21.15.16:0/2736897988 conn(0x7f76700ff440 msgr2=0x7f7670101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.806+0000 7f766fe00640 1 -- 172.21.15.16:0/2736897988 shutdown_connections 2024-09-21T03:46:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.806+0000 7f766fe00640 1 -- 172.21.15.16:0/2736897988 wait complete. 2024-09-21T03:46:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.807+0000 7f766fe00640 1 Processor -- start 2024-09-21T03:46:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.808+0000 7f766fe00640 1 -- start start 2024-09-21T03:46:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.808+0000 7f766fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7670075dd0 0x7f76701a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.808+0000 7f766fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7670077380 0x7f76701a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.808+0000 7f766fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7670114260 0x7f76701a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.808+0000 7f766fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7670119ea0 con 0x7f7670114260 2024-09-21T03:46:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.808+0000 7f766fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7670119d20 con 0x7f7670077380 2024-09-21T03:46:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.808+0000 7f766fe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f767011a020 con 0x7f7670075dd0 2024-09-21T03:46:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.808+0000 7f766f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7670114260 0x7f76701a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.808+0000 7f766ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7670075dd0 0x7f76701a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.808+0000 7f7667e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7670077380 0x7f76701a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.808+0000 7f766ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7670075dd0 0x7f76701a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34674/0 (socket says 172.21.15.16:34674) 2024-09-21T03:46:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.808+0000 7f766ea00640 1 -- 172.21.15.16:0/3754748312 learned_addr learned my addr 172.21.15.16:0/3754748312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:06.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.809+0000 7f766ea00640 1 -- 172.21.15.16:0/3754748312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7670077380 msgr2=0x7f76701a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:06.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.809+0000 7f766ea00640 1 --2- 172.21.15.16:0/3754748312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7670077380 0x7f76701a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.809+0000 7f766ea00640 1 -- 172.21.15.16:0/3754748312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7670114260 msgr2=0x7f76701a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:06.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.809+0000 7f766ea00640 1 --2- 172.21.15.16:0/3754748312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7670114260 0x7f76701a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.809+0000 7f766ea00640 1 -- 172.21.15.16:0/3754748312 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76701a6820 con 0x7f7670075dd0 2024-09-21T03:46:06.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.809+0000 7f7667e00640 1 --2- 172.21.15.16:0/3754748312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7670077380 0x7f76701a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:06.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.809+0000 7f766ea00640 1 --2- 172.21.15.16:0/3754748312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7670075dd0 0x7f76701a08b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7660005940 tx=0x7f7660004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:06.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.810+0000 7f7674a00640 1 -- 172.21.15.16:0/3754748312 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766003f030 con 0x7f7670075dd0 2024-09-21T03:46:06.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.810+0000 7f7674a00640 1 -- 172.21.15.16:0/3754748312 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7660003aa0 con 0x7f7670075dd0 2024-09-21T03:46:06.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.810+0000 7f766fe00640 1 -- 172.21.15.16:0/3754748312 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76701a6ab0 con 0x7f7670075dd0 2024-09-21T03:46:06.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.810+0000 7f7674a00640 1 -- 172.21.15.16:0/3754748312 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660035070 con 0x7f7670075dd0 2024-09-21T03:46:06.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.810+0000 7f766fe00640 1 -- 172.21.15.16:0/3754748312 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76701ad270 con 0x7f7670075dd0 2024-09-21T03:46:06.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.814+0000 7f7674a00640 1 -- 172.21.15.16:0/3754748312 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f7660042030 con 0x7f7670075dd0 2024-09-21T03:46:06.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.814+0000 7f767cac2740 1 Processor -- start 2024-09-21T03:46:06.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.814+0000 7f767cac2740 1 -- start start 2024-09-21T03:46:06.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.814+0000 7f767cac2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585555f60 0x5635858fc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.814+0000 7f767cac2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56358552d0a0 0x5635858fcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.814+0000 7f767cac2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563585535280 0x5635858fd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.814+0000 7f767cac2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563585873d50 con 0x56358552d0a0 2024-09-21T03:46:06.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.814+0000 7f767cac2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563585863180 con 0x563585555f60 2024-09-21T03:46:06.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.814+0000 7f767cac2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563585866f90 con 0x563585535280 2024-09-21T03:46:06.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.815+0000 7f766ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56358552d0a0 0x5635858fcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.815+0000 7f766f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585555f60 0x5635858fc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.815+0000 7f766f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585555f60 0x5635858fc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:32876/0 (socket says 172.21.15.16:32876) 2024-09-21T03:46:06.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.815+0000 7f766f400640 1 -- 172.21.15.16:0/350246994 learned_addr learned my addr 172.21.15.16:0/350246994 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:06.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.815+0000 7f7667e00640 1 --2- 172.21.15.16:0/350246994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563585535280 0x5635858fd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.815+0000 7f766f400640 1 -- 172.21.15.16:0/350246994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563585535280 msgr2=0x5635858fd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:06.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.815+0000 7f766f400640 1 --2- 172.21.15.16:0/350246994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563585535280 0x5635858fd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.815+0000 7f766f400640 1 -- 172.21.15.16:0/350246994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56358552d0a0 msgr2=0x5635858fcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:06.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.815+0000 7f766f400640 1 --2- 172.21.15.16:0/350246994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56358552d0a0 0x5635858fcbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.815+0000 7f766f400640 1 -- 172.21.15.16:0/350246994 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56358582da70 con 0x563585555f60 2024-09-21T03:46:06.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.816+0000 7f766f400640 1 --2- 172.21.15.16:0/350246994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585555f60 0x5635858fc670 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7668010370 tx=0x7f7668010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98da7f817e6a8844 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:06.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.816+0000 7f7674a00640 1 --2- 172.21.15.16:0/3754748312 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f76540b6510 0x7f76540b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.816+0000 7f765fe00640 1 -- 172.21.15.16:0/350246994 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668016070 con 0x563585555f60 2024-09-21T03:46:06.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.816+0000 7f7674a00640 1 -- 172.21.15.16:0/3754748312 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f7660107cf0 con 0x7f7670075dd0 2024-09-21T03:46:06.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.816+0000 7f766ea00640 1 --2- 172.21.15.16:0/3754748312 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f76540b6510 0x7f76540b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.816+0000 7f765fe00640 1 -- 172.21.15.16:0/350246994 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7668004d20 con 0x563585555f60 2024-09-21T03:46:06.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.816+0000 7f765fe00640 1 -- 172.21.15.16:0/350246994 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668005390 con 0x563585555f60 2024-09-21T03:46:06.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.817+0000 7f767cac2740 1 -- 172.21.15.16:0/350246994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585555f60 msgr2=0x5635858fc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:06.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.817+0000 7f767cac2740 1 --2- 172.21.15.16:0/350246994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585555f60 0x5635858fc670 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7668010370 tx=0x7f7668010830 comp rx=0 tx=0).stop 2024-09-21T03:46:06.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.817+0000 7f766ea00640 1 --2- 172.21.15.16:0/3754748312 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f76540b6510 0x7f76540b8a20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f76600ca0f0 tx=0x7f766010b730 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:06.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.817+0000 7f767cac2740 1 -- 172.21.15.16:0/350246994 shutdown_connections 2024-09-21T03:46:06.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.817+0000 7f767cac2740 1 --2- 172.21.15.16:0/350246994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563585535280 0x5635858fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.817+0000 7f767cac2740 1 --2- 172.21.15.16:0/350246994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56358552d0a0 0x5635858fcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.817+0000 7f767cac2740 1 --2- 172.21.15.16:0/350246994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585555f60 0x5635858fc670 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.817+0000 7f767cac2740 1 -- 172.21.15.16:0/350246994 >> 172.21.15.16:0/350246994 conn(0x56358554a350 msgr2=0x5635858fa560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:06.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.817+0000 7f767cac2740 1 -- 172.21.15.16:0/350246994 shutdown_connections 2024-09-21T03:46:06.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.817+0000 7f767cac2740 1 -- 172.21.15.16:0/350246994 wait complete. 2024-09-21T03:46:06.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:06 smithi016 ceph-mon[30375]: pgmap v412: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:46:06.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.818+0000 7f767cac2740 1 -- start start 2024-09-21T03:46:06.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.818+0000 7f767cac2740 1 Processor -- start 2024-09-21T03:46:06.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.819+0000 7f767cac2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56358552d0a0 0x5635859082d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.819+0000 7f767cac2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563585535280 0x563585908850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.819+0000 7f767cac2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585555f60 0x563585914140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.819+0000 7f767cac2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5635856c45f0 con 0x56358552d0a0 2024-09-21T03:46:06.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.819+0000 7f767cac2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5635857fe460 con 0x563585555f60 2024-09-21T03:46:06.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.819+0000 7f767cac2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56358590ae50 con 0x563585535280 2024-09-21T03:46:06.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.819+0000 7f767cac2740 10 client.0 _ll_register_callbacks cb 0x56358580e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:06.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.819+0000 7f766f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56358552d0a0 0x5635859082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.820+0000 7f766f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56358552d0a0 0x5635859082d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53206/0 (socket says 172.21.15.16:53206) 2024-09-21T03:46:06.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.820+0000 7f766f400640 1 -- 172.21.15.16:0/2037455121 learned_addr learned my addr 172.21.15.16:0/2037455121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.820+0000 7f7667e00640 1 --2- 172.21.15.16:0/2037455121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563585535280 0x563585908850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.820+0000 7f766f400640 1 --2- 172.21.15.16:0/2037455121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585555f60 0x563585914140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.820+0000 7f766f400640 1 -- 172.21.15.16:0/2037455121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563585535280 msgr2=0x563585908850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.820+0000 7f766f400640 1 --2- 172.21.15.16:0/2037455121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563585535280 0x563585908850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.820+0000 7f766f400640 1 -- 172.21.15.16:0/2037455121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585555f60 msgr2=0x563585914140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.820+0000 7f766f400640 1 --2- 172.21.15.16:0/2037455121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585555f60 0x563585914140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.820+0000 7f766f400640 1 -- 172.21.15.16:0/2037455121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635855c1540 con 0x56358552d0a0 2024-09-21T03:46:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.820+0000 7f766f400640 1 --2- 172.21.15.16:0/2037455121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56358552d0a0 0x5635859082d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7668012eb0 tx=0x7f7668005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e0dd49043f0971 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:06.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.820+0000 7f765cc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:46:06.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.821+0000 7f765cc00640 1 -- 172.21.15.16:0/2037455121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668016070 con 0x56358552d0a0 2024-09-21T03:46:06.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.821+0000 7f765cc00640 1 -- 172.21.15.16:0/2037455121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f7668022040 con 0x56358552d0a0 2024-09-21T03:46:06.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.821+0000 7f767cac2740 1 -- 172.21.15.16:0/2037455121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563585800a70 con 0x56358552d0a0 2024-09-21T03:46:06.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.821+0000 7f765cc00640 1 -- 172.21.15.16:0/2037455121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7668012720 con 0x56358552d0a0 2024-09-21T03:46:06.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.821+0000 7f767cac2740 10 client.15393 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:06.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.821+0000 7f767cac2740 1 -- 172.21.15.16:0/2037455121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56358581b6f0 con 0x56358552d0a0 2024-09-21T03:46:06.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.822+0000 7f765cc00640 1 -- 172.21.15.16:0/2037455121 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563585800a70 con 0x56358552d0a0 2024-09-21T03:46:06.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.822+0000 7f765cc00640 1 -- 172.21.15.16:0/2037455121 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f766800f040 con 0x56358552d0a0 2024-09-21T03:46:06.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.822+0000 7f767cac2740 10 client.15393 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:06.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.822+0000 7f767cac2740 10 client.15393 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:46:06.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.822+0000 7f767cac2740 20 client.15393 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:06.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.822+0000 7f767cac2740 1 --2- 172.21.15.16:0/2037455121 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5635855148e0 0x56358591a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.822+0000 7f767cac2740 4 client.15393 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:06.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.822+0000 7f767cac2740 1 -- 172.21.15.16:0/2037455121 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563585875ca0 con 0x5635855148e0 2024-09-21T03:46:06.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.823+0000 7f7667e00640 1 --2- 172.21.15.16:0/2037455121 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5635855148e0 0x56358591a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.823+0000 7f7667e00640 1 --2- 172.21.15.16:0/2037455121 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5635855148e0 0x56358591a440 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d442cf638d3e90 server_cookie=9a4e86184953b668 in_seq=0 out_seq=0 2024-09-21T03:46:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.823+0000 7f765cc00640 10 client.15393 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.824+0000 7f765cc00640 1 -- 172.21.15.16:0/2037455121 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563585875ca0 con 0x5635855148e0 2024-09-21T03:46:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.824+0000 7f765cc00640 10 client.15393 handle_command_reply: tid=0 2024-09-21T03:46:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.824+0000 7f767cac2740 1 client.15393 shutdown 2024-09-21T03:46:06.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.824+0000 7f767cac2740 1 -- 172.21.15.16:0/2037455121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56358552d0a0 msgr2=0x5635859082d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:06.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.824+0000 7f767cac2740 1 --2- 172.21.15.16:0/2037455121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56358552d0a0 0x5635859082d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7668012eb0 tx=0x7f7668005c50 comp rx=0 tx=0).stop 2024-09-21T03:46:06.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.825+0000 7f767cac2740 1 -- 172.21.15.16:0/2037455121 shutdown_connections 2024-09-21T03:46:06.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.825+0000 7f767cac2740 1 --2- 172.21.15.16:0/2037455121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585555f60 0x563585914140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.825+0000 7f767cac2740 1 --2- 172.21.15.16:0/2037455121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563585535280 0x563585908850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.825+0000 7f767cac2740 1 --2- 172.21.15.16:0/2037455121 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5635855148e0 0x56358591a440 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.825+0000 7f767cac2740 1 --2- 172.21.15.16:0/2037455121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56358552d0a0 0x5635859082d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.825+0000 7f767cac2740 1 -- 172.21.15.16:0/2037455121 >> 172.21.15.16:0/2037455121 conn(0x56358554a350 msgr2=0x5635858ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:06.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.825+0000 7f765cc00640 0 client.15393 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:06.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.825+0000 7f767cac2740 1 -- 172.21.15.16:0/2037455121 shutdown_connections 2024-09-21T03:46:06.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.825+0000 7f767cac2740 1 -- 172.21.15.16:0/2037455121 wait complete. 2024-09-21T03:46:06.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.825+0000 7f767cac2740 20 client.15393 trim_cache size 0 max 16384 2024-09-21T03:46:06.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.845+0000 7f767cac2740 1 Processor -- start 2024-09-21T03:46:06.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.845+0000 7f767cac2740 1 -- start start 2024-09-21T03:46:06.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.846+0000 7f767cac2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56358552d0a0 0x5635859140b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.846+0000 7f767cac2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563585535280 0x563585906950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.846+0000 7f767cac2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563585555f60 0x563585906e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.846+0000 7f767cac2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563585918780 con 0x563585535280 2024-09-21T03:46:06.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.846+0000 7f767cac2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563585918e50 con 0x56358552d0a0 2024-09-21T03:46:06.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.846+0000 7f767cac2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563585919520 con 0x563585555f60 2024-09-21T03:46:06.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.846+0000 7f7667e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563585535280 0x563585906950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.846+0000 7f7667e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563585535280 0x563585906950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53210/0 (socket says 172.21.15.16:53210) 2024-09-21T03:46:06.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.846+0000 7f7667e00640 1 -- 172.21.15.16:0/3971699856 learned_addr learned my addr 172.21.15.16:0/3971699856 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:06.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.846+0000 7f766f400640 1 --2- 172.21.15.16:0/3971699856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56358552d0a0 0x5635859140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.846+0000 7f766f400640 1 --2- 172.21.15.16:0/3971699856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563585555f60 0x563585906e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.846+0000 7f7667e00640 1 -- 172.21.15.16:0/3971699856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563585555f60 msgr2=0x563585906e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:06.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.847+0000 7f7667e00640 1 --2- 172.21.15.16:0/3971699856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563585555f60 0x563585906e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.847+0000 7f7667e00640 1 -- 172.21.15.16:0/3971699856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56358552d0a0 msgr2=0x5635859140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:06.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.847+0000 7f7667e00640 1 --2- 172.21.15.16:0/3971699856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56358552d0a0 0x5635859140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.847+0000 7f7667e00640 1 -- 172.21.15.16:0/3971699856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563585909390 con 0x563585535280 2024-09-21T03:46:06.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.847+0000 7f766f400640 1 --2- 172.21.15.16:0/3971699856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56358552d0a0 0x5635859140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:06.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.847+0000 7f7667e00640 1 --2- 172.21.15.16:0/3971699856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563585535280 0x563585906950 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f76701163c0 tx=0x7f7658005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93bb84bfa94e4c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:06.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.847+0000 7f765f400640 1 -- 172.21.15.16:0/3971699856 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76580120d0 con 0x563585535280 2024-09-21T03:46:06.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.847+0000 7f765f400640 1 -- 172.21.15.16:0/3971699856 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7658012270 con 0x563585535280 2024-09-21T03:46:06.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.847+0000 7f765f400640 1 -- 172.21.15.16:0/3971699856 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76580047a0 con 0x563585535280 2024-09-21T03:46:06.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.848+0000 7f767cac2740 1 -- 172.21.15.16:0/3971699856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563585535280 msgr2=0x563585906950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:06.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.848+0000 7f767cac2740 1 --2- 172.21.15.16:0/3971699856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563585535280 0x563585906950 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f76701163c0 tx=0x7f7658005f90 comp rx=0 tx=0).stop 2024-09-21T03:46:06.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.848+0000 7f767cac2740 1 -- 172.21.15.16:0/3971699856 shutdown_connections 2024-09-21T03:46:06.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.848+0000 7f767cac2740 1 --2- 172.21.15.16:0/3971699856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563585555f60 0x563585906e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.848+0000 7f767cac2740 1 --2- 172.21.15.16:0/3971699856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563585535280 0x563585906950 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.848+0000 7f767cac2740 1 --2- 172.21.15.16:0/3971699856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56358552d0a0 0x5635859140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.848+0000 7f767cac2740 1 -- 172.21.15.16:0/3971699856 >> 172.21.15.16:0/3971699856 conn(0x56358554a350 msgr2=0x5635859082b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:06.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.848+0000 7f767cac2740 1 -- 172.21.15.16:0/3971699856 shutdown_connections 2024-09-21T03:46:06.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.848+0000 7f767cac2740 1 -- 172.21.15.16:0/3971699856 wait complete. 2024-09-21T03:46:06.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.849+0000 7f767cac2740 1 -- start start 2024-09-21T03:46:06.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.849+0000 7f767cac2740 1 Processor -- start 2024-09-21T03:46:06.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:06 smithi134 ceph-mon[35031]: pgmap v412: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:46:06.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.849+0000 7f767cac2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56358552d0a0 0x563585922d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.849+0000 7f767cac2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585535280 0x56358591c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.849+0000 7f767cac2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563585555f60 0x56358591c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.849+0000 7f767cac2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56358591a2c0 con 0x563585555f60 2024-09-21T03:46:06.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.849+0000 7f767cac2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563585919bf0 con 0x563585535280 2024-09-21T03:46:06.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.849+0000 7f767cac2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56358591a990 con 0x56358552d0a0 2024-09-21T03:46:06.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.849+0000 7f767cac2740 10 client.0 _ll_register_callbacks cb 0x7f7668009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:06.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.850+0000 7f766f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56358552d0a0 0x563585922d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.850+0000 7f766f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563585555f60 0x56358591c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.850+0000 7f766f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563585555f60 0x56358591c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53222/0 (socket says 172.21.15.16:53222) 2024-09-21T03:46:06.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.850+0000 7f766f400640 1 -- 172.21.15.16:0/140896011 learned_addr learned my addr 172.21.15.16:0/140896011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:06.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.850+0000 7f7667e00640 1 --2- 172.21.15.16:0/140896011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585535280 0x56358591c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.850+0000 7f766f400640 1 -- 172.21.15.16:0/140896011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56358552d0a0 msgr2=0x563585922d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:06.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.850+0000 7f766f400640 1 --2- 172.21.15.16:0/140896011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56358552d0a0 0x563585922d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.850+0000 7f766f400640 1 -- 172.21.15.16:0/140896011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585535280 msgr2=0x56358591c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:06.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.850+0000 7f766f400640 1 --2- 172.21.15.16:0/140896011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585535280 0x56358591c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.850+0000 7f766f400640 1 -- 172.21.15.16:0/140896011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56358591cd50 con 0x563585555f60 2024-09-21T03:46:06.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.850+0000 7f7667e00640 1 --2- 172.21.15.16:0/140896011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585535280 0x56358591c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:06.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.851+0000 7f766f400640 1 --2- 172.21.15.16:0/140896011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563585555f60 0x56358591c740 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7668005cc0 tx=0x7f7668010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75844c3eab79a09 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:06.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.851+0000 7f765cc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:46:06.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.851+0000 7f765cc00640 1 -- 172.21.15.16:0/140896011 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668016040 con 0x563585555f60 2024-09-21T03:46:06.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.851+0000 7f767cac2740 1 -- 172.21.15.16:0/140896011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7658012270 con 0x563585555f60 2024-09-21T03:46:06.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.851+0000 7f765cc00640 1 -- 172.21.15.16:0/140896011 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f7668035040 con 0x563585555f60 2024-09-21T03:46:06.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.851+0000 7f765cc00640 1 -- 172.21.15.16:0/140896011 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7668003970 con 0x563585555f60 2024-09-21T03:46:06.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.851+0000 7f767cac2740 10 client.15399 fetch_fsmap learned FSMap version 29 2024-09-21T03:46:06.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.851+0000 7f767cac2740 1 -- 172.21.15.16:0/140896011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563585922760 con 0x563585555f60 2024-09-21T03:46:06.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.852+0000 7f765cc00640 1 -- 172.21.15.16:0/140896011 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7658012270 con 0x563585555f60 2024-09-21T03:46:06.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.852+0000 7f765cc00640 1 -- 172.21.15.16:0/140896011 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f7668004540 con 0x563585555f60 2024-09-21T03:46:06.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.852+0000 7f767cac2740 10 client.15399 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:46:06.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.852+0000 7f767cac2740 10 client.15399 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:46:06.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.852+0000 7f767cac2740 20 client.15399 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:06.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.852+0000 7f767cac2740 1 --2- 172.21.15.16:0/140896011 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5635854e9c90 0x563585988fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:06.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.852+0000 7f767cac2740 4 client.15399 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:46:06.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.852+0000 7f767cac2740 1 -- 172.21.15.16:0/140896011 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "status"}) -- 0x563585871750 con 0x5635854e9c90 2024-09-21T03:46:06.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.853+0000 7f7667e00640 1 --2- 172.21.15.16:0/140896011 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5635854e9c90 0x563585988fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:06.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.853+0000 7f7667e00640 1 --2- 172.21.15.16:0/140896011 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5635854e9c90 0x563585988fb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88a3ee036231729 server_cookie=eb4fc4b856bd0a57 in_seq=0 out_seq=0 2024-09-21T03:46:06.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.853+0000 7f765cc00640 10 client.15399 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:06.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.853+0000 7f765cc00640 1 -- 172.21.15.16:0/140896011 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563585871750 con 0x5635854e9c90 2024-09-21T03:46:06.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.853+0000 7f765cc00640 10 client.15399 handle_command_reply: tid=0 2024-09-21T03:46:06.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.853+0000 7f767cac2740 1 client.15399 shutdown 2024-09-21T03:46:06.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.854+0000 7f767cac2740 1 -- 172.21.15.16:0/140896011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563585555f60 msgr2=0x56358591c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:06.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.854+0000 7f767cac2740 1 --2- 172.21.15.16:0/140896011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563585555f60 0x56358591c740 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7668005cc0 tx=0x7f7668010830 comp rx=0 tx=0).stop 2024-09-21T03:46:06.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.854+0000 7f767cac2740 1 -- 172.21.15.16:0/140896011 shutdown_connections 2024-09-21T03:46:06.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.854+0000 7f767cac2740 1 --2- 172.21.15.16:0/140896011 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5635854e9c90 0x563585988fb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.854+0000 7f767cac2740 1 --2- 172.21.15.16:0/140896011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563585555f60 0x56358591c740 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.854+0000 7f767cac2740 1 --2- 172.21.15.16:0/140896011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563585535280 0x56358591c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.854+0000 7f767cac2740 1 --2- 172.21.15.16:0/140896011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56358552d0a0 0x563585922d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.854+0000 7f767cac2740 1 -- 172.21.15.16:0/140896011 >> 172.21.15.16:0/140896011 conn(0x56358554a350 msgr2=0x5635858f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:06.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.854+0000 7f765cc00640 0 client.15399 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:06.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.854+0000 7f767cac2740 1 -- 172.21.15.16:0/140896011 shutdown_connections 2024-09-21T03:46:06.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.854+0000 7f767cac2740 1 -- 172.21.15.16:0/140896011 wait complete. 2024-09-21T03:46:06.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.854+0000 7f767cac2740 20 client.15399 trim_cache size 0 max 16384 2024-09-21T03:46:06.859 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:46:06.859 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:46:06.859 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-21T03:46:06.859 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24731, 2024-09-21T03:46:06.859 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T03:46:06.859 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T03:46:06.859 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T03:46:06.859 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 176.847345241, 2024-09-21T03:46:06.859 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 29, 2024-09-21T03:46:06.859 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 86, 2024-09-21T03:46:06.859 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 84, 2024-09-21T03:46:06.859 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 193.24991181999999 2024-09-21T03:46:06.859 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:46:06.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.855+0000 7f766fe00640 1 -- 172.21.15.16:0/3754748312 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f76540b6510 msgr2=0x7f76540b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:06.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.855+0000 7f766fe00640 1 --2- 172.21.15.16:0/3754748312 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f76540b6510 0x7f76540b8a20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f76600ca0f0 tx=0x7f766010b730 comp rx=0 tx=0).stop 2024-09-21T03:46:06.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.855+0000 7f766fe00640 1 -- 172.21.15.16:0/3754748312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7670075dd0 msgr2=0x7f76701a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:06.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.855+0000 7f766fe00640 1 --2- 172.21.15.16:0/3754748312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7670075dd0 0x7f76701a08b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7660005940 tx=0x7f7660004270 comp rx=0 tx=0).stop 2024-09-21T03:46:06.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.855+0000 7f766fe00640 1 -- 172.21.15.16:0/3754748312 shutdown_connections 2024-09-21T03:46:06.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.855+0000 7f766fe00640 1 --2- 172.21.15.16:0/3754748312 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f76540b6510 0x7f76540b8a20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.855+0000 7f766fe00640 1 --2- 172.21.15.16:0/3754748312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7670114260 0x7f76701a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.855+0000 7f766fe00640 1 --2- 172.21.15.16:0/3754748312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7670077380 0x7f76701a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.855+0000 7f766fe00640 1 --2- 172.21.15.16:0/3754748312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7670075dd0 0x7f76701a08b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:06.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.855+0000 7f766fe00640 1 -- 172.21.15.16:0/3754748312 >> 172.21.15.16:0/3754748312 conn(0x7f76700ff440 msgr2=0x7f7670110160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:06.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.855+0000 7f766fe00640 1 -- 172.21.15.16:0/3754748312 shutdown_connections 2024-09-21T03:46:06.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:06.855+0000 7f766fe00640 1 -- 172.21.15.16:0/3754748312 wait complete. 2024-09-21T03:46:06.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:06 smithi123 ceph-mon[34673]: pgmap v412: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:46:07.767 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T03:46:07.786 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T03:46:07.787 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-21T03:46:07.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:07.787 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-21T03:46:07.824 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-21T03:46:07.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:07.825 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:46:07.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:07.922 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-21T03:46:08.250 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-21T03:46:08.250 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T03:46:08.250 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:46:08.250 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T03:46:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:08.769 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:08 smithi016 ceph-mon[30375]: pgmap v413: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:46:08.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:08 smithi134 ceph-mon[35031]: pgmap v413: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:46:08.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:08 smithi123 ceph-mon[34673]: pgmap v413: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:46:09.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.269+0000 7f8664000640 1 -- 172.21.15.16:0/1555550434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c103b60 msgr2=0x7f865c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.269+0000 7f8664000640 1 --2- 172.21.15.16:0/1555550434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c103b60 0x7f865c103f70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8650009900 tx=0x7f865002a410 comp rx=0 tx=0).stop 2024-09-21T03:46:09.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.270+0000 7f8664000640 1 -- 172.21.15.16:0/1555550434 shutdown_connections 2024-09-21T03:46:09.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.270+0000 7f8664000640 1 --2- 172.21.15.16:0/1555550434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f865c109df0 0x7f865c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.270+0000 7f8664000640 1 --2- 172.21.15.16:0/1555550434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f865c1094a0 0x7f865c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.270+0000 7f8664000640 1 --2- 172.21.15.16:0/1555550434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c103b60 0x7f865c103f70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.270+0000 7f8664000640 1 -- 172.21.15.16:0/1555550434 >> 172.21.15.16:0/1555550434 conn(0x7f865c07b850 msgr2=0x7f865c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:09.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.270+0000 7f8664000640 1 -- 172.21.15.16:0/1555550434 shutdown_connections 2024-09-21T03:46:09.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.270+0000 7f8664000640 1 -- 172.21.15.16:0/1555550434 wait complete. 2024-09-21T03:46:09.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.271+0000 7f8664000640 1 Processor -- start 2024-09-21T03:46:09.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.271+0000 7f8664000640 1 -- start start 2024-09-21T03:46:09.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.272+0000 7f8664000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f865c103b60 0x7f865c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:09.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.272+0000 7f8664000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f865c1094a0 0x7f865c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:09.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.272+0000 7f8664000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c109df0 0x7f865c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:09.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.272+0000 7f8664000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f865c113b00 con 0x7f865c109df0 2024-09-21T03:46:09.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.272+0000 7f8664000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f865c113980 con 0x7f865c1094a0 2024-09-21T03:46:09.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.272+0000 7f8664000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f865c113c80 con 0x7f865c103b60 2024-09-21T03:46:09.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.272+0000 7f8661600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f865c103b60 0x7f865c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:09.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.272+0000 7f8661600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f865c103b60 0x7f865c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34726/0 (socket says 172.21.15.16:34726) 2024-09-21T03:46:09.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.272+0000 7f8661600640 1 -- 172.21.15.16:0/681054148 learned_addr learned my addr 172.21.15.16:0/681054148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:09.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.272+0000 7f8662000640 1 --2- 172.21.15.16:0/681054148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c109df0 0x7f865c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:09.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.272+0000 7f8660c00640 1 --2- 172.21.15.16:0/681054148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f865c1094a0 0x7f865c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:09.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.273+0000 7f8661600640 1 -- 172.21.15.16:0/681054148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f865c1094a0 msgr2=0x7f865c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.273+0000 7f8661600640 1 --2- 172.21.15.16:0/681054148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f865c1094a0 0x7f865c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.273+0000 7f8661600640 1 -- 172.21.15.16:0/681054148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c109df0 msgr2=0x7f865c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.273+0000 7f8661600640 1 --2- 172.21.15.16:0/681054148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c109df0 0x7f865c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.273+0000 7f8661600640 1 -- 172.21.15.16:0/681054148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c1a8810 con 0x7f865c103b60 2024-09-21T03:46:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.273+0000 7f8661600640 1 --2- 172.21.15.16:0/681054148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f865c103b60 0x7f865c075710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8650005ab0 tx=0x7f8650003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.273+0000 7f865a000640 1 -- 172.21.15.16:0/681054148 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865003f070 con 0x7f865c103b60 2024-09-21T03:46:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.273+0000 7f8664000640 1 -- 172.21.15.16:0/681054148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865c1a8aa0 con 0x7f865c103b60 2024-09-21T03:46:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.273+0000 7f865a000640 1 -- 172.21.15.16:0/681054148 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8650031b60 con 0x7f865c103b60 2024-09-21T03:46:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.273+0000 7f865a000640 1 -- 172.21.15.16:0/681054148 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650035030 con 0x7f865c103b60 2024-09-21T03:46:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.273+0000 7f8664000640 1 -- 172.21.15.16:0/681054148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865c1a8fd0 con 0x7f865c103b60 2024-09-21T03:46:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.275+0000 7f8664000640 1 -- 172.21.15.16:0/681054148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f865c103f70 con 0x7f865c103b60 2024-09-21T03:46:09.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.277+0000 7f865a000640 1 -- 172.21.15.16:0/681054148 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8650042070 con 0x7f865c103b60 2024-09-21T03:46:09.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.279+0000 7f865a000640 1 --2- 172.21.15.16:0/681054148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f86440b6510 0x7f86440b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:09.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.279+0000 7f865a000640 1 -- 172.21.15.16:0/681054148 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f8650108450 con 0x7f865c103b60 2024-09-21T03:46:09.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.279+0000 7f8660c00640 1 --2- 172.21.15.16:0/681054148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f86440b6510 0x7f86440b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:09.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.280+0000 7f8660c00640 1 --2- 172.21.15.16:0/681054148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f86440b6510 0x7f86440b8a20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f865c1091d0 tx=0x7f8648005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:09.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.281+0000 7f865a000640 1 -- 172.21.15.16:0/681054148 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f86500d2cb0 con 0x7f865c103b60 2024-09-21T03:46:09.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.523+0000 7f8664000640 1 -- 172.21.15.16:0/681054148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f865c076a10 con 0x7f865c103b60 2024-09-21T03:46:09.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.524+0000 7f865a000640 1 -- 172.21.15.16:0/681054148 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7f865c076a10 con 0x7f865c103b60 2024-09-21T03:46:09.526 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T03:46:09.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.527+0000 7f8664000640 1 -- 172.21.15.16:0/681054148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f86440b6510 msgr2=0x7f86440b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.527+0000 7f8664000640 1 --2- 172.21.15.16:0/681054148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f86440b6510 0x7f86440b8a20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f865c1091d0 tx=0x7f8648005f90 comp rx=0 tx=0).stop 2024-09-21T03:46:09.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.527+0000 7f8664000640 1 -- 172.21.15.16:0/681054148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f865c103b60 msgr2=0x7f865c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:09.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.527+0000 7f8664000640 1 --2- 172.21.15.16:0/681054148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f865c103b60 0x7f865c075710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8650005ab0 tx=0x7f8650003c10 comp rx=0 tx=0).stop 2024-09-21T03:46:09.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.527+0000 7f8664000640 1 -- 172.21.15.16:0/681054148 shutdown_connections 2024-09-21T03:46:09.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.527+0000 7f8664000640 1 --2- 172.21.15.16:0/681054148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f86440b6510 0x7f86440b8a20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.527+0000 7f8664000640 1 --2- 172.21.15.16:0/681054148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c109df0 0x7f865c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.527+0000 7f8664000640 1 --2- 172.21.15.16:0/681054148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f865c1094a0 0x7f865c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.527+0000 7f8664000640 1 --2- 172.21.15.16:0/681054148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f865c103b60 0x7f865c075710 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:09.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.527+0000 7f8664000640 1 -- 172.21.15.16:0/681054148 >> 172.21.15.16:0/681054148 conn(0x7f865c07b850 msgr2=0x7f865c107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:09.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.527+0000 7f8664000640 1 -- 172.21.15.16:0/681054148 shutdown_connections 2024-09-21T03:46:09.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:09.527+0000 7f8664000640 1 -- 172.21.15.16:0/681054148 wait complete. 2024-09-21T03:46:09.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:09 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/681054148' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:46:09.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:09 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/681054148' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:46:09.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:09 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/681054148' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:46:10.379 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T03:46:10.379 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:46:10.379 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-21T03:46:10.379 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:46:10.379 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:46:10.379 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T03:46:10.379 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T03:46:10.380 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:46:10.380 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:46:10.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:10.380 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-21T03:46:10.417 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:46:10.417 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:46:10.417 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:46:10.418 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:46:10.418 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-21T03:46:10.418 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:46:10.418 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:46:10.418 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:46:10.418 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-21T03:46:10.418 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:46:10.418 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:46:10.418 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-21T03:46:10.418 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:46:10.418 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:46:10.418 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-21T03:46:10.418 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:46:10.418 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41973sec preferred_lft 41973sec 2024-09-21T03:46:10.419 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-21T03:46:10.419 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:46:10.419 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:46:10.419 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:46:10.419 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-21T03:46:10.419 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T03:46:10.420 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:46:10.420 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-21T03:46:10.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-09-21T03:46:10.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-09-21T03:46:10.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-09-21T03:46:10.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T03:46:10.420 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-21T03:46:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:46:10.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:10.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:10.648 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:46:10.703 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-21T03:46:10.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:10.706 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-21T03:46:10.742 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-21T03:46:10.742 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:46:10.743 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:46:10.743 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:46:10.743 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:46:10.743 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:46:10.743 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-21T03:46:10.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:46:10.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:46:10.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:46:10.743 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-21T03:46:10.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:10 smithi134 ceph-mon[35031]: pgmap v414: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:46:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:46:10.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:10 smithi016 ceph-mon[30375]: pgmap v414: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:46:10.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:10 smithi123 ceph-mon[34673]: pgmap v414: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T03:46:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:10.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:10.943 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-21T03:46:10.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:10.979 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-21T03:46:11.055 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:46:11.055 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-21T03:46:11.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:46:11.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T03:46:11.055 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-21T03:46:11.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:46:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:11.236 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T03:46:11.236 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:46:11.237 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-21T03:46:11.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T03:46:11.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T03:46:11.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T03:46:11.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T03:46:11.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T03:46:11.237 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-21T03:46:11.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:46:11.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:11.465 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:46:11.465 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-21T03:46:11.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-21T03:46:11.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T03:46:11.465 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-21T03:46:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:46:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:11.638 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:46:11.638 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:46:11.674 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:46:11.674 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:46:11.674 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:46:11.753 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-21T03:46:11.753 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-21T03:46:11.753 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/bc0903f4-b2b4-4deb-8269-0a424bf4f740 2024-09-21T03:46:11.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:11.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/bc0903f4-b2b4-4deb-8269-0a424bf4f740 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-21T03:46:11.953 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-21T03:46:11.953 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-09-21T03:46:11.953 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-21T03:46:12.582 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-21T03:46:12.582 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-09-21T03:46:12.582 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-21T03:46:12.582 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.123:6789,172.21.15.134:6789" 2024-09-21T03:46:12.583 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.123:6789,172.21.15.134:6789:/volumes/qa/sv_1/bc0903f4-b2b4-4deb-8269-0a424bf4f740 2024-09-21T03:46:12.583 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=2b324840-77ca-11ef-baf5-efdc52797490" will pass to kernel 2024-09-21T03:46:12.584 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:46:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:46:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:46:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:46:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:46:12.588 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:46:12.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:46:12.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:46:12.653 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:46:12.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:46:12.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:12.746 INFO:teuthology.orchestra.run.smithi016.stdout:client25409 2024-09-21T03:46:12.746 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:46:12.746 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:46:12.783 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:46:12.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:12 smithi134 ceph-mon[35031]: pgmap v415: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:46:12.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:46:12.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:12.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:12 smithi123 ceph-mon[34673]: pgmap v415: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:46:12.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:12 smithi016 ceph-mon[30375]: pgmap v415: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:46:12.914 INFO:teuthology.orchestra.run.smithi016.stdout:client25409 2024-09-21T03:46:12.914 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:46:12.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:46:12.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:13.004 INFO:teuthology.orchestra.run.smithi016.stdout:client25409 2024-09-21T03:46:13.005 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/2b324840-77ca-11ef-baf5-efdc52797490.client25409/status') 2024-09-21T03:46:13.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:46:13.105 INFO:teuthology.orchestra.run.smithi016.stdout:instance: client.25409 (3)172.21.15.16:0/2073732573 2024-09-21T03:46:13.105 INFO:teuthology.orchestra.run.smithi016.stdout:blocklisted: false 2024-09-21T03:46:13.106 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records in 2024-09-21T03:46:13.106 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records out 2024-09-21T03:46:13.106 INFO:teuthology.orchestra.run.smithi016.stderr:71 bytes copied, 7.2507e-05 s, 979 kB/s 2024-09-21T03:46:13.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:46:13.116 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T03:46:13.132 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T03:46:13.132 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-21T03:46:13.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-21T03:46:13.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.352+0000 7fce65400640 1 -- 172.21.15.16:0/3483229441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce601034f0 msgr2=0x7fce60103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:13.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.352+0000 7fce65400640 1 --2- 172.21.15.16:0/3483229441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce601034f0 0x7fce60103960 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fce50009900 tx=0x7fce5002a410 comp rx=0 tx=0).stop 2024-09-21T03:46:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.352+0000 7fce65400640 1 -- 172.21.15.16:0/3483229441 shutdown_connections 2024-09-21T03:46:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.352+0000 7fce65400640 1 --2- 172.21.15.16:0/3483229441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce60104f10 0x7fce6010b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.352+0000 7fce65400640 1 --2- 172.21.15.16:0/3483229441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce60104570 0x7fce60104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.352+0000 7fce65400640 1 --2- 172.21.15.16:0/3483229441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce601034f0 0x7fce60103960 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.352+0000 7fce65400640 1 -- 172.21.15.16:0/3483229441 >> 172.21.15.16:0/3483229441 conn(0x7fce600fee30 msgr2=0x7fce601012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.352+0000 7fce65400640 1 -- 172.21.15.16:0/3483229441 shutdown_connections 2024-09-21T03:46:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.352+0000 7fce65400640 1 -- 172.21.15.16:0/3483229441 wait complete. 2024-09-21T03:46:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.353+0000 7fce65400640 1 Processor -- start 2024-09-21T03:46:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.354+0000 7fce65400640 1 -- start start 2024-09-21T03:46:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.354+0000 7fce65400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce601034f0 0x7fce6019bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.354+0000 7fce65400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce60104570 0x7fce6019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.354+0000 7fce65400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce60104f10 0x7fce6019d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.354+0000 7fce65400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce6010f110 con 0x7fce60104f10 2024-09-21T03:46:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.354+0000 7fce65400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fce6010ef90 con 0x7fce601034f0 2024-09-21T03:46:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.354+0000 7fce65400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce6010f290 con 0x7fce60104570 2024-09-21T03:46:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.354+0000 7fce57e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce60104570 0x7fce6019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.354+0000 7fce57e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce60104570 0x7fce6019c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34656/0 (socket says 172.21.15.16:34656) 2024-09-21T03:46:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.354+0000 7fce57e00640 1 -- 172.21.15.16:0/2352703979 learned_addr learned my addr 172.21.15.16:0/2352703979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.354+0000 7fce64a00640 1 --2- 172.21.15.16:0/2352703979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce60104f10 0x7fce6019d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.354+0000 7fce5fe00640 1 --2- 172.21.15.16:0/2352703979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce601034f0 0x7fce6019bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.355+0000 7fce57e00640 1 -- 172.21.15.16:0/2352703979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce601034f0 msgr2=0x7fce6019bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.355+0000 7fce57e00640 1 --2- 172.21.15.16:0/2352703979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce601034f0 0x7fce6019bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.355+0000 7fce57e00640 1 -- 172.21.15.16:0/2352703979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce60104f10 msgr2=0x7fce6019d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.355+0000 7fce57e00640 1 --2- 172.21.15.16:0/2352703979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce60104f10 0x7fce6019d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.355+0000 7fce57e00640 1 -- 172.21.15.16:0/2352703979 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6019da30 con 0x7fce60104570 2024-09-21T03:46:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.355+0000 7fce57e00640 1 --2- 172.21.15.16:0/2352703979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce60104570 0x7fce6019c660 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fce4800e9f0 tx=0x7fce4800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.355+0000 7fce5d600640 1 -- 172.21.15.16:0/2352703979 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4800cd80 con 0x7fce60104570 2024-09-21T03:46:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.355+0000 7fce5d600640 1 -- 172.21.15.16:0/2352703979 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce480108e0 con 0x7fce60104570 2024-09-21T03:46:13.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.355+0000 7fce5d600640 1 -- 172.21.15.16:0/2352703979 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48018980 con 0x7fce60104570 2024-09-21T03:46:13.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.355+0000 7fce65400640 1 -- 172.21.15.16:0/2352703979 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6019dd20 con 0x7fce60104570 2024-09-21T03:46:13.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.355+0000 7fce65400640 1 -- 172.21.15.16:0/2352703979 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce601a4410 con 0x7fce60104570 2024-09-21T03:46:13.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.357+0000 7fce65400640 1 -- 172.21.15.16:0/2352703979 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce601a45e0 con 0x7fce60104570 2024-09-21T03:46:13.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.359+0000 7fce5d600640 1 -- 172.21.15.16:0/2352703979 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fce48010430 con 0x7fce60104570 2024-09-21T03:46:13.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.361+0000 7fce5d600640 1 --2- 172.21.15.16:0/2352703979 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fce440b6590 0x7fce440b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:13.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.361+0000 7fce5d600640 1 -- 172.21.15.16:0/2352703979 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fce480e7e20 con 0x7fce60104570 2024-09-21T03:46:13.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.361+0000 7fce5fe00640 1 --2- 172.21.15.16:0/2352703979 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fce440b6590 0x7fce440b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:13.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.362+0000 7fce5fe00640 1 --2- 172.21.15.16:0/2352703979 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fce440b6590 0x7fce440b8aa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fce50000c00 tx=0x7fce50031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:13.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.363+0000 7fce5d600640 1 -- 172.21.15.16:0/2352703979 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fce480b2680 con 0x7fce60104570 2024-09-21T03:46:13.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:13.604+0000 7fce65400640 1 -- 172.21.15.16:0/2352703979 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fce601a47b0 con 0x7fce60104570 2024-09-21T03:46:14.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:14.592+0000 7fce5d600640 1 -- 172.21.15.16:0/2352703979 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v30) ==== 117+0+0 (secure 0 0 0) 0x7fce480b75c0 con 0x7fce60104570 2024-09-21T03:46:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:14.595+0000 7fce65400640 1 -- 172.21.15.16:0/2352703979 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fce440b6590 msgr2=0x7fce440b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:14.595+0000 7fce65400640 1 --2- 172.21.15.16:0/2352703979 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fce440b6590 0x7fce440b8aa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fce50000c00 tx=0x7fce50031400 comp rx=0 tx=0).stop 2024-09-21T03:46:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:14.595+0000 7fce65400640 1 -- 172.21.15.16:0/2352703979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce60104570 msgr2=0x7fce6019c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:14.595+0000 7fce65400640 1 --2- 172.21.15.16:0/2352703979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce60104570 0x7fce6019c660 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fce4800e9f0 tx=0x7fce4800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:14.595+0000 7fce65400640 1 -- 172.21.15.16:0/2352703979 shutdown_connections 2024-09-21T03:46:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:14.595+0000 7fce65400640 1 --2- 172.21.15.16:0/2352703979 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fce440b6590 0x7fce440b8aa0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:14.595+0000 7fce65400640 1 --2- 172.21.15.16:0/2352703979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce60104f10 0x7fce6019d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:14.595+0000 7fce65400640 1 --2- 172.21.15.16:0/2352703979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce60104570 0x7fce6019c660 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:14.595+0000 7fce65400640 1 --2- 172.21.15.16:0/2352703979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce601034f0 0x7fce6019bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:14.595+0000 7fce65400640 1 -- 172.21.15.16:0/2352703979 >> 172.21.15.16:0/2352703979 conn(0x7fce600fee30 msgr2=0x7fce600755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:14.595+0000 7fce65400640 1 -- 172.21.15.16:0/2352703979 shutdown_connections 2024-09-21T03:46:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:14.595+0000 7fce65400640 1 -- 172.21.15.16:0/2352703979 wait complete. 2024-09-21T03:46:14.620 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T03:46:14.637 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-21T03:46:14.770 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-21T03:46:14.771 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-21T03:46:14.775 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-21T03:46:14.829 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T03:46:14.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:46:14.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:14 smithi134 ceph-mon[35031]: pgmap v416: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:46:14.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:14 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2352703979' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T03:46:14.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:14 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T03:46:14.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:14 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:46:14.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:14 smithi016 ceph-mon[30375]: pgmap v416: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:46:14.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:14 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2352703979' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T03:46:14.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:14 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T03:46:14.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:14 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:46:14.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:14 smithi123 ceph-mon[34673]: pgmap v416: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:46:14.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:14 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2352703979' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T03:46:14.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:14 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T03:46:14.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:14 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:46:15.222 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:15.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:15 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T03:46:15.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:15 smithi016 ceph-mon[30375]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:46:15.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:15 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:46:15.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:15 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:46:15.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:15 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:46:15.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:15 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:46:15.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:15 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:46:15.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:15 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:46:15.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:15 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:46:15.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:15 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T03:46:15.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:15 smithi134 ceph-mon[35031]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:46:15.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:15 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:46:15.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:15 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:46:15.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:15 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:46:15.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:15 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:46:15.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:15 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:46:15.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:15 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:46:15.853 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:15 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:46:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.874+0000 7fea31200640 1 -- 172.21.15.16:0/992973740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea2c1092d0 msgr2=0x7fea2c1096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.874+0000 7fea31200640 1 --2- 172.21.15.16:0/992973740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea2c1092d0 0x7fea2c1096e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fea14009860 tx=0x7fea1402a340 comp rx=0 tx=0).stop 2024-09-21T03:46:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.874+0000 7fea31200640 1 -- 172.21.15.16:0/992973740 shutdown_connections 2024-09-21T03:46:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.874+0000 7fea31200640 1 --2- 172.21.15.16:0/992973740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c109c20 0x7fea2c110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:15.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.874+0000 7fea31200640 1 --2- 172.21.15.16:0/992973740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea2c1092d0 0x7fea2c1096e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:15.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.874+0000 7fea31200640 1 --2- 172.21.15.16:0/992973740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea2c103990 0x7fea2c103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:15.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.874+0000 7fea31200640 1 -- 172.21.15.16:0/992973740 >> 172.21.15.16:0/992973740 conn(0x7fea2c0ff3e0 msgr2=0x7fea2c101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:15.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.874+0000 7fea31200640 1 -- 172.21.15.16:0/992973740 shutdown_connections 2024-09-21T03:46:15.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.874+0000 7fea31200640 1 -- 172.21.15.16:0/992973740 wait complete. 2024-09-21T03:46:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.875+0000 7fea31200640 1 Processor -- start 2024-09-21T03:46:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.875+0000 7fea31200640 1 -- start start 2024-09-21T03:46:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.876+0000 7fea31200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c103990 0x7fea2c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.876+0000 7fea31200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea2c1092d0 0x7fea2c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.876+0000 7fea31200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea2c109c20 0x7fea2c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.876+0000 7fea31200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1138e0 con 0x7fea2c103990 2024-09-21T03:46:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.876+0000 7fea31200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c113760 con 0x7fea2c109c20 2024-09-21T03:46:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.876+0000 7fea31200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c113a60 con 0x7fea2c1092d0 2024-09-21T03:46:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.876+0000 7fea23e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea2c1092d0 0x7fea2c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.876+0000 7fea23e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea2c1092d0 0x7fea2c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34666/0 (socket says 172.21.15.16:34666) 2024-09-21T03:46:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.876+0000 7fea23e00640 1 -- 172.21.15.16:0/2599046254 learned_addr learned my addr 172.21.15.16:0/2599046254 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.876+0000 7fea2b400640 1 --2- 172.21.15.16:0/2599046254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c103990 0x7fea2c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.876+0000 7fea2be00640 1 --2- 172.21.15.16:0/2599046254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea2c109c20 0x7fea2c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.876+0000 7fea23e00640 1 -- 172.21.15.16:0/2599046254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea2c109c20 msgr2=0x7fea2c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.876+0000 7fea23e00640 1 --2- 172.21.15.16:0/2599046254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea2c109c20 0x7fea2c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.876+0000 7fea23e00640 1 -- 172.21.15.16:0/2599046254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c103990 msgr2=0x7fea2c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.876+0000 7fea23e00640 1 --2- 172.21.15.16:0/2599046254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c103990 0x7fea2c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.876+0000 7fea23e00640 1 -- 172.21.15.16:0/2599046254 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2c1a8630 con 0x7fea2c1092d0 2024-09-21T03:46:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.877+0000 7fea2b400640 1 --2- 172.21.15.16:0/2599046254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c103990 0x7fea2c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.877+0000 7fea23e00640 1 --2- 172.21.15.16:0/2599046254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea2c1092d0 0x7fea2c075c50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fea14035040 tx=0x7fea14032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.877+0000 7fea28c00640 1 -- 172.21.15.16:0/2599046254 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14040020 con 0x7fea2c1092d0 2024-09-21T03:46:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.877+0000 7fea31200640 1 -- 172.21.15.16:0/2599046254 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2c1a88c0 con 0x7fea2c1092d0 2024-09-21T03:46:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.877+0000 7fea28c00640 1 -- 172.21.15.16:0/2599046254 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea14030e10 con 0x7fea2c1092d0 2024-09-21T03:46:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.877+0000 7fea28c00640 1 -- 172.21.15.16:0/2599046254 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14037040 con 0x7fea2c1092d0 2024-09-21T03:46:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.877+0000 7fea31200640 1 -- 172.21.15.16:0/2599046254 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea2c1a8df0 con 0x7fea2c1092d0 2024-09-21T03:46:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.879+0000 7fea31200640 1 -- 172.21.15.16:0/2599046254 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea2c103da0 con 0x7fea2c1092d0 2024-09-21T03:46:15.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.881+0000 7fea28c00640 1 -- 172.21.15.16:0/2599046254 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fea14043020 con 0x7fea2c1092d0 2024-09-21T03:46:15.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.883+0000 7fea28c00640 1 --2- 172.21.15.16:0/2599046254 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fea100b6510 0x7fea100b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:15.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.883+0000 7fea28c00640 1 -- 172.21.15.16:0/2599046254 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fea14107390 con 0x7fea2c1092d0 2024-09-21T03:46:15.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.883+0000 7fea2b400640 1 --2- 172.21.15.16:0/2599046254 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fea100b6510 0x7fea100b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:15.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.884+0000 7fea2b400640 1 --2- 172.21.15.16:0/2599046254 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fea100b6510 0x7fea100b8a20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fea2c077160 tx=0x7fea1c005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:15.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:15.884+0000 7fea28c00640 1 -- 172.21.15.16:0/2599046254 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fea1404a020 con 0x7fea2c1092d0 2024-09-21T03:46:15.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:15 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T03:46:15.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:15 smithi123 ceph-mon[34673]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:46:15.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:15 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:46:15.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:15 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:46:15.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:15 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:46:15.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:15 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:46:15.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:15 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:46:15.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:15 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:46:15.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:15 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:46:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:16.080+0000 7fea31200640 1 -- 172.21.15.16:0/2599046254 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fea2c063800 con 0x7fea100b6510 2024-09-21T03:46:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:16.087+0000 7fea28c00640 1 -- 172.21.15.16:0/2599046254 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340527 (secure 0 0 0) 0x7fea2c063800 con 0x7fea100b6510 2024-09-21T03:46:16.093 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:46:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-21T03:46:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:16.091+0000 7fea31200640 1 -- 172.21.15.16:0/2599046254 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fea100b6510 msgr2=0x7fea100b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:16.091+0000 7fea31200640 1 --2- 172.21.15.16:0/2599046254 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fea100b6510 0x7fea100b8a20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fea2c077160 tx=0x7fea1c005f90 comp rx=0 tx=0).stop 2024-09-21T03:46:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:16.091+0000 7fea31200640 1 -- 172.21.15.16:0/2599046254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea2c1092d0 msgr2=0x7fea2c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:16.091+0000 7fea31200640 1 --2- 172.21.15.16:0/2599046254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea2c1092d0 0x7fea2c075c50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fea14035040 tx=0x7fea14032d50 comp rx=0 tx=0).stop 2024-09-21T03:46:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:16.091+0000 7fea31200640 1 -- 172.21.15.16:0/2599046254 shutdown_connections 2024-09-21T03:46:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:16.091+0000 7fea31200640 1 --2- 172.21.15.16:0/2599046254 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fea100b6510 0x7fea100b8a20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:16.091+0000 7fea31200640 1 --2- 172.21.15.16:0/2599046254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea2c109c20 0x7fea2c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:16.091+0000 7fea31200640 1 --2- 172.21.15.16:0/2599046254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea2c1092d0 0x7fea2c075c50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:16.091+0000 7fea31200640 1 --2- 172.21.15.16:0/2599046254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c103990 0x7fea2c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:16.091+0000 7fea31200640 1 -- 172.21.15.16:0/2599046254 >> 172.21.15.16:0/2599046254 conn(0x7fea2c0ff3e0 msgr2=0x7fea2c101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:16.091+0000 7fea31200640 1 -- 172.21.15.16:0/2599046254 shutdown_connections 2024-09-21T03:46:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:16.091+0000 7fea31200640 1 -- 172.21.15.16:0/2599046254 wait complete. 2024-09-21T03:46:16.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:16 smithi134 ceph-mon[35031]: pgmap v417: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:46:16.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:16 smithi134 ceph-mon[35031]: from='client.25405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:16.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:16 smithi123 ceph-mon[34673]: pgmap v417: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:46:16.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:16 smithi123 ceph-mon[34673]: from='client.25405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:16 smithi016 ceph-mon[30375]: pgmap v417: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:46:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:16 smithi016 ceph-mon[30375]: from='client.25405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:16.986 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-09-21T03:46:15.383407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665494,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1387,"num_read_kb":798,"num_write":384,"num_write_kb":2305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2951952,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":816,"ondisk_log_size":816,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379756,"kb_used_data":13868,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562068,"statfs":{"total":1151940427776,"available":1151551557632,"internally_reserved":0,"allocated":14200832,"data_stored":7971396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":622,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":52,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003573"},"pg_stats":[{"pgid":"2.1f","version":"84'13","reported_seq":184,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:08.741963+0000","last_change":"2024-09-21T03:43:04.980132+0000","last_active":"2024-09-21T03:46:08.741963+0000","last_peered":"2024-09-21T03:46:08.741963+0000","last_clean":"2024-09-21T03:46:08.741963+0000","last_became_active":"2024-09-21T03:43:04.979274+0000","last_became_peered":"2024-09-21T03:43:04.979274+0000","last_unstale":"2024-09-21T03:46:08.741963+0000","last_undegraded":"2024-09-21T03:46:08.741963+0000","last_fullsized":"2024-09-21T03:46:08.741963+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:35.047129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.727333+0000","last_change":"2024-09-21T03:43:08.011576+0000","last_active":"2024-09-21T03:45:23.727333+0000","last_peered":"2024-09-21T03:45:23.727333+0000","last_clean":"2024-09-21T03:45:23.727333+0000","last_became_active":"2024-09-21T03:43:08.011188+0000","last_became_peered":"2024-09-21T03:43:08.011188+0000","last_unstale":"2024-09-21T03:45:23.727333+0000","last_undegraded":"2024-09-21T03:45:23.727333+0000","last_fullsized":"2024-09-21T03:45:23.727333+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:34.477639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678062+0000","last_change":"2024-09-21T03:43:54.493183+0000","last_active":"2024-09-21T03:44:54.678062+0000","last_peered":"2024-09-21T03:44:54.678062+0000","last_clean":"2024-09-21T03:44:54.678062+0000","last_became_active":"2024-09-21T03:43:54.492605+0000","last_became_peered":"2024-09-21T03:43:54.492605+0000","last_unstale":"2024-09-21T03:44:54.678062+0000","last_undegraded":"2024-09-21T03:44:54.678062+0000","last_fullsized":"2024-09-21T03:44:54.678062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:26.634042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.672352+0000","last_change":"2024-09-21T03:43:05.141790+0000","last_active":"2024-09-21T03:44:54.672352+0000","last_peered":"2024-09-21T03:44:54.672352+0000","last_clean":"2024-09-21T03:44:54.672352+0000","last_became_active":"2024-09-21T03:43:05.141299+0000","last_became_peered":"2024-09-21T03:43:05.141299+0000","last_unstale":"2024-09-21T03:44:54.672352+0000","last_undegraded":"2024-09-21T03:44:54.672352+0000","last_fullsized":"2024-09-21T03:44:54.672352+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:48.844615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.747911+0000","last_change":"2024-09-21T03:43:08.018245+0000","last_active":"2024-09-21T03:45:23.747911+0000","last_peered":"2024-09-21T03:45:23.747911+0000","last_clean":"2024-09-21T03:45:23.747911+0000","last_became_active":"2024-09-21T03:43:08.015719+0000","last_became_peered":"2024-09-21T03:43:08.015719+0000","last_unstale":"2024-09-21T03:45:23.747911+0000","last_undegraded":"2024-09-21T03:45:23.747911+0000","last_fullsized":"2024-09-21T03:45:23.747911+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:39.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766982+0000","last_change":"2024-09-21T03:43:54.493279+0000","last_active":"2024-09-21T03:44:54.766982+0000","last_peered":"2024-09-21T03:44:54.766982+0000","last_clean":"2024-09-21T03:44:54.766982+0000","last_became_active":"2024-09-21T03:43:54.492592+0000","last_became_peered":"2024-09-21T03:43:54.492592+0000","last_unstale":"2024-09-21T03:44:54.766982+0000","last_undegraded":"2024-09-21T03:44:54.766982+0000","last_fullsized":"2024-09-21T03:44:54.766982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:28.520990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.693998+0000","last_change":"2024-09-21T03:44:54.693431+0000","last_active":"2024-09-21T03:44:54.693998+0000","last_peered":"2024-09-21T03:44:54.693998+0000","last_clean":"2024-09-21T03:44:54.693998+0000","last_became_active":"2024-09-21T03:44:54.680324+0000","last_became_peered":"2024-09-21T03:44:54.680324+0000","last_unstale":"2024-09-21T03:44:54.693998+0000","last_undegraded":"2024-09-21T03:44:54.693998+0000","last_fullsized":"2024-09-21T03:44:54.693998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:43.730697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.739651+0000","last_change":"2024-09-21T03:43:08.009018+0000","last_active":"2024-09-21T03:45:23.739651+0000","last_peered":"2024-09-21T03:45:23.739651+0000","last_clean":"2024-09-21T03:45:23.739651+0000","last_became_active":"2024-09-21T03:43:08.008614+0000","last_became_peered":"2024-09-21T03:43:08.008614+0000","last_unstale":"2024-09-21T03:45:23.739651+0000","last_undegraded":"2024-09-21T03:45:23.739651+0000","last_fullsized":"2024-09-21T03:45:23.739651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:31.000616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768752+0000","last_change":"2024-09-21T03:43:18.092750+0000","last_active":"2024-09-21T03:44:54.768752+0000","last_peered":"2024-09-21T03:44:54.768752+0000","last_clean":"2024-09-21T03:44:54.768752+0000","last_became_active":"2024-09-21T03:43:18.092122+0000","last_became_peered":"2024-09-21T03:43:18.092122+0000","last_unstale":"2024-09-21T03:44:54.768752+0000","last_undegraded":"2024-09-21T03:44:54.768752+0000","last_fullsized":"2024-09-21T03:44:54.768752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:15.032388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675595+0000","last_change":"2024-09-21T03:43:04.978045+0000","last_active":"2024-09-21T03:44:54.675595+0000","last_peered":"2024-09-21T03:44:54.675595+0000","last_clean":"2024-09-21T03:44:54.675595+0000","last_became_active":"2024-09-21T03:43:04.977565+0000","last_became_peered":"2024-09-21T03:43:04.977565+0000","last_unstale":"2024-09-21T03:44:54.675595+0000","last_undegraded":"2024-09-21T03:44:54.675595+0000","last_fullsized":"2024-09-21T03:44:54.675595+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:42.537892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.740686+0000","last_change":"2024-09-21T03:43:08.007094+0000","last_active":"2024-09-21T03:45:23.740686+0000","last_peered":"2024-09-21T03:45:23.740686+0000","last_clean":"2024-09-21T03:45:23.740686+0000","last_became_active":"2024-09-21T03:43:08.006666+0000","last_became_peered":"2024-09-21T03:43:08.006666+0000","last_unstale":"2024-09-21T03:45:23.740686+0000","last_undegraded":"2024-09-21T03:45:23.740686+0000","last_fullsized":"2024-09-21T03:45:23.740686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:59.247459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767130+0000","last_change":"2024-09-21T03:43:18.081141+0000","last_active":"2024-09-21T03:44:54.767130+0000","last_peered":"2024-09-21T03:44:54.767130+0000","last_clean":"2024-09-21T03:44:54.767130+0000","last_became_active":"2024-09-21T03:43:18.080623+0000","last_became_peered":"2024-09-21T03:43:18.080623+0000","last_unstale":"2024-09-21T03:44:54.767130+0000","last_undegraded":"2024-09-21T03:44:54.767130+0000","last_fullsized":"2024-09-21T03:44:54.767130+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:44.932673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.741475+0000","last_change":"2024-09-21T03:43:08.008602+0000","last_active":"2024-09-21T03:45:23.741475+0000","last_peered":"2024-09-21T03:45:23.741475+0000","last_clean":"2024-09-21T03:45:23.741475+0000","last_became_active":"2024-09-21T03:43:08.008220+0000","last_became_peered":"2024-09-21T03:43:08.008220+0000","last_unstale":"2024-09-21T03:45:23.741475+0000","last_undegraded":"2024-09-21T03:45:23.741475+0000","last_fullsized":"2024-09-21T03:45:23.741475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:28.286016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.774443+0000","last_change":"2024-09-21T03:44:54.773918+0000","last_active":"2024-09-21T03:44:54.774443+0000","last_peered":"2024-09-21T03:44:54.774443+0000","last_clean":"2024-09-21T03:44:54.774443+0000","last_became_active":"2024-09-21T03:44:54.769721+0000","last_became_peered":"2024-09-21T03:44:54.769721+0000","last_unstale":"2024-09-21T03:44:54.774443+0000","last_undegraded":"2024-09-21T03:44:54.774443+0000","last_fullsized":"2024-09-21T03:44:54.774443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:05.656051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767195+0000","last_change":"2024-09-21T03:43:18.483523+0000","last_active":"2024-09-21T03:44:54.767195+0000","last_peered":"2024-09-21T03:44:54.767195+0000","last_clean":"2024-09-21T03:44:54.767195+0000","last_became_active":"2024-09-21T03:43:18.482833+0000","last_became_peered":"2024-09-21T03:43:18.482833+0000","last_unstale":"2024-09-21T03:44:54.767195+0000","last_undegraded":"2024-09-21T03:44:54.767195+0000","last_fullsized":"2024-09-21T03:44:54.767195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:48.236569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.736655+0000","last_change":"2024-09-21T03:43:08.022592+0000","last_active":"2024-09-21T03:45:23.736655+0000","last_peered":"2024-09-21T03:45:23.736655+0000","last_clean":"2024-09-21T03:45:23.736655+0000","last_became_active":"2024-09-21T03:43:08.018065+0000","last_became_peered":"2024-09-21T03:43:08.018065+0000","last_unstale":"2024-09-21T03:45:23.736655+0000","last_undegraded":"2024-09-21T03:45:23.736655+0000","last_fullsized":"2024-09-21T03:45:23.736655+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:41.524402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679262+0000","last_change":"2024-09-21T03:43:04.978788+0000","last_active":"2024-09-21T03:44:54.679262+0000","last_peered":"2024-09-21T03:44:54.679262+0000","last_clean":"2024-09-21T03:44:54.679262+0000","last_became_active":"2024-09-21T03:43:04.978382+0000","last_became_peered":"2024-09-21T03:43:04.978382+0000","last_unstale":"2024-09-21T03:44:54.679262+0000","last_undegraded":"2024-09-21T03:44:54.679262+0000","last_fullsized":"2024-09-21T03:44:54.679262+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:12.571217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768259+0000","last_change":"2024-09-21T03:43:54.493347+0000","last_active":"2024-09-21T03:44:54.768259+0000","last_peered":"2024-09-21T03:44:54.768259+0000","last_clean":"2024-09-21T03:44:54.768259+0000","last_became_active":"2024-09-21T03:43:54.492598+0000","last_became_peered":"2024-09-21T03:43:54.492598+0000","last_unstale":"2024-09-21T03:44:54.768259+0000","last_undegraded":"2024-09-21T03:44:54.768259+0000","last_fullsized":"2024-09-21T03:44:54.768259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:34.788296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743945+0000","last_change":"2024-09-21T03:43:08.014368+0000","last_active":"2024-09-21T03:45:23.743945+0000","last_peered":"2024-09-21T03:45:23.743945+0000","last_clean":"2024-09-21T03:45:23.743945+0000","last_became_active":"2024-09-21T03:43:08.013952+0000","last_became_peered":"2024-09-21T03:43:08.013952+0000","last_unstale":"2024-09-21T03:45:23.743945+0000","last_undegraded":"2024-09-21T03:45:23.743945+0000","last_fullsized":"2024-09-21T03:45:23.743945+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:16.893588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671998+0000","last_change":"2024-09-21T03:43:04.976162+0000","last_active":"2024-09-21T03:44:54.671998+0000","last_peered":"2024-09-21T03:44:54.671998+0000","last_clean":"2024-09-21T03:44:54.671998+0000","last_became_active":"2024-09-21T03:43:04.973790+0000","last_became_peered":"2024-09-21T03:43:04.973790+0000","last_unstale":"2024-09-21T03:44:54.671998+0000","last_undegraded":"2024-09-21T03:44:54.671998+0000","last_fullsized":"2024-09-21T03:44:54.671998+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:47.838229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670339+0000","last_change":"2024-09-21T03:43:18.084344+0000","last_active":"2024-09-21T03:44:54.670339+0000","last_peered":"2024-09-21T03:44:54.670339+0000","last_clean":"2024-09-21T03:44:54.670339+0000","last_became_active":"2024-09-21T03:43:18.083232+0000","last_became_peered":"2024-09-21T03:43:18.083232+0000","last_unstale":"2024-09-21T03:44:54.670339+0000","last_undegraded":"2024-09-21T03:44:54.670339+0000","last_fullsized":"2024-09-21T03:44:54.670339+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:46.254405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.690064+0000","last_change":"2024-09-21T03:43:08.009932+0000","last_active":"2024-09-21T03:45:23.690064+0000","last_peered":"2024-09-21T03:45:23.690064+0000","last_clean":"2024-09-21T03:45:23.690064+0000","last_became_active":"2024-09-21T03:43:08.009492+0000","last_became_peered":"2024-09-21T03:43:08.009492+0000","last_unstale":"2024-09-21T03:45:23.690064+0000","last_undegraded":"2024-09-21T03:45:23.690064+0000","last_fullsized":"2024-09-21T03:45:23.690064+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:58.679348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768057+0000","last_change":"2024-09-21T03:43:05.141927+0000","last_active":"2024-09-21T03:44:54.768057+0000","last_peered":"2024-09-21T03:44:54.768057+0000","last_clean":"2024-09-21T03:44:54.768057+0000","last_became_active":"2024-09-21T03:43:05.141516+0000","last_became_peered":"2024-09-21T03:43:05.141516+0000","last_unstale":"2024-09-21T03:44:54.768057+0000","last_undegraded":"2024-09-21T03:44:54.768057+0000","last_fullsized":"2024-09-21T03:44:54.768057+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:35.666981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767396+0000","last_change":"2024-09-21T03:43:18.484478+0000","last_active":"2024-09-21T03:44:54.767396+0000","last_peered":"2024-09-21T03:44:54.767396+0000","last_clean":"2024-09-21T03:44:54.767396+0000","last_became_active":"2024-09-21T03:43:18.483890+0000","last_became_peered":"2024-09-21T03:43:18.483890+0000","last_unstale":"2024-09-21T03:44:54.767396+0000","last_undegraded":"2024-09-21T03:44:54.767396+0000","last_fullsized":"2024-09-21T03:44:54.767396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:03.542783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.748379+0000","last_change":"2024-09-21T03:43:08.017785+0000","last_active":"2024-09-21T03:45:23.748379+0000","last_peered":"2024-09-21T03:45:23.748379+0000","last_clean":"2024-09-21T03:45:23.748379+0000","last_became_active":"2024-09-21T03:43:08.017389+0000","last_became_peered":"2024-09-21T03:43:08.017389+0000","last_unstale":"2024-09-21T03:45:23.748379+0000","last_undegraded":"2024-09-21T03:45:23.748379+0000","last_fullsized":"2024-09-21T03:45:23.748379+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:37.557061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":54,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767655+0000","last_change":"2024-09-21T03:43:05.567829+0000","last_active":"2024-09-21T03:44:54.767655+0000","last_peered":"2024-09-21T03:44:54.767655+0000","last_clean":"2024-09-21T03:44:54.767655+0000","last_became_active":"2024-09-21T03:43:05.567401+0000","last_became_peered":"2024-09-21T03:43:05.567401+0000","last_unstale":"2024-09-21T03:44:54.767655+0000","last_undegraded":"2024-09-21T03:44:54.767655+0000","last_fullsized":"2024-09-21T03:44:54.767655+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:03.954280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678873+0000","last_change":"2024-09-21T03:43:18.080064+0000","last_active":"2024-09-21T03:44:54.678873+0000","last_peered":"2024-09-21T03:44:54.678873+0000","last_clean":"2024-09-21T03:44:54.678873+0000","last_became_active":"2024-09-21T03:43:18.079569+0000","last_became_peered":"2024-09-21T03:43:18.079569+0000","last_unstale":"2024-09-21T03:44:54.678873+0000","last_undegraded":"2024-09-21T03:44:54.678873+0000","last_fullsized":"2024-09-21T03:44:54.678873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:29.568476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743198+0000","last_change":"2024-09-21T03:43:08.011603+0000","last_active":"2024-09-21T03:45:23.743198+0000","last_peered":"2024-09-21T03:45:23.743198+0000","last_clean":"2024-09-21T03:45:23.743198+0000","last_became_active":"2024-09-21T03:43:08.011078+0000","last_became_peered":"2024-09-21T03:43:08.011078+0000","last_unstale":"2024-09-21T03:45:23.743198+0000","last_undegraded":"2024-09-21T03:45:23.743198+0000","last_fullsized":"2024-09-21T03:45:23.743198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:44.764713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'16","reported_seq":62,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:15.640997+0000","last_change":"2024-09-21T03:43:04.979223+0000","last_active":"2024-09-21T03:45:15.640997+0000","last_peered":"2024-09-21T03:45:15.640997+0000","last_clean":"2024-09-21T03:45:15.640997+0000","last_became_active":"2024-09-21T03:43:04.978675+0000","last_became_peered":"2024-09-21T03:43:04.978675+0000","last_unstale":"2024-09-21T03:45:15.640997+0000","last_undegraded":"2024-09-21T03:45:15.640997+0000","last_fullsized":"2024-09-21T03:45:15.640997+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:09.717921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671365+0000","last_change":"2024-09-21T03:43:18.482334+0000","last_active":"2024-09-21T03:44:54.671365+0000","last_peered":"2024-09-21T03:44:54.671365+0000","last_clean":"2024-09-21T03:44:54.671365+0000","last_became_active":"2024-09-21T03:43:18.481881+0000","last_became_peered":"2024-09-21T03:43:18.481881+0000","last_unstale":"2024-09-21T03:44:54.671365+0000","last_undegraded":"2024-09-21T03:44:54.671365+0000","last_fullsized":"2024-09-21T03:44:54.671365+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:02:32.497348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.730466+0000","last_change":"2024-09-21T03:43:07.993771+0000","last_active":"2024-09-21T03:45:23.730466+0000","last_peered":"2024-09-21T03:45:23.730466+0000","last_clean":"2024-09-21T03:45:23.730466+0000","last_became_active":"2024-09-21T03:43:07.993268+0000","last_became_peered":"2024-09-21T03:43:07.993268+0000","last_unstale":"2024-09-21T03:45:23.730466+0000","last_undegraded":"2024-09-21T03:45:23.730466+0000","last_fullsized":"2024-09-21T03:45:23.730466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:42.348561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675923+0000","last_change":"2024-09-21T03:43:05.141786+0000","last_active":"2024-09-21T03:44:54.675923+0000","last_peered":"2024-09-21T03:44:54.675923+0000","last_clean":"2024-09-21T03:44:54.675923+0000","last_became_active":"2024-09-21T03:43:05.141307+0000","last_became_peered":"2024-09-21T03:43:05.141307+0000","last_unstale":"2024-09-21T03:44:54.675923+0000","last_undegraded":"2024-09-21T03:44:54.675923+0000","last_fullsized":"2024-09-21T03:44:54.675923+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:12.139398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677370+0000","last_change":"2024-09-21T03:43:18.087848+0000","last_active":"2024-09-21T03:44:54.677370+0000","last_peered":"2024-09-21T03:44:54.677370+0000","last_clean":"2024-09-21T03:44:54.677370+0000","last_became_active":"2024-09-21T03:43:18.087400+0000","last_became_peered":"2024-09-21T03:43:18.087400+0000","last_unstale":"2024-09-21T03:44:54.677370+0000","last_undegraded":"2024-09-21T03:44:54.677370+0000","last_fullsized":"2024-09-21T03:44:54.677370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:46.902979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"86'4","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.730419+0000","last_change":"2024-09-21T03:43:08.010051+0000","last_active":"2024-09-21T03:45:23.730419+0000","last_peered":"2024-09-21T03:45:23.730419+0000","last_clean":"2024-09-21T03:45:23.730419+0000","last_became_active":"2024-09-21T03:43:08.007700+0000","last_became_peered":"2024-09-21T03:43:08.007700+0000","last_unstale":"2024-09-21T03:45:23.730419+0000","last_undegraded":"2024-09-21T03:45:23.730419+0000","last_fullsized":"2024-09-21T03:45:23.730419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:31.828193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670543+0000","last_change":"2024-09-21T03:43:05.140490+0000","last_active":"2024-09-21T03:44:54.670543+0000","last_peered":"2024-09-21T03:44:54.670543+0000","last_clean":"2024-09-21T03:44:54.670543+0000","last_became_active":"2024-09-21T03:43:05.139979+0000","last_became_peered":"2024-09-21T03:43:05.139979+0000","last_unstale":"2024-09-21T03:44:54.670543+0000","last_undegraded":"2024-09-21T03:44:54.670543+0000","last_fullsized":"2024-09-21T03:44:54.670543+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:20.957596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.674914+0000","last_change":"2024-09-21T03:43:18.482777+0000","last_active":"2024-09-21T03:44:54.674914+0000","last_peered":"2024-09-21T03:44:54.674914+0000","last_clean":"2024-09-21T03:44:54.674914+0000","last_became_active":"2024-09-21T03:43:18.482244+0000","last_became_peered":"2024-09-21T03:43:18.482244+0000","last_unstale":"2024-09-21T03:44:54.674914+0000","last_undegraded":"2024-09-21T03:44:54.674914+0000","last_fullsized":"2024-09-21T03:44:54.674914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:41.997309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.734311+0000","last_change":"2024-09-21T03:43:08.019087+0000","last_active":"2024-09-21T03:45:23.734311+0000","last_peered":"2024-09-21T03:45:23.734311+0000","last_clean":"2024-09-21T03:45:23.734311+0000","last_became_active":"2024-09-21T03:43:08.018467+0000","last_became_peered":"2024-09-21T03:43:08.018467+0000","last_unstale":"2024-09-21T03:45:23.734311+0000","last_undegraded":"2024-09-21T03:45:23.734311+0000","last_fullsized":"2024-09-21T03:45:23.734311+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:25.824955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'6","reported_seq":183,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:13.751456+0000","last_change":"2024-09-21T03:43:04.973433+0000","last_active":"2024-09-21T03:46:13.751456+0000","last_peered":"2024-09-21T03:46:13.751456+0000","last_clean":"2024-09-21T03:46:13.751456+0000","last_became_active":"2024-09-21T03:43:04.973033+0000","last_became_peered":"2024-09-21T03:43:04.973033+0000","last_unstale":"2024-09-21T03:46:13.751456+0000","last_undegraded":"2024-09-21T03:46:13.751456+0000","last_fullsized":"2024-09-21T03:46:13.751456+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:25.462635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":899,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766900+0000","last_change":"2024-09-21T03:43:18.082385+0000","last_active":"2024-09-21T03:44:54.766900+0000","last_peered":"2024-09-21T03:44:54.766900+0000","last_clean":"2024-09-21T03:44:54.766900+0000","last_became_active":"2024-09-21T03:43:18.081958+0000","last_became_peered":"2024-09-21T03:43:18.081958+0000","last_unstale":"2024-09-21T03:44:54.766900+0000","last_undegraded":"2024-09-21T03:44:54.766900+0000","last_fullsized":"2024-09-21T03:44:54.766900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:36.554838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.739812+0000","last_change":"2024-09-21T03:43:08.008307+0000","last_active":"2024-09-21T03:45:23.739812+0000","last_peered":"2024-09-21T03:45:23.739812+0000","last_clean":"2024-09-21T03:45:23.739812+0000","last_became_active":"2024-09-21T03:43:08.006736+0000","last_became_peered":"2024-09-21T03:43:08.006736+0000","last_unstale":"2024-09-21T03:45:23.739812+0000","last_undegraded":"2024-09-21T03:45:23.739812+0000","last_fullsized":"2024-09-21T03:45:23.739812+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:31.694655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":179,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:11.746814+0000","last_change":"2024-09-21T03:43:05.140263+0000","last_active":"2024-09-21T03:46:11.746814+0000","last_peered":"2024-09-21T03:46:11.746814+0000","last_clean":"2024-09-21T03:46:11.746814+0000","last_became_active":"2024-09-21T03:43:05.139869+0000","last_became_peered":"2024-09-21T03:43:05.139869+0000","last_unstale":"2024-09-21T03:46:11.746814+0000","last_undegraded":"2024-09-21T03:46:11.746814+0000","last_fullsized":"2024-09-21T03:46:11.746814+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:19.313677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766889+0000","last_change":"2024-09-21T03:43:18.483795+0000","last_active":"2024-09-21T03:44:54.766889+0000","last_peered":"2024-09-21T03:44:54.766889+0000","last_clean":"2024-09-21T03:44:54.766889+0000","last_became_active":"2024-09-21T03:43:18.483159+0000","last_became_peered":"2024-09-21T03:43:18.483159+0000","last_unstale":"2024-09-21T03:44:54.766889+0000","last_undegraded":"2024-09-21T03:44:54.766889+0000","last_fullsized":"2024-09-21T03:44:54.766889+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:56.277956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.726790+0000","last_change":"2024-09-21T03:43:08.001661+0000","last_active":"2024-09-21T03:45:23.726790+0000","last_peered":"2024-09-21T03:45:23.726790+0000","last_clean":"2024-09-21T03:45:23.726790+0000","last_became_active":"2024-09-21T03:43:08.001254+0000","last_became_peered":"2024-09-21T03:43:08.001254+0000","last_unstale":"2024-09-21T03:45:23.726790+0000","last_undegraded":"2024-09-21T03:45:23.726790+0000","last_fullsized":"2024-09-21T03:45:23.726790+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:19.020542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678080+0000","last_change":"2024-09-21T03:43:04.979127+0000","last_active":"2024-09-21T03:44:54.678080+0000","last_peered":"2024-09-21T03:44:54.678080+0000","last_clean":"2024-09-21T03:44:54.678080+0000","last_became_active":"2024-09-21T03:43:04.978705+0000","last_became_peered":"2024-09-21T03:43:04.978705+0000","last_unstale":"2024-09-21T03:44:54.678080+0000","last_undegraded":"2024-09-21T03:44:54.678080+0000","last_fullsized":"2024-09-21T03:44:54.678080+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:48.548002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766993+0000","last_change":"2024-09-21T03:43:18.483682+0000","last_active":"2024-09-21T03:44:54.766993+0000","last_peered":"2024-09-21T03:44:54.766993+0000","last_clean":"2024-09-21T03:44:54.766993+0000","last_became_active":"2024-09-21T03:43:18.483124+0000","last_became_peered":"2024-09-21T03:43:18.483124+0000","last_unstale":"2024-09-21T03:44:54.766993+0000","last_undegraded":"2024-09-21T03:44:54.766993+0000","last_fullsized":"2024-09-21T03:44:54.766993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:12.513795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.739926+0000","last_change":"2024-09-21T03:43:08.013193+0000","last_active":"2024-09-21T03:45:23.739926+0000","last_peered":"2024-09-21T03:45:23.739926+0000","last_clean":"2024-09-21T03:45:23.739926+0000","last_became_active":"2024-09-21T03:43:08.010360+0000","last_became_peered":"2024-09-21T03:43:08.010360+0000","last_unstale":"2024-09-21T03:45:23.739926+0000","last_undegraded":"2024-09-21T03:45:23.739926+0000","last_fullsized":"2024-09-21T03:45:23.739926+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:28.101646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768945+0000","last_change":"2024-09-21T03:43:04.971782+0000","last_active":"2024-09-21T03:44:54.768945+0000","last_peered":"2024-09-21T03:44:54.768945+0000","last_clean":"2024-09-21T03:44:54.768945+0000","last_became_active":"2024-09-21T03:43:04.971375+0000","last_became_peered":"2024-09-21T03:43:04.971375+0000","last_unstale":"2024-09-21T03:44:54.768945+0000","last_undegraded":"2024-09-21T03:44:54.768945+0000","last_fullsized":"2024-09-21T03:44:54.768945+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:49.573490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'92","reported_seq":115,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678174+0000","last_change":"2024-09-21T03:41:42.914435+0000","last_active":"2024-09-21T03:44:54.678174+0000","last_peered":"2024-09-21T03:44:54.678174+0000","last_clean":"2024-09-21T03:44:54.678174+0000","last_became_active":"2024-09-21T03:41:42.909490+0000","last_became_peered":"2024-09-21T03:41:42.909490+0000","last_unstale":"2024-09-21T03:44:54.678174+0000","last_undegraded":"2024-09-21T03:44:54.678174+0000","last_fullsized":"2024-09-21T03:44:54.678174+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:56.507027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:56.507027+0000","last_clean_scrub_stamp":"2024-09-21T03:37:56.507027+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:34.564062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678296+0000","last_change":"2024-09-21T03:43:18.084701+0000","last_active":"2024-09-21T03:44:54.678296+0000","last_peered":"2024-09-21T03:44:54.678296+0000","last_clean":"2024-09-21T03:44:54.678296+0000","last_became_active":"2024-09-21T03:43:18.084251+0000","last_became_peered":"2024-09-21T03:43:18.084251+0000","last_unstale":"2024-09-21T03:44:54.678296+0000","last_undegraded":"2024-09-21T03:44:54.678296+0000","last_fullsized":"2024-09-21T03:44:54.678296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:15.660276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.741599+0000","last_change":"2024-09-21T03:43:08.012195+0000","last_active":"2024-09-21T03:45:23.741599+0000","last_peered":"2024-09-21T03:45:23.741599+0000","last_clean":"2024-09-21T03:45:23.741599+0000","last_became_active":"2024-09-21T03:43:08.011670+0000","last_became_peered":"2024-09-21T03:43:08.011670+0000","last_unstale":"2024-09-21T03:45:23.741599+0000","last_undegraded":"2024-09-21T03:45:23.741599+0000","last_fullsized":"2024-09-21T03:45:23.741599+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:14.290074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'7","reported_seq":182,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:09.745005+0000","last_change":"2024-09-21T03:43:05.567898+0000","last_active":"2024-09-21T03:46:09.745005+0000","last_peered":"2024-09-21T03:46:09.745005+0000","last_clean":"2024-09-21T03:46:09.745005+0000","last_became_active":"2024-09-21T03:43:05.567367+0000","last_became_peered":"2024-09-21T03:43:05.567367+0000","last_unstale":"2024-09-21T03:46:09.745005+0000","last_undegraded":"2024-09-21T03:46:09.745005+0000","last_fullsized":"2024-09-21T03:46:09.745005+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:55.120838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768187+0000","last_change":"2024-09-21T03:43:18.073942+0000","last_active":"2024-09-21T03:44:54.768187+0000","last_peered":"2024-09-21T03:44:54.768187+0000","last_clean":"2024-09-21T03:44:54.768187+0000","last_became_active":"2024-09-21T03:43:18.073323+0000","last_became_peered":"2024-09-21T03:43:18.073323+0000","last_unstale":"2024-09-21T03:44:54.768187+0000","last_undegraded":"2024-09-21T03:44:54.768187+0000","last_fullsized":"2024-09-21T03:44:54.768187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:41.677143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.742436+0000","last_change":"2024-09-21T03:43:08.016636+0000","last_active":"2024-09-21T03:45:23.742436+0000","last_peered":"2024-09-21T03:45:23.742436+0000","last_clean":"2024-09-21T03:45:23.742436+0000","last_became_active":"2024-09-21T03:43:08.016235+0000","last_became_peered":"2024-09-21T03:43:08.016235+0000","last_unstale":"2024-09-21T03:45:23.742436+0000","last_undegraded":"2024-09-21T03:45:23.742436+0000","last_fullsized":"2024-09-21T03:45:23.742436+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:35.365830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767144+0000","last_change":"2024-09-21T03:43:04.980911+0000","last_active":"2024-09-21T03:44:54.767144+0000","last_peered":"2024-09-21T03:44:54.767144+0000","last_clean":"2024-09-21T03:44:54.767144+0000","last_became_active":"2024-09-21T03:43:04.980512+0000","last_became_peered":"2024-09-21T03:43:04.980512+0000","last_unstale":"2024-09-21T03:44:54.767144+0000","last_undegraded":"2024-09-21T03:44:54.767144+0000","last_fullsized":"2024-09-21T03:44:54.767144+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:04.136345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679934+0000","last_change":"2024-09-21T03:43:18.085874+0000","last_active":"2024-09-21T03:44:54.679934+0000","last_peered":"2024-09-21T03:44:54.679934+0000","last_clean":"2024-09-21T03:44:54.679934+0000","last_became_active":"2024-09-21T03:43:18.085381+0000","last_became_peered":"2024-09-21T03:43:18.085381+0000","last_unstale":"2024-09-21T03:44:54.679934+0000","last_undegraded":"2024-09-21T03:44:54.679934+0000","last_fullsized":"2024-09-21T03:44:54.679934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:02.829202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.734618+0000","last_change":"2024-09-21T03:43:08.014776+0000","last_active":"2024-09-21T03:45:23.734618+0000","last_peered":"2024-09-21T03:45:23.734618+0000","last_clean":"2024-09-21T03:45:23.734618+0000","last_became_active":"2024-09-21T03:43:08.014191+0000","last_became_peered":"2024-09-21T03:43:08.014191+0000","last_unstale":"2024-09-21T03:45:23.734618+0000","last_undegraded":"2024-09-21T03:45:23.734618+0000","last_fullsized":"2024-09-21T03:45:23.734618+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:19.114838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766945+0000","last_change":"2024-09-21T03:43:04.973714+0000","last_active":"2024-09-21T03:44:54.766945+0000","last_peered":"2024-09-21T03:44:54.766945+0000","last_clean":"2024-09-21T03:44:54.766945+0000","last_became_active":"2024-09-21T03:43:04.973348+0000","last_became_peered":"2024-09-21T03:43:04.973348+0000","last_unstale":"2024-09-21T03:44:54.766945+0000","last_undegraded":"2024-09-21T03:44:54.766945+0000","last_fullsized":"2024-09-21T03:44:54.766945+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:00.872589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766379+0000","last_change":"2024-09-21T03:43:18.082643+0000","last_active":"2024-09-21T03:44:54.766379+0000","last_peered":"2024-09-21T03:44:54.766379+0000","last_clean":"2024-09-21T03:44:54.766379+0000","last_became_active":"2024-09-21T03:43:18.082157+0000","last_became_peered":"2024-09-21T03:43:18.082157+0000","last_unstale":"2024-09-21T03:44:54.766379+0000","last_undegraded":"2024-09-21T03:44:54.766379+0000","last_fullsized":"2024-09-21T03:44:54.766379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:52.652014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.725796+0000","last_change":"2024-09-21T03:43:08.010913+0000","last_active":"2024-09-21T03:45:23.725796+0000","last_peered":"2024-09-21T03:45:23.725796+0000","last_clean":"2024-09-21T03:45:23.725796+0000","last_became_active":"2024-09-21T03:43:08.010483+0000","last_became_peered":"2024-09-21T03:43:08.010483+0000","last_unstale":"2024-09-21T03:45:23.725796+0000","last_undegraded":"2024-09-21T03:45:23.725796+0000","last_fullsized":"2024-09-21T03:45:23.725796+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:52.158346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675973+0000","last_change":"2024-09-21T03:43:04.980260+0000","last_active":"2024-09-21T03:44:54.675973+0000","last_peered":"2024-09-21T03:44:54.675973+0000","last_clean":"2024-09-21T03:44:54.675973+0000","last_became_active":"2024-09-21T03:43:04.978999+0000","last_became_peered":"2024-09-21T03:43:04.978999+0000","last_unstale":"2024-09-21T03:44:54.675973+0000","last_undegraded":"2024-09-21T03:44:54.675973+0000","last_fullsized":"2024-09-21T03:44:54.675973+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:33.682591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766371+0000","last_change":"2024-09-21T03:43:18.083665+0000","last_active":"2024-09-21T03:44:54.766371+0000","last_peered":"2024-09-21T03:44:54.766371+0000","last_clean":"2024-09-21T03:44:54.766371+0000","last_became_active":"2024-09-21T03:43:18.083215+0000","last_became_peered":"2024-09-21T03:43:18.083215+0000","last_unstale":"2024-09-21T03:44:54.766371+0000","last_undegraded":"2024-09-21T03:44:54.766371+0000","last_fullsized":"2024-09-21T03:44:54.766371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:55.968968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.739802+0000","last_change":"2024-09-21T03:43:08.010570+0000","last_active":"2024-09-21T03:45:23.739802+0000","last_peered":"2024-09-21T03:45:23.739802+0000","last_clean":"2024-09-21T03:45:23.739802+0000","last_became_active":"2024-09-21T03:43:08.010163+0000","last_became_peered":"2024-09-21T03:43:08.010163+0000","last_unstale":"2024-09-21T03:45:23.739802+0000","last_undegraded":"2024-09-21T03:45:23.739802+0000","last_fullsized":"2024-09-21T03:45:23.739802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:17.368534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.707142+0000","last_change":"2024-09-21T03:44:54.706609+0000","last_active":"2024-09-21T03:44:54.707142+0000","last_peered":"2024-09-21T03:44:54.707142+0000","last_clean":"2024-09-21T03:44:54.707142+0000","last_became_active":"2024-09-21T03:44:54.681060+0000","last_became_peered":"2024-09-21T03:44:54.681060+0000","last_unstale":"2024-09-21T03:44:54.707142+0000","last_undegraded":"2024-09-21T03:44:54.707142+0000","last_fullsized":"2024-09-21T03:44:54.707142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:49.797107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766945+0000","last_change":"2024-09-21T03:43:18.089516+0000","last_active":"2024-09-21T03:44:54.766945+0000","last_peered":"2024-09-21T03:44:54.766945+0000","last_clean":"2024-09-21T03:44:54.766945+0000","last_became_active":"2024-09-21T03:43:18.089097+0000","last_became_peered":"2024-09-21T03:43:18.089097+0000","last_unstale":"2024-09-21T03:44:54.766945+0000","last_undegraded":"2024-09-21T03:44:54.766945+0000","last_fullsized":"2024-09-21T03:44:54.766945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:50.428401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.732694+0000","last_change":"2024-09-21T03:43:08.015762+0000","last_active":"2024-09-21T03:45:23.732694+0000","last_peered":"2024-09-21T03:45:23.732694+0000","last_clean":"2024-09-21T03:45:23.732694+0000","last_became_active":"2024-09-21T03:43:08.015389+0000","last_became_peered":"2024-09-21T03:43:08.015389+0000","last_unstale":"2024-09-21T03:45:23.732694+0000","last_undegraded":"2024-09-21T03:45:23.732694+0000","last_fullsized":"2024-09-21T03:45:23.732694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:00.341479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'5","reported_seq":182,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:13.751560+0000","last_change":"2024-09-21T03:43:04.973173+0000","last_active":"2024-09-21T03:46:13.751560+0000","last_peered":"2024-09-21T03:46:13.751560+0000","last_clean":"2024-09-21T03:46:13.751560+0000","last_became_active":"2024-09-21T03:43:04.972740+0000","last_became_peered":"2024-09-21T03:43:04.972740+0000","last_unstale":"2024-09-21T03:46:13.751560+0000","last_undegraded":"2024-09-21T03:46:13.751560+0000","last_fullsized":"2024-09-21T03:46:13.751560+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:26.329220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670519+0000","last_change":"2024-09-21T03:43:18.093730+0000","last_active":"2024-09-21T03:44:54.670519+0000","last_peered":"2024-09-21T03:44:54.670519+0000","last_clean":"2024-09-21T03:44:54.670519+0000","last_became_active":"2024-09-21T03:43:18.089621+0000","last_became_peered":"2024-09-21T03:43:18.089621+0000","last_unstale":"2024-09-21T03:44:54.670519+0000","last_undegraded":"2024-09-21T03:44:54.670519+0000","last_fullsized":"2024-09-21T03:44:54.670519+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:05.227038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":62,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.745319+0000","last_change":"2024-09-21T03:43:54.465751+0000","last_active":"2024-09-21T03:45:23.745319+0000","last_peered":"2024-09-21T03:45:23.745319+0000","last_clean":"2024-09-21T03:45:23.745319+0000","last_became_active":"2024-09-21T03:43:54.465333+0000","last_became_peered":"2024-09-21T03:43:54.465333+0000","last_unstale":"2024-09-21T03:45:23.745319+0000","last_undegraded":"2024-09-21T03:45:23.745319+0000","last_fullsized":"2024-09-21T03:45:23.745319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:34.307699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766735+0000","last_change":"2024-09-21T03:43:04.979385+0000","last_active":"2024-09-21T03:44:54.766735+0000","last_peered":"2024-09-21T03:44:54.766735+0000","last_clean":"2024-09-21T03:44:54.766735+0000","last_became_active":"2024-09-21T03:43:04.978764+0000","last_became_peered":"2024-09-21T03:43:04.978764+0000","last_unstale":"2024-09-21T03:44:54.766735+0000","last_undegraded":"2024-09-21T03:44:54.766735+0000","last_fullsized":"2024-09-21T03:44:54.766735+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:34.256966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677476+0000","last_change":"2024-09-21T03:43:18.095937+0000","last_active":"2024-09-21T03:44:54.677476+0000","last_peered":"2024-09-21T03:44:54.677476+0000","last_clean":"2024-09-21T03:44:54.677476+0000","last_became_active":"2024-09-21T03:43:18.095531+0000","last_became_peered":"2024-09-21T03:43:18.095531+0000","last_unstale":"2024-09-21T03:44:54.677476+0000","last_undegraded":"2024-09-21T03:44:54.677476+0000","last_fullsized":"2024-09-21T03:44:54.677476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:18.089661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.742877+0000","last_change":"2024-09-21T03:43:08.011342+0000","last_active":"2024-09-21T03:45:23.742877+0000","last_peered":"2024-09-21T03:45:23.742877+0000","last_clean":"2024-09-21T03:45:23.742877+0000","last_became_active":"2024-09-21T03:43:08.010963+0000","last_became_peered":"2024-09-21T03:43:08.010963+0000","last_unstale":"2024-09-21T03:45:23.742877+0000","last_undegraded":"2024-09-21T03:45:23.742877+0000","last_fullsized":"2024-09-21T03:45:23.742877+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:05.970863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":57,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679564+0000","last_change":"2024-09-21T03:43:04.972896+0000","last_active":"2024-09-21T03:44:54.679564+0000","last_peered":"2024-09-21T03:44:54.679564+0000","last_clean":"2024-09-21T03:44:54.679564+0000","last_became_active":"2024-09-21T03:43:04.972535+0000","last_became_peered":"2024-09-21T03:43:04.972535+0000","last_unstale":"2024-09-21T03:44:54.679564+0000","last_undegraded":"2024-09-21T03:44:54.679564+0000","last_fullsized":"2024-09-21T03:44:54.679564+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:36.362019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766929+0000","last_change":"2024-09-21T03:43:18.081300+0000","last_active":"2024-09-21T03:44:54.766929+0000","last_peered":"2024-09-21T03:44:54.766929+0000","last_clean":"2024-09-21T03:44:54.766929+0000","last_became_active":"2024-09-21T03:43:18.080874+0000","last_became_peered":"2024-09-21T03:43:18.080874+0000","last_unstale":"2024-09-21T03:44:54.766929+0000","last_undegraded":"2024-09-21T03:44:54.766929+0000","last_fullsized":"2024-09-21T03:44:54.766929+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:13.246348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.718671+0000","last_change":"2024-09-21T03:43:08.014954+0000","last_active":"2024-09-21T03:45:23.718671+0000","last_peered":"2024-09-21T03:45:23.718671+0000","last_clean":"2024-09-21T03:45:23.718671+0000","last_became_active":"2024-09-21T03:43:08.014576+0000","last_became_peered":"2024-09-21T03:43:08.014576+0000","last_unstale":"2024-09-21T03:45:23.718671+0000","last_undegraded":"2024-09-21T03:45:23.718671+0000","last_fullsized":"2024-09-21T03:45:23.718671+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:13.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'40","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:12.749967+0000","last_change":"2024-09-21T03:43:04.973305+0000","last_active":"2024-09-21T03:46:12.749967+0000","last_peered":"2024-09-21T03:46:12.749967+0000","last_clean":"2024-09-21T03:46:12.749967+0000","last_became_active":"2024-09-21T03:43:04.972891+0000","last_became_peered":"2024-09-21T03:43:04.972891+0000","last_unstale":"2024-09-21T03:46:12.749967+0000","last_undegraded":"2024-09-21T03:46:12.749967+0000","last_fullsized":"2024-09-21T03:46:12.749967+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:15.653769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67627,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":265,"num_read_kb":199,"num_write":42,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670066+0000","last_change":"2024-09-21T03:43:18.093613+0000","last_active":"2024-09-21T03:44:54.670066+0000","last_peered":"2024-09-21T03:44:54.670066+0000","last_clean":"2024-09-21T03:44:54.670066+0000","last_became_active":"2024-09-21T03:43:18.093195+0000","last_became_peered":"2024-09-21T03:43:18.093195+0000","last_unstale":"2024-09-21T03:44:54.670066+0000","last_undegraded":"2024-09-21T03:44:54.670066+0000","last_fullsized":"2024-09-21T03:44:54.670066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:17.825071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.734749+0000","last_change":"2024-09-21T03:43:08.014314+0000","last_active":"2024-09-21T03:45:23.734749+0000","last_peered":"2024-09-21T03:45:23.734749+0000","last_clean":"2024-09-21T03:45:23.734749+0000","last_became_active":"2024-09-21T03:43:08.012925+0000","last_became_peered":"2024-09-21T03:43:08.012925+0000","last_unstale":"2024-09-21T03:45:23.734749+0000","last_undegraded":"2024-09-21T03:45:23.734749+0000","last_fullsized":"2024-09-21T03:45:23.734749+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:47:28.599444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767094+0000","last_change":"2024-09-21T03:43:05.141584+0000","last_active":"2024-09-21T03:44:54.767094+0000","last_peered":"2024-09-21T03:44:54.767094+0000","last_clean":"2024-09-21T03:44:54.767094+0000","last_became_active":"2024-09-21T03:43:05.141200+0000","last_became_peered":"2024-09-21T03:43:05.141200+0000","last_unstale":"2024-09-21T03:44:54.767094+0000","last_undegraded":"2024-09-21T03:44:54.767094+0000","last_fullsized":"2024-09-21T03:44:54.767094+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:04.357970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768559+0000","last_change":"2024-09-21T03:43:18.092184+0000","last_active":"2024-09-21T03:44:54.768559+0000","last_peered":"2024-09-21T03:44:54.768559+0000","last_clean":"2024-09-21T03:44:54.768559+0000","last_became_active":"2024-09-21T03:43:18.091629+0000","last_became_peered":"2024-09-21T03:43:18.091629+0000","last_unstale":"2024-09-21T03:44:54.768559+0000","last_undegraded":"2024-09-21T03:44:54.768559+0000","last_fullsized":"2024-09-21T03:44:54.768559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:39.448009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.748221+0000","last_change":"2024-09-21T03:43:08.018335+0000","last_active":"2024-09-21T03:45:23.748221+0000","last_peered":"2024-09-21T03:45:23.748221+0000","last_clean":"2024-09-21T03:45:23.748221+0000","last_became_active":"2024-09-21T03:43:08.014005+0000","last_became_peered":"2024-09-21T03:43:08.014005+0000","last_unstale":"2024-09-21T03:45:23.748221+0000","last_undegraded":"2024-09-21T03:45:23.748221+0000","last_fullsized":"2024-09-21T03:45:23.748221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:35.749381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678554+0000","last_change":"2024-09-21T03:43:04.978609+0000","last_active":"2024-09-21T03:44:54.678554+0000","last_peered":"2024-09-21T03:44:54.678554+0000","last_clean":"2024-09-21T03:44:54.678554+0000","last_became_active":"2024-09-21T03:43:04.978146+0000","last_became_peered":"2024-09-21T03:43:04.978146+0000","last_unstale":"2024-09-21T03:44:54.678554+0000","last_undegraded":"2024-09-21T03:44:54.678554+0000","last_fullsized":"2024-09-21T03:44:54.678554+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:41.651745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670529+0000","last_change":"2024-09-21T03:43:18.089399+0000","last_active":"2024-09-21T03:44:54.670529+0000","last_peered":"2024-09-21T03:44:54.670529+0000","last_clean":"2024-09-21T03:44:54.670529+0000","last_became_active":"2024-09-21T03:43:18.086156+0000","last_became_peered":"2024-09-21T03:43:18.086156+0000","last_unstale":"2024-09-21T03:44:54.670529+0000","last_undegraded":"2024-09-21T03:44:54.670529+0000","last_fullsized":"2024-09-21T03:44:54.670529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:41.955341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.737922+0000","last_change":"2024-09-21T03:43:08.008773+0000","last_active":"2024-09-21T03:45:23.737922+0000","last_peered":"2024-09-21T03:45:23.737922+0000","last_clean":"2024-09-21T03:45:23.737922+0000","last_became_active":"2024-09-21T03:43:08.008270+0000","last_became_peered":"2024-09-21T03:43:08.008270+0000","last_unstale":"2024-09-21T03:45:23.737922+0000","last_undegraded":"2024-09-21T03:45:23.737922+0000","last_fullsized":"2024-09-21T03:45:23.737922+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:57.901701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'5","reported_seq":305,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:12.749481+0000","last_change":"2024-09-21T03:43:04.978296+0000","last_active":"2024-09-21T03:46:12.749481+0000","last_peered":"2024-09-21T03:46:12.749481+0000","last_clean":"2024-09-21T03:46:12.749481+0000","last_became_active":"2024-09-21T03:43:04.977782+0000","last_became_peered":"2024-09-21T03:43:04.977782+0000","last_unstale":"2024-09-21T03:46:12.749481+0000","last_undegraded":"2024-09-21T03:46:12.749481+0000","last_fullsized":"2024-09-21T03:46:12.749481+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:57.748737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":254,"num_read_kb":254,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671551+0000","last_change":"2024-09-21T03:43:54.495536+0000","last_active":"2024-09-21T03:44:54.671551+0000","last_peered":"2024-09-21T03:44:54.671551+0000","last_clean":"2024-09-21T03:44:54.671551+0000","last_became_active":"2024-09-21T03:43:54.493492+0000","last_became_peered":"2024-09-21T03:43:54.493492+0000","last_unstale":"2024-09-21T03:44:54.671551+0000","last_undegraded":"2024-09-21T03:44:54.671551+0000","last_fullsized":"2024-09-21T03:44:54.671551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:11.291911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679588+0000","last_change":"2024-09-21T03:43:18.077741+0000","last_active":"2024-09-21T03:44:54.679588+0000","last_peered":"2024-09-21T03:44:54.679588+0000","last_clean":"2024-09-21T03:44:54.679588+0000","last_became_active":"2024-09-21T03:43:18.076619+0000","last_became_peered":"2024-09-21T03:43:18.076619+0000","last_unstale":"2024-09-21T03:44:54.679588+0000","last_undegraded":"2024-09-21T03:44:54.679588+0000","last_fullsized":"2024-09-21T03:44:54.679588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:33.297020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.729664+0000","last_change":"2024-09-21T03:43:08.008838+0000","last_active":"2024-09-21T03:45:23.729664+0000","last_peered":"2024-09-21T03:45:23.729664+0000","last_clean":"2024-09-21T03:45:23.729664+0000","last_became_active":"2024-09-21T03:43:08.008422+0000","last_became_peered":"2024-09-21T03:43:08.008422+0000","last_unstale":"2024-09-21T03:45:23.729664+0000","last_undegraded":"2024-09-21T03:45:23.729664+0000","last_fullsized":"2024-09-21T03:45:23.729664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:00:17.881443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766641+0000","last_change":"2024-09-21T03:43:04.979289+0000","last_active":"2024-09-21T03:44:54.766641+0000","last_peered":"2024-09-21T03:44:54.766641+0000","last_clean":"2024-09-21T03:44:54.766641+0000","last_became_active":"2024-09-21T03:43:04.978843+0000","last_became_peered":"2024-09-21T03:43:04.978843+0000","last_unstale":"2024-09-21T03:44:54.766641+0000","last_undegraded":"2024-09-21T03:44:54.766641+0000","last_fullsized":"2024-09-21T03:44:54.766641+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:50.560607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677251+0000","last_change":"2024-09-21T03:43:18.096870+0000","last_active":"2024-09-21T03:44:54.677251+0000","last_peered":"2024-09-21T03:44:54.677251+0000","last_clean":"2024-09-21T03:44:54.677251+0000","last_became_active":"2024-09-21T03:43:18.096424+0000","last_became_peered":"2024-09-21T03:43:18.096424+0000","last_unstale":"2024-09-21T03:44:54.677251+0000","last_undegraded":"2024-09-21T03:44:54.677251+0000","last_fullsized":"2024-09-21T03:44:54.677251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:39.532368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.729991+0000","last_change":"2024-09-21T03:43:54.492499+0000","last_active":"2024-09-21T03:45:23.729991+0000","last_peered":"2024-09-21T03:45:23.729991+0000","last_clean":"2024-09-21T03:45:23.729991+0000","last_became_active":"2024-09-21T03:43:54.492043+0000","last_became_peered":"2024-09-21T03:43:54.492043+0000","last_unstale":"2024-09-21T03:45:23.729991+0000","last_undegraded":"2024-09-21T03:45:23.729991+0000","last_fullsized":"2024-09-21T03:45:23.729991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:25.687090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766447+0000","last_change":"2024-09-21T03:43:04.974265+0000","last_active":"2024-09-21T03:44:54.766447+0000","last_peered":"2024-09-21T03:44:54.766447+0000","last_clean":"2024-09-21T03:44:54.766447+0000","last_became_active":"2024-09-21T03:43:04.973383+0000","last_became_peered":"2024-09-21T03:43:04.973383+0000","last_unstale":"2024-09-21T03:44:54.766447+0000","last_undegraded":"2024-09-21T03:44:54.766447+0000","last_fullsized":"2024-09-21T03:44:54.766447+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:02.312378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768086+0000","last_change":"2024-09-21T03:43:04.980353+0000","last_active":"2024-09-21T03:44:54.768086+0000","last_peered":"2024-09-21T03:44:54.768086+0000","last_clean":"2024-09-21T03:44:54.768086+0000","last_became_active":"2024-09-21T03:43:04.979922+0000","last_became_peered":"2024-09-21T03:43:04.979922+0000","last_unstale":"2024-09-21T03:44:54.768086+0000","last_undegraded":"2024-09-21T03:44:54.768086+0000","last_fullsized":"2024-09-21T03:44:54.768086+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:17.636536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.740065+0000","last_change":"2024-09-21T03:43:08.009225+0000","last_active":"2024-09-21T03:45:23.740065+0000","last_peered":"2024-09-21T03:45:23.740065+0000","last_clean":"2024-09-21T03:45:23.740065+0000","last_became_active":"2024-09-21T03:43:08.008830+0000","last_became_peered":"2024-09-21T03:43:08.008830+0000","last_unstale":"2024-09-21T03:45:23.740065+0000","last_undegraded":"2024-09-21T03:45:23.740065+0000","last_fullsized":"2024-09-21T03:45:23.740065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:38.047763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679550+0000","last_change":"2024-09-21T03:43:18.089302+0000","last_active":"2024-09-21T03:44:54.679550+0000","last_peered":"2024-09-21T03:44:54.679550+0000","last_clean":"2024-09-21T03:44:54.679550+0000","last_became_active":"2024-09-21T03:43:18.088867+0000","last_became_peered":"2024-09-21T03:43:18.088867+0000","last_unstale":"2024-09-21T03:44:54.679550+0000","last_undegraded":"2024-09-21T03:44:54.679550+0000","last_fullsized":"2024-09-21T03:44:54.679550+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:08.960044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":188,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:12.750217+0000","last_change":"2024-09-21T03:43:54.464317+0000","last_active":"2024-09-21T03:46:12.750217+0000","last_peered":"2024-09-21T03:46:12.750217+0000","last_clean":"2024-09-21T03:46:12.750217+0000","last_became_active":"2024-09-21T03:43:54.463870+0000","last_became_peered":"2024-09-21T03:43:54.463870+0000","last_unstale":"2024-09-21T03:46:12.750217+0000","last_undegraded":"2024-09-21T03:46:12.750217+0000","last_fullsized":"2024-09-21T03:46:12.750217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:14.581726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.727426+0000","last_change":"2024-09-21T03:43:08.009398+0000","last_active":"2024-09-21T03:45:23.727426+0000","last_peered":"2024-09-21T03:45:23.727426+0000","last_clean":"2024-09-21T03:45:23.727426+0000","last_became_active":"2024-09-21T03:43:08.009000+0000","last_became_peered":"2024-09-21T03:43:08.009000+0000","last_unstale":"2024-09-21T03:45:23.727426+0000","last_undegraded":"2024-09-21T03:45:23.727426+0000","last_fullsized":"2024-09-21T03:45:23.727426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:07.278613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679894+0000","last_change":"2024-09-21T03:43:18.081834+0000","last_active":"2024-09-21T03:44:54.679894+0000","last_peered":"2024-09-21T03:44:54.679894+0000","last_clean":"2024-09-21T03:44:54.679894+0000","last_became_active":"2024-09-21T03:43:18.081328+0000","last_became_peered":"2024-09-21T03:43:18.081328+0000","last_unstale":"2024-09-21T03:44:54.679894+0000","last_undegraded":"2024-09-21T03:44:54.679894+0000","last_fullsized":"2024-09-21T03:44:54.679894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:37.638826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.725948+0000","last_change":"2024-09-21T03:43:08.005144+0000","last_active":"2024-09-21T03:45:23.725948+0000","last_peered":"2024-09-21T03:45:23.725948+0000","last_clean":"2024-09-21T03:45:23.725948+0000","last_became_active":"2024-09-21T03:43:08.003895+0000","last_became_peered":"2024-09-21T03:43:08.003895+0000","last_unstale":"2024-09-21T03:45:23.725948+0000","last_undegraded":"2024-09-21T03:45:23.725948+0000","last_fullsized":"2024-09-21T03:45:23.725948+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:25.822315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.728904+0000","last_change":"2024-09-21T03:43:08.000218+0000","last_active":"2024-09-21T03:45:23.728904+0000","last_peered":"2024-09-21T03:45:23.728904+0000","last_clean":"2024-09-21T03:45:23.728904+0000","last_became_active":"2024-09-21T03:43:07.999840+0000","last_became_peered":"2024-09-21T03:43:07.999840+0000","last_unstale":"2024-09-21T03:45:23.728904+0000","last_undegraded":"2024-09-21T03:45:23.728904+0000","last_fullsized":"2024-09-21T03:45:23.728904+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:55.391706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.744272+0000","last_change":"2024-09-21T03:43:08.012075+0000","last_active":"2024-09-21T03:45:23.744272+0000","last_peered":"2024-09-21T03:45:23.744272+0000","last_clean":"2024-09-21T03:45:23.744272+0000","last_became_active":"2024-09-21T03:43:08.011666+0000","last_became_peered":"2024-09-21T03:43:08.011666+0000","last_unstale":"2024-09-21T03:45:23.744272+0000","last_undegraded":"2024-09-21T03:45:23.744272+0000","last_fullsized":"2024-09-21T03:45:23.744272+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:42.124948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743198+0000","last_change":"2024-09-21T03:43:08.021082+0000","last_active":"2024-09-21T03:45:23.743198+0000","last_peered":"2024-09-21T03:45:23.743198+0000","last_clean":"2024-09-21T03:45:23.743198+0000","last_became_active":"2024-09-21T03:43:08.020674+0000","last_became_peered":"2024-09-21T03:43:08.020674+0000","last_unstale":"2024-09-21T03:45:23.743198+0000","last_undegraded":"2024-09-21T03:45:23.743198+0000","last_fullsized":"2024-09-21T03:45:23.743198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:13.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.744549+0000","last_change":"2024-09-21T03:43:08.004199+0000","last_active":"2024-09-21T03:45:23.744549+0000","last_peered":"2024-09-21T03:45:23.744549+0000","last_clean":"2024-09-21T03:45:23.744549+0000","last_became_active":"2024-09-21T03:43:08.003806+0000","last_became_peered":"2024-09-21T03:43:08.003806+0000","last_unstale":"2024-09-21T03:45:23.744549+0000","last_undegraded":"2024-09-21T03:45:23.744549+0000","last_fullsized":"2024-09-21T03:45:23.744549+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:13.941130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.723624+0000","last_change":"2024-09-21T03:43:08.007237+0000","last_active":"2024-09-21T03:45:23.723624+0000","last_peered":"2024-09-21T03:45:23.723624+0000","last_clean":"2024-09-21T03:45:23.723624+0000","last_became_active":"2024-09-21T03:43:08.006805+0000","last_became_peered":"2024-09-21T03:43:08.006805+0000","last_unstale":"2024-09-21T03:45:23.723624+0000","last_undegraded":"2024-09-21T03:45:23.723624+0000","last_fullsized":"2024-09-21T03:45:23.723624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:26.274494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.729741+0000","last_change":"2024-09-21T03:43:08.007934+0000","last_active":"2024-09-21T03:45:23.729741+0000","last_peered":"2024-09-21T03:45:23.729741+0000","last_clean":"2024-09-21T03:45:23.729741+0000","last_became_active":"2024-09-21T03:43:08.007549+0000","last_became_peered":"2024-09-21T03:43:08.007549+0000","last_unstale":"2024-09-21T03:45:23.729741+0000","last_undegraded":"2024-09-21T03:45:23.729741+0000","last_fullsized":"2024-09-21T03:45:23.729741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:58.158255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.714180+0000","last_change":"2024-09-21T03:43:54.493665+0000","last_active":"2024-09-21T03:45:23.714180+0000","last_peered":"2024-09-21T03:45:23.714180+0000","last_clean":"2024-09-21T03:45:23.714180+0000","last_became_active":"2024-09-21T03:43:54.493245+0000","last_became_peered":"2024-09-21T03:43:54.493245+0000","last_unstale":"2024-09-21T03:45:23.714180+0000","last_undegraded":"2024-09-21T03:45:23.714180+0000","last_fullsized":"2024-09-21T03:45:23.714180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:35.449852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.731123+0000","last_change":"2024-09-21T03:43:54.463419+0000","last_active":"2024-09-21T03:45:23.731123+0000","last_peered":"2024-09-21T03:45:23.731123+0000","last_clean":"2024-09-21T03:45:23.731123+0000","last_became_active":"2024-09-21T03:43:54.462951+0000","last_became_peered":"2024-09-21T03:43:54.462951+0000","last_unstale":"2024-09-21T03:45:23.731123+0000","last_undegraded":"2024-09-21T03:45:23.731123+0000","last_fullsized":"2024-09-21T03:45:23.731123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:23.620275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.717641+0000","last_change":"2024-09-21T03:43:08.011436+0000","last_active":"2024-09-21T03:45:23.717641+0000","last_peered":"2024-09-21T03:45:23.717641+0000","last_clean":"2024-09-21T03:45:23.717641+0000","last_became_active":"2024-09-21T03:43:08.011025+0000","last_became_peered":"2024-09-21T03:43:08.011025+0000","last_unstale":"2024-09-21T03:45:23.717641+0000","last_undegraded":"2024-09-21T03:45:23.717641+0000","last_fullsized":"2024-09-21T03:45:23.717641+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:31.411858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743351+0000","last_change":"2024-09-21T03:43:08.016378+0000","last_active":"2024-09-21T03:45:23.743351+0000","last_peered":"2024-09-21T03:45:23.743351+0000","last_clean":"2024-09-21T03:45:23.743351+0000","last_became_active":"2024-09-21T03:43:08.016013+0000","last_became_peered":"2024-09-21T03:43:08.016013+0000","last_unstale":"2024-09-21T03:45:23.743351+0000","last_undegraded":"2024-09-21T03:45:23.743351+0000","last_fullsized":"2024-09-21T03:45:23.743351+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:01.865669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743940+0000","last_change":"2024-09-21T03:43:08.007862+0000","last_active":"2024-09-21T03:45:23.743940+0000","last_peered":"2024-09-21T03:45:23.743940+0000","last_clean":"2024-09-21T03:45:23.743940+0000","last_became_active":"2024-09-21T03:43:08.007433+0000","last_became_peered":"2024-09-21T03:43:08.007433+0000","last_unstale":"2024-09-21T03:45:23.743940+0000","last_undegraded":"2024-09-21T03:45:23.743940+0000","last_fullsized":"2024-09-21T03:45:23.743940+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:22:03.364767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.723330+0000","last_change":"2024-09-21T03:43:08.010542+0000","last_active":"2024-09-21T03:45:23.723330+0000","last_peered":"2024-09-21T03:45:23.723330+0000","last_clean":"2024-09-21T03:45:23.723330+0000","last_became_active":"2024-09-21T03:43:08.010195+0000","last_became_peered":"2024-09-21T03:43:08.010195+0000","last_unstale":"2024-09-21T03:45:23.723330+0000","last_undegraded":"2024-09-21T03:45:23.723330+0000","last_fullsized":"2024-09-21T03:45:23.723330+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:52.850657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.728086+0000","last_change":"2024-09-21T03:43:08.006168+0000","last_active":"2024-09-21T03:45:23.728086+0000","last_peered":"2024-09-21T03:45:23.728086+0000","last_clean":"2024-09-21T03:45:23.728086+0000","last_became_active":"2024-09-21T03:43:08.005283+0000","last_became_peered":"2024-09-21T03:43:08.005283+0000","last_unstale":"2024-09-21T03:45:23.728086+0000","last_undegraded":"2024-09-21T03:45:23.728086+0000","last_fullsized":"2024-09-21T03:45:23.728086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:59.852412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.731547+0000","last_change":"2024-09-21T03:43:08.005870+0000","last_active":"2024-09-21T03:45:23.731547+0000","last_peered":"2024-09-21T03:45:23.731547+0000","last_clean":"2024-09-21T03:45:23.731547+0000","last_became_active":"2024-09-21T03:43:08.005485+0000","last_became_peered":"2024-09-21T03:43:08.005485+0000","last_unstale":"2024-09-21T03:45:23.731547+0000","last_undegraded":"2024-09-21T03:45:23.731547+0000","last_fullsized":"2024-09-21T03:45:23.731547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:41.387865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.741258+0000","last_change":"2024-09-21T03:43:08.015176+0000","last_active":"2024-09-21T03:45:23.741258+0000","last_peered":"2024-09-21T03:45:23.741258+0000","last_clean":"2024-09-21T03:45:23.741258+0000","last_became_active":"2024-09-21T03:43:08.014451+0000","last_became_peered":"2024-09-21T03:43:08.014451+0000","last_unstale":"2024-09-21T03:45:23.741258+0000","last_undegraded":"2024-09-21T03:45:23.741258+0000","last_fullsized":"2024-09-21T03:45:23.741258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:49.524800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.747012+0000","last_change":"2024-09-21T03:43:08.016798+0000","last_active":"2024-09-21T03:45:23.747012+0000","last_peered":"2024-09-21T03:45:23.747012+0000","last_clean":"2024-09-21T03:45:23.747012+0000","last_became_active":"2024-09-21T03:43:08.014842+0000","last_became_peered":"2024-09-21T03:43:08.014842+0000","last_unstale":"2024-09-21T03:45:23.747012+0000","last_undegraded":"2024-09-21T03:45:23.747012+0000","last_fullsized":"2024-09-21T03:45:23.747012+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:19.290618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.746263+0000","last_change":"2024-09-21T03:43:08.021285+0000","last_active":"2024-09-21T03:45:23.746263+0000","last_peered":"2024-09-21T03:45:23.746263+0000","last_clean":"2024-09-21T03:45:23.746263+0000","last_became_active":"2024-09-21T03:43:08.020904+0000","last_became_peered":"2024-09-21T03:43:08.020904+0000","last_unstale":"2024-09-21T03:45:23.746263+0000","last_undegraded":"2024-09-21T03:45:23.746263+0000","last_fullsized":"2024-09-21T03:45:23.746263+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:56.589089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.745437+0000","last_change":"2024-09-21T03:43:08.017599+0000","last_active":"2024-09-21T03:45:23.745437+0000","last_peered":"2024-09-21T03:45:23.745437+0000","last_clean":"2024-09-21T03:45:23.745437+0000","last_became_active":"2024-09-21T03:43:08.016138+0000","last_became_peered":"2024-09-21T03:43:08.016138+0000","last_unstale":"2024-09-21T03:45:23.745437+0000","last_undegraded":"2024-09-21T03:45:23.745437+0000","last_fullsized":"2024-09-21T03:45:23.745437+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:09.902246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.732277+0000","last_change":"2024-09-21T03:43:08.009598+0000","last_active":"2024-09-21T03:45:23.732277+0000","last_peered":"2024-09-21T03:45:23.732277+0000","last_clean":"2024-09-21T03:45:23.732277+0000","last_became_active":"2024-09-21T03:43:08.007488+0000","last_became_peered":"2024-09-21T03:43:08.007488+0000","last_unstale":"2024-09-21T03:45:23.732277+0000","last_undegraded":"2024-09-21T03:45:23.732277+0000","last_fullsized":"2024-09-21T03:45:23.732277+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:23.192761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.732050+0000","last_change":"2024-09-21T03:43:08.016450+0000","last_active":"2024-09-21T03:45:23.732050+0000","last_peered":"2024-09-21T03:45:23.732050+0000","last_clean":"2024-09-21T03:45:23.732050+0000","last_became_active":"2024-09-21T03:43:08.016037+0000","last_became_peered":"2024-09-21T03:43:08.016037+0000","last_unstale":"2024-09-21T03:45:23.732050+0000","last_undegraded":"2024-09-21T03:45:23.732050+0000","last_fullsized":"2024-09-21T03:45:23.732050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:39.109408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.726221+0000","last_change":"2024-09-21T03:43:08.010298+0000","last_active":"2024-09-21T03:45:23.726221+0000","last_peered":"2024-09-21T03:45:23.726221+0000","last_clean":"2024-09-21T03:45:23.726221+0000","last_became_active":"2024-09-21T03:43:08.009847+0000","last_became_peered":"2024-09-21T03:43:08.009847+0000","last_unstale":"2024-09-21T03:45:23.726221+0000","last_undegraded":"2024-09-21T03:45:23.726221+0000","last_fullsized":"2024-09-21T03:45:23.726221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:37.463456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.736715+0000","last_change":"2024-09-21T03:43:08.017834+0000","last_active":"2024-09-21T03:45:23.736715+0000","last_peered":"2024-09-21T03:45:23.736715+0000","last_clean":"2024-09-21T03:45:23.736715+0000","last_became_active":"2024-09-21T03:43:08.016910+0000","last_became_peered":"2024-09-21T03:43:08.016910+0000","last_unstale":"2024-09-21T03:45:23.736715+0000","last_undegraded":"2024-09-21T03:45:23.736715+0000","last_fullsized":"2024-09-21T03:45:23.736715+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:50.528651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.737971+0000","last_change":"2024-09-21T03:43:08.013945+0000","last_active":"2024-09-21T03:45:23.737971+0000","last_peered":"2024-09-21T03:45:23.737971+0000","last_clean":"2024-09-21T03:45:23.737971+0000","last_became_active":"2024-09-21T03:43:08.013598+0000","last_became_peered":"2024-09-21T03:43:08.013598+0000","last_unstale":"2024-09-21T03:45:23.737971+0000","last_undegraded":"2024-09-21T03:45:23.737971+0000","last_fullsized":"2024-09-21T03:45:23.737971+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:58.182938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.726374+0000","last_change":"2024-09-21T03:43:08.018824+0000","last_active":"2024-09-21T03:45:23.726374+0000","last_peered":"2024-09-21T03:45:23.726374+0000","last_clean":"2024-09-21T03:45:23.726374+0000","last_became_active":"2024-09-21T03:43:08.017196+0000","last_became_peered":"2024-09-21T03:43:08.017196+0000","last_unstale":"2024-09-21T03:45:23.726374+0000","last_undegraded":"2024-09-21T03:45:23.726374+0000","last_fullsized":"2024-09-21T03:45:23.726374+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:02.357523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743021+0000","last_change":"2024-09-21T03:43:08.003514+0000","last_active":"2024-09-21T03:45:23.743021+0000","last_peered":"2024-09-21T03:45:23.743021+0000","last_clean":"2024-09-21T03:45:23.743021+0000","last_became_active":"2024-09-21T03:43:08.003131+0000","last_became_peered":"2024-09-21T03:43:08.003131+0000","last_unstale":"2024-09-21T03:45:23.743021+0000","last_undegraded":"2024-09-21T03:45:23.743021+0000","last_fullsized":"2024-09-21T03:45:23.743021+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:23.170302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.735332+0000","last_change":"2024-09-21T03:43:08.015890+0000","last_active":"2024-09-21T03:45:23.735332+0000","last_peered":"2024-09-21T03:45:23.735332+0000","last_clean":"2024-09-21T03:45:23.735332+0000","last_became_active":"2024-09-21T03:43:08.015519+0000","last_became_peered":"2024-09-21T03:43:08.015519+0000","last_unstale":"2024-09-21T03:45:23.735332+0000","last_undegraded":"2024-09-21T03:45:23.735332+0000","last_fullsized":"2024-09-21T03:45:23.735332+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:10.875923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.727227+0000","last_change":"2024-09-21T03:43:08.012261+0000","last_active":"2024-09-21T03:45:23.727227+0000","last_peered":"2024-09-21T03:45:23.727227+0000","last_clean":"2024-09-21T03:45:23.727227+0000","last_became_active":"2024-09-21T03:43:08.011783+0000","last_became_peered":"2024-09-21T03:43:08.011783+0000","last_unstale":"2024-09-21T03:45:23.727227+0000","last_undegraded":"2024-09-21T03:45:23.727227+0000","last_fullsized":"2024-09-21T03:45:23.727227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:07.011340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.724685+0000","last_change":"2024-09-21T03:43:54.494826+0000","last_active":"2024-09-21T03:45:23.724685+0000","last_peered":"2024-09-21T03:45:23.724685+0000","last_clean":"2024-09-21T03:45:23.724685+0000","last_became_active":"2024-09-21T03:43:54.494404+0000","last_became_peered":"2024-09-21T03:43:54.494404+0000","last_unstale":"2024-09-21T03:45:23.724685+0000","last_undegraded":"2024-09-21T03:45:23.724685+0000","last_fullsized":"2024-09-21T03:45:23.724685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:42.109053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.740074+0000","last_change":"2024-09-21T03:43:08.016214+0000","last_active":"2024-09-21T03:45:23.740074+0000","last_peered":"2024-09-21T03:45:23.740074+0000","last_clean":"2024-09-21T03:45:23.740074+0000","last_became_active":"2024-09-21T03:43:08.014815+0000","last_became_peered":"2024-09-21T03:43:08.014815+0000","last_unstale":"2024-09-21T03:45:23.740074+0000","last_undegraded":"2024-09-21T03:45:23.740074+0000","last_fullsized":"2024-09-21T03:45:23.740074+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:04.742829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.745698+0000","last_change":"2024-09-21T03:43:08.019269+0000","last_active":"2024-09-21T03:45:23.745698+0000","last_peered":"2024-09-21T03:45:23.745698+0000","last_clean":"2024-09-21T03:45:23.745698+0000","last_became_active":"2024-09-21T03:43:08.017962+0000","last_became_peered":"2024-09-21T03:43:08.017962+0000","last_unstale":"2024-09-21T03:45:23.745698+0000","last_undegraded":"2024-09-21T03:45:23.745698+0000","last_fullsized":"2024-09-21T03:45:23.745698+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:05.950697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.738799+0000","last_change":"2024-09-21T03:43:08.014790+0000","last_active":"2024-09-21T03:45:23.738799+0000","last_peered":"2024-09-21T03:45:23.738799+0000","last_clean":"2024-09-21T03:45:23.738799+0000","last_became_active":"2024-09-21T03:43:08.013891+0000","last_became_peered":"2024-09-21T03:43:08.013891+0000","last_unstale":"2024-09-21T03:45:23.738799+0000","last_undegraded":"2024-09-21T03:45:23.738799+0000","last_fullsized":"2024-09-21T03:45:23.738799+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:46.045675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74892,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1311,"num_read_kb":734,"num_write":259,"num_write_kb":153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":223743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":501273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972410,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28688,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716464,"statfs":{"total":95995035648,"available":95965659136,"internally_reserved":0,"allocated":1703936,"data_stored":1158857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135934,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":500559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332165,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":500825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495691,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36924,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708228,"statfs":{"total":95995035648,"available":95957225472,"internally_reserved":0,"allocated":1617920,"data_stored":1097354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659216,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":496374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855446,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36932,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708220,"statfs":{"total":95995035648,"available":95957217280,"internally_reserved":0,"allocated":1626112,"data_stored":1092625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018972,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":500759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215200,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":495771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444135,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":564539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":564409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":498051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:46:16.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:46:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:18.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.024+0000 7f3e41000640 1 -- 172.21.15.16:0/1277743654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e3c1039b0 msgr2=0x7f3e3c103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:18.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.024+0000 7f3e41000640 1 --2- 172.21.15.16:0/1277743654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e3c1039b0 0x7f3e3c103dc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009900 tx=0x7f3e2c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.024+0000 7f3e41000640 1 -- 172.21.15.16:0/1277743654 shutdown_connections 2024-09-21T03:46:18.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.024+0000 7f3e41000640 1 --2- 172.21.15.16:0/1277743654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e3c109c40 0x7f3e3c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.024+0000 7f3e41000640 1 --2- 172.21.15.16:0/1277743654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e3c1092f0 0x7f3e3c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.024+0000 7f3e41000640 1 --2- 172.21.15.16:0/1277743654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e3c1039b0 0x7f3e3c103dc0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.024+0000 7f3e41000640 1 -- 172.21.15.16:0/1277743654 >> 172.21.15.16:0/1277743654 conn(0x7f3e3c0ff400 msgr2=0x7f3e3c101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:18.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.024+0000 7f3e41000640 1 -- 172.21.15.16:0/1277743654 shutdown_connections 2024-09-21T03:46:18.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.025+0000 7f3e41000640 1 -- 172.21.15.16:0/1277743654 wait complete. 2024-09-21T03:46:18.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.026+0000 7f3e41000640 1 Processor -- start 2024-09-21T03:46:18.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.026+0000 7f3e41000640 1 -- start start 2024-09-21T03:46:18.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.026+0000 7f3e41000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e3c1039b0 0x7f3e3c19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:18.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.026+0000 7f3e41000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e3c1092f0 0x7f3e3c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:18.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.026+0000 7f3e41000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e3c109c40 0x7f3e3c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:18.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.026+0000 7f3e41000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c113950 con 0x7f3e3c1092f0 2024-09-21T03:46:18.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.026+0000 7f3e41000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c1137d0 con 0x7f3e3c1039b0 2024-09-21T03:46:18.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.026+0000 7f3e41000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c113ad0 con 0x7f3e3c109c40 2024-09-21T03:46:18.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.027+0000 7f3e3b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e3c1039b0 0x7f3e3c19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:18.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.027+0000 7f3e3aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e3c1092f0 0x7f3e3c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:18.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.027+0000 7f3e3b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e3c1039b0 0x7f3e3c19a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37224/0 (socket says 172.21.15.16:37224) 2024-09-21T03:46:18.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.027+0000 7f3e3b400640 1 -- 172.21.15.16:0/1305076792 learned_addr learned my addr 172.21.15.16:0/1305076792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:18.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.027+0000 7f3e3be00640 1 --2- 172.21.15.16:0/1305076792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e3c109c40 0x7f3e3c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:18.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.027+0000 7f3e3b400640 1 -- 172.21.15.16:0/1305076792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e3c109c40 msgr2=0x7f3e3c19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:18.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.027+0000 7f3e3b400640 1 --2- 172.21.15.16:0/1305076792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e3c109c40 0x7f3e3c19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.027+0000 7f3e3b400640 1 -- 172.21.15.16:0/1305076792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e3c1092f0 msgr2=0x7f3e3c19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:18.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.027+0000 7f3e3b400640 1 --2- 172.21.15.16:0/1305076792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e3c1092f0 0x7f3e3c19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.027+0000 7f3e3b400640 1 -- 172.21.15.16:0/1305076792 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c1a02f0 con 0x7f3e3c1039b0 2024-09-21T03:46:18.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.027+0000 7f3e3be00640 1 --2- 172.21.15.16:0/1305076792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e3c109c40 0x7f3e3c19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:18.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.027+0000 7f3e3aa00640 1 --2- 172.21.15.16:0/1305076792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e3c1092f0 0x7f3e3c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:18.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.028+0000 7f3e3b400640 1 --2- 172.21.15.16:0/1305076792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e3c1039b0 0x7f3e3c19a380 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c005ab0 tx=0x7f3e2c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:18.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.028+0000 7f3e33e00640 1 -- 172.21.15.16:0/1305076792 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c03f070 con 0x7f3e3c1039b0 2024-09-21T03:46:18.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.028+0000 7f3e41000640 1 -- 172.21.15.16:0/1305076792 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e3c1a0580 con 0x7f3e3c1039b0 2024-09-21T03:46:18.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.028+0000 7f3e33e00640 1 -- 172.21.15.16:0/1305076792 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e2c031b60 con 0x7f3e3c1039b0 2024-09-21T03:46:18.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.028+0000 7f3e33e00640 1 -- 172.21.15.16:0/1305076792 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c035030 con 0x7f3e3c1039b0 2024-09-21T03:46:18.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.028+0000 7f3e41000640 1 -- 172.21.15.16:0/1305076792 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e3c1a6d40 con 0x7f3e3c1039b0 2024-09-21T03:46:18.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.029+0000 7f3e41000640 1 -- 172.21.15.16:0/1305076792 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e3c103dc0 con 0x7f3e3c1039b0 2024-09-21T03:46:18.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.032+0000 7f3e33e00640 1 -- 172.21.15.16:0/1305076792 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3e2c042070 con 0x7f3e3c1039b0 2024-09-21T03:46:18.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.034+0000 7f3e33e00640 1 --2- 172.21.15.16:0/1305076792 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3e200b6590 0x7f3e200b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:18.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.034+0000 7f3e33e00640 1 -- 172.21.15.16:0/1305076792 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f3e2c1083c0 con 0x7f3e3c1039b0 2024-09-21T03:46:18.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.034+0000 7f3e3aa00640 1 --2- 172.21.15.16:0/1305076792 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3e200b6590 0x7f3e200b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:18.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.035+0000 7f3e3aa00640 1 --2- 172.21.15.16:0/1305076792 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3e200b6590 0x7f3e200b8aa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c108df0 tx=0x7f3e28005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:18.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.036+0000 7f3e33e00640 1 -- 172.21.15.16:0/1305076792 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3e2c0d2c20 con 0x7f3e3c1039b0 2024-09-21T03:46:18.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.234+0000 7f3e41000640 1 -- 172.21.15.16:0/1305076792 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3e3c10ba70 con 0x7f3e200b6590 2024-09-21T03:46:18.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.240+0000 7f3e33e00640 1 -- 172.21.15.16:0/1305076792 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340527 (secure 0 0 0) 0x7f3e3c10ba70 con 0x7f3e200b6590 2024-09-21T03:46:18.245 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:46:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-21T03:46:18.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.245+0000 7f3e41000640 1 -- 172.21.15.16:0/1305076792 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3e200b6590 msgr2=0x7f3e200b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:18.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.245+0000 7f3e41000640 1 --2- 172.21.15.16:0/1305076792 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3e200b6590 0x7f3e200b8aa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c108df0 tx=0x7f3e28005f90 comp rx=0 tx=0).stop 2024-09-21T03:46:18.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.245+0000 7f3e41000640 1 -- 172.21.15.16:0/1305076792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e3c1039b0 msgr2=0x7f3e3c19a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:18.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.245+0000 7f3e41000640 1 --2- 172.21.15.16:0/1305076792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e3c1039b0 0x7f3e3c19a380 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c005ab0 tx=0x7f3e2c003c10 comp rx=0 tx=0).stop 2024-09-21T03:46:18.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.245+0000 7f3e41000640 1 -- 172.21.15.16:0/1305076792 shutdown_connections 2024-09-21T03:46:18.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.245+0000 7f3e41000640 1 --2- 172.21.15.16:0/1305076792 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3e200b6590 0x7f3e200b8aa0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.245+0000 7f3e41000640 1 --2- 172.21.15.16:0/1305076792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e3c109c40 0x7f3e3c19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.245+0000 7f3e41000640 1 --2- 172.21.15.16:0/1305076792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e3c1092f0 0x7f3e3c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.245+0000 7f3e41000640 1 --2- 172.21.15.16:0/1305076792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e3c1039b0 0x7f3e3c19a380 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:18.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.245+0000 7f3e41000640 1 -- 172.21.15.16:0/1305076792 >> 172.21.15.16:0/1305076792 conn(0x7f3e3c0ff400 msgr2=0x7f3e3c101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:18.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.245+0000 7f3e41000640 1 -- 172.21.15.16:0/1305076792 shutdown_connections 2024-09-21T03:46:18.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:18.245+0000 7f3e41000640 1 -- 172.21.15.16:0/1305076792 wait complete. 2024-09-21T03:46:18.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:18 smithi134 ceph-mon[35031]: pgmap v418: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-21T03:46:18.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:18 smithi123 ceph-mon[34673]: pgmap v418: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-21T03:46:18.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:18 smithi016 ceph-mon[30375]: pgmap v418: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-21T03:46:18.996 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-09-21T03:46:17.383996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665494,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1397,"num_read_kb":803,"num_write":389,"num_write_kb":2308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2951952,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":819,"ondisk_log_size":819,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379764,"kb_used_data":13876,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562060,"statfs":{"total":1151940427776,"available":1151551549440,"internally_reserved":0,"allocated":14209024,"data_stored":7981027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":622,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":57,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003585"},"pg_stats":[{"pgid":"2.1f","version":"86'14","reported_seq":190,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:13.750840+0000","last_change":"2024-09-21T03:43:04.980132+0000","last_active":"2024-09-21T03:46:13.750840+0000","last_peered":"2024-09-21T03:46:13.750840+0000","last_clean":"2024-09-21T03:46:13.750840+0000","last_became_active":"2024-09-21T03:43:04.979274+0000","last_became_peered":"2024-09-21T03:43:04.979274+0000","last_unstale":"2024-09-21T03:46:13.750840+0000","last_undegraded":"2024-09-21T03:46:13.750840+0000","last_fullsized":"2024-09-21T03:46:13.750840+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:35.047129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":130,"num_read_kb":130,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.727333+0000","last_change":"2024-09-21T03:43:08.011576+0000","last_active":"2024-09-21T03:45:23.727333+0000","last_peered":"2024-09-21T03:45:23.727333+0000","last_clean":"2024-09-21T03:45:23.727333+0000","last_became_active":"2024-09-21T03:43:08.011188+0000","last_became_peered":"2024-09-21T03:43:08.011188+0000","last_unstale":"2024-09-21T03:45:23.727333+0000","last_undegraded":"2024-09-21T03:45:23.727333+0000","last_fullsized":"2024-09-21T03:45:23.727333+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:34.477639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678062+0000","last_change":"2024-09-21T03:43:54.493183+0000","last_active":"2024-09-21T03:44:54.678062+0000","last_peered":"2024-09-21T03:44:54.678062+0000","last_clean":"2024-09-21T03:44:54.678062+0000","last_became_active":"2024-09-21T03:43:54.492605+0000","last_became_peered":"2024-09-21T03:43:54.492605+0000","last_unstale":"2024-09-21T03:44:54.678062+0000","last_undegraded":"2024-09-21T03:44:54.678062+0000","last_fullsized":"2024-09-21T03:44:54.678062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:26.634042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.672352+0000","last_change":"2024-09-21T03:43:05.141790+0000","last_active":"2024-09-21T03:44:54.672352+0000","last_peered":"2024-09-21T03:44:54.672352+0000","last_clean":"2024-09-21T03:44:54.672352+0000","last_became_active":"2024-09-21T03:43:05.141299+0000","last_became_peered":"2024-09-21T03:43:05.141299+0000","last_unstale":"2024-09-21T03:44:54.672352+0000","last_undegraded":"2024-09-21T03:44:54.672352+0000","last_fullsized":"2024-09-21T03:44:54.672352+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:48.844615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.747911+0000","last_change":"2024-09-21T03:43:08.018245+0000","last_active":"2024-09-21T03:45:23.747911+0000","last_peered":"2024-09-21T03:45:23.747911+0000","last_clean":"2024-09-21T03:45:23.747911+0000","last_became_active":"2024-09-21T03:43:08.015719+0000","last_became_peered":"2024-09-21T03:43:08.015719+0000","last_unstale":"2024-09-21T03:45:23.747911+0000","last_undegraded":"2024-09-21T03:45:23.747911+0000","last_fullsized":"2024-09-21T03:45:23.747911+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:39.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766982+0000","last_change":"2024-09-21T03:43:54.493279+0000","last_active":"2024-09-21T03:44:54.766982+0000","last_peered":"2024-09-21T03:44:54.766982+0000","last_clean":"2024-09-21T03:44:54.766982+0000","last_became_active":"2024-09-21T03:43:54.492592+0000","last_became_peered":"2024-09-21T03:43:54.492592+0000","last_unstale":"2024-09-21T03:44:54.766982+0000","last_undegraded":"2024-09-21T03:44:54.766982+0000","last_fullsized":"2024-09-21T03:44:54.766982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:28.520990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'13","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.941089+0000","last_change":"2024-09-21T03:44:54.693431+0000","last_active":"2024-09-21T03:44:56.941089+0000","last_peered":"2024-09-21T03:44:56.941089+0000","last_clean":"2024-09-21T03:44:56.941089+0000","last_became_active":"2024-09-21T03:44:54.680324+0000","last_became_peered":"2024-09-21T03:44:54.680324+0000","last_unstale":"2024-09-21T03:44:56.941089+0000","last_undegraded":"2024-09-21T03:44:56.941089+0000","last_fullsized":"2024-09-21T03:44:56.941089+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:43.730697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.739651+0000","last_change":"2024-09-21T03:43:08.009018+0000","last_active":"2024-09-21T03:45:23.739651+0000","last_peered":"2024-09-21T03:45:23.739651+0000","last_clean":"2024-09-21T03:45:23.739651+0000","last_became_active":"2024-09-21T03:43:08.008614+0000","last_became_peered":"2024-09-21T03:43:08.008614+0000","last_unstale":"2024-09-21T03:45:23.739651+0000","last_undegraded":"2024-09-21T03:45:23.739651+0000","last_fullsized":"2024-09-21T03:45:23.739651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:31.000616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768752+0000","last_change":"2024-09-21T03:43:18.092750+0000","last_active":"2024-09-21T03:44:54.768752+0000","last_peered":"2024-09-21T03:44:54.768752+0000","last_clean":"2024-09-21T03:44:54.768752+0000","last_became_active":"2024-09-21T03:43:18.092122+0000","last_became_peered":"2024-09-21T03:43:18.092122+0000","last_unstale":"2024-09-21T03:44:54.768752+0000","last_undegraded":"2024-09-21T03:44:54.768752+0000","last_fullsized":"2024-09-21T03:44:54.768752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:15.032388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675595+0000","last_change":"2024-09-21T03:43:04.978045+0000","last_active":"2024-09-21T03:44:54.675595+0000","last_peered":"2024-09-21T03:44:54.675595+0000","last_clean":"2024-09-21T03:44:54.675595+0000","last_became_active":"2024-09-21T03:43:04.977565+0000","last_became_peered":"2024-09-21T03:43:04.977565+0000","last_unstale":"2024-09-21T03:44:54.675595+0000","last_undegraded":"2024-09-21T03:44:54.675595+0000","last_fullsized":"2024-09-21T03:44:54.675595+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:42.537892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.740686+0000","last_change":"2024-09-21T03:43:08.007094+0000","last_active":"2024-09-21T03:45:23.740686+0000","last_peered":"2024-09-21T03:45:23.740686+0000","last_clean":"2024-09-21T03:45:23.740686+0000","last_became_active":"2024-09-21T03:43:08.006666+0000","last_became_peered":"2024-09-21T03:43:08.006666+0000","last_unstale":"2024-09-21T03:45:23.740686+0000","last_undegraded":"2024-09-21T03:45:23.740686+0000","last_fullsized":"2024-09-21T03:45:23.740686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:59.247459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767130+0000","last_change":"2024-09-21T03:43:18.081141+0000","last_active":"2024-09-21T03:44:54.767130+0000","last_peered":"2024-09-21T03:44:54.767130+0000","last_clean":"2024-09-21T03:44:54.767130+0000","last_became_active":"2024-09-21T03:43:18.080623+0000","last_became_peered":"2024-09-21T03:43:18.080623+0000","last_unstale":"2024-09-21T03:44:54.767130+0000","last_undegraded":"2024-09-21T03:44:54.767130+0000","last_fullsized":"2024-09-21T03:44:54.767130+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:44.932673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.741475+0000","last_change":"2024-09-21T03:43:08.008602+0000","last_active":"2024-09-21T03:45:23.741475+0000","last_peered":"2024-09-21T03:45:23.741475+0000","last_clean":"2024-09-21T03:45:23.741475+0000","last_became_active":"2024-09-21T03:43:08.008220+0000","last_became_peered":"2024-09-21T03:43:08.008220+0000","last_unstale":"2024-09-21T03:45:23.741475+0000","last_undegraded":"2024-09-21T03:45:23.741475+0000","last_fullsized":"2024-09-21T03:45:23.741475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:28.286016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.774443+0000","last_change":"2024-09-21T03:44:54.773918+0000","last_active":"2024-09-21T03:44:54.774443+0000","last_peered":"2024-09-21T03:44:54.774443+0000","last_clean":"2024-09-21T03:44:54.774443+0000","last_became_active":"2024-09-21T03:44:54.769721+0000","last_became_peered":"2024-09-21T03:44:54.769721+0000","last_unstale":"2024-09-21T03:44:54.774443+0000","last_undegraded":"2024-09-21T03:44:54.774443+0000","last_fullsized":"2024-09-21T03:44:54.774443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:05.656051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767195+0000","last_change":"2024-09-21T03:43:18.483523+0000","last_active":"2024-09-21T03:44:54.767195+0000","last_peered":"2024-09-21T03:44:54.767195+0000","last_clean":"2024-09-21T03:44:54.767195+0000","last_became_active":"2024-09-21T03:43:18.482833+0000","last_became_peered":"2024-09-21T03:43:18.482833+0000","last_unstale":"2024-09-21T03:44:54.767195+0000","last_undegraded":"2024-09-21T03:44:54.767195+0000","last_fullsized":"2024-09-21T03:44:54.767195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:48.236569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.736655+0000","last_change":"2024-09-21T03:43:08.022592+0000","last_active":"2024-09-21T03:45:23.736655+0000","last_peered":"2024-09-21T03:45:23.736655+0000","last_clean":"2024-09-21T03:45:23.736655+0000","last_became_active":"2024-09-21T03:43:08.018065+0000","last_became_peered":"2024-09-21T03:43:08.018065+0000","last_unstale":"2024-09-21T03:45:23.736655+0000","last_undegraded":"2024-09-21T03:45:23.736655+0000","last_fullsized":"2024-09-21T03:45:23.736655+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:41.524402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679262+0000","last_change":"2024-09-21T03:43:04.978788+0000","last_active":"2024-09-21T03:44:54.679262+0000","last_peered":"2024-09-21T03:44:54.679262+0000","last_clean":"2024-09-21T03:44:54.679262+0000","last_became_active":"2024-09-21T03:43:04.978382+0000","last_became_peered":"2024-09-21T03:43:04.978382+0000","last_unstale":"2024-09-21T03:44:54.679262+0000","last_undegraded":"2024-09-21T03:44:54.679262+0000","last_fullsized":"2024-09-21T03:44:54.679262+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:12.571217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768259+0000","last_change":"2024-09-21T03:43:54.493347+0000","last_active":"2024-09-21T03:44:54.768259+0000","last_peered":"2024-09-21T03:44:54.768259+0000","last_clean":"2024-09-21T03:44:54.768259+0000","last_became_active":"2024-09-21T03:43:54.492598+0000","last_became_peered":"2024-09-21T03:43:54.492598+0000","last_unstale":"2024-09-21T03:44:54.768259+0000","last_undegraded":"2024-09-21T03:44:54.768259+0000","last_fullsized":"2024-09-21T03:44:54.768259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:34.788296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743945+0000","last_change":"2024-09-21T03:43:08.014368+0000","last_active":"2024-09-21T03:45:23.743945+0000","last_peered":"2024-09-21T03:45:23.743945+0000","last_clean":"2024-09-21T03:45:23.743945+0000","last_became_active":"2024-09-21T03:43:08.013952+0000","last_became_peered":"2024-09-21T03:43:08.013952+0000","last_unstale":"2024-09-21T03:45:23.743945+0000","last_undegraded":"2024-09-21T03:45:23.743945+0000","last_fullsized":"2024-09-21T03:45:23.743945+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:16.893588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671998+0000","last_change":"2024-09-21T03:43:04.976162+0000","last_active":"2024-09-21T03:44:54.671998+0000","last_peered":"2024-09-21T03:44:54.671998+0000","last_clean":"2024-09-21T03:44:54.671998+0000","last_became_active":"2024-09-21T03:43:04.973790+0000","last_became_peered":"2024-09-21T03:43:04.973790+0000","last_unstale":"2024-09-21T03:44:54.671998+0000","last_undegraded":"2024-09-21T03:44:54.671998+0000","last_fullsized":"2024-09-21T03:44:54.671998+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:47.838229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670339+0000","last_change":"2024-09-21T03:43:18.084344+0000","last_active":"2024-09-21T03:44:54.670339+0000","last_peered":"2024-09-21T03:44:54.670339+0000","last_clean":"2024-09-21T03:44:54.670339+0000","last_became_active":"2024-09-21T03:43:18.083232+0000","last_became_peered":"2024-09-21T03:43:18.083232+0000","last_unstale":"2024-09-21T03:44:54.670339+0000","last_undegraded":"2024-09-21T03:44:54.670339+0000","last_fullsized":"2024-09-21T03:44:54.670339+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:46.254405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.690064+0000","last_change":"2024-09-21T03:43:08.009932+0000","last_active":"2024-09-21T03:45:23.690064+0000","last_peered":"2024-09-21T03:45:23.690064+0000","last_clean":"2024-09-21T03:45:23.690064+0000","last_became_active":"2024-09-21T03:43:08.009492+0000","last_became_peered":"2024-09-21T03:43:08.009492+0000","last_unstale":"2024-09-21T03:45:23.690064+0000","last_undegraded":"2024-09-21T03:45:23.690064+0000","last_fullsized":"2024-09-21T03:45:23.690064+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:58.679348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768057+0000","last_change":"2024-09-21T03:43:05.141927+0000","last_active":"2024-09-21T03:44:54.768057+0000","last_peered":"2024-09-21T03:44:54.768057+0000","last_clean":"2024-09-21T03:44:54.768057+0000","last_became_active":"2024-09-21T03:43:05.141516+0000","last_became_peered":"2024-09-21T03:43:05.141516+0000","last_unstale":"2024-09-21T03:44:54.768057+0000","last_undegraded":"2024-09-21T03:44:54.768057+0000","last_fullsized":"2024-09-21T03:44:54.768057+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:35.666981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767396+0000","last_change":"2024-09-21T03:43:18.484478+0000","last_active":"2024-09-21T03:44:54.767396+0000","last_peered":"2024-09-21T03:44:54.767396+0000","last_clean":"2024-09-21T03:44:54.767396+0000","last_became_active":"2024-09-21T03:43:18.483890+0000","last_became_peered":"2024-09-21T03:43:18.483890+0000","last_unstale":"2024-09-21T03:44:54.767396+0000","last_undegraded":"2024-09-21T03:44:54.767396+0000","last_fullsized":"2024-09-21T03:44:54.767396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:03.542783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.748379+0000","last_change":"2024-09-21T03:43:08.017785+0000","last_active":"2024-09-21T03:45:23.748379+0000","last_peered":"2024-09-21T03:45:23.748379+0000","last_clean":"2024-09-21T03:45:23.748379+0000","last_became_active":"2024-09-21T03:43:08.017389+0000","last_became_peered":"2024-09-21T03:43:08.017389+0000","last_unstale":"2024-09-21T03:45:23.748379+0000","last_undegraded":"2024-09-21T03:45:23.748379+0000","last_fullsized":"2024-09-21T03:45:23.748379+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:37.557061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":54,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767655+0000","last_change":"2024-09-21T03:43:05.567829+0000","last_active":"2024-09-21T03:44:54.767655+0000","last_peered":"2024-09-21T03:44:54.767655+0000","last_clean":"2024-09-21T03:44:54.767655+0000","last_became_active":"2024-09-21T03:43:05.567401+0000","last_became_peered":"2024-09-21T03:43:05.567401+0000","last_unstale":"2024-09-21T03:44:54.767655+0000","last_undegraded":"2024-09-21T03:44:54.767655+0000","last_fullsized":"2024-09-21T03:44:54.767655+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:03.954280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678873+0000","last_change":"2024-09-21T03:43:18.080064+0000","last_active":"2024-09-21T03:44:54.678873+0000","last_peered":"2024-09-21T03:44:54.678873+0000","last_clean":"2024-09-21T03:44:54.678873+0000","last_became_active":"2024-09-21T03:43:18.079569+0000","last_became_peered":"2024-09-21T03:43:18.079569+0000","last_unstale":"2024-09-21T03:44:54.678873+0000","last_undegraded":"2024-09-21T03:44:54.678873+0000","last_fullsized":"2024-09-21T03:44:54.678873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:29.568476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743198+0000","last_change":"2024-09-21T03:43:08.011603+0000","last_active":"2024-09-21T03:45:23.743198+0000","last_peered":"2024-09-21T03:45:23.743198+0000","last_clean":"2024-09-21T03:45:23.743198+0000","last_became_active":"2024-09-21T03:43:08.011078+0000","last_became_peered":"2024-09-21T03:43:08.011078+0000","last_unstale":"2024-09-21T03:45:23.743198+0000","last_undegraded":"2024-09-21T03:45:23.743198+0000","last_fullsized":"2024-09-21T03:45:23.743198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:44.764713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'17","reported_seq":63,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:24.032154+0000","last_change":"2024-09-21T03:43:04.979223+0000","last_active":"2024-09-21T03:45:24.032154+0000","last_peered":"2024-09-21T03:45:24.032154+0000","last_clean":"2024-09-21T03:45:24.032154+0000","last_became_active":"2024-09-21T03:43:04.978675+0000","last_became_peered":"2024-09-21T03:43:04.978675+0000","last_unstale":"2024-09-21T03:45:24.032154+0000","last_undegraded":"2024-09-21T03:45:24.032154+0000","last_fullsized":"2024-09-21T03:45:24.032154+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:09.717921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671365+0000","last_change":"2024-09-21T03:43:18.482334+0000","last_active":"2024-09-21T03:44:54.671365+0000","last_peered":"2024-09-21T03:44:54.671365+0000","last_clean":"2024-09-21T03:44:54.671365+0000","last_became_active":"2024-09-21T03:43:18.481881+0000","last_became_peered":"2024-09-21T03:43:18.481881+0000","last_unstale":"2024-09-21T03:44:54.671365+0000","last_undegraded":"2024-09-21T03:44:54.671365+0000","last_fullsized":"2024-09-21T03:44:54.671365+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:02:32.497348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.730466+0000","last_change":"2024-09-21T03:43:07.993771+0000","last_active":"2024-09-21T03:45:23.730466+0000","last_peered":"2024-09-21T03:45:23.730466+0000","last_clean":"2024-09-21T03:45:23.730466+0000","last_became_active":"2024-09-21T03:43:07.993268+0000","last_became_peered":"2024-09-21T03:43:07.993268+0000","last_unstale":"2024-09-21T03:45:23.730466+0000","last_undegraded":"2024-09-21T03:45:23.730466+0000","last_fullsized":"2024-09-21T03:45:23.730466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:42.348561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675923+0000","last_change":"2024-09-21T03:43:05.141786+0000","last_active":"2024-09-21T03:44:54.675923+0000","last_peered":"2024-09-21T03:44:54.675923+0000","last_clean":"2024-09-21T03:44:54.675923+0000","last_became_active":"2024-09-21T03:43:05.141307+0000","last_became_peered":"2024-09-21T03:43:05.141307+0000","last_unstale":"2024-09-21T03:44:54.675923+0000","last_undegraded":"2024-09-21T03:44:54.675923+0000","last_fullsized":"2024-09-21T03:44:54.675923+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:12.139398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677370+0000","last_change":"2024-09-21T03:43:18.087848+0000","last_active":"2024-09-21T03:44:54.677370+0000","last_peered":"2024-09-21T03:44:54.677370+0000","last_clean":"2024-09-21T03:44:54.677370+0000","last_became_active":"2024-09-21T03:43:18.087400+0000","last_became_peered":"2024-09-21T03:43:18.087400+0000","last_unstale":"2024-09-21T03:44:54.677370+0000","last_undegraded":"2024-09-21T03:44:54.677370+0000","last_fullsized":"2024-09-21T03:44:54.677370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:46.902979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"86'4","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.730419+0000","last_change":"2024-09-21T03:43:08.010051+0000","last_active":"2024-09-21T03:45:23.730419+0000","last_peered":"2024-09-21T03:45:23.730419+0000","last_clean":"2024-09-21T03:45:23.730419+0000","last_became_active":"2024-09-21T03:43:08.007700+0000","last_became_peered":"2024-09-21T03:43:08.007700+0000","last_unstale":"2024-09-21T03:45:23.730419+0000","last_undegraded":"2024-09-21T03:45:23.730419+0000","last_fullsized":"2024-09-21T03:45:23.730419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:31.828193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670543+0000","last_change":"2024-09-21T03:43:05.140490+0000","last_active":"2024-09-21T03:44:54.670543+0000","last_peered":"2024-09-21T03:44:54.670543+0000","last_clean":"2024-09-21T03:44:54.670543+0000","last_became_active":"2024-09-21T03:43:05.139979+0000","last_became_peered":"2024-09-21T03:43:05.139979+0000","last_unstale":"2024-09-21T03:44:54.670543+0000","last_undegraded":"2024-09-21T03:44:54.670543+0000","last_fullsized":"2024-09-21T03:44:54.670543+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:20.957596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.674914+0000","last_change":"2024-09-21T03:43:18.482777+0000","last_active":"2024-09-21T03:44:54.674914+0000","last_peered":"2024-09-21T03:44:54.674914+0000","last_clean":"2024-09-21T03:44:54.674914+0000","last_became_active":"2024-09-21T03:43:18.482244+0000","last_became_peered":"2024-09-21T03:43:18.482244+0000","last_unstale":"2024-09-21T03:44:54.674914+0000","last_undegraded":"2024-09-21T03:44:54.674914+0000","last_fullsized":"2024-09-21T03:44:54.674914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:41.997309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.734311+0000","last_change":"2024-09-21T03:43:08.019087+0000","last_active":"2024-09-21T03:45:23.734311+0000","last_peered":"2024-09-21T03:45:23.734311+0000","last_clean":"2024-09-21T03:45:23.734311+0000","last_became_active":"2024-09-21T03:43:08.018467+0000","last_became_peered":"2024-09-21T03:43:08.018467+0000","last_unstale":"2024-09-21T03:45:23.734311+0000","last_undegraded":"2024-09-21T03:45:23.734311+0000","last_fullsized":"2024-09-21T03:45:23.734311+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:25.824955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'6","reported_seq":183,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:13.751456+0000","last_change":"2024-09-21T03:43:04.973433+0000","last_active":"2024-09-21T03:46:13.751456+0000","last_peered":"2024-09-21T03:46:13.751456+0000","last_clean":"2024-09-21T03:46:13.751456+0000","last_became_active":"2024-09-21T03:43:04.973033+0000","last_became_peered":"2024-09-21T03:43:04.973033+0000","last_unstale":"2024-09-21T03:46:13.751456+0000","last_undegraded":"2024-09-21T03:46:13.751456+0000","last_fullsized":"2024-09-21T03:46:13.751456+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:25.462635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":899,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766900+0000","last_change":"2024-09-21T03:43:18.082385+0000","last_active":"2024-09-21T03:44:54.766900+0000","last_peered":"2024-09-21T03:44:54.766900+0000","last_clean":"2024-09-21T03:44:54.766900+0000","last_became_active":"2024-09-21T03:43:18.081958+0000","last_became_peered":"2024-09-21T03:43:18.081958+0000","last_unstale":"2024-09-21T03:44:54.766900+0000","last_undegraded":"2024-09-21T03:44:54.766900+0000","last_fullsized":"2024-09-21T03:44:54.766900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:36.554838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.739812+0000","last_change":"2024-09-21T03:43:08.008307+0000","last_active":"2024-09-21T03:45:23.739812+0000","last_peered":"2024-09-21T03:45:23.739812+0000","last_clean":"2024-09-21T03:45:23.739812+0000","last_became_active":"2024-09-21T03:43:08.006736+0000","last_became_peered":"2024-09-21T03:43:08.006736+0000","last_unstale":"2024-09-21T03:45:23.739812+0000","last_undegraded":"2024-09-21T03:45:23.739812+0000","last_fullsized":"2024-09-21T03:45:23.739812+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:31.694655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":179,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:11.746814+0000","last_change":"2024-09-21T03:43:05.140263+0000","last_active":"2024-09-21T03:46:11.746814+0000","last_peered":"2024-09-21T03:46:11.746814+0000","last_clean":"2024-09-21T03:46:11.746814+0000","last_became_active":"2024-09-21T03:43:05.139869+0000","last_became_peered":"2024-09-21T03:43:05.139869+0000","last_unstale":"2024-09-21T03:46:11.746814+0000","last_undegraded":"2024-09-21T03:46:11.746814+0000","last_fullsized":"2024-09-21T03:46:11.746814+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:19.313677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766889+0000","last_change":"2024-09-21T03:43:18.483795+0000","last_active":"2024-09-21T03:44:54.766889+0000","last_peered":"2024-09-21T03:44:54.766889+0000","last_clean":"2024-09-21T03:44:54.766889+0000","last_became_active":"2024-09-21T03:43:18.483159+0000","last_became_peered":"2024-09-21T03:43:18.483159+0000","last_unstale":"2024-09-21T03:44:54.766889+0000","last_undegraded":"2024-09-21T03:44:54.766889+0000","last_fullsized":"2024-09-21T03:44:54.766889+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:56.277956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.726790+0000","last_change":"2024-09-21T03:43:08.001661+0000","last_active":"2024-09-21T03:45:23.726790+0000","last_peered":"2024-09-21T03:45:23.726790+0000","last_clean":"2024-09-21T03:45:23.726790+0000","last_became_active":"2024-09-21T03:43:08.001254+0000","last_became_peered":"2024-09-21T03:43:08.001254+0000","last_unstale":"2024-09-21T03:45:23.726790+0000","last_undegraded":"2024-09-21T03:45:23.726790+0000","last_fullsized":"2024-09-21T03:45:23.726790+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:19.020542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678080+0000","last_change":"2024-09-21T03:43:04.979127+0000","last_active":"2024-09-21T03:44:54.678080+0000","last_peered":"2024-09-21T03:44:54.678080+0000","last_clean":"2024-09-21T03:44:54.678080+0000","last_became_active":"2024-09-21T03:43:04.978705+0000","last_became_peered":"2024-09-21T03:43:04.978705+0000","last_unstale":"2024-09-21T03:44:54.678080+0000","last_undegraded":"2024-09-21T03:44:54.678080+0000","last_fullsized":"2024-09-21T03:44:54.678080+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:48.548002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766993+0000","last_change":"2024-09-21T03:43:18.483682+0000","last_active":"2024-09-21T03:44:54.766993+0000","last_peered":"2024-09-21T03:44:54.766993+0000","last_clean":"2024-09-21T03:44:54.766993+0000","last_became_active":"2024-09-21T03:43:18.483124+0000","last_became_peered":"2024-09-21T03:43:18.483124+0000","last_unstale":"2024-09-21T03:44:54.766993+0000","last_undegraded":"2024-09-21T03:44:54.766993+0000","last_fullsized":"2024-09-21T03:44:54.766993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:12.513795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.739926+0000","last_change":"2024-09-21T03:43:08.013193+0000","last_active":"2024-09-21T03:45:23.739926+0000","last_peered":"2024-09-21T03:45:23.739926+0000","last_clean":"2024-09-21T03:45:23.739926+0000","last_became_active":"2024-09-21T03:43:08.010360+0000","last_became_peered":"2024-09-21T03:43:08.010360+0000","last_unstale":"2024-09-21T03:45:23.739926+0000","last_undegraded":"2024-09-21T03:45:23.739926+0000","last_fullsized":"2024-09-21T03:45:23.739926+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:28.101646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768945+0000","last_change":"2024-09-21T03:43:04.971782+0000","last_active":"2024-09-21T03:44:54.768945+0000","last_peered":"2024-09-21T03:44:54.768945+0000","last_clean":"2024-09-21T03:44:54.768945+0000","last_became_active":"2024-09-21T03:43:04.971375+0000","last_became_peered":"2024-09-21T03:43:04.971375+0000","last_unstale":"2024-09-21T03:44:54.768945+0000","last_undegraded":"2024-09-21T03:44:54.768945+0000","last_fullsized":"2024-09-21T03:44:54.768945+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:49.573490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'92","reported_seq":115,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678174+0000","last_change":"2024-09-21T03:41:42.914435+0000","last_active":"2024-09-21T03:44:54.678174+0000","last_peered":"2024-09-21T03:44:54.678174+0000","last_clean":"2024-09-21T03:44:54.678174+0000","last_became_active":"2024-09-21T03:41:42.909490+0000","last_became_peered":"2024-09-21T03:41:42.909490+0000","last_unstale":"2024-09-21T03:44:54.678174+0000","last_undegraded":"2024-09-21T03:44:54.678174+0000","last_fullsized":"2024-09-21T03:44:54.678174+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:56.507027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:56.507027+0000","last_clean_scrub_stamp":"2024-09-21T03:37:56.507027+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:34.564062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678296+0000","last_change":"2024-09-21T03:43:18.084701+0000","last_active":"2024-09-21T03:44:54.678296+0000","last_peered":"2024-09-21T03:44:54.678296+0000","last_clean":"2024-09-21T03:44:54.678296+0000","last_became_active":"2024-09-21T03:43:18.084251+0000","last_became_peered":"2024-09-21T03:43:18.084251+0000","last_unstale":"2024-09-21T03:44:54.678296+0000","last_undegraded":"2024-09-21T03:44:54.678296+0000","last_fullsized":"2024-09-21T03:44:54.678296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:15.660276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.741599+0000","last_change":"2024-09-21T03:43:08.012195+0000","last_active":"2024-09-21T03:45:23.741599+0000","last_peered":"2024-09-21T03:45:23.741599+0000","last_clean":"2024-09-21T03:45:23.741599+0000","last_became_active":"2024-09-21T03:43:08.011670+0000","last_became_peered":"2024-09-21T03:43:08.011670+0000","last_unstale":"2024-09-21T03:45:23.741599+0000","last_undegraded":"2024-09-21T03:45:23.741599+0000","last_fullsized":"2024-09-21T03:45:23.741599+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:14.290074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'7","reported_seq":187,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:14.752915+0000","last_change":"2024-09-21T03:43:05.567898+0000","last_active":"2024-09-21T03:46:14.752915+0000","last_peered":"2024-09-21T03:46:14.752915+0000","last_clean":"2024-09-21T03:46:14.752915+0000","last_became_active":"2024-09-21T03:43:05.567367+0000","last_became_peered":"2024-09-21T03:43:05.567367+0000","last_unstale":"2024-09-21T03:46:14.752915+0000","last_undegraded":"2024-09-21T03:46:14.752915+0000","last_fullsized":"2024-09-21T03:46:14.752915+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:55.120838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768187+0000","last_change":"2024-09-21T03:43:18.073942+0000","last_active":"2024-09-21T03:44:54.768187+0000","last_peered":"2024-09-21T03:44:54.768187+0000","last_clean":"2024-09-21T03:44:54.768187+0000","last_became_active":"2024-09-21T03:43:18.073323+0000","last_became_peered":"2024-09-21T03:43:18.073323+0000","last_unstale":"2024-09-21T03:44:54.768187+0000","last_undegraded":"2024-09-21T03:44:54.768187+0000","last_fullsized":"2024-09-21T03:44:54.768187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:41.677143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.742436+0000","last_change":"2024-09-21T03:43:08.016636+0000","last_active":"2024-09-21T03:45:23.742436+0000","last_peered":"2024-09-21T03:45:23.742436+0000","last_clean":"2024-09-21T03:45:23.742436+0000","last_became_active":"2024-09-21T03:43:08.016235+0000","last_became_peered":"2024-09-21T03:43:08.016235+0000","last_unstale":"2024-09-21T03:45:23.742436+0000","last_undegraded":"2024-09-21T03:45:23.742436+0000","last_fullsized":"2024-09-21T03:45:23.742436+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:35.365830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767144+0000","last_change":"2024-09-21T03:43:04.980911+0000","last_active":"2024-09-21T03:44:54.767144+0000","last_peered":"2024-09-21T03:44:54.767144+0000","last_clean":"2024-09-21T03:44:54.767144+0000","last_became_active":"2024-09-21T03:43:04.980512+0000","last_became_peered":"2024-09-21T03:43:04.980512+0000","last_unstale":"2024-09-21T03:44:54.767144+0000","last_undegraded":"2024-09-21T03:44:54.767144+0000","last_fullsized":"2024-09-21T03:44:54.767144+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:04.136345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679934+0000","last_change":"2024-09-21T03:43:18.085874+0000","last_active":"2024-09-21T03:44:54.679934+0000","last_peered":"2024-09-21T03:44:54.679934+0000","last_clean":"2024-09-21T03:44:54.679934+0000","last_became_active":"2024-09-21T03:43:18.085381+0000","last_became_peered":"2024-09-21T03:43:18.085381+0000","last_unstale":"2024-09-21T03:44:54.679934+0000","last_undegraded":"2024-09-21T03:44:54.679934+0000","last_fullsized":"2024-09-21T03:44:54.679934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:02.829202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.734618+0000","last_change":"2024-09-21T03:43:08.014776+0000","last_active":"2024-09-21T03:45:23.734618+0000","last_peered":"2024-09-21T03:45:23.734618+0000","last_clean":"2024-09-21T03:45:23.734618+0000","last_became_active":"2024-09-21T03:43:08.014191+0000","last_became_peered":"2024-09-21T03:43:08.014191+0000","last_unstale":"2024-09-21T03:45:23.734618+0000","last_undegraded":"2024-09-21T03:45:23.734618+0000","last_fullsized":"2024-09-21T03:45:23.734618+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:19.114838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766945+0000","last_change":"2024-09-21T03:43:04.973714+0000","last_active":"2024-09-21T03:44:54.766945+0000","last_peered":"2024-09-21T03:44:54.766945+0000","last_clean":"2024-09-21T03:44:54.766945+0000","last_became_active":"2024-09-21T03:43:04.973348+0000","last_became_peered":"2024-09-21T03:43:04.973348+0000","last_unstale":"2024-09-21T03:44:54.766945+0000","last_undegraded":"2024-09-21T03:44:54.766945+0000","last_fullsized":"2024-09-21T03:44:54.766945+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:00.872589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766379+0000","last_change":"2024-09-21T03:43:18.082643+0000","last_active":"2024-09-21T03:44:54.766379+0000","last_peered":"2024-09-21T03:44:54.766379+0000","last_clean":"2024-09-21T03:44:54.766379+0000","last_became_active":"2024-09-21T03:43:18.082157+0000","last_became_peered":"2024-09-21T03:43:18.082157+0000","last_unstale":"2024-09-21T03:44:54.766379+0000","last_undegraded":"2024-09-21T03:44:54.766379+0000","last_fullsized":"2024-09-21T03:44:54.766379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:52.652014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.725796+0000","last_change":"2024-09-21T03:43:08.010913+0000","last_active":"2024-09-21T03:45:23.725796+0000","last_peered":"2024-09-21T03:45:23.725796+0000","last_clean":"2024-09-21T03:45:23.725796+0000","last_became_active":"2024-09-21T03:43:08.010483+0000","last_became_peered":"2024-09-21T03:43:08.010483+0000","last_unstale":"2024-09-21T03:45:23.725796+0000","last_undegraded":"2024-09-21T03:45:23.725796+0000","last_fullsized":"2024-09-21T03:45:23.725796+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:52.158346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675973+0000","last_change":"2024-09-21T03:43:04.980260+0000","last_active":"2024-09-21T03:44:54.675973+0000","last_peered":"2024-09-21T03:44:54.675973+0000","last_clean":"2024-09-21T03:44:54.675973+0000","last_became_active":"2024-09-21T03:43:04.978999+0000","last_became_peered":"2024-09-21T03:43:04.978999+0000","last_unstale":"2024-09-21T03:44:54.675973+0000","last_undegraded":"2024-09-21T03:44:54.675973+0000","last_fullsized":"2024-09-21T03:44:54.675973+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:33.682591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766371+0000","last_change":"2024-09-21T03:43:18.083665+0000","last_active":"2024-09-21T03:44:54.766371+0000","last_peered":"2024-09-21T03:44:54.766371+0000","last_clean":"2024-09-21T03:44:54.766371+0000","last_became_active":"2024-09-21T03:43:18.083215+0000","last_became_peered":"2024-09-21T03:43:18.083215+0000","last_unstale":"2024-09-21T03:44:54.766371+0000","last_undegraded":"2024-09-21T03:44:54.766371+0000","last_fullsized":"2024-09-21T03:44:54.766371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:55.968968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.739802+0000","last_change":"2024-09-21T03:43:08.010570+0000","last_active":"2024-09-21T03:45:23.739802+0000","last_peered":"2024-09-21T03:45:23.739802+0000","last_clean":"2024-09-21T03:45:23.739802+0000","last_became_active":"2024-09-21T03:43:08.010163+0000","last_became_peered":"2024-09-21T03:43:08.010163+0000","last_unstale":"2024-09-21T03:45:23.739802+0000","last_undegraded":"2024-09-21T03:45:23.739802+0000","last_fullsized":"2024-09-21T03:45:23.739802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:17.368534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.707142+0000","last_change":"2024-09-21T03:44:54.706609+0000","last_active":"2024-09-21T03:44:54.707142+0000","last_peered":"2024-09-21T03:44:54.707142+0000","last_clean":"2024-09-21T03:44:54.707142+0000","last_became_active":"2024-09-21T03:44:54.681060+0000","last_became_peered":"2024-09-21T03:44:54.681060+0000","last_unstale":"2024-09-21T03:44:54.707142+0000","last_undegraded":"2024-09-21T03:44:54.707142+0000","last_fullsized":"2024-09-21T03:44:54.707142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:49.797107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766945+0000","last_change":"2024-09-21T03:43:18.089516+0000","last_active":"2024-09-21T03:44:54.766945+0000","last_peered":"2024-09-21T03:44:54.766945+0000","last_clean":"2024-09-21T03:44:54.766945+0000","last_became_active":"2024-09-21T03:43:18.089097+0000","last_became_peered":"2024-09-21T03:43:18.089097+0000","last_unstale":"2024-09-21T03:44:54.766945+0000","last_undegraded":"2024-09-21T03:44:54.766945+0000","last_fullsized":"2024-09-21T03:44:54.766945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:50.428401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.732694+0000","last_change":"2024-09-21T03:43:08.015762+0000","last_active":"2024-09-21T03:45:23.732694+0000","last_peered":"2024-09-21T03:45:23.732694+0000","last_clean":"2024-09-21T03:45:23.732694+0000","last_became_active":"2024-09-21T03:43:08.015389+0000","last_became_peered":"2024-09-21T03:43:08.015389+0000","last_unstale":"2024-09-21T03:45:23.732694+0000","last_undegraded":"2024-09-21T03:45:23.732694+0000","last_fullsized":"2024-09-21T03:45:23.732694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:00.341479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'5","reported_seq":182,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:13.751560+0000","last_change":"2024-09-21T03:43:04.973173+0000","last_active":"2024-09-21T03:46:13.751560+0000","last_peered":"2024-09-21T03:46:13.751560+0000","last_clean":"2024-09-21T03:46:13.751560+0000","last_became_active":"2024-09-21T03:43:04.972740+0000","last_became_peered":"2024-09-21T03:43:04.972740+0000","last_unstale":"2024-09-21T03:46:13.751560+0000","last_undegraded":"2024-09-21T03:46:13.751560+0000","last_fullsized":"2024-09-21T03:46:13.751560+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:26.329220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670519+0000","last_change":"2024-09-21T03:43:18.093730+0000","last_active":"2024-09-21T03:44:54.670519+0000","last_peered":"2024-09-21T03:44:54.670519+0000","last_clean":"2024-09-21T03:44:54.670519+0000","last_became_active":"2024-09-21T03:43:18.089621+0000","last_became_peered":"2024-09-21T03:43:18.089621+0000","last_unstale":"2024-09-21T03:44:54.670519+0000","last_undegraded":"2024-09-21T03:44:54.670519+0000","last_fullsized":"2024-09-21T03:44:54.670519+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:05.227038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":62,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.745319+0000","last_change":"2024-09-21T03:43:54.465751+0000","last_active":"2024-09-21T03:45:23.745319+0000","last_peered":"2024-09-21T03:45:23.745319+0000","last_clean":"2024-09-21T03:45:23.745319+0000","last_became_active":"2024-09-21T03:43:54.465333+0000","last_became_peered":"2024-09-21T03:43:54.465333+0000","last_unstale":"2024-09-21T03:45:23.745319+0000","last_undegraded":"2024-09-21T03:45:23.745319+0000","last_fullsized":"2024-09-21T03:45:23.745319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:34.307699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766735+0000","last_change":"2024-09-21T03:43:04.979385+0000","last_active":"2024-09-21T03:44:54.766735+0000","last_peered":"2024-09-21T03:44:54.766735+0000","last_clean":"2024-09-21T03:44:54.766735+0000","last_became_active":"2024-09-21T03:43:04.978764+0000","last_became_peered":"2024-09-21T03:43:04.978764+0000","last_unstale":"2024-09-21T03:44:54.766735+0000","last_undegraded":"2024-09-21T03:44:54.766735+0000","last_fullsized":"2024-09-21T03:44:54.766735+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:34.256966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677476+0000","last_change":"2024-09-21T03:43:18.095937+0000","last_active":"2024-09-21T03:44:54.677476+0000","last_peered":"2024-09-21T03:44:54.677476+0000","last_clean":"2024-09-21T03:44:54.677476+0000","last_became_active":"2024-09-21T03:43:18.095531+0000","last_became_peered":"2024-09-21T03:43:18.095531+0000","last_unstale":"2024-09-21T03:44:54.677476+0000","last_undegraded":"2024-09-21T03:44:54.677476+0000","last_fullsized":"2024-09-21T03:44:54.677476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:18.089661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.742877+0000","last_change":"2024-09-21T03:43:08.011342+0000","last_active":"2024-09-21T03:45:23.742877+0000","last_peered":"2024-09-21T03:45:23.742877+0000","last_clean":"2024-09-21T03:45:23.742877+0000","last_became_active":"2024-09-21T03:43:08.010963+0000","last_became_peered":"2024-09-21T03:43:08.010963+0000","last_unstale":"2024-09-21T03:45:23.742877+0000","last_undegraded":"2024-09-21T03:45:23.742877+0000","last_fullsized":"2024-09-21T03:45:23.742877+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:05.970863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":57,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679564+0000","last_change":"2024-09-21T03:43:04.972896+0000","last_active":"2024-09-21T03:44:54.679564+0000","last_peered":"2024-09-21T03:44:54.679564+0000","last_clean":"2024-09-21T03:44:54.679564+0000","last_became_active":"2024-09-21T03:43:04.972535+0000","last_became_peered":"2024-09-21T03:43:04.972535+0000","last_unstale":"2024-09-21T03:44:54.679564+0000","last_undegraded":"2024-09-21T03:44:54.679564+0000","last_fullsized":"2024-09-21T03:44:54.679564+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:36.362019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766929+0000","last_change":"2024-09-21T03:43:18.081300+0000","last_active":"2024-09-21T03:44:54.766929+0000","last_peered":"2024-09-21T03:44:54.766929+0000","last_clean":"2024-09-21T03:44:54.766929+0000","last_became_active":"2024-09-21T03:43:18.080874+0000","last_became_peered":"2024-09-21T03:43:18.080874+0000","last_unstale":"2024-09-21T03:44:54.766929+0000","last_undegraded":"2024-09-21T03:44:54.766929+0000","last_fullsized":"2024-09-21T03:44:54.766929+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:13.246348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.718671+0000","last_change":"2024-09-21T03:43:08.014954+0000","last_active":"2024-09-21T03:45:23.718671+0000","last_peered":"2024-09-21T03:45:23.718671+0000","last_clean":"2024-09-21T03:45:23.718671+0000","last_became_active":"2024-09-21T03:43:08.014576+0000","last_became_peered":"2024-09-21T03:43:08.014576+0000","last_unstale":"2024-09-21T03:45:23.718671+0000","last_undegraded":"2024-09-21T03:45:23.718671+0000","last_fullsized":"2024-09-21T03:45:23.718671+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:13.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'40","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:12.749967+0000","last_change":"2024-09-21T03:43:04.973305+0000","last_active":"2024-09-21T03:46:12.749967+0000","last_peered":"2024-09-21T03:46:12.749967+0000","last_clean":"2024-09-21T03:46:12.749967+0000","last_became_active":"2024-09-21T03:43:04.972891+0000","last_became_peered":"2024-09-21T03:43:04.972891+0000","last_unstale":"2024-09-21T03:46:12.749967+0000","last_undegraded":"2024-09-21T03:46:12.749967+0000","last_fullsized":"2024-09-21T03:46:12.749967+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:15.653769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67627,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":265,"num_read_kb":199,"num_write":42,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670066+0000","last_change":"2024-09-21T03:43:18.093613+0000","last_active":"2024-09-21T03:44:54.670066+0000","last_peered":"2024-09-21T03:44:54.670066+0000","last_clean":"2024-09-21T03:44:54.670066+0000","last_became_active":"2024-09-21T03:43:18.093195+0000","last_became_peered":"2024-09-21T03:43:18.093195+0000","last_unstale":"2024-09-21T03:44:54.670066+0000","last_undegraded":"2024-09-21T03:44:54.670066+0000","last_fullsized":"2024-09-21T03:44:54.670066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:17.825071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.734749+0000","last_change":"2024-09-21T03:43:08.014314+0000","last_active":"2024-09-21T03:45:23.734749+0000","last_peered":"2024-09-21T03:45:23.734749+0000","last_clean":"2024-09-21T03:45:23.734749+0000","last_became_active":"2024-09-21T03:43:08.012925+0000","last_became_peered":"2024-09-21T03:43:08.012925+0000","last_unstale":"2024-09-21T03:45:23.734749+0000","last_undegraded":"2024-09-21T03:45:23.734749+0000","last_fullsized":"2024-09-21T03:45:23.734749+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:47:28.599444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767094+0000","last_change":"2024-09-21T03:43:05.141584+0000","last_active":"2024-09-21T03:44:54.767094+0000","last_peered":"2024-09-21T03:44:54.767094+0000","last_clean":"2024-09-21T03:44:54.767094+0000","last_became_active":"2024-09-21T03:43:05.141200+0000","last_became_peered":"2024-09-21T03:43:05.141200+0000","last_unstale":"2024-09-21T03:44:54.767094+0000","last_undegraded":"2024-09-21T03:44:54.767094+0000","last_fullsized":"2024-09-21T03:44:54.767094+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:04.357970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768559+0000","last_change":"2024-09-21T03:43:18.092184+0000","last_active":"2024-09-21T03:44:54.768559+0000","last_peered":"2024-09-21T03:44:54.768559+0000","last_clean":"2024-09-21T03:44:54.768559+0000","last_became_active":"2024-09-21T03:43:18.091629+0000","last_became_peered":"2024-09-21T03:43:18.091629+0000","last_unstale":"2024-09-21T03:44:54.768559+0000","last_undegraded":"2024-09-21T03:44:54.768559+0000","last_fullsized":"2024-09-21T03:44:54.768559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:39.448009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.748221+0000","last_change":"2024-09-21T03:43:08.018335+0000","last_active":"2024-09-21T03:45:23.748221+0000","last_peered":"2024-09-21T03:45:23.748221+0000","last_clean":"2024-09-21T03:45:23.748221+0000","last_became_active":"2024-09-21T03:43:08.014005+0000","last_became_peered":"2024-09-21T03:43:08.014005+0000","last_unstale":"2024-09-21T03:45:23.748221+0000","last_undegraded":"2024-09-21T03:45:23.748221+0000","last_fullsized":"2024-09-21T03:45:23.748221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:35.749381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678554+0000","last_change":"2024-09-21T03:43:04.978609+0000","last_active":"2024-09-21T03:44:54.678554+0000","last_peered":"2024-09-21T03:44:54.678554+0000","last_clean":"2024-09-21T03:44:54.678554+0000","last_became_active":"2024-09-21T03:43:04.978146+0000","last_became_peered":"2024-09-21T03:43:04.978146+0000","last_unstale":"2024-09-21T03:44:54.678554+0000","last_undegraded":"2024-09-21T03:44:54.678554+0000","last_fullsized":"2024-09-21T03:44:54.678554+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:41.651745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670529+0000","last_change":"2024-09-21T03:43:18.089399+0000","last_active":"2024-09-21T03:44:54.670529+0000","last_peered":"2024-09-21T03:44:54.670529+0000","last_clean":"2024-09-21T03:44:54.670529+0000","last_became_active":"2024-09-21T03:43:18.086156+0000","last_became_peered":"2024-09-21T03:43:18.086156+0000","last_unstale":"2024-09-21T03:44:54.670529+0000","last_undegraded":"2024-09-21T03:44:54.670529+0000","last_fullsized":"2024-09-21T03:44:54.670529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:41.955341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.737922+0000","last_change":"2024-09-21T03:43:08.008773+0000","last_active":"2024-09-21T03:45:23.737922+0000","last_peered":"2024-09-21T03:45:23.737922+0000","last_clean":"2024-09-21T03:45:23.737922+0000","last_became_active":"2024-09-21T03:43:08.008270+0000","last_became_peered":"2024-09-21T03:43:08.008270+0000","last_unstale":"2024-09-21T03:45:23.737922+0000","last_undegraded":"2024-09-21T03:45:23.737922+0000","last_fullsized":"2024-09-21T03:45:23.737922+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:57.901701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'5","reported_seq":305,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:12.749481+0000","last_change":"2024-09-21T03:43:04.978296+0000","last_active":"2024-09-21T03:46:12.749481+0000","last_peered":"2024-09-21T03:46:12.749481+0000","last_clean":"2024-09-21T03:46:12.749481+0000","last_became_active":"2024-09-21T03:43:04.977782+0000","last_became_peered":"2024-09-21T03:43:04.977782+0000","last_unstale":"2024-09-21T03:46:12.749481+0000","last_undegraded":"2024-09-21T03:46:12.749481+0000","last_fullsized":"2024-09-21T03:46:12.749481+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:57.748737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":254,"num_read_kb":254,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671551+0000","last_change":"2024-09-21T03:43:54.495536+0000","last_active":"2024-09-21T03:44:54.671551+0000","last_peered":"2024-09-21T03:44:54.671551+0000","last_clean":"2024-09-21T03:44:54.671551+0000","last_became_active":"2024-09-21T03:43:54.493492+0000","last_became_peered":"2024-09-21T03:43:54.493492+0000","last_unstale":"2024-09-21T03:44:54.671551+0000","last_undegraded":"2024-09-21T03:44:54.671551+0000","last_fullsized":"2024-09-21T03:44:54.671551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:11.291911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679588+0000","last_change":"2024-09-21T03:43:18.077741+0000","last_active":"2024-09-21T03:44:54.679588+0000","last_peered":"2024-09-21T03:44:54.679588+0000","last_clean":"2024-09-21T03:44:54.679588+0000","last_became_active":"2024-09-21T03:43:18.076619+0000","last_became_peered":"2024-09-21T03:43:18.076619+0000","last_unstale":"2024-09-21T03:44:54.679588+0000","last_undegraded":"2024-09-21T03:44:54.679588+0000","last_fullsized":"2024-09-21T03:44:54.679588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:33.297020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.729664+0000","last_change":"2024-09-21T03:43:08.008838+0000","last_active":"2024-09-21T03:45:23.729664+0000","last_peered":"2024-09-21T03:45:23.729664+0000","last_clean":"2024-09-21T03:45:23.729664+0000","last_became_active":"2024-09-21T03:43:08.008422+0000","last_became_peered":"2024-09-21T03:43:08.008422+0000","last_unstale":"2024-09-21T03:45:23.729664+0000","last_undegraded":"2024-09-21T03:45:23.729664+0000","last_fullsized":"2024-09-21T03:45:23.729664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:00:17.881443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766641+0000","last_change":"2024-09-21T03:43:04.979289+0000","last_active":"2024-09-21T03:44:54.766641+0000","last_peered":"2024-09-21T03:44:54.766641+0000","last_clean":"2024-09-21T03:44:54.766641+0000","last_became_active":"2024-09-21T03:43:04.978843+0000","last_became_peered":"2024-09-21T03:43:04.978843+0000","last_unstale":"2024-09-21T03:44:54.766641+0000","last_undegraded":"2024-09-21T03:44:54.766641+0000","last_fullsized":"2024-09-21T03:44:54.766641+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:50.560607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677251+0000","last_change":"2024-09-21T03:43:18.096870+0000","last_active":"2024-09-21T03:44:54.677251+0000","last_peered":"2024-09-21T03:44:54.677251+0000","last_clean":"2024-09-21T03:44:54.677251+0000","last_became_active":"2024-09-21T03:43:18.096424+0000","last_became_peered":"2024-09-21T03:43:18.096424+0000","last_unstale":"2024-09-21T03:44:54.677251+0000","last_undegraded":"2024-09-21T03:44:54.677251+0000","last_fullsized":"2024-09-21T03:44:54.677251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:39.532368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.729991+0000","last_change":"2024-09-21T03:43:54.492499+0000","last_active":"2024-09-21T03:45:23.729991+0000","last_peered":"2024-09-21T03:45:23.729991+0000","last_clean":"2024-09-21T03:45:23.729991+0000","last_became_active":"2024-09-21T03:43:54.492043+0000","last_became_peered":"2024-09-21T03:43:54.492043+0000","last_unstale":"2024-09-21T03:45:23.729991+0000","last_undegraded":"2024-09-21T03:45:23.729991+0000","last_fullsized":"2024-09-21T03:45:23.729991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:25.687090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766447+0000","last_change":"2024-09-21T03:43:04.974265+0000","last_active":"2024-09-21T03:44:54.766447+0000","last_peered":"2024-09-21T03:44:54.766447+0000","last_clean":"2024-09-21T03:44:54.766447+0000","last_became_active":"2024-09-21T03:43:04.973383+0000","last_became_peered":"2024-09-21T03:43:04.973383+0000","last_unstale":"2024-09-21T03:44:54.766447+0000","last_undegraded":"2024-09-21T03:44:54.766447+0000","last_fullsized":"2024-09-21T03:44:54.766447+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:02.312378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768086+0000","last_change":"2024-09-21T03:43:04.980353+0000","last_active":"2024-09-21T03:44:54.768086+0000","last_peered":"2024-09-21T03:44:54.768086+0000","last_clean":"2024-09-21T03:44:54.768086+0000","last_became_active":"2024-09-21T03:43:04.979922+0000","last_became_peered":"2024-09-21T03:43:04.979922+0000","last_unstale":"2024-09-21T03:44:54.768086+0000","last_undegraded":"2024-09-21T03:44:54.768086+0000","last_fullsized":"2024-09-21T03:44:54.768086+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:17.636536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.740065+0000","last_change":"2024-09-21T03:43:08.009225+0000","last_active":"2024-09-21T03:45:23.740065+0000","last_peered":"2024-09-21T03:45:23.740065+0000","last_clean":"2024-09-21T03:45:23.740065+0000","last_became_active":"2024-09-21T03:43:08.008830+0000","last_became_peered":"2024-09-21T03:43:08.008830+0000","last_unstale":"2024-09-21T03:45:23.740065+0000","last_undegraded":"2024-09-21T03:45:23.740065+0000","last_fullsized":"2024-09-21T03:45:23.740065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:38.047763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679550+0000","last_change":"2024-09-21T03:43:18.089302+0000","last_active":"2024-09-21T03:44:54.679550+0000","last_peered":"2024-09-21T03:44:54.679550+0000","last_clean":"2024-09-21T03:44:54.679550+0000","last_became_active":"2024-09-21T03:43:18.088867+0000","last_became_peered":"2024-09-21T03:43:18.088867+0000","last_unstale":"2024-09-21T03:44:54.679550+0000","last_undegraded":"2024-09-21T03:44:54.679550+0000","last_fullsized":"2024-09-21T03:44:54.679550+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:08.960044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":188,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:12.750217+0000","last_change":"2024-09-21T03:43:54.464317+0000","last_active":"2024-09-21T03:46:12.750217+0000","last_peered":"2024-09-21T03:46:12.750217+0000","last_clean":"2024-09-21T03:46:12.750217+0000","last_became_active":"2024-09-21T03:43:54.463870+0000","last_became_peered":"2024-09-21T03:43:54.463870+0000","last_unstale":"2024-09-21T03:46:12.750217+0000","last_undegraded":"2024-09-21T03:46:12.750217+0000","last_fullsized":"2024-09-21T03:46:12.750217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:14.581726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.727426+0000","last_change":"2024-09-21T03:43:08.009398+0000","last_active":"2024-09-21T03:45:23.727426+0000","last_peered":"2024-09-21T03:45:23.727426+0000","last_clean":"2024-09-21T03:45:23.727426+0000","last_became_active":"2024-09-21T03:43:08.009000+0000","last_became_peered":"2024-09-21T03:43:08.009000+0000","last_unstale":"2024-09-21T03:45:23.727426+0000","last_undegraded":"2024-09-21T03:45:23.727426+0000","last_fullsized":"2024-09-21T03:45:23.727426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:07.278613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679894+0000","last_change":"2024-09-21T03:43:18.081834+0000","last_active":"2024-09-21T03:44:54.679894+0000","last_peered":"2024-09-21T03:44:54.679894+0000","last_clean":"2024-09-21T03:44:54.679894+0000","last_became_active":"2024-09-21T03:43:18.081328+0000","last_became_peered":"2024-09-21T03:43:18.081328+0000","last_unstale":"2024-09-21T03:44:54.679894+0000","last_undegraded":"2024-09-21T03:44:54.679894+0000","last_fullsized":"2024-09-21T03:44:54.679894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:37.638826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.725948+0000","last_change":"2024-09-21T03:43:08.005144+0000","last_active":"2024-09-21T03:45:23.725948+0000","last_peered":"2024-09-21T03:45:23.725948+0000","last_clean":"2024-09-21T03:45:23.725948+0000","last_became_active":"2024-09-21T03:43:08.003895+0000","last_became_peered":"2024-09-21T03:43:08.003895+0000","last_unstale":"2024-09-21T03:45:23.725948+0000","last_undegraded":"2024-09-21T03:45:23.725948+0000","last_fullsized":"2024-09-21T03:45:23.725948+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:25.822315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.728904+0000","last_change":"2024-09-21T03:43:08.000218+0000","last_active":"2024-09-21T03:45:23.728904+0000","last_peered":"2024-09-21T03:45:23.728904+0000","last_clean":"2024-09-21T03:45:23.728904+0000","last_became_active":"2024-09-21T03:43:07.999840+0000","last_became_peered":"2024-09-21T03:43:07.999840+0000","last_unstale":"2024-09-21T03:45:23.728904+0000","last_undegraded":"2024-09-21T03:45:23.728904+0000","last_fullsized":"2024-09-21T03:45:23.728904+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:55.391706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.744272+0000","last_change":"2024-09-21T03:43:08.012075+0000","last_active":"2024-09-21T03:45:23.744272+0000","last_peered":"2024-09-21T03:45:23.744272+0000","last_clean":"2024-09-21T03:45:23.744272+0000","last_became_active":"2024-09-21T03:43:08.011666+0000","last_became_peered":"2024-09-21T03:43:08.011666+0000","last_unstale":"2024-09-21T03:45:23.744272+0000","last_undegraded":"2024-09-21T03:45:23.744272+0000","last_fullsized":"2024-09-21T03:45:23.744272+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:42.124948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743198+0000","last_change":"2024-09-21T03:43:08.021082+0000","last_active":"2024-09-21T03:45:23.743198+0000","last_peered":"2024-09-21T03:45:23.743198+0000","last_clean":"2024-09-21T03:45:23.743198+0000","last_became_active":"2024-09-21T03:43:08.020674+0000","last_became_peered":"2024-09-21T03:43:08.020674+0000","last_unstale":"2024-09-21T03:45:23.743198+0000","last_undegraded":"2024-09-21T03:45:23.743198+0000","last_fullsized":"2024-09-21T03:45:23.743198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:13.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.744549+0000","last_change":"2024-09-21T03:43:08.004199+0000","last_active":"2024-09-21T03:45:23.744549+0000","last_peered":"2024-09-21T03:45:23.744549+0000","last_clean":"2024-09-21T03:45:23.744549+0000","last_became_active":"2024-09-21T03:43:08.003806+0000","last_became_peered":"2024-09-21T03:43:08.003806+0000","last_unstale":"2024-09-21T03:45:23.744549+0000","last_undegraded":"2024-09-21T03:45:23.744549+0000","last_fullsized":"2024-09-21T03:45:23.744549+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:13.941130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.723624+0000","last_change":"2024-09-21T03:43:08.007237+0000","last_active":"2024-09-21T03:45:23.723624+0000","last_peered":"2024-09-21T03:45:23.723624+0000","last_clean":"2024-09-21T03:45:23.723624+0000","last_became_active":"2024-09-21T03:43:08.006805+0000","last_became_peered":"2024-09-21T03:43:08.006805+0000","last_unstale":"2024-09-21T03:45:23.723624+0000","last_undegraded":"2024-09-21T03:45:23.723624+0000","last_fullsized":"2024-09-21T03:45:23.723624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:26.274494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.729741+0000","last_change":"2024-09-21T03:43:08.007934+0000","last_active":"2024-09-21T03:45:23.729741+0000","last_peered":"2024-09-21T03:45:23.729741+0000","last_clean":"2024-09-21T03:45:23.729741+0000","last_became_active":"2024-09-21T03:43:08.007549+0000","last_became_peered":"2024-09-21T03:43:08.007549+0000","last_unstale":"2024-09-21T03:45:23.729741+0000","last_undegraded":"2024-09-21T03:45:23.729741+0000","last_fullsized":"2024-09-21T03:45:23.729741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:58.158255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.714180+0000","last_change":"2024-09-21T03:43:54.493665+0000","last_active":"2024-09-21T03:45:23.714180+0000","last_peered":"2024-09-21T03:45:23.714180+0000","last_clean":"2024-09-21T03:45:23.714180+0000","last_became_active":"2024-09-21T03:43:54.493245+0000","last_became_peered":"2024-09-21T03:43:54.493245+0000","last_unstale":"2024-09-21T03:45:23.714180+0000","last_undegraded":"2024-09-21T03:45:23.714180+0000","last_fullsized":"2024-09-21T03:45:23.714180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:35.449852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.731123+0000","last_change":"2024-09-21T03:43:54.463419+0000","last_active":"2024-09-21T03:45:23.731123+0000","last_peered":"2024-09-21T03:45:23.731123+0000","last_clean":"2024-09-21T03:45:23.731123+0000","last_became_active":"2024-09-21T03:43:54.462951+0000","last_became_peered":"2024-09-21T03:43:54.462951+0000","last_unstale":"2024-09-21T03:45:23.731123+0000","last_undegraded":"2024-09-21T03:45:23.731123+0000","last_fullsized":"2024-09-21T03:45:23.731123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:23.620275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.717641+0000","last_change":"2024-09-21T03:43:08.011436+0000","last_active":"2024-09-21T03:45:23.717641+0000","last_peered":"2024-09-21T03:45:23.717641+0000","last_clean":"2024-09-21T03:45:23.717641+0000","last_became_active":"2024-09-21T03:43:08.011025+0000","last_became_peered":"2024-09-21T03:43:08.011025+0000","last_unstale":"2024-09-21T03:45:23.717641+0000","last_undegraded":"2024-09-21T03:45:23.717641+0000","last_fullsized":"2024-09-21T03:45:23.717641+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:31.411858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743351+0000","last_change":"2024-09-21T03:43:08.016378+0000","last_active":"2024-09-21T03:45:23.743351+0000","last_peered":"2024-09-21T03:45:23.743351+0000","last_clean":"2024-09-21T03:45:23.743351+0000","last_became_active":"2024-09-21T03:43:08.016013+0000","last_became_peered":"2024-09-21T03:43:08.016013+0000","last_unstale":"2024-09-21T03:45:23.743351+0000","last_undegraded":"2024-09-21T03:45:23.743351+0000","last_fullsized":"2024-09-21T03:45:23.743351+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:01.865669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743940+0000","last_change":"2024-09-21T03:43:08.007862+0000","last_active":"2024-09-21T03:45:23.743940+0000","last_peered":"2024-09-21T03:45:23.743940+0000","last_clean":"2024-09-21T03:45:23.743940+0000","last_became_active":"2024-09-21T03:43:08.007433+0000","last_became_peered":"2024-09-21T03:43:08.007433+0000","last_unstale":"2024-09-21T03:45:23.743940+0000","last_undegraded":"2024-09-21T03:45:23.743940+0000","last_fullsized":"2024-09-21T03:45:23.743940+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:22:03.364767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.723330+0000","last_change":"2024-09-21T03:43:08.010542+0000","last_active":"2024-09-21T03:45:23.723330+0000","last_peered":"2024-09-21T03:45:23.723330+0000","last_clean":"2024-09-21T03:45:23.723330+0000","last_became_active":"2024-09-21T03:43:08.010195+0000","last_became_peered":"2024-09-21T03:43:08.010195+0000","last_unstale":"2024-09-21T03:45:23.723330+0000","last_undegraded":"2024-09-21T03:45:23.723330+0000","last_fullsized":"2024-09-21T03:45:23.723330+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:52.850657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.728086+0000","last_change":"2024-09-21T03:43:08.006168+0000","last_active":"2024-09-21T03:45:23.728086+0000","last_peered":"2024-09-21T03:45:23.728086+0000","last_clean":"2024-09-21T03:45:23.728086+0000","last_became_active":"2024-09-21T03:43:08.005283+0000","last_became_peered":"2024-09-21T03:43:08.005283+0000","last_unstale":"2024-09-21T03:45:23.728086+0000","last_undegraded":"2024-09-21T03:45:23.728086+0000","last_fullsized":"2024-09-21T03:45:23.728086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:59.852412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.731547+0000","last_change":"2024-09-21T03:43:08.005870+0000","last_active":"2024-09-21T03:45:23.731547+0000","last_peered":"2024-09-21T03:45:23.731547+0000","last_clean":"2024-09-21T03:45:23.731547+0000","last_became_active":"2024-09-21T03:43:08.005485+0000","last_became_peered":"2024-09-21T03:43:08.005485+0000","last_unstale":"2024-09-21T03:45:23.731547+0000","last_undegraded":"2024-09-21T03:45:23.731547+0000","last_fullsized":"2024-09-21T03:45:23.731547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:41.387865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.741258+0000","last_change":"2024-09-21T03:43:08.015176+0000","last_active":"2024-09-21T03:45:23.741258+0000","last_peered":"2024-09-21T03:45:23.741258+0000","last_clean":"2024-09-21T03:45:23.741258+0000","last_became_active":"2024-09-21T03:43:08.014451+0000","last_became_peered":"2024-09-21T03:43:08.014451+0000","last_unstale":"2024-09-21T03:45:23.741258+0000","last_undegraded":"2024-09-21T03:45:23.741258+0000","last_fullsized":"2024-09-21T03:45:23.741258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:49.524800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.747012+0000","last_change":"2024-09-21T03:43:08.016798+0000","last_active":"2024-09-21T03:45:23.747012+0000","last_peered":"2024-09-21T03:45:23.747012+0000","last_clean":"2024-09-21T03:45:23.747012+0000","last_became_active":"2024-09-21T03:43:08.014842+0000","last_became_peered":"2024-09-21T03:43:08.014842+0000","last_unstale":"2024-09-21T03:45:23.747012+0000","last_undegraded":"2024-09-21T03:45:23.747012+0000","last_fullsized":"2024-09-21T03:45:23.747012+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:19.290618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.746263+0000","last_change":"2024-09-21T03:43:08.021285+0000","last_active":"2024-09-21T03:45:23.746263+0000","last_peered":"2024-09-21T03:45:23.746263+0000","last_clean":"2024-09-21T03:45:23.746263+0000","last_became_active":"2024-09-21T03:43:08.020904+0000","last_became_peered":"2024-09-21T03:43:08.020904+0000","last_unstale":"2024-09-21T03:45:23.746263+0000","last_undegraded":"2024-09-21T03:45:23.746263+0000","last_fullsized":"2024-09-21T03:45:23.746263+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:56.589089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.745437+0000","last_change":"2024-09-21T03:43:08.017599+0000","last_active":"2024-09-21T03:45:23.745437+0000","last_peered":"2024-09-21T03:45:23.745437+0000","last_clean":"2024-09-21T03:45:23.745437+0000","last_became_active":"2024-09-21T03:43:08.016138+0000","last_became_peered":"2024-09-21T03:43:08.016138+0000","last_unstale":"2024-09-21T03:45:23.745437+0000","last_undegraded":"2024-09-21T03:45:23.745437+0000","last_fullsized":"2024-09-21T03:45:23.745437+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:09.902246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.732277+0000","last_change":"2024-09-21T03:43:08.009598+0000","last_active":"2024-09-21T03:45:23.732277+0000","last_peered":"2024-09-21T03:45:23.732277+0000","last_clean":"2024-09-21T03:45:23.732277+0000","last_became_active":"2024-09-21T03:43:08.007488+0000","last_became_peered":"2024-09-21T03:43:08.007488+0000","last_unstale":"2024-09-21T03:45:23.732277+0000","last_undegraded":"2024-09-21T03:45:23.732277+0000","last_fullsized":"2024-09-21T03:45:23.732277+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:23.192761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.732050+0000","last_change":"2024-09-21T03:43:08.016450+0000","last_active":"2024-09-21T03:45:23.732050+0000","last_peered":"2024-09-21T03:45:23.732050+0000","last_clean":"2024-09-21T03:45:23.732050+0000","last_became_active":"2024-09-21T03:43:08.016037+0000","last_became_peered":"2024-09-21T03:43:08.016037+0000","last_unstale":"2024-09-21T03:45:23.732050+0000","last_undegraded":"2024-09-21T03:45:23.732050+0000","last_fullsized":"2024-09-21T03:45:23.732050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:39.109408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.726221+0000","last_change":"2024-09-21T03:43:08.010298+0000","last_active":"2024-09-21T03:45:23.726221+0000","last_peered":"2024-09-21T03:45:23.726221+0000","last_clean":"2024-09-21T03:45:23.726221+0000","last_became_active":"2024-09-21T03:43:08.009847+0000","last_became_peered":"2024-09-21T03:43:08.009847+0000","last_unstale":"2024-09-21T03:45:23.726221+0000","last_undegraded":"2024-09-21T03:45:23.726221+0000","last_fullsized":"2024-09-21T03:45:23.726221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:37.463456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.736715+0000","last_change":"2024-09-21T03:43:08.017834+0000","last_active":"2024-09-21T03:45:23.736715+0000","last_peered":"2024-09-21T03:45:23.736715+0000","last_clean":"2024-09-21T03:45:23.736715+0000","last_became_active":"2024-09-21T03:43:08.016910+0000","last_became_peered":"2024-09-21T03:43:08.016910+0000","last_unstale":"2024-09-21T03:45:23.736715+0000","last_undegraded":"2024-09-21T03:45:23.736715+0000","last_fullsized":"2024-09-21T03:45:23.736715+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:50.528651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.737971+0000","last_change":"2024-09-21T03:43:08.013945+0000","last_active":"2024-09-21T03:45:23.737971+0000","last_peered":"2024-09-21T03:45:23.737971+0000","last_clean":"2024-09-21T03:45:23.737971+0000","last_became_active":"2024-09-21T03:43:08.013598+0000","last_became_peered":"2024-09-21T03:43:08.013598+0000","last_unstale":"2024-09-21T03:45:23.737971+0000","last_undegraded":"2024-09-21T03:45:23.737971+0000","last_fullsized":"2024-09-21T03:45:23.737971+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:58.182938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.726374+0000","last_change":"2024-09-21T03:43:08.018824+0000","last_active":"2024-09-21T03:45:23.726374+0000","last_peered":"2024-09-21T03:45:23.726374+0000","last_clean":"2024-09-21T03:45:23.726374+0000","last_became_active":"2024-09-21T03:43:08.017196+0000","last_became_peered":"2024-09-21T03:43:08.017196+0000","last_unstale":"2024-09-21T03:45:23.726374+0000","last_undegraded":"2024-09-21T03:45:23.726374+0000","last_fullsized":"2024-09-21T03:45:23.726374+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:02.357523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743021+0000","last_change":"2024-09-21T03:43:08.003514+0000","last_active":"2024-09-21T03:45:23.743021+0000","last_peered":"2024-09-21T03:45:23.743021+0000","last_clean":"2024-09-21T03:45:23.743021+0000","last_became_active":"2024-09-21T03:43:08.003131+0000","last_became_peered":"2024-09-21T03:43:08.003131+0000","last_unstale":"2024-09-21T03:45:23.743021+0000","last_undegraded":"2024-09-21T03:45:23.743021+0000","last_fullsized":"2024-09-21T03:45:23.743021+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:23.170302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.735332+0000","last_change":"2024-09-21T03:43:08.015890+0000","last_active":"2024-09-21T03:45:23.735332+0000","last_peered":"2024-09-21T03:45:23.735332+0000","last_clean":"2024-09-21T03:45:23.735332+0000","last_became_active":"2024-09-21T03:43:08.015519+0000","last_became_peered":"2024-09-21T03:43:08.015519+0000","last_unstale":"2024-09-21T03:45:23.735332+0000","last_undegraded":"2024-09-21T03:45:23.735332+0000","last_fullsized":"2024-09-21T03:45:23.735332+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:10.875923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.727227+0000","last_change":"2024-09-21T03:43:08.012261+0000","last_active":"2024-09-21T03:45:23.727227+0000","last_peered":"2024-09-21T03:45:23.727227+0000","last_clean":"2024-09-21T03:45:23.727227+0000","last_became_active":"2024-09-21T03:43:08.011783+0000","last_became_peered":"2024-09-21T03:43:08.011783+0000","last_unstale":"2024-09-21T03:45:23.727227+0000","last_undegraded":"2024-09-21T03:45:23.727227+0000","last_fullsized":"2024-09-21T03:45:23.727227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:07.011340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.724685+0000","last_change":"2024-09-21T03:43:54.494826+0000","last_active":"2024-09-21T03:45:23.724685+0000","last_peered":"2024-09-21T03:45:23.724685+0000","last_clean":"2024-09-21T03:45:23.724685+0000","last_became_active":"2024-09-21T03:43:54.494404+0000","last_became_peered":"2024-09-21T03:43:54.494404+0000","last_unstale":"2024-09-21T03:45:23.724685+0000","last_undegraded":"2024-09-21T03:45:23.724685+0000","last_fullsized":"2024-09-21T03:45:23.724685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:42.109053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.740074+0000","last_change":"2024-09-21T03:43:08.016214+0000","last_active":"2024-09-21T03:45:23.740074+0000","last_peered":"2024-09-21T03:45:23.740074+0000","last_clean":"2024-09-21T03:45:23.740074+0000","last_became_active":"2024-09-21T03:43:08.014815+0000","last_became_peered":"2024-09-21T03:43:08.014815+0000","last_unstale":"2024-09-21T03:45:23.740074+0000","last_undegraded":"2024-09-21T03:45:23.740074+0000","last_fullsized":"2024-09-21T03:45:23.740074+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:04.742829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.745698+0000","last_change":"2024-09-21T03:43:08.019269+0000","last_active":"2024-09-21T03:45:23.745698+0000","last_peered":"2024-09-21T03:45:23.745698+0000","last_clean":"2024-09-21T03:45:23.745698+0000","last_became_active":"2024-09-21T03:43:08.017962+0000","last_became_peered":"2024-09-21T03:43:08.017962+0000","last_unstale":"2024-09-21T03:45:23.745698+0000","last_undegraded":"2024-09-21T03:45:23.745698+0000","last_fullsized":"2024-09-21T03:45:23.745698+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:05.950697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.738799+0000","last_change":"2024-09-21T03:43:08.014790+0000","last_active":"2024-09-21T03:45:23.738799+0000","last_peered":"2024-09-21T03:45:23.738799+0000","last_clean":"2024-09-21T03:45:23.738799+0000","last_became_active":"2024-09-21T03:43:08.013891+0000","last_became_peered":"2024-09-21T03:43:08.013891+0000","last_unstale":"2024-09-21T03:45:23.738799+0000","last_undegraded":"2024-09-21T03:45:23.738799+0000","last_fullsized":"2024-09-21T03:45:23.738799+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:46.045675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74892,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1321,"num_read_kb":739,"num_write":264,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":598016,"data_stored":233374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808886,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":501273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972410,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28688,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716464,"statfs":{"total":95995035648,"available":95965659136,"internally_reserved":0,"allocated":1703936,"data_stored":1158857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135935,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":500559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332165,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":500825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495691,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36924,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708228,"statfs":{"total":95995035648,"available":95957225472,"internally_reserved":0,"allocated":1617920,"data_stored":1097354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659216,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":496374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855446,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36932,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708220,"statfs":{"total":95995035648,"available":95957217280,"internally_reserved":0,"allocated":1626112,"data_stored":1092625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018972,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":500759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215201,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":495771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444136,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":569199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":569380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":498051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":73858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:46:19.002 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T03:46:19.002 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:46:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:19.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:19 smithi016 ceph-mon[30375]: from='client.25427 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:19.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:19 smithi123 ceph-mon[34673]: from='client.25427 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.038+0000 7f8e59000640 1 -- 172.21.15.16:0/1970055256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e541094a0 msgr2=0x7f8e541098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.038+0000 7f8e59000640 1 --2- 172.21.15.16:0/1970055256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e541094a0 0x7f8e541098b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009900 tx=0x7f8e3c02a410 comp rx=0 tx=0).stop 2024-09-21T03:46:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.038+0000 7f8e59000640 1 -- 172.21.15.16:0/1970055256 shutdown_connections 2024-09-21T03:46:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.038+0000 7f8e59000640 1 --2- 172.21.15.16:0/1970055256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e54109df0 0x7f8e54110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.038+0000 7f8e59000640 1 --2- 172.21.15.16:0/1970055256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e541094a0 0x7f8e541098b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.038+0000 7f8e59000640 1 --2- 172.21.15.16:0/1970055256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e54103b60 0x7f8e54103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.038+0000 7f8e59000640 1 -- 172.21.15.16:0/1970055256 >> 172.21.15.16:0/1970055256 conn(0x7f8e5407b850 msgr2=0x7f8e5407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.038+0000 7f8e59000640 1 -- 172.21.15.16:0/1970055256 shutdown_connections 2024-09-21T03:46:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.038+0000 7f8e59000640 1 -- 172.21.15.16:0/1970055256 wait complete. 2024-09-21T03:46:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.039+0000 7f8e59000640 1 Processor -- start 2024-09-21T03:46:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.039+0000 7f8e59000640 1 -- start start 2024-09-21T03:46:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.040+0000 7f8e59000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e54103b60 0x7f8e5419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.040+0000 7f8e59000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e541094a0 0x7f8e5419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.040+0000 7f8e59000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e54109df0 0x7f8e5419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.040+0000 7f8e59000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8e54113b00 con 0x7f8e54103b60 2024-09-21T03:46:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.040+0000 7f8e59000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e54113980 con 0x7f8e54109df0 2024-09-21T03:46:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.040+0000 7f8e59000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e54113c80 con 0x7f8e541094a0 2024-09-21T03:46:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.040+0000 7f8e51e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e54103b60 0x7f8e5419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.040+0000 7f8e4be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e541094a0 0x7f8e5419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.040+0000 7f8e51e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e54103b60 0x7f8e5419a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48890/0 (socket says 172.21.15.16:48890) 2024-09-21T03:46:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.040+0000 7f8e51e00640 1 -- 172.21.15.16:0/3929256185 learned_addr learned my addr 172.21.15.16:0/3929256185 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.040+0000 7f8e52800640 1 --2- 172.21.15.16:0/3929256185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e54109df0 0x7f8e5419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.040+0000 7f8e51e00640 1 -- 172.21.15.16:0/3929256185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e541094a0 msgr2=0x7f8e5419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.040+0000 7f8e51e00640 1 --2- 172.21.15.16:0/3929256185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e541094a0 0x7f8e5419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.040+0000 7f8e51e00640 1 -- 172.21.15.16:0/3929256185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e54109df0 msgr2=0x7f8e5419fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.041+0000 7f8e51e00640 1 --2- 172.21.15.16:0/3929256185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e54109df0 0x7f8e5419fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.041+0000 7f8e51e00640 1 -- 172.21.15.16:0/3929256185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e541a0540 con 0x7f8e54103b60 2024-09-21T03:46:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.041+0000 7f8e52800640 1 --2- 172.21.15.16:0/3929256185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e54109df0 0x7f8e5419fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.041+0000 7f8e51e00640 1 --2- 172.21.15.16:0/3929256185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e54103b60 0x7f8e5419a5d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400e9f0 tx=0x7f8e4400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.041+0000 7f8e4aa00640 1 -- 172.21.15.16:0/3929256185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4400cd80 con 0x7f8e54103b60 2024-09-21T03:46:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.041+0000 7f8e4aa00640 1 -- 172.21.15.16:0/3929256185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e440108e0 con 0x7f8e54103b60 2024-09-21T03:46:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.041+0000 7f8e4aa00640 1 -- 172.21.15.16:0/3929256185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44018980 con 0x7f8e54103b60 2024-09-21T03:46:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.041+0000 7f8e59000640 1 -- 172.21.15.16:0/3929256185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e541a0830 con 0x7f8e54103b60 2024-09-21T03:46:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.041+0000 7f8e59000640 1 -- 172.21.15.16:0/3929256185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e541a6f90 con 0x7f8e54103b60 2024-09-21T03:46:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.043+0000 7f8e59000640 1 -- 172.21.15.16:0/3929256185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e54104de0 con 0x7f8e54103b60 2024-09-21T03:46:20.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.045+0000 7f8e4aa00640 1 -- 172.21.15.16:0/3929256185 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8e44010430 con 0x7f8e54103b60 2024-09-21T03:46:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.046+0000 7f8e4aa00640 1 --2- 172.21.15.16:0/3929256185 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8e380b6590 0x7f8e380b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.047+0000 7f8e4aa00640 1 -- 172.21.15.16:0/3929256185 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f8e440e7980 con 0x7f8e54103b60 2024-09-21T03:46:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.047+0000 7f8e4be00640 1 --2- 172.21.15.16:0/3929256185 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8e380b6590 0x7f8e380b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.048+0000 7f8e4be00640 1 --2- 172.21.15.16:0/3929256185 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8e380b6590 0x7f8e380b8aa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c02aa00 tx=0x7f8e3c005a20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.051+0000 7f8e4aa00640 1 -- 172.21.15.16:0/3929256185 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8e440b21e0 con 0x7f8e54103b60 2024-09-21T03:46:20.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:19 smithi134 ceph-mon[35031]: from='client.25427 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:20.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.247+0000 7f8e59000640 1 -- 172.21.15.16:0/3929256185 --> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8e54063800 con 0x7f8e380b6590 2024-09-21T03:46:20.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.253+0000 7f8e4aa00640 1 -- 172.21.15.16:0/3929256185 <== mgr.14154 v2:172.21.15.16:6800/3097741949 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340529 (secure 0 0 0) 0x7f8e54063800 con 0x7f8e380b6590 2024-09-21T03:46:20.259 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:46:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-21T03:46:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.258+0000 7f8e59000640 1 -- 172.21.15.16:0/3929256185 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8e380b6590 msgr2=0x7f8e380b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.258+0000 7f8e59000640 1 --2- 172.21.15.16:0/3929256185 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8e380b6590 0x7f8e380b8aa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c02aa00 tx=0x7f8e3c005a20 comp rx=0 tx=0).stop 2024-09-21T03:46:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.258+0000 7f8e59000640 1 -- 172.21.15.16:0/3929256185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e54103b60 msgr2=0x7f8e5419a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.258+0000 7f8e59000640 1 --2- 172.21.15.16:0/3929256185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e54103b60 0x7f8e5419a5d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400e9f0 tx=0x7f8e4400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:46:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.258+0000 7f8e59000640 1 -- 172.21.15.16:0/3929256185 shutdown_connections 2024-09-21T03:46:20.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.258+0000 7f8e59000640 1 --2- 172.21.15.16:0/3929256185 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8e380b6590 0x7f8e380b8aa0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:20.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.258+0000 7f8e59000640 1 --2- 172.21.15.16:0/3929256185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e54109df0 0x7f8e5419fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:20.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.258+0000 7f8e59000640 1 --2- 172.21.15.16:0/3929256185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e541094a0 0x7f8e5419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:20.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.258+0000 7f8e59000640 1 --2- 172.21.15.16:0/3929256185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e54103b60 0x7f8e5419a5d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:20.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.258+0000 7f8e59000640 1 -- 172.21.15.16:0/3929256185 >> 172.21.15.16:0/3929256185 conn(0x7f8e5407b850 msgr2=0x7f8e54107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:20.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.258+0000 7f8e59000640 1 -- 172.21.15.16:0/3929256185 shutdown_connections 2024-09-21T03:46:20.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:20.258+0000 7f8e59000640 1 -- 172.21.15.16:0/3929256185 wait complete. 2024-09-21T03:46:20.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:20 smithi123 ceph-mon[34673]: pgmap v419: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-21T03:46:20.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:20 smithi016 ceph-mon[30375]: pgmap v419: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-21T03:46:20.982 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-09-21T03:46:19.384717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669843,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1428,"num_read_kb":828,"num_write":390,"num_write_kb":2313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2951952,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":820,"ondisk_log_size":820,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379768,"kb_used_data":13880,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562056,"statfs":{"total":1151940427776,"available":1151551545344,"internally_reserved":0,"allocated":14213120,"data_stored":7985687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4971,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":133,"num_read_kb":77,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003639"},"pg_stats":[{"pgid":"2.1f","version":"86'14","reported_seq":190,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:13.750840+0000","last_change":"2024-09-21T03:43:04.980132+0000","last_active":"2024-09-21T03:46:13.750840+0000","last_peered":"2024-09-21T03:46:13.750840+0000","last_clean":"2024-09-21T03:46:13.750840+0000","last_became_active":"2024-09-21T03:43:04.979274+0000","last_became_peered":"2024-09-21T03:43:04.979274+0000","last_unstale":"2024-09-21T03:46:13.750840+0000","last_undegraded":"2024-09-21T03:46:13.750840+0000","last_fullsized":"2024-09-21T03:46:13.750840+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:35.047129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":130,"num_read_kb":130,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.727333+0000","last_change":"2024-09-21T03:43:08.011576+0000","last_active":"2024-09-21T03:45:23.727333+0000","last_peered":"2024-09-21T03:45:23.727333+0000","last_clean":"2024-09-21T03:45:23.727333+0000","last_became_active":"2024-09-21T03:43:08.011188+0000","last_became_peered":"2024-09-21T03:43:08.011188+0000","last_unstale":"2024-09-21T03:45:23.727333+0000","last_undegraded":"2024-09-21T03:45:23.727333+0000","last_fullsized":"2024-09-21T03:45:23.727333+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:34.477639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678062+0000","last_change":"2024-09-21T03:43:54.493183+0000","last_active":"2024-09-21T03:44:54.678062+0000","last_peered":"2024-09-21T03:44:54.678062+0000","last_clean":"2024-09-21T03:44:54.678062+0000","last_became_active":"2024-09-21T03:43:54.492605+0000","last_became_peered":"2024-09-21T03:43:54.492605+0000","last_unstale":"2024-09-21T03:44:54.678062+0000","last_undegraded":"2024-09-21T03:44:54.678062+0000","last_fullsized":"2024-09-21T03:44:54.678062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:26.634042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.672352+0000","last_change":"2024-09-21T03:43:05.141790+0000","last_active":"2024-09-21T03:44:54.672352+0000","last_peered":"2024-09-21T03:44:54.672352+0000","last_clean":"2024-09-21T03:44:54.672352+0000","last_became_active":"2024-09-21T03:43:05.141299+0000","last_became_peered":"2024-09-21T03:43:05.141299+0000","last_unstale":"2024-09-21T03:44:54.672352+0000","last_undegraded":"2024-09-21T03:44:54.672352+0000","last_fullsized":"2024-09-21T03:44:54.672352+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:48.844615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.747911+0000","last_change":"2024-09-21T03:43:08.018245+0000","last_active":"2024-09-21T03:45:23.747911+0000","last_peered":"2024-09-21T03:45:23.747911+0000","last_clean":"2024-09-21T03:45:23.747911+0000","last_became_active":"2024-09-21T03:43:08.015719+0000","last_became_peered":"2024-09-21T03:43:08.015719+0000","last_unstale":"2024-09-21T03:45:23.747911+0000","last_undegraded":"2024-09-21T03:45:23.747911+0000","last_fullsized":"2024-09-21T03:45:23.747911+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:39.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766982+0000","last_change":"2024-09-21T03:43:54.493279+0000","last_active":"2024-09-21T03:44:54.766982+0000","last_peered":"2024-09-21T03:44:54.766982+0000","last_clean":"2024-09-21T03:44:54.766982+0000","last_became_active":"2024-09-21T03:43:54.492592+0000","last_became_peered":"2024-09-21T03:43:54.492592+0000","last_unstale":"2024-09-21T03:44:54.766982+0000","last_undegraded":"2024-09-21T03:44:54.766982+0000","last_fullsized":"2024-09-21T03:44:54.766982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:28.520990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'13","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:56.941089+0000","last_change":"2024-09-21T03:44:54.693431+0000","last_active":"2024-09-21T03:44:56.941089+0000","last_peered":"2024-09-21T03:44:56.941089+0000","last_clean":"2024-09-21T03:44:56.941089+0000","last_became_active":"2024-09-21T03:44:54.680324+0000","last_became_peered":"2024-09-21T03:44:54.680324+0000","last_unstale":"2024-09-21T03:44:56.941089+0000","last_undegraded":"2024-09-21T03:44:56.941089+0000","last_fullsized":"2024-09-21T03:44:56.941089+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:43.730697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.739651+0000","last_change":"2024-09-21T03:43:08.009018+0000","last_active":"2024-09-21T03:45:23.739651+0000","last_peered":"2024-09-21T03:45:23.739651+0000","last_clean":"2024-09-21T03:45:23.739651+0000","last_became_active":"2024-09-21T03:43:08.008614+0000","last_became_peered":"2024-09-21T03:43:08.008614+0000","last_unstale":"2024-09-21T03:45:23.739651+0000","last_undegraded":"2024-09-21T03:45:23.739651+0000","last_fullsized":"2024-09-21T03:45:23.739651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:31.000616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768752+0000","last_change":"2024-09-21T03:43:18.092750+0000","last_active":"2024-09-21T03:44:54.768752+0000","last_peered":"2024-09-21T03:44:54.768752+0000","last_clean":"2024-09-21T03:44:54.768752+0000","last_became_active":"2024-09-21T03:43:18.092122+0000","last_became_peered":"2024-09-21T03:43:18.092122+0000","last_unstale":"2024-09-21T03:44:54.768752+0000","last_undegraded":"2024-09-21T03:44:54.768752+0000","last_fullsized":"2024-09-21T03:44:54.768752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:15.032388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675595+0000","last_change":"2024-09-21T03:43:04.978045+0000","last_active":"2024-09-21T03:44:54.675595+0000","last_peered":"2024-09-21T03:44:54.675595+0000","last_clean":"2024-09-21T03:44:54.675595+0000","last_became_active":"2024-09-21T03:43:04.977565+0000","last_became_peered":"2024-09-21T03:43:04.977565+0000","last_unstale":"2024-09-21T03:44:54.675595+0000","last_undegraded":"2024-09-21T03:44:54.675595+0000","last_fullsized":"2024-09-21T03:44:54.675595+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:42.537892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.740686+0000","last_change":"2024-09-21T03:43:08.007094+0000","last_active":"2024-09-21T03:45:23.740686+0000","last_peered":"2024-09-21T03:45:23.740686+0000","last_clean":"2024-09-21T03:45:23.740686+0000","last_became_active":"2024-09-21T03:43:08.006666+0000","last_became_peered":"2024-09-21T03:43:08.006666+0000","last_unstale":"2024-09-21T03:45:23.740686+0000","last_undegraded":"2024-09-21T03:45:23.740686+0000","last_fullsized":"2024-09-21T03:45:23.740686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:35:59.247459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767130+0000","last_change":"2024-09-21T03:43:18.081141+0000","last_active":"2024-09-21T03:44:54.767130+0000","last_peered":"2024-09-21T03:44:54.767130+0000","last_clean":"2024-09-21T03:44:54.767130+0000","last_became_active":"2024-09-21T03:43:18.080623+0000","last_became_peered":"2024-09-21T03:43:18.080623+0000","last_unstale":"2024-09-21T03:44:54.767130+0000","last_undegraded":"2024-09-21T03:44:54.767130+0000","last_fullsized":"2024-09-21T03:44:54.767130+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:44.932673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.741475+0000","last_change":"2024-09-21T03:43:08.008602+0000","last_active":"2024-09-21T03:45:23.741475+0000","last_peered":"2024-09-21T03:45:23.741475+0000","last_clean":"2024-09-21T03:45:23.741475+0000","last_became_active":"2024-09-21T03:43:08.008220+0000","last_became_peered":"2024-09-21T03:43:08.008220+0000","last_unstale":"2024-09-21T03:45:23.741475+0000","last_undegraded":"2024-09-21T03:45:23.741475+0000","last_fullsized":"2024-09-21T03:45:23.741475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:28.286016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.774443+0000","last_change":"2024-09-21T03:44:54.773918+0000","last_active":"2024-09-21T03:44:54.774443+0000","last_peered":"2024-09-21T03:44:54.774443+0000","last_clean":"2024-09-21T03:44:54.774443+0000","last_became_active":"2024-09-21T03:44:54.769721+0000","last_became_peered":"2024-09-21T03:44:54.769721+0000","last_unstale":"2024-09-21T03:44:54.774443+0000","last_undegraded":"2024-09-21T03:44:54.774443+0000","last_fullsized":"2024-09-21T03:44:54.774443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:05.656051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767195+0000","last_change":"2024-09-21T03:43:18.483523+0000","last_active":"2024-09-21T03:44:54.767195+0000","last_peered":"2024-09-21T03:44:54.767195+0000","last_clean":"2024-09-21T03:44:54.767195+0000","last_became_active":"2024-09-21T03:43:18.482833+0000","last_became_peered":"2024-09-21T03:43:18.482833+0000","last_unstale":"2024-09-21T03:44:54.767195+0000","last_undegraded":"2024-09-21T03:44:54.767195+0000","last_fullsized":"2024-09-21T03:44:54.767195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:48.236569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.736655+0000","last_change":"2024-09-21T03:43:08.022592+0000","last_active":"2024-09-21T03:45:23.736655+0000","last_peered":"2024-09-21T03:45:23.736655+0000","last_clean":"2024-09-21T03:45:23.736655+0000","last_became_active":"2024-09-21T03:43:08.018065+0000","last_became_peered":"2024-09-21T03:43:08.018065+0000","last_unstale":"2024-09-21T03:45:23.736655+0000","last_undegraded":"2024-09-21T03:45:23.736655+0000","last_fullsized":"2024-09-21T03:45:23.736655+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:41.524402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679262+0000","last_change":"2024-09-21T03:43:04.978788+0000","last_active":"2024-09-21T03:44:54.679262+0000","last_peered":"2024-09-21T03:44:54.679262+0000","last_clean":"2024-09-21T03:44:54.679262+0000","last_became_active":"2024-09-21T03:43:04.978382+0000","last_became_peered":"2024-09-21T03:43:04.978382+0000","last_unstale":"2024-09-21T03:44:54.679262+0000","last_undegraded":"2024-09-21T03:44:54.679262+0000","last_fullsized":"2024-09-21T03:44:54.679262+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:12.571217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768259+0000","last_change":"2024-09-21T03:43:54.493347+0000","last_active":"2024-09-21T03:44:54.768259+0000","last_peered":"2024-09-21T03:44:54.768259+0000","last_clean":"2024-09-21T03:44:54.768259+0000","last_became_active":"2024-09-21T03:43:54.492598+0000","last_became_peered":"2024-09-21T03:43:54.492598+0000","last_unstale":"2024-09-21T03:44:54.768259+0000","last_undegraded":"2024-09-21T03:44:54.768259+0000","last_fullsized":"2024-09-21T03:44:54.768259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:34.788296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743945+0000","last_change":"2024-09-21T03:43:08.014368+0000","last_active":"2024-09-21T03:45:23.743945+0000","last_peered":"2024-09-21T03:45:23.743945+0000","last_clean":"2024-09-21T03:45:23.743945+0000","last_became_active":"2024-09-21T03:43:08.013952+0000","last_became_peered":"2024-09-21T03:43:08.013952+0000","last_unstale":"2024-09-21T03:45:23.743945+0000","last_undegraded":"2024-09-21T03:45:23.743945+0000","last_fullsized":"2024-09-21T03:45:23.743945+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:16.893588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671998+0000","last_change":"2024-09-21T03:43:04.976162+0000","last_active":"2024-09-21T03:44:54.671998+0000","last_peered":"2024-09-21T03:44:54.671998+0000","last_clean":"2024-09-21T03:44:54.671998+0000","last_became_active":"2024-09-21T03:43:04.973790+0000","last_became_peered":"2024-09-21T03:43:04.973790+0000","last_unstale":"2024-09-21T03:44:54.671998+0000","last_undegraded":"2024-09-21T03:44:54.671998+0000","last_fullsized":"2024-09-21T03:44:54.671998+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:47.838229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670339+0000","last_change":"2024-09-21T03:43:18.084344+0000","last_active":"2024-09-21T03:44:54.670339+0000","last_peered":"2024-09-21T03:44:54.670339+0000","last_clean":"2024-09-21T03:44:54.670339+0000","last_became_active":"2024-09-21T03:43:18.083232+0000","last_became_peered":"2024-09-21T03:43:18.083232+0000","last_unstale":"2024-09-21T03:44:54.670339+0000","last_undegraded":"2024-09-21T03:44:54.670339+0000","last_fullsized":"2024-09-21T03:44:54.670339+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:46.254405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.690064+0000","last_change":"2024-09-21T03:43:08.009932+0000","last_active":"2024-09-21T03:45:23.690064+0000","last_peered":"2024-09-21T03:45:23.690064+0000","last_clean":"2024-09-21T03:45:23.690064+0000","last_became_active":"2024-09-21T03:43:08.009492+0000","last_became_peered":"2024-09-21T03:43:08.009492+0000","last_unstale":"2024-09-21T03:45:23.690064+0000","last_undegraded":"2024-09-21T03:45:23.690064+0000","last_fullsized":"2024-09-21T03:45:23.690064+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:58.679348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768057+0000","last_change":"2024-09-21T03:43:05.141927+0000","last_active":"2024-09-21T03:44:54.768057+0000","last_peered":"2024-09-21T03:44:54.768057+0000","last_clean":"2024-09-21T03:44:54.768057+0000","last_became_active":"2024-09-21T03:43:05.141516+0000","last_became_peered":"2024-09-21T03:43:05.141516+0000","last_unstale":"2024-09-21T03:44:54.768057+0000","last_undegraded":"2024-09-21T03:44:54.768057+0000","last_fullsized":"2024-09-21T03:44:54.768057+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:35.666981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767396+0000","last_change":"2024-09-21T03:43:18.484478+0000","last_active":"2024-09-21T03:44:54.767396+0000","last_peered":"2024-09-21T03:44:54.767396+0000","last_clean":"2024-09-21T03:44:54.767396+0000","last_became_active":"2024-09-21T03:43:18.483890+0000","last_became_peered":"2024-09-21T03:43:18.483890+0000","last_unstale":"2024-09-21T03:44:54.767396+0000","last_undegraded":"2024-09-21T03:44:54.767396+0000","last_fullsized":"2024-09-21T03:44:54.767396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:03.542783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.748379+0000","last_change":"2024-09-21T03:43:08.017785+0000","last_active":"2024-09-21T03:45:23.748379+0000","last_peered":"2024-09-21T03:45:23.748379+0000","last_clean":"2024-09-21T03:45:23.748379+0000","last_became_active":"2024-09-21T03:43:08.017389+0000","last_became_peered":"2024-09-21T03:43:08.017389+0000","last_unstale":"2024-09-21T03:45:23.748379+0000","last_undegraded":"2024-09-21T03:45:23.748379+0000","last_fullsized":"2024-09-21T03:45:23.748379+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:37.557061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":54,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767655+0000","last_change":"2024-09-21T03:43:05.567829+0000","last_active":"2024-09-21T03:44:54.767655+0000","last_peered":"2024-09-21T03:44:54.767655+0000","last_clean":"2024-09-21T03:44:54.767655+0000","last_became_active":"2024-09-21T03:43:05.567401+0000","last_became_peered":"2024-09-21T03:43:05.567401+0000","last_unstale":"2024-09-21T03:44:54.767655+0000","last_undegraded":"2024-09-21T03:44:54.767655+0000","last_fullsized":"2024-09-21T03:44:54.767655+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:03.954280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678873+0000","last_change":"2024-09-21T03:43:18.080064+0000","last_active":"2024-09-21T03:44:54.678873+0000","last_peered":"2024-09-21T03:44:54.678873+0000","last_clean":"2024-09-21T03:44:54.678873+0000","last_became_active":"2024-09-21T03:43:18.079569+0000","last_became_peered":"2024-09-21T03:43:18.079569+0000","last_unstale":"2024-09-21T03:44:54.678873+0000","last_undegraded":"2024-09-21T03:44:54.678873+0000","last_fullsized":"2024-09-21T03:44:54.678873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:29.568476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743198+0000","last_change":"2024-09-21T03:43:08.011603+0000","last_active":"2024-09-21T03:45:23.743198+0000","last_peered":"2024-09-21T03:45:23.743198+0000","last_clean":"2024-09-21T03:45:23.743198+0000","last_became_active":"2024-09-21T03:43:08.011078+0000","last_became_peered":"2024-09-21T03:43:08.011078+0000","last_unstale":"2024-09-21T03:45:23.743198+0000","last_undegraded":"2024-09-21T03:45:23.743198+0000","last_fullsized":"2024-09-21T03:45:23.743198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:44.764713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'17","reported_seq":63,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:24.032154+0000","last_change":"2024-09-21T03:43:04.979223+0000","last_active":"2024-09-21T03:45:24.032154+0000","last_peered":"2024-09-21T03:45:24.032154+0000","last_clean":"2024-09-21T03:45:24.032154+0000","last_became_active":"2024-09-21T03:43:04.978675+0000","last_became_peered":"2024-09-21T03:43:04.978675+0000","last_unstale":"2024-09-21T03:45:24.032154+0000","last_undegraded":"2024-09-21T03:45:24.032154+0000","last_fullsized":"2024-09-21T03:45:24.032154+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:09.717921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671365+0000","last_change":"2024-09-21T03:43:18.482334+0000","last_active":"2024-09-21T03:44:54.671365+0000","last_peered":"2024-09-21T03:44:54.671365+0000","last_clean":"2024-09-21T03:44:54.671365+0000","last_became_active":"2024-09-21T03:43:18.481881+0000","last_became_peered":"2024-09-21T03:43:18.481881+0000","last_unstale":"2024-09-21T03:44:54.671365+0000","last_undegraded":"2024-09-21T03:44:54.671365+0000","last_fullsized":"2024-09-21T03:44:54.671365+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:02:32.497348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.730466+0000","last_change":"2024-09-21T03:43:07.993771+0000","last_active":"2024-09-21T03:45:23.730466+0000","last_peered":"2024-09-21T03:45:23.730466+0000","last_clean":"2024-09-21T03:45:23.730466+0000","last_became_active":"2024-09-21T03:43:07.993268+0000","last_became_peered":"2024-09-21T03:43:07.993268+0000","last_unstale":"2024-09-21T03:45:23.730466+0000","last_undegraded":"2024-09-21T03:45:23.730466+0000","last_fullsized":"2024-09-21T03:45:23.730466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:42.348561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675923+0000","last_change":"2024-09-21T03:43:05.141786+0000","last_active":"2024-09-21T03:44:54.675923+0000","last_peered":"2024-09-21T03:44:54.675923+0000","last_clean":"2024-09-21T03:44:54.675923+0000","last_became_active":"2024-09-21T03:43:05.141307+0000","last_became_peered":"2024-09-21T03:43:05.141307+0000","last_unstale":"2024-09-21T03:44:54.675923+0000","last_undegraded":"2024-09-21T03:44:54.675923+0000","last_fullsized":"2024-09-21T03:44:54.675923+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:12.139398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677370+0000","last_change":"2024-09-21T03:43:18.087848+0000","last_active":"2024-09-21T03:44:54.677370+0000","last_peered":"2024-09-21T03:44:54.677370+0000","last_clean":"2024-09-21T03:44:54.677370+0000","last_became_active":"2024-09-21T03:43:18.087400+0000","last_became_peered":"2024-09-21T03:43:18.087400+0000","last_unstale":"2024-09-21T03:44:54.677370+0000","last_undegraded":"2024-09-21T03:44:54.677370+0000","last_fullsized":"2024-09-21T03:44:54.677370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:46.902979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"86'4","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.730419+0000","last_change":"2024-09-21T03:43:08.010051+0000","last_active":"2024-09-21T03:45:23.730419+0000","last_peered":"2024-09-21T03:45:23.730419+0000","last_clean":"2024-09-21T03:45:23.730419+0000","last_became_active":"2024-09-21T03:43:08.007700+0000","last_became_peered":"2024-09-21T03:43:08.007700+0000","last_unstale":"2024-09-21T03:45:23.730419+0000","last_undegraded":"2024-09-21T03:45:23.730419+0000","last_fullsized":"2024-09-21T03:45:23.730419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:31.828193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670543+0000","last_change":"2024-09-21T03:43:05.140490+0000","last_active":"2024-09-21T03:44:54.670543+0000","last_peered":"2024-09-21T03:44:54.670543+0000","last_clean":"2024-09-21T03:44:54.670543+0000","last_became_active":"2024-09-21T03:43:05.139979+0000","last_became_peered":"2024-09-21T03:43:05.139979+0000","last_unstale":"2024-09-21T03:44:54.670543+0000","last_undegraded":"2024-09-21T03:44:54.670543+0000","last_fullsized":"2024-09-21T03:44:54.670543+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:20.957596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.674914+0000","last_change":"2024-09-21T03:43:18.482777+0000","last_active":"2024-09-21T03:44:54.674914+0000","last_peered":"2024-09-21T03:44:54.674914+0000","last_clean":"2024-09-21T03:44:54.674914+0000","last_became_active":"2024-09-21T03:43:18.482244+0000","last_became_peered":"2024-09-21T03:43:18.482244+0000","last_unstale":"2024-09-21T03:44:54.674914+0000","last_undegraded":"2024-09-21T03:44:54.674914+0000","last_fullsized":"2024-09-21T03:44:54.674914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:41.997309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.734311+0000","last_change":"2024-09-21T03:43:08.019087+0000","last_active":"2024-09-21T03:45:23.734311+0000","last_peered":"2024-09-21T03:45:23.734311+0000","last_clean":"2024-09-21T03:45:23.734311+0000","last_became_active":"2024-09-21T03:43:08.018467+0000","last_became_peered":"2024-09-21T03:43:08.018467+0000","last_unstale":"2024-09-21T03:45:23.734311+0000","last_undegraded":"2024-09-21T03:45:23.734311+0000","last_fullsized":"2024-09-21T03:45:23.734311+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:25.824955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'6","reported_seq":183,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:13.751456+0000","last_change":"2024-09-21T03:43:04.973433+0000","last_active":"2024-09-21T03:46:13.751456+0000","last_peered":"2024-09-21T03:46:13.751456+0000","last_clean":"2024-09-21T03:46:13.751456+0000","last_became_active":"2024-09-21T03:43:04.973033+0000","last_became_peered":"2024-09-21T03:43:04.973033+0000","last_unstale":"2024-09-21T03:46:13.751456+0000","last_undegraded":"2024-09-21T03:46:13.751456+0000","last_fullsized":"2024-09-21T03:46:13.751456+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:25.462635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":899,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766900+0000","last_change":"2024-09-21T03:43:18.082385+0000","last_active":"2024-09-21T03:44:54.766900+0000","last_peered":"2024-09-21T03:44:54.766900+0000","last_clean":"2024-09-21T03:44:54.766900+0000","last_became_active":"2024-09-21T03:43:18.081958+0000","last_became_peered":"2024-09-21T03:43:18.081958+0000","last_unstale":"2024-09-21T03:44:54.766900+0000","last_undegraded":"2024-09-21T03:44:54.766900+0000","last_fullsized":"2024-09-21T03:44:54.766900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:36.554838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.739812+0000","last_change":"2024-09-21T03:43:08.008307+0000","last_active":"2024-09-21T03:45:23.739812+0000","last_peered":"2024-09-21T03:45:23.739812+0000","last_clean":"2024-09-21T03:45:23.739812+0000","last_became_active":"2024-09-21T03:43:08.006736+0000","last_became_peered":"2024-09-21T03:43:08.006736+0000","last_unstale":"2024-09-21T03:45:23.739812+0000","last_undegraded":"2024-09-21T03:45:23.739812+0000","last_fullsized":"2024-09-21T03:45:23.739812+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:31.694655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":184,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:16.755358+0000","last_change":"2024-09-21T03:43:05.140263+0000","last_active":"2024-09-21T03:46:16.755358+0000","last_peered":"2024-09-21T03:46:16.755358+0000","last_clean":"2024-09-21T03:46:16.755358+0000","last_became_active":"2024-09-21T03:43:05.139869+0000","last_became_peered":"2024-09-21T03:43:05.139869+0000","last_unstale":"2024-09-21T03:46:16.755358+0000","last_undegraded":"2024-09-21T03:46:16.755358+0000","last_fullsized":"2024-09-21T03:46:16.755358+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:19.313677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766889+0000","last_change":"2024-09-21T03:43:18.483795+0000","last_active":"2024-09-21T03:44:54.766889+0000","last_peered":"2024-09-21T03:44:54.766889+0000","last_clean":"2024-09-21T03:44:54.766889+0000","last_became_active":"2024-09-21T03:43:18.483159+0000","last_became_peered":"2024-09-21T03:43:18.483159+0000","last_unstale":"2024-09-21T03:44:54.766889+0000","last_undegraded":"2024-09-21T03:44:54.766889+0000","last_fullsized":"2024-09-21T03:44:54.766889+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:56.277956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.726790+0000","last_change":"2024-09-21T03:43:08.001661+0000","last_active":"2024-09-21T03:45:23.726790+0000","last_peered":"2024-09-21T03:45:23.726790+0000","last_clean":"2024-09-21T03:45:23.726790+0000","last_became_active":"2024-09-21T03:43:08.001254+0000","last_became_peered":"2024-09-21T03:43:08.001254+0000","last_unstale":"2024-09-21T03:45:23.726790+0000","last_undegraded":"2024-09-21T03:45:23.726790+0000","last_fullsized":"2024-09-21T03:45:23.726790+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:19.020542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678080+0000","last_change":"2024-09-21T03:43:04.979127+0000","last_active":"2024-09-21T03:44:54.678080+0000","last_peered":"2024-09-21T03:44:54.678080+0000","last_clean":"2024-09-21T03:44:54.678080+0000","last_became_active":"2024-09-21T03:43:04.978705+0000","last_became_peered":"2024-09-21T03:43:04.978705+0000","last_unstale":"2024-09-21T03:44:54.678080+0000","last_undegraded":"2024-09-21T03:44:54.678080+0000","last_fullsized":"2024-09-21T03:44:54.678080+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:48.548002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766993+0000","last_change":"2024-09-21T03:43:18.483682+0000","last_active":"2024-09-21T03:44:54.766993+0000","last_peered":"2024-09-21T03:44:54.766993+0000","last_clean":"2024-09-21T03:44:54.766993+0000","last_became_active":"2024-09-21T03:43:18.483124+0000","last_became_peered":"2024-09-21T03:43:18.483124+0000","last_unstale":"2024-09-21T03:44:54.766993+0000","last_undegraded":"2024-09-21T03:44:54.766993+0000","last_fullsized":"2024-09-21T03:44:54.766993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:12.513795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.739926+0000","last_change":"2024-09-21T03:43:08.013193+0000","last_active":"2024-09-21T03:45:23.739926+0000","last_peered":"2024-09-21T03:45:23.739926+0000","last_clean":"2024-09-21T03:45:23.739926+0000","last_became_active":"2024-09-21T03:43:08.010360+0000","last_became_peered":"2024-09-21T03:43:08.010360+0000","last_unstale":"2024-09-21T03:45:23.739926+0000","last_undegraded":"2024-09-21T03:45:23.739926+0000","last_fullsized":"2024-09-21T03:45:23.739926+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:28.101646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768945+0000","last_change":"2024-09-21T03:43:04.971782+0000","last_active":"2024-09-21T03:44:54.768945+0000","last_peered":"2024-09-21T03:44:54.768945+0000","last_clean":"2024-09-21T03:44:54.768945+0000","last_became_active":"2024-09-21T03:43:04.971375+0000","last_became_peered":"2024-09-21T03:43:04.971375+0000","last_unstale":"2024-09-21T03:44:54.768945+0000","last_undegraded":"2024-09-21T03:44:54.768945+0000","last_fullsized":"2024-09-21T03:44:54.768945+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:49.573490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'92","reported_seq":115,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678174+0000","last_change":"2024-09-21T03:41:42.914435+0000","last_active":"2024-09-21T03:44:54.678174+0000","last_peered":"2024-09-21T03:44:54.678174+0000","last_clean":"2024-09-21T03:44:54.678174+0000","last_became_active":"2024-09-21T03:41:42.909490+0000","last_became_peered":"2024-09-21T03:41:42.909490+0000","last_unstale":"2024-09-21T03:44:54.678174+0000","last_undegraded":"2024-09-21T03:44:54.678174+0000","last_fullsized":"2024-09-21T03:44:54.678174+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:56.507027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:56.507027+0000","last_clean_scrub_stamp":"2024-09-21T03:37:56.507027+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:34.564062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678296+0000","last_change":"2024-09-21T03:43:18.084701+0000","last_active":"2024-09-21T03:44:54.678296+0000","last_peered":"2024-09-21T03:44:54.678296+0000","last_clean":"2024-09-21T03:44:54.678296+0000","last_became_active":"2024-09-21T03:43:18.084251+0000","last_became_peered":"2024-09-21T03:43:18.084251+0000","last_unstale":"2024-09-21T03:44:54.678296+0000","last_undegraded":"2024-09-21T03:44:54.678296+0000","last_fullsized":"2024-09-21T03:44:54.678296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:15.660276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.741599+0000","last_change":"2024-09-21T03:43:08.012195+0000","last_active":"2024-09-21T03:45:23.741599+0000","last_peered":"2024-09-21T03:45:23.741599+0000","last_clean":"2024-09-21T03:45:23.741599+0000","last_became_active":"2024-09-21T03:43:08.011670+0000","last_became_peered":"2024-09-21T03:43:08.011670+0000","last_unstale":"2024-09-21T03:45:23.741599+0000","last_undegraded":"2024-09-21T03:45:23.741599+0000","last_fullsized":"2024-09-21T03:45:23.741599+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:14.290074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'7","reported_seq":187,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:14.752915+0000","last_change":"2024-09-21T03:43:05.567898+0000","last_active":"2024-09-21T03:46:14.752915+0000","last_peered":"2024-09-21T03:46:14.752915+0000","last_clean":"2024-09-21T03:46:14.752915+0000","last_became_active":"2024-09-21T03:43:05.567367+0000","last_became_peered":"2024-09-21T03:43:05.567367+0000","last_unstale":"2024-09-21T03:46:14.752915+0000","last_undegraded":"2024-09-21T03:46:14.752915+0000","last_fullsized":"2024-09-21T03:46:14.752915+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:55.120838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768187+0000","last_change":"2024-09-21T03:43:18.073942+0000","last_active":"2024-09-21T03:44:54.768187+0000","last_peered":"2024-09-21T03:44:54.768187+0000","last_clean":"2024-09-21T03:44:54.768187+0000","last_became_active":"2024-09-21T03:43:18.073323+0000","last_became_peered":"2024-09-21T03:43:18.073323+0000","last_unstale":"2024-09-21T03:44:54.768187+0000","last_undegraded":"2024-09-21T03:44:54.768187+0000","last_fullsized":"2024-09-21T03:44:54.768187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:41.677143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.742436+0000","last_change":"2024-09-21T03:43:08.016636+0000","last_active":"2024-09-21T03:45:23.742436+0000","last_peered":"2024-09-21T03:45:23.742436+0000","last_clean":"2024-09-21T03:45:23.742436+0000","last_became_active":"2024-09-21T03:43:08.016235+0000","last_became_peered":"2024-09-21T03:43:08.016235+0000","last_unstale":"2024-09-21T03:45:23.742436+0000","last_undegraded":"2024-09-21T03:45:23.742436+0000","last_fullsized":"2024-09-21T03:45:23.742436+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:35.365830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767144+0000","last_change":"2024-09-21T03:43:04.980911+0000","last_active":"2024-09-21T03:44:54.767144+0000","last_peered":"2024-09-21T03:44:54.767144+0000","last_clean":"2024-09-21T03:44:54.767144+0000","last_became_active":"2024-09-21T03:43:04.980512+0000","last_became_peered":"2024-09-21T03:43:04.980512+0000","last_unstale":"2024-09-21T03:44:54.767144+0000","last_undegraded":"2024-09-21T03:44:54.767144+0000","last_fullsized":"2024-09-21T03:44:54.767144+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:04.136345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679934+0000","last_change":"2024-09-21T03:43:18.085874+0000","last_active":"2024-09-21T03:44:54.679934+0000","last_peered":"2024-09-21T03:44:54.679934+0000","last_clean":"2024-09-21T03:44:54.679934+0000","last_became_active":"2024-09-21T03:43:18.085381+0000","last_became_peered":"2024-09-21T03:43:18.085381+0000","last_unstale":"2024-09-21T03:44:54.679934+0000","last_undegraded":"2024-09-21T03:44:54.679934+0000","last_fullsized":"2024-09-21T03:44:54.679934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:02.829202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.734618+0000","last_change":"2024-09-21T03:43:08.014776+0000","last_active":"2024-09-21T03:45:23.734618+0000","last_peered":"2024-09-21T03:45:23.734618+0000","last_clean":"2024-09-21T03:45:23.734618+0000","last_became_active":"2024-09-21T03:43:08.014191+0000","last_became_peered":"2024-09-21T03:43:08.014191+0000","last_unstale":"2024-09-21T03:45:23.734618+0000","last_undegraded":"2024-09-21T03:45:23.734618+0000","last_fullsized":"2024-09-21T03:45:23.734618+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:19.114838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766945+0000","last_change":"2024-09-21T03:43:04.973714+0000","last_active":"2024-09-21T03:44:54.766945+0000","last_peered":"2024-09-21T03:44:54.766945+0000","last_clean":"2024-09-21T03:44:54.766945+0000","last_became_active":"2024-09-21T03:43:04.973348+0000","last_became_peered":"2024-09-21T03:43:04.973348+0000","last_unstale":"2024-09-21T03:44:54.766945+0000","last_undegraded":"2024-09-21T03:44:54.766945+0000","last_fullsized":"2024-09-21T03:44:54.766945+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:00.872589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766379+0000","last_change":"2024-09-21T03:43:18.082643+0000","last_active":"2024-09-21T03:44:54.766379+0000","last_peered":"2024-09-21T03:44:54.766379+0000","last_clean":"2024-09-21T03:44:54.766379+0000","last_became_active":"2024-09-21T03:43:18.082157+0000","last_became_peered":"2024-09-21T03:43:18.082157+0000","last_unstale":"2024-09-21T03:44:54.766379+0000","last_undegraded":"2024-09-21T03:44:54.766379+0000","last_fullsized":"2024-09-21T03:44:54.766379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:52.652014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.725796+0000","last_change":"2024-09-21T03:43:08.010913+0000","last_active":"2024-09-21T03:45:23.725796+0000","last_peered":"2024-09-21T03:45:23.725796+0000","last_clean":"2024-09-21T03:45:23.725796+0000","last_became_active":"2024-09-21T03:43:08.010483+0000","last_became_peered":"2024-09-21T03:43:08.010483+0000","last_unstale":"2024-09-21T03:45:23.725796+0000","last_undegraded":"2024-09-21T03:45:23.725796+0000","last_fullsized":"2024-09-21T03:45:23.725796+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:52.158346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.675973+0000","last_change":"2024-09-21T03:43:04.980260+0000","last_active":"2024-09-21T03:44:54.675973+0000","last_peered":"2024-09-21T03:44:54.675973+0000","last_clean":"2024-09-21T03:44:54.675973+0000","last_became_active":"2024-09-21T03:43:04.978999+0000","last_became_peered":"2024-09-21T03:43:04.978999+0000","last_unstale":"2024-09-21T03:44:54.675973+0000","last_undegraded":"2024-09-21T03:44:54.675973+0000","last_fullsized":"2024-09-21T03:44:54.675973+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:33.682591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766371+0000","last_change":"2024-09-21T03:43:18.083665+0000","last_active":"2024-09-21T03:44:54.766371+0000","last_peered":"2024-09-21T03:44:54.766371+0000","last_clean":"2024-09-21T03:44:54.766371+0000","last_became_active":"2024-09-21T03:43:18.083215+0000","last_became_peered":"2024-09-21T03:43:18.083215+0000","last_unstale":"2024-09-21T03:44:54.766371+0000","last_undegraded":"2024-09-21T03:44:54.766371+0000","last_fullsized":"2024-09-21T03:44:54.766371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:55.968968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.739802+0000","last_change":"2024-09-21T03:43:08.010570+0000","last_active":"2024-09-21T03:45:23.739802+0000","last_peered":"2024-09-21T03:45:23.739802+0000","last_clean":"2024-09-21T03:45:23.739802+0000","last_became_active":"2024-09-21T03:43:08.010163+0000","last_became_peered":"2024-09-21T03:43:08.010163+0000","last_unstale":"2024-09-21T03:45:23.739802+0000","last_undegraded":"2024-09-21T03:45:23.739802+0000","last_fullsized":"2024-09-21T03:45:23.739802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:59:17.368534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.707142+0000","last_change":"2024-09-21T03:44:54.706609+0000","last_active":"2024-09-21T03:44:54.707142+0000","last_peered":"2024-09-21T03:44:54.707142+0000","last_clean":"2024-09-21T03:44:54.707142+0000","last_became_active":"2024-09-21T03:44:54.681060+0000","last_became_peered":"2024-09-21T03:44:54.681060+0000","last_unstale":"2024-09-21T03:44:54.707142+0000","last_undegraded":"2024-09-21T03:44:54.707142+0000","last_fullsized":"2024-09-21T03:44:54.707142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:49.797107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766945+0000","last_change":"2024-09-21T03:43:18.089516+0000","last_active":"2024-09-21T03:44:54.766945+0000","last_peered":"2024-09-21T03:44:54.766945+0000","last_clean":"2024-09-21T03:44:54.766945+0000","last_became_active":"2024-09-21T03:43:18.089097+0000","last_became_peered":"2024-09-21T03:43:18.089097+0000","last_unstale":"2024-09-21T03:44:54.766945+0000","last_undegraded":"2024-09-21T03:44:54.766945+0000","last_fullsized":"2024-09-21T03:44:54.766945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:50.428401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.732694+0000","last_change":"2024-09-21T03:43:08.015762+0000","last_active":"2024-09-21T03:45:23.732694+0000","last_peered":"2024-09-21T03:45:23.732694+0000","last_clean":"2024-09-21T03:45:23.732694+0000","last_became_active":"2024-09-21T03:43:08.015389+0000","last_became_peered":"2024-09-21T03:43:08.015389+0000","last_unstale":"2024-09-21T03:45:23.732694+0000","last_undegraded":"2024-09-21T03:45:23.732694+0000","last_fullsized":"2024-09-21T03:45:23.732694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:00.341479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'5","reported_seq":182,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:13.751560+0000","last_change":"2024-09-21T03:43:04.973173+0000","last_active":"2024-09-21T03:46:13.751560+0000","last_peered":"2024-09-21T03:46:13.751560+0000","last_clean":"2024-09-21T03:46:13.751560+0000","last_became_active":"2024-09-21T03:43:04.972740+0000","last_became_peered":"2024-09-21T03:43:04.972740+0000","last_unstale":"2024-09-21T03:46:13.751560+0000","last_undegraded":"2024-09-21T03:46:13.751560+0000","last_fullsized":"2024-09-21T03:46:13.751560+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:26.329220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670519+0000","last_change":"2024-09-21T03:43:18.093730+0000","last_active":"2024-09-21T03:44:54.670519+0000","last_peered":"2024-09-21T03:44:54.670519+0000","last_clean":"2024-09-21T03:44:54.670519+0000","last_became_active":"2024-09-21T03:43:18.089621+0000","last_became_peered":"2024-09-21T03:43:18.089621+0000","last_unstale":"2024-09-21T03:44:54.670519+0000","last_undegraded":"2024-09-21T03:44:54.670519+0000","last_fullsized":"2024-09-21T03:44:54.670519+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:05.227038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":62,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.745319+0000","last_change":"2024-09-21T03:43:54.465751+0000","last_active":"2024-09-21T03:45:23.745319+0000","last_peered":"2024-09-21T03:45:23.745319+0000","last_clean":"2024-09-21T03:45:23.745319+0000","last_became_active":"2024-09-21T03:43:54.465333+0000","last_became_peered":"2024-09-21T03:43:54.465333+0000","last_unstale":"2024-09-21T03:45:23.745319+0000","last_undegraded":"2024-09-21T03:45:23.745319+0000","last_fullsized":"2024-09-21T03:45:23.745319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:34.307699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766735+0000","last_change":"2024-09-21T03:43:04.979385+0000","last_active":"2024-09-21T03:44:54.766735+0000","last_peered":"2024-09-21T03:44:54.766735+0000","last_clean":"2024-09-21T03:44:54.766735+0000","last_became_active":"2024-09-21T03:43:04.978764+0000","last_became_peered":"2024-09-21T03:43:04.978764+0000","last_unstale":"2024-09-21T03:44:54.766735+0000","last_undegraded":"2024-09-21T03:44:54.766735+0000","last_fullsized":"2024-09-21T03:44:54.766735+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:34.256966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677476+0000","last_change":"2024-09-21T03:43:18.095937+0000","last_active":"2024-09-21T03:44:54.677476+0000","last_peered":"2024-09-21T03:44:54.677476+0000","last_clean":"2024-09-21T03:44:54.677476+0000","last_became_active":"2024-09-21T03:43:18.095531+0000","last_became_peered":"2024-09-21T03:43:18.095531+0000","last_unstale":"2024-09-21T03:44:54.677476+0000","last_undegraded":"2024-09-21T03:44:54.677476+0000","last_fullsized":"2024-09-21T03:44:54.677476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:18.089661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.742877+0000","last_change":"2024-09-21T03:43:08.011342+0000","last_active":"2024-09-21T03:45:23.742877+0000","last_peered":"2024-09-21T03:45:23.742877+0000","last_clean":"2024-09-21T03:45:23.742877+0000","last_became_active":"2024-09-21T03:43:08.010963+0000","last_became_peered":"2024-09-21T03:43:08.010963+0000","last_unstale":"2024-09-21T03:45:23.742877+0000","last_undegraded":"2024-09-21T03:45:23.742877+0000","last_fullsized":"2024-09-21T03:45:23.742877+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:05.970863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":57,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679564+0000","last_change":"2024-09-21T03:43:04.972896+0000","last_active":"2024-09-21T03:44:54.679564+0000","last_peered":"2024-09-21T03:44:54.679564+0000","last_clean":"2024-09-21T03:44:54.679564+0000","last_became_active":"2024-09-21T03:43:04.972535+0000","last_became_peered":"2024-09-21T03:43:04.972535+0000","last_unstale":"2024-09-21T03:44:54.679564+0000","last_undegraded":"2024-09-21T03:44:54.679564+0000","last_fullsized":"2024-09-21T03:44:54.679564+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:36.362019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766929+0000","last_change":"2024-09-21T03:43:18.081300+0000","last_active":"2024-09-21T03:44:54.766929+0000","last_peered":"2024-09-21T03:44:54.766929+0000","last_clean":"2024-09-21T03:44:54.766929+0000","last_became_active":"2024-09-21T03:43:18.080874+0000","last_became_peered":"2024-09-21T03:43:18.080874+0000","last_unstale":"2024-09-21T03:44:54.766929+0000","last_undegraded":"2024-09-21T03:44:54.766929+0000","last_fullsized":"2024-09-21T03:44:54.766929+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:13.246348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.718671+0000","last_change":"2024-09-21T03:43:08.014954+0000","last_active":"2024-09-21T03:45:23.718671+0000","last_peered":"2024-09-21T03:45:23.718671+0000","last_clean":"2024-09-21T03:45:23.718671+0000","last_became_active":"2024-09-21T03:43:08.014576+0000","last_became_peered":"2024-09-21T03:43:08.014576+0000","last_unstale":"2024-09-21T03:45:23.718671+0000","last_undegraded":"2024-09-21T03:45:23.718671+0000","last_fullsized":"2024-09-21T03:45:23.718671+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:13.517510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'41","reported_seq":363,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:17.757063+0000","last_change":"2024-09-21T03:43:04.973305+0000","last_active":"2024-09-21T03:46:17.757063+0000","last_peered":"2024-09-21T03:46:17.757063+0000","last_clean":"2024-09-21T03:46:17.757063+0000","last_became_active":"2024-09-21T03:43:04.972891+0000","last_became_peered":"2024-09-21T03:43:04.972891+0000","last_unstale":"2024-09-21T03:46:17.757063+0000","last_undegraded":"2024-09-21T03:46:17.757063+0000","last_fullsized":"2024-09-21T03:46:17.757063+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:15.653769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71976,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":276,"num_read_kb":209,"num_write":43,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670066+0000","last_change":"2024-09-21T03:43:18.093613+0000","last_active":"2024-09-21T03:44:54.670066+0000","last_peered":"2024-09-21T03:44:54.670066+0000","last_clean":"2024-09-21T03:44:54.670066+0000","last_became_active":"2024-09-21T03:43:18.093195+0000","last_became_peered":"2024-09-21T03:43:18.093195+0000","last_unstale":"2024-09-21T03:44:54.670066+0000","last_undegraded":"2024-09-21T03:44:54.670066+0000","last_fullsized":"2024-09-21T03:44:54.670066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:17.825071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.734749+0000","last_change":"2024-09-21T03:43:08.014314+0000","last_active":"2024-09-21T03:45:23.734749+0000","last_peered":"2024-09-21T03:45:23.734749+0000","last_clean":"2024-09-21T03:45:23.734749+0000","last_became_active":"2024-09-21T03:43:08.012925+0000","last_became_peered":"2024-09-21T03:43:08.012925+0000","last_unstale":"2024-09-21T03:45:23.734749+0000","last_undegraded":"2024-09-21T03:45:23.734749+0000","last_fullsized":"2024-09-21T03:45:23.734749+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:47:28.599444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.767094+0000","last_change":"2024-09-21T03:43:05.141584+0000","last_active":"2024-09-21T03:44:54.767094+0000","last_peered":"2024-09-21T03:44:54.767094+0000","last_clean":"2024-09-21T03:44:54.767094+0000","last_became_active":"2024-09-21T03:43:05.141200+0000","last_became_peered":"2024-09-21T03:43:05.141200+0000","last_unstale":"2024-09-21T03:44:54.767094+0000","last_undegraded":"2024-09-21T03:44:54.767094+0000","last_fullsized":"2024-09-21T03:44:54.767094+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:04.357970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768559+0000","last_change":"2024-09-21T03:43:18.092184+0000","last_active":"2024-09-21T03:44:54.768559+0000","last_peered":"2024-09-21T03:44:54.768559+0000","last_clean":"2024-09-21T03:44:54.768559+0000","last_became_active":"2024-09-21T03:43:18.091629+0000","last_became_peered":"2024-09-21T03:43:18.091629+0000","last_unstale":"2024-09-21T03:44:54.768559+0000","last_undegraded":"2024-09-21T03:44:54.768559+0000","last_fullsized":"2024-09-21T03:44:54.768559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:39.448009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.748221+0000","last_change":"2024-09-21T03:43:08.018335+0000","last_active":"2024-09-21T03:45:23.748221+0000","last_peered":"2024-09-21T03:45:23.748221+0000","last_clean":"2024-09-21T03:45:23.748221+0000","last_became_active":"2024-09-21T03:43:08.014005+0000","last_became_peered":"2024-09-21T03:43:08.014005+0000","last_unstale":"2024-09-21T03:45:23.748221+0000","last_undegraded":"2024-09-21T03:45:23.748221+0000","last_fullsized":"2024-09-21T03:45:23.748221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:35.749381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.678554+0000","last_change":"2024-09-21T03:43:04.978609+0000","last_active":"2024-09-21T03:44:54.678554+0000","last_peered":"2024-09-21T03:44:54.678554+0000","last_clean":"2024-09-21T03:44:54.678554+0000","last_became_active":"2024-09-21T03:43:04.978146+0000","last_became_peered":"2024-09-21T03:43:04.978146+0000","last_unstale":"2024-09-21T03:44:54.678554+0000","last_undegraded":"2024-09-21T03:44:54.678554+0000","last_fullsized":"2024-09-21T03:44:54.678554+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:41.651745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.670529+0000","last_change":"2024-09-21T03:43:18.089399+0000","last_active":"2024-09-21T03:44:54.670529+0000","last_peered":"2024-09-21T03:44:54.670529+0000","last_clean":"2024-09-21T03:44:54.670529+0000","last_became_active":"2024-09-21T03:43:18.086156+0000","last_became_peered":"2024-09-21T03:43:18.086156+0000","last_unstale":"2024-09-21T03:44:54.670529+0000","last_undegraded":"2024-09-21T03:44:54.670529+0000","last_fullsized":"2024-09-21T03:44:54.670529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:41.955341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.737922+0000","last_change":"2024-09-21T03:43:08.008773+0000","last_active":"2024-09-21T03:45:23.737922+0000","last_peered":"2024-09-21T03:45:23.737922+0000","last_clean":"2024-09-21T03:45:23.737922+0000","last_became_active":"2024-09-21T03:43:08.008270+0000","last_became_peered":"2024-09-21T03:43:08.008270+0000","last_unstale":"2024-09-21T03:45:23.737922+0000","last_undegraded":"2024-09-21T03:45:23.737922+0000","last_fullsized":"2024-09-21T03:45:23.737922+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:57.901701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'5","reported_seq":315,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:17.757080+0000","last_change":"2024-09-21T03:43:04.978296+0000","last_active":"2024-09-21T03:46:17.757080+0000","last_peered":"2024-09-21T03:46:17.757080+0000","last_clean":"2024-09-21T03:46:17.757080+0000","last_became_active":"2024-09-21T03:43:04.977782+0000","last_became_peered":"2024-09-21T03:43:04.977782+0000","last_unstale":"2024-09-21T03:46:17.757080+0000","last_undegraded":"2024-09-21T03:46:17.757080+0000","last_fullsized":"2024-09-21T03:46:17.757080+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:57.748737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":264,"num_read_kb":264,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.671551+0000","last_change":"2024-09-21T03:43:54.495536+0000","last_active":"2024-09-21T03:44:54.671551+0000","last_peered":"2024-09-21T03:44:54.671551+0000","last_clean":"2024-09-21T03:44:54.671551+0000","last_became_active":"2024-09-21T03:43:54.493492+0000","last_became_peered":"2024-09-21T03:43:54.493492+0000","last_unstale":"2024-09-21T03:44:54.671551+0000","last_undegraded":"2024-09-21T03:44:54.671551+0000","last_fullsized":"2024-09-21T03:44:54.671551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:11.291911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679588+0000","last_change":"2024-09-21T03:43:18.077741+0000","last_active":"2024-09-21T03:44:54.679588+0000","last_peered":"2024-09-21T03:44:54.679588+0000","last_clean":"2024-09-21T03:44:54.679588+0000","last_became_active":"2024-09-21T03:43:18.076619+0000","last_became_peered":"2024-09-21T03:43:18.076619+0000","last_unstale":"2024-09-21T03:44:54.679588+0000","last_undegraded":"2024-09-21T03:44:54.679588+0000","last_fullsized":"2024-09-21T03:44:54.679588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:33.297020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.729664+0000","last_change":"2024-09-21T03:43:08.008838+0000","last_active":"2024-09-21T03:45:23.729664+0000","last_peered":"2024-09-21T03:45:23.729664+0000","last_clean":"2024-09-21T03:45:23.729664+0000","last_became_active":"2024-09-21T03:43:08.008422+0000","last_became_peered":"2024-09-21T03:43:08.008422+0000","last_unstale":"2024-09-21T03:45:23.729664+0000","last_undegraded":"2024-09-21T03:45:23.729664+0000","last_fullsized":"2024-09-21T03:45:23.729664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:00:17.881443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766641+0000","last_change":"2024-09-21T03:43:04.979289+0000","last_active":"2024-09-21T03:44:54.766641+0000","last_peered":"2024-09-21T03:44:54.766641+0000","last_clean":"2024-09-21T03:44:54.766641+0000","last_became_active":"2024-09-21T03:43:04.978843+0000","last_became_peered":"2024-09-21T03:43:04.978843+0000","last_unstale":"2024-09-21T03:44:54.766641+0000","last_undegraded":"2024-09-21T03:44:54.766641+0000","last_fullsized":"2024-09-21T03:44:54.766641+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:50.560607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.677251+0000","last_change":"2024-09-21T03:43:18.096870+0000","last_active":"2024-09-21T03:44:54.677251+0000","last_peered":"2024-09-21T03:44:54.677251+0000","last_clean":"2024-09-21T03:44:54.677251+0000","last_became_active":"2024-09-21T03:43:18.096424+0000","last_became_peered":"2024-09-21T03:43:18.096424+0000","last_unstale":"2024-09-21T03:44:54.677251+0000","last_undegraded":"2024-09-21T03:44:54.677251+0000","last_fullsized":"2024-09-21T03:44:54.677251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:39.532368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.729991+0000","last_change":"2024-09-21T03:43:54.492499+0000","last_active":"2024-09-21T03:45:23.729991+0000","last_peered":"2024-09-21T03:45:23.729991+0000","last_clean":"2024-09-21T03:45:23.729991+0000","last_became_active":"2024-09-21T03:43:54.492043+0000","last_became_peered":"2024-09-21T03:43:54.492043+0000","last_unstale":"2024-09-21T03:45:23.729991+0000","last_undegraded":"2024-09-21T03:45:23.729991+0000","last_fullsized":"2024-09-21T03:45:23.729991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:25.687090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.766447+0000","last_change":"2024-09-21T03:43:04.974265+0000","last_active":"2024-09-21T03:44:54.766447+0000","last_peered":"2024-09-21T03:44:54.766447+0000","last_clean":"2024-09-21T03:44:54.766447+0000","last_became_active":"2024-09-21T03:43:04.973383+0000","last_became_peered":"2024-09-21T03:43:04.973383+0000","last_unstale":"2024-09-21T03:44:54.766447+0000","last_undegraded":"2024-09-21T03:44:54.766447+0000","last_fullsized":"2024-09-21T03:44:54.766447+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:02.312378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.768086+0000","last_change":"2024-09-21T03:43:04.980353+0000","last_active":"2024-09-21T03:44:54.768086+0000","last_peered":"2024-09-21T03:44:54.768086+0000","last_clean":"2024-09-21T03:44:54.768086+0000","last_became_active":"2024-09-21T03:43:04.979922+0000","last_became_peered":"2024-09-21T03:43:04.979922+0000","last_unstale":"2024-09-21T03:44:54.768086+0000","last_undegraded":"2024-09-21T03:44:54.768086+0000","last_fullsized":"2024-09-21T03:44:54.768086+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:17.636536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.740065+0000","last_change":"2024-09-21T03:43:08.009225+0000","last_active":"2024-09-21T03:45:23.740065+0000","last_peered":"2024-09-21T03:45:23.740065+0000","last_clean":"2024-09-21T03:45:23.740065+0000","last_became_active":"2024-09-21T03:43:08.008830+0000","last_became_peered":"2024-09-21T03:43:08.008830+0000","last_unstale":"2024-09-21T03:45:23.740065+0000","last_undegraded":"2024-09-21T03:45:23.740065+0000","last_fullsized":"2024-09-21T03:45:23.740065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:38.047763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679550+0000","last_change":"2024-09-21T03:43:18.089302+0000","last_active":"2024-09-21T03:44:54.679550+0000","last_peered":"2024-09-21T03:44:54.679550+0000","last_clean":"2024-09-21T03:44:54.679550+0000","last_became_active":"2024-09-21T03:43:18.088867+0000","last_became_peered":"2024-09-21T03:43:18.088867+0000","last_unstale":"2024-09-21T03:44:54.679550+0000","last_undegraded":"2024-09-21T03:44:54.679550+0000","last_fullsized":"2024-09-21T03:44:54.679550+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:08.960044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":193,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:46:17.757833+0000","last_change":"2024-09-21T03:43:54.464317+0000","last_active":"2024-09-21T03:46:17.757833+0000","last_peered":"2024-09-21T03:46:17.757833+0000","last_clean":"2024-09-21T03:46:17.757833+0000","last_became_active":"2024-09-21T03:43:54.463870+0000","last_became_peered":"2024-09-21T03:43:54.463870+0000","last_unstale":"2024-09-21T03:46:17.757833+0000","last_undegraded":"2024-09-21T03:46:17.757833+0000","last_fullsized":"2024-09-21T03:46:17.757833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:03.944062+0000","last_clean_scrub_stamp":"2024-09-21T03:43:03.944062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:14.581726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":135,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.727426+0000","last_change":"2024-09-21T03:43:08.009398+0000","last_active":"2024-09-21T03:45:23.727426+0000","last_peered":"2024-09-21T03:45:23.727426+0000","last_clean":"2024-09-21T03:45:23.727426+0000","last_became_active":"2024-09-21T03:43:08.009000+0000","last_became_peered":"2024-09-21T03:43:08.009000+0000","last_unstale":"2024-09-21T03:45:23.727426+0000","last_undegraded":"2024-09-21T03:45:23.727426+0000","last_fullsized":"2024-09-21T03:45:23.727426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:07.278613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:44:54.679894+0000","last_change":"2024-09-21T03:43:18.081834+0000","last_active":"2024-09-21T03:44:54.679894+0000","last_peered":"2024-09-21T03:44:54.679894+0000","last_clean":"2024-09-21T03:44:54.679894+0000","last_became_active":"2024-09-21T03:43:18.081328+0000","last_became_peered":"2024-09-21T03:43:18.081328+0000","last_unstale":"2024-09-21T03:44:54.679894+0000","last_undegraded":"2024-09-21T03:44:54.679894+0000","last_fullsized":"2024-09-21T03:44:54.679894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:17.044093+0000","last_clean_scrub_stamp":"2024-09-21T03:43:17.044093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:37.638826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.725948+0000","last_change":"2024-09-21T03:43:08.005144+0000","last_active":"2024-09-21T03:45:23.725948+0000","last_peered":"2024-09-21T03:45:23.725948+0000","last_clean":"2024-09-21T03:45:23.725948+0000","last_became_active":"2024-09-21T03:43:08.003895+0000","last_became_peered":"2024-09-21T03:43:08.003895+0000","last_unstale":"2024-09-21T03:45:23.725948+0000","last_undegraded":"2024-09-21T03:45:23.725948+0000","last_fullsized":"2024-09-21T03:45:23.725948+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:25.822315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.728904+0000","last_change":"2024-09-21T03:43:08.000218+0000","last_active":"2024-09-21T03:45:23.728904+0000","last_peered":"2024-09-21T03:45:23.728904+0000","last_clean":"2024-09-21T03:45:23.728904+0000","last_became_active":"2024-09-21T03:43:07.999840+0000","last_became_peered":"2024-09-21T03:43:07.999840+0000","last_unstale":"2024-09-21T03:45:23.728904+0000","last_undegraded":"2024-09-21T03:45:23.728904+0000","last_fullsized":"2024-09-21T03:45:23.728904+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:55.391706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.744272+0000","last_change":"2024-09-21T03:43:08.012075+0000","last_active":"2024-09-21T03:45:23.744272+0000","last_peered":"2024-09-21T03:45:23.744272+0000","last_clean":"2024-09-21T03:45:23.744272+0000","last_became_active":"2024-09-21T03:43:08.011666+0000","last_became_peered":"2024-09-21T03:43:08.011666+0000","last_unstale":"2024-09-21T03:45:23.744272+0000","last_undegraded":"2024-09-21T03:45:23.744272+0000","last_fullsized":"2024-09-21T03:45:23.744272+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:42.124948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743198+0000","last_change":"2024-09-21T03:43:08.021082+0000","last_active":"2024-09-21T03:45:23.743198+0000","last_peered":"2024-09-21T03:45:23.743198+0000","last_clean":"2024-09-21T03:45:23.743198+0000","last_became_active":"2024-09-21T03:43:08.020674+0000","last_became_peered":"2024-09-21T03:43:08.020674+0000","last_unstale":"2024-09-21T03:45:23.743198+0000","last_undegraded":"2024-09-21T03:45:23.743198+0000","last_fullsized":"2024-09-21T03:45:23.743198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:13.747594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.744549+0000","last_change":"2024-09-21T03:43:08.004199+0000","last_active":"2024-09-21T03:45:23.744549+0000","last_peered":"2024-09-21T03:45:23.744549+0000","last_clean":"2024-09-21T03:45:23.744549+0000","last_became_active":"2024-09-21T03:43:08.003806+0000","last_became_peered":"2024-09-21T03:43:08.003806+0000","last_unstale":"2024-09-21T03:45:23.744549+0000","last_undegraded":"2024-09-21T03:45:23.744549+0000","last_fullsized":"2024-09-21T03:45:23.744549+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:13.941130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.723624+0000","last_change":"2024-09-21T03:43:08.007237+0000","last_active":"2024-09-21T03:45:23.723624+0000","last_peered":"2024-09-21T03:45:23.723624+0000","last_clean":"2024-09-21T03:45:23.723624+0000","last_became_active":"2024-09-21T03:43:08.006805+0000","last_became_peered":"2024-09-21T03:43:08.006805+0000","last_unstale":"2024-09-21T03:45:23.723624+0000","last_undegraded":"2024-09-21T03:45:23.723624+0000","last_fullsized":"2024-09-21T03:45:23.723624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:26.274494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.729741+0000","last_change":"2024-09-21T03:43:08.007934+0000","last_active":"2024-09-21T03:45:23.729741+0000","last_peered":"2024-09-21T03:45:23.729741+0000","last_clean":"2024-09-21T03:45:23.729741+0000","last_became_active":"2024-09-21T03:43:08.007549+0000","last_became_peered":"2024-09-21T03:43:08.007549+0000","last_unstale":"2024-09-21T03:45:23.729741+0000","last_undegraded":"2024-09-21T03:45:23.729741+0000","last_fullsized":"2024-09-21T03:45:23.729741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:58.158255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.714180+0000","last_change":"2024-09-21T03:43:54.493665+0000","last_active":"2024-09-21T03:45:23.714180+0000","last_peered":"2024-09-21T03:45:23.714180+0000","last_clean":"2024-09-21T03:45:23.714180+0000","last_became_active":"2024-09-21T03:43:54.493245+0000","last_became_peered":"2024-09-21T03:43:54.493245+0000","last_unstale":"2024-09-21T03:45:23.714180+0000","last_undegraded":"2024-09-21T03:45:23.714180+0000","last_fullsized":"2024-09-21T03:45:23.714180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:35.449852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.731123+0000","last_change":"2024-09-21T03:43:54.463419+0000","last_active":"2024-09-21T03:45:23.731123+0000","last_peered":"2024-09-21T03:45:23.731123+0000","last_clean":"2024-09-21T03:45:23.731123+0000","last_became_active":"2024-09-21T03:43:54.462951+0000","last_became_peered":"2024-09-21T03:43:54.462951+0000","last_unstale":"2024-09-21T03:45:23.731123+0000","last_undegraded":"2024-09-21T03:45:23.731123+0000","last_fullsized":"2024-09-21T03:45:23.731123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:23.620275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.717641+0000","last_change":"2024-09-21T03:43:08.011436+0000","last_active":"2024-09-21T03:45:23.717641+0000","last_peered":"2024-09-21T03:45:23.717641+0000","last_clean":"2024-09-21T03:45:23.717641+0000","last_became_active":"2024-09-21T03:43:08.011025+0000","last_became_peered":"2024-09-21T03:43:08.011025+0000","last_unstale":"2024-09-21T03:45:23.717641+0000","last_undegraded":"2024-09-21T03:45:23.717641+0000","last_fullsized":"2024-09-21T03:45:23.717641+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:31.411858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743351+0000","last_change":"2024-09-21T03:43:08.016378+0000","last_active":"2024-09-21T03:45:23.743351+0000","last_peered":"2024-09-21T03:45:23.743351+0000","last_clean":"2024-09-21T03:45:23.743351+0000","last_became_active":"2024-09-21T03:43:08.016013+0000","last_became_peered":"2024-09-21T03:43:08.016013+0000","last_unstale":"2024-09-21T03:45:23.743351+0000","last_undegraded":"2024-09-21T03:45:23.743351+0000","last_fullsized":"2024-09-21T03:45:23.743351+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:01.865669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743940+0000","last_change":"2024-09-21T03:43:08.007862+0000","last_active":"2024-09-21T03:45:23.743940+0000","last_peered":"2024-09-21T03:45:23.743940+0000","last_clean":"2024-09-21T03:45:23.743940+0000","last_became_active":"2024-09-21T03:43:08.007433+0000","last_became_peered":"2024-09-21T03:43:08.007433+0000","last_unstale":"2024-09-21T03:45:23.743940+0000","last_undegraded":"2024-09-21T03:45:23.743940+0000","last_fullsized":"2024-09-21T03:45:23.743940+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:22:03.364767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.723330+0000","last_change":"2024-09-21T03:43:08.010542+0000","last_active":"2024-09-21T03:45:23.723330+0000","last_peered":"2024-09-21T03:45:23.723330+0000","last_clean":"2024-09-21T03:45:23.723330+0000","last_became_active":"2024-09-21T03:43:08.010195+0000","last_became_peered":"2024-09-21T03:43:08.010195+0000","last_unstale":"2024-09-21T03:45:23.723330+0000","last_undegraded":"2024-09-21T03:45:23.723330+0000","last_fullsized":"2024-09-21T03:45:23.723330+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:52.850657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.728086+0000","last_change":"2024-09-21T03:43:08.006168+0000","last_active":"2024-09-21T03:45:23.728086+0000","last_peered":"2024-09-21T03:45:23.728086+0000","last_clean":"2024-09-21T03:45:23.728086+0000","last_became_active":"2024-09-21T03:43:08.005283+0000","last_became_peered":"2024-09-21T03:43:08.005283+0000","last_unstale":"2024-09-21T03:45:23.728086+0000","last_undegraded":"2024-09-21T03:45:23.728086+0000","last_fullsized":"2024-09-21T03:45:23.728086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:59.852412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.731547+0000","last_change":"2024-09-21T03:43:08.005870+0000","last_active":"2024-09-21T03:45:23.731547+0000","last_peered":"2024-09-21T03:45:23.731547+0000","last_clean":"2024-09-21T03:45:23.731547+0000","last_became_active":"2024-09-21T03:43:08.005485+0000","last_became_peered":"2024-09-21T03:43:08.005485+0000","last_unstale":"2024-09-21T03:45:23.731547+0000","last_undegraded":"2024-09-21T03:45:23.731547+0000","last_fullsized":"2024-09-21T03:45:23.731547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:41.387865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.741258+0000","last_change":"2024-09-21T03:43:08.015176+0000","last_active":"2024-09-21T03:45:23.741258+0000","last_peered":"2024-09-21T03:45:23.741258+0000","last_clean":"2024-09-21T03:45:23.741258+0000","last_became_active":"2024-09-21T03:43:08.014451+0000","last_became_peered":"2024-09-21T03:43:08.014451+0000","last_unstale":"2024-09-21T03:45:23.741258+0000","last_undegraded":"2024-09-21T03:45:23.741258+0000","last_fullsized":"2024-09-21T03:45:23.741258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:49.524800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.747012+0000","last_change":"2024-09-21T03:43:08.016798+0000","last_active":"2024-09-21T03:45:23.747012+0000","last_peered":"2024-09-21T03:45:23.747012+0000","last_clean":"2024-09-21T03:45:23.747012+0000","last_became_active":"2024-09-21T03:43:08.014842+0000","last_became_peered":"2024-09-21T03:43:08.014842+0000","last_unstale":"2024-09-21T03:45:23.747012+0000","last_undegraded":"2024-09-21T03:45:23.747012+0000","last_fullsized":"2024-09-21T03:45:23.747012+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:19.290618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.746263+0000","last_change":"2024-09-21T03:43:08.021285+0000","last_active":"2024-09-21T03:45:23.746263+0000","last_peered":"2024-09-21T03:45:23.746263+0000","last_clean":"2024-09-21T03:45:23.746263+0000","last_became_active":"2024-09-21T03:43:08.020904+0000","last_became_peered":"2024-09-21T03:43:08.020904+0000","last_unstale":"2024-09-21T03:45:23.746263+0000","last_undegraded":"2024-09-21T03:45:23.746263+0000","last_fullsized":"2024-09-21T03:45:23.746263+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:56.589089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.745437+0000","last_change":"2024-09-21T03:43:08.017599+0000","last_active":"2024-09-21T03:45:23.745437+0000","last_peered":"2024-09-21T03:45:23.745437+0000","last_clean":"2024-09-21T03:45:23.745437+0000","last_became_active":"2024-09-21T03:43:08.016138+0000","last_became_peered":"2024-09-21T03:43:08.016138+0000","last_unstale":"2024-09-21T03:45:23.745437+0000","last_undegraded":"2024-09-21T03:45:23.745437+0000","last_fullsized":"2024-09-21T03:45:23.745437+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:09.902246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.732277+0000","last_change":"2024-09-21T03:43:08.009598+0000","last_active":"2024-09-21T03:45:23.732277+0000","last_peered":"2024-09-21T03:45:23.732277+0000","last_clean":"2024-09-21T03:45:23.732277+0000","last_became_active":"2024-09-21T03:43:08.007488+0000","last_became_peered":"2024-09-21T03:43:08.007488+0000","last_unstale":"2024-09-21T03:45:23.732277+0000","last_undegraded":"2024-09-21T03:45:23.732277+0000","last_fullsized":"2024-09-21T03:45:23.732277+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:23.192761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.732050+0000","last_change":"2024-09-21T03:43:08.016450+0000","last_active":"2024-09-21T03:45:23.732050+0000","last_peered":"2024-09-21T03:45:23.732050+0000","last_clean":"2024-09-21T03:45:23.732050+0000","last_became_active":"2024-09-21T03:43:08.016037+0000","last_became_peered":"2024-09-21T03:43:08.016037+0000","last_unstale":"2024-09-21T03:45:23.732050+0000","last_undegraded":"2024-09-21T03:45:23.732050+0000","last_fullsized":"2024-09-21T03:45:23.732050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:39.109408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.726221+0000","last_change":"2024-09-21T03:43:08.010298+0000","last_active":"2024-09-21T03:45:23.726221+0000","last_peered":"2024-09-21T03:45:23.726221+0000","last_clean":"2024-09-21T03:45:23.726221+0000","last_became_active":"2024-09-21T03:43:08.009847+0000","last_became_peered":"2024-09-21T03:43:08.009847+0000","last_unstale":"2024-09-21T03:45:23.726221+0000","last_undegraded":"2024-09-21T03:45:23.726221+0000","last_fullsized":"2024-09-21T03:45:23.726221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:37.463456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.736715+0000","last_change":"2024-09-21T03:43:08.017834+0000","last_active":"2024-09-21T03:45:23.736715+0000","last_peered":"2024-09-21T03:45:23.736715+0000","last_clean":"2024-09-21T03:45:23.736715+0000","last_became_active":"2024-09-21T03:43:08.016910+0000","last_became_peered":"2024-09-21T03:43:08.016910+0000","last_unstale":"2024-09-21T03:45:23.736715+0000","last_undegraded":"2024-09-21T03:45:23.736715+0000","last_fullsized":"2024-09-21T03:45:23.736715+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:50.528651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.737971+0000","last_change":"2024-09-21T03:43:08.013945+0000","last_active":"2024-09-21T03:45:23.737971+0000","last_peered":"2024-09-21T03:45:23.737971+0000","last_clean":"2024-09-21T03:45:23.737971+0000","last_became_active":"2024-09-21T03:43:08.013598+0000","last_became_peered":"2024-09-21T03:43:08.013598+0000","last_unstale":"2024-09-21T03:45:23.737971+0000","last_undegraded":"2024-09-21T03:45:23.737971+0000","last_fullsized":"2024-09-21T03:45:23.737971+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:58.182938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.726374+0000","last_change":"2024-09-21T03:43:08.018824+0000","last_active":"2024-09-21T03:45:23.726374+0000","last_peered":"2024-09-21T03:45:23.726374+0000","last_clean":"2024-09-21T03:45:23.726374+0000","last_became_active":"2024-09-21T03:43:08.017196+0000","last_became_peered":"2024-09-21T03:43:08.017196+0000","last_unstale":"2024-09-21T03:45:23.726374+0000","last_undegraded":"2024-09-21T03:45:23.726374+0000","last_fullsized":"2024-09-21T03:45:23.726374+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:02.357523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.743021+0000","last_change":"2024-09-21T03:43:08.003514+0000","last_active":"2024-09-21T03:45:23.743021+0000","last_peered":"2024-09-21T03:45:23.743021+0000","last_clean":"2024-09-21T03:45:23.743021+0000","last_became_active":"2024-09-21T03:43:08.003131+0000","last_became_peered":"2024-09-21T03:43:08.003131+0000","last_unstale":"2024-09-21T03:45:23.743021+0000","last_undegraded":"2024-09-21T03:45:23.743021+0000","last_fullsized":"2024-09-21T03:45:23.743021+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:23.170302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.735332+0000","last_change":"2024-09-21T03:43:08.015890+0000","last_active":"2024-09-21T03:45:23.735332+0000","last_peered":"2024-09-21T03:45:23.735332+0000","last_clean":"2024-09-21T03:45:23.735332+0000","last_became_active":"2024-09-21T03:43:08.015519+0000","last_became_peered":"2024-09-21T03:43:08.015519+0000","last_unstale":"2024-09-21T03:45:23.735332+0000","last_undegraded":"2024-09-21T03:45:23.735332+0000","last_fullsized":"2024-09-21T03:45:23.735332+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:10.875923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.727227+0000","last_change":"2024-09-21T03:43:08.012261+0000","last_active":"2024-09-21T03:45:23.727227+0000","last_peered":"2024-09-21T03:45:23.727227+0000","last_clean":"2024-09-21T03:45:23.727227+0000","last_became_active":"2024-09-21T03:43:08.011783+0000","last_became_peered":"2024-09-21T03:43:08.011783+0000","last_unstale":"2024-09-21T03:45:23.727227+0000","last_undegraded":"2024-09-21T03:45:23.727227+0000","last_fullsized":"2024-09-21T03:45:23.727227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:07.011340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.724685+0000","last_change":"2024-09-21T03:43:54.494826+0000","last_active":"2024-09-21T03:45:23.724685+0000","last_peered":"2024-09-21T03:45:23.724685+0000","last_clean":"2024-09-21T03:45:23.724685+0000","last_became_active":"2024-09-21T03:43:54.494404+0000","last_became_peered":"2024-09-21T03:43:54.494404+0000","last_unstale":"2024-09-21T03:45:23.724685+0000","last_undegraded":"2024-09-21T03:45:23.724685+0000","last_fullsized":"2024-09-21T03:45:23.724685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:42.109053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.740074+0000","last_change":"2024-09-21T03:43:08.016214+0000","last_active":"2024-09-21T03:45:23.740074+0000","last_peered":"2024-09-21T03:45:23.740074+0000","last_clean":"2024-09-21T03:45:23.740074+0000","last_became_active":"2024-09-21T03:43:08.014815+0000","last_became_peered":"2024-09-21T03:43:08.014815+0000","last_unstale":"2024-09-21T03:45:23.740074+0000","last_undegraded":"2024-09-21T03:45:23.740074+0000","last_fullsized":"2024-09-21T03:45:23.740074+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:04.742829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.745698+0000","last_change":"2024-09-21T03:43:08.019269+0000","last_active":"2024-09-21T03:45:23.745698+0000","last_peered":"2024-09-21T03:45:23.745698+0000","last_clean":"2024-09-21T03:45:23.745698+0000","last_became_active":"2024-09-21T03:43:08.017962+0000","last_became_peered":"2024-09-21T03:43:08.017962+0000","last_unstale":"2024-09-21T03:45:23.745698+0000","last_undegraded":"2024-09-21T03:45:23.745698+0000","last_fullsized":"2024-09-21T03:45:23.745698+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:05.950697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"86'8","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T03:45:23.738799+0000","last_change":"2024-09-21T03:43:08.014790+0000","last_active":"2024-09-21T03:45:23.738799+0000","last_peered":"2024-09-21T03:45:23.738799+0000","last_clean":"2024-09-21T03:45:23.738799+0000","last_became_active":"2024-09-21T03:43:08.013891+0000","last_became_peered":"2024-09-21T03:43:08.013891+0000","last_unstale":"2024-09-21T03:45:23.738799+0000","last_undegraded":"2024-09-21T03:45:23.738799+0000","last_fullsized":"2024-09-21T03:45:23.738799+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:43:06.969734+0000","last_clean_scrub_stamp":"2024-09-21T03:43:06.969734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:46.045675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79241,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1352,"num_read_kb":764,"num_write":265,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":237723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808886,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":501273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972411,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1708032,"data_stored":1163517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135935,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":500559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332165,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":500825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495691,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36924,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708228,"statfs":{"total":95995035648,"available":95957225472,"internally_reserved":0,"allocated":1617920,"data_stored":1097354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659217,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":496374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855447,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36932,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708220,"statfs":{"total":95995035648,"available":95957217280,"internally_reserved":0,"allocated":1626112,"data_stored":1092625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018972,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":500759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215201,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":495771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444136,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":569199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":569380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":498051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":73858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":73712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:46:20.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:46:21.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:20 smithi134 ceph-mon[35031]: pgmap v419: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-21T03:46:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:21.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:21 smithi016 ceph-mon[30375]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:21.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:21 smithi123 ceph-mon[34673]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.982+0000 7f3825000640 1 -- 172.21.15.16:0/3671491172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38201039b0 msgr2=0x7f3820103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.982+0000 7f3825000640 1 --2- 172.21.15.16:0/3671491172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38201039b0 0x7f3820103dc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3810009900 tx=0x7f381002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:46:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.982+0000 7f3825000640 1 -- 172.21.15.16:0/3671491172 shutdown_connections 2024-09-21T03:46:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.982+0000 7f3825000640 1 --2- 172.21.15.16:0/3671491172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3820109c40 0x7f3820110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.982+0000 7f3825000640 1 --2- 172.21.15.16:0/3671491172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38201092f0 0x7f3820109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.982+0000 7f3825000640 1 --2- 172.21.15.16:0/3671491172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38201039b0 0x7f3820103dc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.982+0000 7f3825000640 1 -- 172.21.15.16:0/3671491172 >> 172.21.15.16:0/3671491172 conn(0x7f38200ff400 msgr2=0x7f3820101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.982+0000 7f3825000640 1 -- 172.21.15.16:0/3671491172 shutdown_connections 2024-09-21T03:46:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.982+0000 7f3825000640 1 -- 172.21.15.16:0/3671491172 wait complete. 2024-09-21T03:46:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.984+0000 7f3825000640 1 Processor -- start 2024-09-21T03:46:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.984+0000 7f3825000640 1 -- start start 2024-09-21T03:46:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.984+0000 7f3825000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38201039b0 0x7f3820075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.984+0000 7f3825000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38201092f0 0x7f3820075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.985+0000 7f3825000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3820109c40 0x7f3820076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.985+0000 7f3825000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3820113900 con 0x7f3820109c40 2024-09-21T03:46:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.985+0000 7f3825000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3820113780 con 0x7f38201039b0 2024-09-21T03:46:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.985+0000 7f3825000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3820113a80 con 0x7f38201092f0 2024-09-21T03:46:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.985+0000 7f381fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3820109c40 0x7f3820076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.985+0000 7f381f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38201039b0 0x7f3820075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.985+0000 7f381fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3820109c40 0x7f3820076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55028/0 (socket says 172.21.15.16:55028) 2024-09-21T03:46:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.985+0000 7f381f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38201039b0 0x7f3820075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44208/0 (socket says 172.21.15.16:44208) 2024-09-21T03:46:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.985+0000 7f381fe00640 1 -- 172.21.15.16:0/1955149825 learned_addr learned my addr 172.21.15.16:0/1955149825 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.985+0000 7f381ea00640 1 --2- 172.21.15.16:0/1955149825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38201092f0 0x7f3820075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.986+0000 7f381fe00640 1 -- 172.21.15.16:0/1955149825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38201092f0 msgr2=0x7f3820075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.986+0000 7f381fe00640 1 --2- 172.21.15.16:0/1955149825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38201092f0 0x7f3820075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.986+0000 7f381fe00640 1 -- 172.21.15.16:0/1955149825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38201039b0 msgr2=0x7f3820075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.986+0000 7f381fe00640 1 --2- 172.21.15.16:0/1955149825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38201039b0 0x7f3820075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.986+0000 7f381fe00640 1 -- 172.21.15.16:0/1955149825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38201a8590 con 0x7f3820109c40 2024-09-21T03:46:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.986+0000 7f381ea00640 1 --2- 172.21.15.16:0/1955149825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38201092f0 0x7f3820075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.986+0000 7f381fe00640 1 --2- 172.21.15.16:0/1955149825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3820109c40 0x7f3820076190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f381800eaa0 tx=0x7f381800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.986+0000 7f3817e00640 1 -- 172.21.15.16:0/1955149825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381800cdb0 con 0x7f3820109c40 2024-09-21T03:46:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.986+0000 7f3817e00640 1 -- 172.21.15.16:0/1955149825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3818005020 con 0x7f3820109c40 2024-09-21T03:46:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.986+0000 7f3825000640 1 -- 172.21.15.16:0/1955149825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38201a8880 con 0x7f3820109c40 2024-09-21T03:46:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.987+0000 7f3817e00640 1 -- 172.21.15.16:0/1955149825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38180116d0 con 0x7f3820109c40 2024-09-21T03:46:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.987+0000 7f3825000640 1 -- 172.21.15.16:0/1955149825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38201a8dc0 con 0x7f3820109c40 2024-09-21T03:46:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.988+0000 7f3825000640 1 -- 172.21.15.16:0/1955149825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3820103dc0 con 0x7f3820109c40 2024-09-21T03:46:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.991+0000 7f3817e00640 1 -- 172.21.15.16:0/1955149825 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f38180051d0 con 0x7f3820109c40 2024-09-21T03:46:21.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.993+0000 7f3817e00640 1 --2- 172.21.15.16:0/1955149825 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38040b6590 0x7f38040b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:21.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.993+0000 7f3817e00640 1 -- 172.21.15.16:0/1955149825 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f38180e7730 con 0x7f3820109c40 2024-09-21T03:46:21.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.993+0000 7f381f400640 1 --2- 172.21.15.16:0/1955149825 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38040b6590 0x7f38040b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.994+0000 7f381f400640 1 --2- 172.21.15.16:0/1955149825 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38040b6590 0x7f38040b8aa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3810000c00 tx=0x7f3810031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:21.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:21.997+0000 7f3817e00640 1 -- 172.21.15.16:0/1955149825 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3818016030 con 0x7f3820109c40 2024-09-21T03:46:22.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:21 smithi134 ceph-mon[35031]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:46:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:22.239+0000 7f3825000640 1 -- 172.21.15.16:0/1955149825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f382002e640 con 0x7f3820109c40 2024-09-21T03:46:22.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:22.240+0000 7f3817e00640 1 -- 172.21.15.16:0/1955149825 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f38180b1f90 con 0x7f3820109c40 2024-09-21T03:46:22.242 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:46:22.242 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:46:22.242 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:46:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:22.243+0000 7f3825000640 1 -- 172.21.15.16:0/1955149825 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38040b6590 msgr2=0x7f38040b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:22.243+0000 7f3825000640 1 --2- 172.21.15.16:0/1955149825 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38040b6590 0x7f38040b8aa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3810000c00 tx=0x7f3810031400 comp rx=0 tx=0).stop 2024-09-21T03:46:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:22.243+0000 7f3825000640 1 -- 172.21.15.16:0/1955149825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3820109c40 msgr2=0x7f3820076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:22.243+0000 7f3825000640 1 --2- 172.21.15.16:0/1955149825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3820109c40 0x7f3820076190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f381800eaa0 tx=0x7f381800ef60 comp rx=0 tx=0).stop 2024-09-21T03:46:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:22.243+0000 7f3825000640 1 -- 172.21.15.16:0/1955149825 shutdown_connections 2024-09-21T03:46:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:22.243+0000 7f3825000640 1 --2- 172.21.15.16:0/1955149825 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f38040b6590 0x7f38040b8aa0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:22.243+0000 7f3825000640 1 --2- 172.21.15.16:0/1955149825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3820109c40 0x7f3820076190 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:22.243+0000 7f3825000640 1 --2- 172.21.15.16:0/1955149825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38201092f0 0x7f3820075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:22.243+0000 7f3825000640 1 --2- 172.21.15.16:0/1955149825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38201039b0 0x7f3820075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:22.243+0000 7f3825000640 1 -- 172.21.15.16:0/1955149825 >> 172.21.15.16:0/1955149825 conn(0x7f38200ff400 msgr2=0x7f3820101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:22.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:22.243+0000 7f3825000640 1 -- 172.21.15.16:0/1955149825 shutdown_connections 2024-09-21T03:46:22.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:22.243+0000 7f3825000640 1 -- 172.21.15.16:0/1955149825 wait complete. 2024-09-21T03:46:22.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:22 smithi123 ceph-mon[34673]: pgmap v420: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-21T03:46:22.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:22 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1955149825' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:22.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:22 smithi016 ceph-mon[30375]: pgmap v420: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-21T03:46:22.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:22 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1955149825' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:23.009 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:46:23.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:22 smithi134 ceph-mon[35031]: pgmap v420: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-21T03:46:23.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:22 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1955149825' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:23.391 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.000+0000 7f5234c00640 1 -- 172.21.15.16:0/1591775818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f523010be40 msgr2=0x7f5230112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.000+0000 7f5234c00640 1 --2- 172.21.15.16:0/1591775818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f523010be40 0x7f5230112670 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5228009ca0 tx=0x7f522802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:46:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.000+0000 7f5234c00640 1 -- 172.21.15.16:0/1591775818 shutdown_connections 2024-09-21T03:46:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.000+0000 7f5234c00640 1 --2- 172.21.15.16:0/1591775818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f523010be40 0x7f5230112670 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.000+0000 7f5234c00640 1 --2- 172.21.15.16:0/1591775818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f523010b4f0 0x7f523010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.000+0000 7f5234c00640 1 --2- 172.21.15.16:0/1591775818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5230105bb0 0x7f5230105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.000+0000 7f5234c00640 1 -- 172.21.15.16:0/1591775818 >> 172.21.15.16:0/1591775818 conn(0x7f5230101680 msgr2=0x7f5230103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.000+0000 7f5234c00640 1 -- 172.21.15.16:0/1591775818 shutdown_connections 2024-09-21T03:46:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.000+0000 7f5234c00640 1 -- 172.21.15.16:0/1591775818 wait complete. 2024-09-21T03:46:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.001+0000 7f5234c00640 1 Processor -- start 2024-09-21T03:46:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.001+0000 7f5234c00640 1 -- start start 2024-09-21T03:46:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.002+0000 7f5234c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5230105bb0 0x7f523019c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.002+0000 7f5234c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f523010b4f0 0x7f523019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.002+0000 7f5234c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f523010be40 0x7f52301a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.002+0000 7f5234c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5230115b00 con 0x7f523010b4f0 2024-09-21T03:46:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.002+0000 7f5234c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5230115980 con 0x7f5230105bb0 2024-09-21T03:46:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.002+0000 7f5234c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5230115c80 con 0x7f523010be40 2024-09-21T03:46:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.002+0000 7f522f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5230105bb0 0x7f523019c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.002+0000 7f5227e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f523010b4f0 0x7f523019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.002+0000 7f522f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5230105bb0 0x7f523019c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44232/0 (socket says 172.21.15.16:44232) 2024-09-21T03:46:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.002+0000 7f522f400640 1 -- 172.21.15.16:0/571444001 learned_addr learned my addr 172.21.15.16:0/571444001 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.002+0000 7f522fe00640 1 --2- 172.21.15.16:0/571444001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f523010be40 0x7f52301a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.003+0000 7f522f400640 1 -- 172.21.15.16:0/571444001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f523010be40 msgr2=0x7f52301a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.003+0000 7f522f400640 1 --2- 172.21.15.16:0/571444001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f523010be40 0x7f52301a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.003+0000 7f522f400640 1 -- 172.21.15.16:0/571444001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f523010b4f0 msgr2=0x7f523019ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.003+0000 7f522f400640 1 --2- 172.21.15.16:0/571444001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f523010b4f0 0x7f523019ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.003+0000 7f522f400640 1 -- 172.21.15.16:0/571444001 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52301a2440 con 0x7f5230105bb0 2024-09-21T03:46:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.003+0000 7f522f400640 1 --2- 172.21.15.16:0/571444001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5230105bb0 0x7f523019c4d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f522000e9f0 tx=0x7f522000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.003+0000 7f522cc00640 1 -- 172.21.15.16:0/571444001 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522000cd80 con 0x7f5230105bb0 2024-09-21T03:46:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.003+0000 7f5234c00640 1 -- 172.21.15.16:0/571444001 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52301a2730 con 0x7f5230105bb0 2024-09-21T03:46:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.003+0000 7f522cc00640 1 -- 172.21.15.16:0/571444001 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52200108e0 con 0x7f5230105bb0 2024-09-21T03:46:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.003+0000 7f522cc00640 1 -- 172.21.15.16:0/571444001 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5220018980 con 0x7f5230105bb0 2024-09-21T03:46:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.004+0000 7f5234c00640 1 -- 172.21.15.16:0/571444001 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52301a8e90 con 0x7f5230105bb0 2024-09-21T03:46:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.005+0000 7f5234c00640 1 -- 172.21.15.16:0/571444001 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5230105fc0 con 0x7f5230105bb0 2024-09-21T03:46:24.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.007+0000 7f522cc00640 1 -- 172.21.15.16:0/571444001 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5220010430 con 0x7f5230105bb0 2024-09-21T03:46:24.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.008+0000 7f522cc00640 1 --2- 172.21.15.16:0/571444001 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f52140b6590 0x7f52140b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:24.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.009+0000 7f522cc00640 1 -- 172.21.15.16:0/571444001 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f52200e7e50 con 0x7f5230105bb0 2024-09-21T03:46:24.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.009+0000 7f5227e00640 1 --2- 172.21.15.16:0/571444001 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f52140b6590 0x7f52140b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.010+0000 7f5227e00640 1 --2- 172.21.15.16:0/571444001 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f52140b6590 0x7f52140b8aa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5230104ed0 tx=0x7f5218005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.010+0000 7f522cc00640 1 -- 172.21.15.16:0/571444001 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f52200b26b0 con 0x7f5230105bb0 2024-09-21T03:46:24.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.248+0000 7f5234c00640 1 -- 172.21.15.16:0/571444001 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f523019d600 con 0x7f5230105bb0 2024-09-21T03:46:24.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.249+0000 7f522cc00640 1 -- 172.21.15.16:0/571444001 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f52200b75f0 con 0x7f5230105bb0 2024-09-21T03:46:24.252 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:46:24.252 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:46:24.252 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:46:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.252+0000 7f5234c00640 1 -- 172.21.15.16:0/571444001 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f52140b6590 msgr2=0x7f52140b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.252+0000 7f5234c00640 1 --2- 172.21.15.16:0/571444001 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f52140b6590 0x7f52140b8aa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5230104ed0 tx=0x7f5218005f90 comp rx=0 tx=0).stop 2024-09-21T03:46:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.252+0000 7f5234c00640 1 -- 172.21.15.16:0/571444001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5230105bb0 msgr2=0x7f523019c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.252+0000 7f5234c00640 1 --2- 172.21.15.16:0/571444001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5230105bb0 0x7f523019c4d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f522000e9f0 tx=0x7f522000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:46:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.252+0000 7f5234c00640 1 -- 172.21.15.16:0/571444001 shutdown_connections 2024-09-21T03:46:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.252+0000 7f5234c00640 1 --2- 172.21.15.16:0/571444001 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f52140b6590 0x7f52140b8aa0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.252+0000 7f5234c00640 1 --2- 172.21.15.16:0/571444001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f523010be40 0x7f52301a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.252+0000 7f5234c00640 1 --2- 172.21.15.16:0/571444001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f523010b4f0 0x7f523019ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.252+0000 7f5234c00640 1 --2- 172.21.15.16:0/571444001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5230105bb0 0x7f523019c4d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.252+0000 7f5234c00640 1 -- 172.21.15.16:0/571444001 >> 172.21.15.16:0/571444001 conn(0x7f5230101680 msgr2=0x7f523010ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.252+0000 7f5234c00640 1 -- 172.21.15.16:0/571444001 shutdown_connections 2024-09-21T03:46:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:24.252+0000 7f5234c00640 1 -- 172.21.15.16:0/571444001 wait complete. 2024-09-21T03:46:24.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:24 smithi123 ceph-mon[34673]: pgmap v421: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-21T03:46:24.933 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:24 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/571444001' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:24.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:24 smithi016 ceph-mon[30375]: pgmap v421: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-21T03:46:24.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:24 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/571444001' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:25.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:46:25.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:24 smithi134 ceph-mon[35031]: pgmap v421: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-21T03:46:25.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:24 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/571444001' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:25.423 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.092+0000 7f0817200640 1 -- 172.21.15.16:0/644104981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08181092f0 msgr2=0x7f0818109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.092+0000 7f0817200640 1 --2- 172.21.15.16:0/644104981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08181092f0 0x7f0818109700 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0804009900 tx=0x7f080402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:46:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.092+0000 7f0817200640 1 -- 172.21.15.16:0/644104981 shutdown_connections 2024-09-21T03:46:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.092+0000 7f0817200640 1 --2- 172.21.15.16:0/644104981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0818109c40 0x7f0818110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.092+0000 7f0817200640 1 --2- 172.21.15.16:0/644104981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08181092f0 0x7f0818109700 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.092+0000 7f0817200640 1 --2- 172.21.15.16:0/644104981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08181039b0 0x7f0818103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.092+0000 7f0817200640 1 -- 172.21.15.16:0/644104981 >> 172.21.15.16:0/644104981 conn(0x7f08180ff400 msgr2=0x7f0818101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.092+0000 7f0817200640 1 -- 172.21.15.16:0/644104981 shutdown_connections 2024-09-21T03:46:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.093+0000 7f0817200640 1 -- 172.21.15.16:0/644104981 wait complete. 2024-09-21T03:46:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.094+0000 7f0817200640 1 Processor -- start 2024-09-21T03:46:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.094+0000 7f0817200640 1 -- start start 2024-09-21T03:46:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.094+0000 7f0817200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08181039b0 0x7f081819c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.094+0000 7f0817200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08181092f0 0x7f081819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.094+0000 7f0817200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0818109c40 0x7f08181a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.094+0000 7f0817200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0818113950 con 0x7f0818109c40 2024-09-21T03:46:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.094+0000 7f0817200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08181137d0 con 0x7f08181092f0 2024-09-21T03:46:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.094+0000 7f0817200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0818113ad0 con 0x7f08181039b0 2024-09-21T03:46:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.095+0000 7f0816800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0818109c40 0x7f08181a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.095+0000 7f0816800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0818109c40 0x7f08181a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55058/0 (socket says 172.21.15.16:55058) 2024-09-21T03:46:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.095+0000 7f0816800640 1 -- 172.21.15.16:0/4166817148 learned_addr learned my addr 172.21.15.16:0/4166817148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.095+0000 7f0815400640 1 --2- 172.21.15.16:0/4166817148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08181092f0 0x7f081819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.095+0000 7f0815e00640 1 --2- 172.21.15.16:0/4166817148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08181039b0 0x7f081819c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.095+0000 7f0816800640 1 -- 172.21.15.16:0/4166817148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08181039b0 msgr2=0x7f081819c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.095+0000 7f0816800640 1 --2- 172.21.15.16:0/4166817148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08181039b0 0x7f081819c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.095+0000 7f0816800640 1 -- 172.21.15.16:0/4166817148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08181092f0 msgr2=0x7f081819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.095+0000 7f0816800640 1 --2- 172.21.15.16:0/4166817148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08181092f0 0x7f081819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.095+0000 7f0816800640 1 -- 172.21.15.16:0/4166817148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08181a2490 con 0x7f0818109c40 2024-09-21T03:46:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.096+0000 7f0816800640 1 --2- 172.21.15.16:0/4166817148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0818109c40 0x7f08181a1d60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f081000eb10 tx=0x7f081000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.096+0000 7f080ea00640 1 -- 172.21.15.16:0/4166817148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810004b20 con 0x7f0818109c40 2024-09-21T03:46:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.096+0000 7f080ea00640 1 -- 172.21.15.16:0/4166817148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0810004cc0 con 0x7f0818109c40 2024-09-21T03:46:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.096+0000 7f080ea00640 1 -- 172.21.15.16:0/4166817148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810010670 con 0x7f0818109c40 2024-09-21T03:46:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.096+0000 7f0817200640 1 -- 172.21.15.16:0/4166817148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08181a2780 con 0x7f0818109c40 2024-09-21T03:46:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.096+0000 7f0817200640 1 -- 172.21.15.16:0/4166817148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08181a8ee0 con 0x7f0818109c40 2024-09-21T03:46:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.097+0000 7f0817200640 1 -- 172.21.15.16:0/4166817148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0818103dc0 con 0x7f0818109c40 2024-09-21T03:46:26.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.099+0000 7f080ea00640 1 -- 172.21.15.16:0/4166817148 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0810005020 con 0x7f0818109c40 2024-09-21T03:46:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.102+0000 7f080ea00640 1 --2- 172.21.15.16:0/4166817148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f07fc0b6510 0x7f07fc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:26.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.102+0000 7f080ea00640 1 -- 172.21.15.16:0/4166817148 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f08100e7c60 con 0x7f0818109c40 2024-09-21T03:46:26.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.102+0000 7f0815e00640 1 --2- 172.21.15.16:0/4166817148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f07fc0b6510 0x7f07fc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:26.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.103+0000 7f0815e00640 1 --2- 172.21.15.16:0/4166817148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f07fc0b6510 0x7f07fc0b8a20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f08080095f0 tx=0x7f0808005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.106+0000 7f080ea00640 1 -- 172.21.15.16:0/4166817148 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f08100ae0e0 con 0x7f0818109c40 2024-09-21T03:46:26.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.303+0000 7f0817200640 1 -- 172.21.15.16:0/4166817148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08181a9180 con 0x7f0818109c40 2024-09-21T03:46:26.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.305+0000 7f080ea00640 1 -- 172.21.15.16:0/4166817148 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24902 (secure 0 0 0) 0x7f08100b24c0 con 0x7f0818109c40 2024-09-21T03:46:26.308 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:46:26.309 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:33:53.220358+0000","modified":"2024-09-21T03:44:54.658912+0000","last_up_change":"2024-09-21T03:42:06.614247+0000","last_in_change":"2024-09-21T03:41:43.379404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:37:55.708915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:43:03.567522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:43:06.048083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:43:16.061356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6a7e4b9e-078f-4ae9-ab9e-37be995e4deb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6803","nonce":311974551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6805","nonce":311974551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6809","nonce":311974551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6807","nonce":311974551}]},"public_addr":"172.21.15.16:6803/311974551","cluster_addr":"172.21.15.16:6805/311974551","heartbeat_back_addr":"172.21.15.16:6809/311974551","heartbeat_front_addr":"172.21.15.16:6807/311974551","state":["exists","up"]},{"osd":1,"uuid":"3a497a64-e1a8-43ef-8509-0a79515da0aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6801","nonce":2152486571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6803","nonce":2152486571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6807","nonce":2152486571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6805","nonce":2152486571}]},"public_addr":"172.21.15.123:6801/2152486571","cluster_addr":"172.21.15.123:6803/2152486571","heartbeat_back_addr":"172.21.15.123:6807/2152486571","heartbeat_front_addr":"172.21.15.123:6805/2152486571","state":["exists","up"]},{"osd":2,"uuid":"1c39518e-6cce-4a0f-9ee8-55c60ad325f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6801","nonce":2812351279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6803","nonce":2812351279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6807","nonce":2812351279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6805","nonce":2812351279}]},"public_addr":"172.21.15.134:6801/2812351279","cluster_addr":"172.21.15.134:6803/2812351279","heartbeat_back_addr":"172.21.15.134:6807/2812351279","heartbeat_front_addr":"172.21.15.134:6805/2812351279","state":["exists","up"]},{"osd":3,"uuid":"0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6811","nonce":2366216026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6813","nonce":2366216026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6817","nonce":2366216026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6815","nonce":2366216026}]},"public_addr":"172.21.15.16:6811/2366216026","cluster_addr":"172.21.15.16:6813/2366216026","heartbeat_back_addr":"172.21.15.16:6817/2366216026","heartbeat_front_addr":"172.21.15.16:6815/2366216026","state":["exists","up"]},{"osd":4,"uuid":"ad5e6d0e-31e8-4779-9e74-f215032777ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6809","nonce":2354279116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6811","nonce":2354279116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6815","nonce":2354279116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6813","nonce":2354279116}]},"public_addr":"172.21.15.123:6809/2354279116","cluster_addr":"172.21.15.123:6811/2354279116","heartbeat_back_addr":"172.21.15.123:6815/2354279116","heartbeat_front_addr":"172.21.15.123:6813/2354279116","state":["exists","up"]},{"osd":5,"uuid":"ca09bf88-1772-4dc0-8711-cbf9a5b31977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6809","nonce":2678416182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6811","nonce":2678416182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6815","nonce":2678416182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6813","nonce":2678416182}]},"public_addr":"172.21.15.134:6809/2678416182","cluster_addr":"172.21.15.134:6811/2678416182","heartbeat_back_addr":"172.21.15.134:6815/2678416182","heartbeat_front_addr":"172.21.15.134:6813/2678416182","state":["exists","up"]},{"osd":6,"uuid":"0d41ed4e-8699-4d88-9254-12d3b84941e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6819","nonce":3377807744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6821","nonce":3377807744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6825","nonce":3377807744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6823","nonce":3377807744}]},"public_addr":"172.21.15.16:6819/3377807744","cluster_addr":"172.21.15.16:6821/3377807744","heartbeat_back_addr":"172.21.15.16:6825/3377807744","heartbeat_front_addr":"172.21.15.16:6823/3377807744","state":["exists","up"]},{"osd":7,"uuid":"1541a272-c524-4f12-9b94-763f69bb1c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6817","nonce":1463412739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6819","nonce":1463412739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6823","nonce":1463412739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6821","nonce":1463412739}]},"public_addr":"172.21.15.123:6817/1463412739","cluster_addr":"172.21.15.123:6819/1463412739","heartbeat_back_addr":"172.21.15.123:6823/1463412739","heartbeat_front_addr":"172.21.15.123:6821/1463412739","state":["exists","up"]},{"osd":8,"uuid":"cc00014c-eb99-49ab-984c-0bc26ef85130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6817","nonce":2644021450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6819","nonce":2644021450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6823","nonce":2644021450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6821","nonce":2644021450}]},"public_addr":"172.21.15.134:6817/2644021450","cluster_addr":"172.21.15.134:6819/2644021450","heartbeat_back_addr":"172.21.15.134:6823/2644021450","heartbeat_front_addr":"172.21.15.134:6821/2644021450","state":["exists","up"]},{"osd":9,"uuid":"3cbfff74-31ff-4ffd-844b-54ec8563ba32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6827","nonce":1653164371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6829","nonce":1653164371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6833","nonce":1653164371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6831","nonce":1653164371}]},"public_addr":"172.21.15.16:6827/1653164371","cluster_addr":"172.21.15.16:6829/1653164371","heartbeat_back_addr":"172.21.15.16:6833/1653164371","heartbeat_front_addr":"172.21.15.16:6831/1653164371","state":["exists","up"]},{"osd":10,"uuid":"0a1caaad-70cd-4c2c-88d3-e9a5cc885655","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6825","nonce":3682234496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6827","nonce":3682234496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6831","nonce":3682234496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6829","nonce":3682234496}]},"public_addr":"172.21.15.123:6825/3682234496","cluster_addr":"172.21.15.123:6827/3682234496","heartbeat_back_addr":"172.21.15.123:6831/3682234496","heartbeat_front_addr":"172.21.15.123:6829/3682234496","state":["exists","up"]},{"osd":11,"uuid":"3aff3258-5a38-41ff-aac3-4a082feffdae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6825","nonce":584758685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6827","nonce":584758685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6831","nonce":584758685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6829","nonce":584758685}]},"public_addr":"172.21.15.134:6825/584758685","cluster_addr":"172.21.15.134:6827/584758685","heartbeat_back_addr":"172.21.15.134:6831/584758685","heartbeat_front_addr":"172.21.15.134:6829/584758685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:02.177731+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:28.847562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:54.272384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:24.338596+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:52.430371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:17.832737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:50.397890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:16.842245+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:42.102933+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:13.074732+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:40.057905+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:04.898190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/557742783":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/3238676548":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6801/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/2357336233":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6801/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6800/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/3806844703":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/2351581621":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6800/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/355731724":"2024-09-22T03:34:22.549156+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:46:26.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.308+0000 7f0817200640 1 -- 172.21.15.16:0/4166817148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f07fc0b6510 msgr2=0x7f07fc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:26.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.308+0000 7f0817200640 1 --2- 172.21.15.16:0/4166817148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f07fc0b6510 0x7f07fc0b8a20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f08080095f0 tx=0x7f0808005f90 comp rx=0 tx=0).stop 2024-09-21T03:46:26.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.308+0000 7f0817200640 1 -- 172.21.15.16:0/4166817148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0818109c40 msgr2=0x7f08181a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:26.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.308+0000 7f0817200640 1 --2- 172.21.15.16:0/4166817148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0818109c40 0x7f08181a1d60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f081000eb10 tx=0x7f081000c2d0 comp rx=0 tx=0).stop 2024-09-21T03:46:26.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.308+0000 7f0817200640 1 -- 172.21.15.16:0/4166817148 shutdown_connections 2024-09-21T03:46:26.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.308+0000 7f0817200640 1 --2- 172.21.15.16:0/4166817148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f07fc0b6510 0x7f07fc0b8a20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:26.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.308+0000 7f0817200640 1 --2- 172.21.15.16:0/4166817148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0818109c40 0x7f08181a1d60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:26.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.308+0000 7f0817200640 1 --2- 172.21.15.16:0/4166817148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08181092f0 0x7f081819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:26.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.308+0000 7f0817200640 1 --2- 172.21.15.16:0/4166817148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08181039b0 0x7f081819c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:26.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.308+0000 7f0817200640 1 -- 172.21.15.16:0/4166817148 >> 172.21.15.16:0/4166817148 conn(0x7f08180ff400 msgr2=0x7f0818108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:26.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.308+0000 7f0817200640 1 -- 172.21.15.16:0/4166817148 shutdown_connections 2024-09-21T03:46:26.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:26.308+0000 7f0817200640 1 -- 172.21.15.16:0/4166817148 wait complete. 2024-09-21T03:46:26.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:26 smithi123 ceph-mon[34673]: pgmap v422: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:46:26.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:26 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/4166817148' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:46:26.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:26 smithi016 ceph-mon[30375]: pgmap v422: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:46:26.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:26 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/4166817148' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:46:27.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:26 smithi134 ceph-mon[35031]: pgmap v422: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:46:27.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:26 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/4166817148' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:46:27.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph daemon mon.a --format=json config get run_dir 2024-09-21T03:46:27.493 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.040+0000 7fdfa9a00640 1 -- 172.21.15.16:0/2296014009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa41034e0 msgr2=0x7fdfa4103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.040+0000 7fdfa9a00640 1 --2- 172.21.15.16:0/2296014009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa41034e0 0x7fdfa4103950 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdf94009900 tx=0x7fdf9402a410 comp rx=0 tx=0).stop 2024-09-21T03:46:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.040+0000 7fdfa9a00640 1 -- 172.21.15.16:0/2296014009 shutdown_connections 2024-09-21T03:46:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.040+0000 7fdfa9a00640 1 --2- 172.21.15.16:0/2296014009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa4104f00 0x7fdfa410b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.040+0000 7fdfa9a00640 1 --2- 172.21.15.16:0/2296014009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfa4104560 0x7fdfa4104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.040+0000 7fdfa9a00640 1 --2- 172.21.15.16:0/2296014009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa41034e0 0x7fdfa4103950 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.040+0000 7fdfa9a00640 1 -- 172.21.15.16:0/2296014009 >> 172.21.15.16:0/2296014009 conn(0x7fdfa40fee20 msgr2=0x7fdfa41012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.040+0000 7fdfa9a00640 1 -- 172.21.15.16:0/2296014009 shutdown_connections 2024-09-21T03:46:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.040+0000 7fdfa9a00640 1 -- 172.21.15.16:0/2296014009 wait complete. 2024-09-21T03:46:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.041+0000 7fdfa9a00640 1 Processor -- start 2024-09-21T03:46:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.041+0000 7fdfa9a00640 1 -- start start 2024-09-21T03:46:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.042+0000 7fdfa9a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa41034e0 0x7fdfa4076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.042+0000 7fdfa9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa4104560 0x7fdfa4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.042+0000 7fdfa9a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfa4104f00 0x7fdfa4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.042+0000 7fdfa9a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa410f1a0 con 0x7fdfa4104560 2024-09-21T03:46:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.042+0000 7fdfa9a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa410f020 con 0x7fdfa41034e0 2024-09-21T03:46:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.042+0000 7fdfa9a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa410f320 con 0x7fdfa4104f00 2024-09-21T03:46:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.042+0000 7fdfa3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa4104560 0x7fdfa4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.042+0000 7fdfa3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa4104560 0x7fdfa4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55074/0 (socket says 172.21.15.16:55074) 2024-09-21T03:46:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.042+0000 7fdfa3400640 1 -- 172.21.15.16:0/3223637073 learned_addr learned my addr 172.21.15.16:0/3223637073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.042+0000 7fdfa3e00640 1 --2- 172.21.15.16:0/3223637073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa41034e0 0x7fdfa4076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.042+0000 7fdfa9000640 1 --2- 172.21.15.16:0/3223637073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfa4104f00 0x7fdfa4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.042+0000 7fdfa3400640 1 -- 172.21.15.16:0/3223637073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfa4104f00 msgr2=0x7fdfa4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.042+0000 7fdfa3400640 1 --2- 172.21.15.16:0/3223637073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfa4104f00 0x7fdfa4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.042+0000 7fdfa3400640 1 -- 172.21.15.16:0/3223637073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa41034e0 msgr2=0x7fdfa4076e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.042+0000 7fdfa3400640 1 --2- 172.21.15.16:0/3223637073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa41034e0 0x7fdfa4076e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.042+0000 7fdfa3400640 1 -- 172.21.15.16:0/3223637073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa4076380 con 0x7fdfa4104560 2024-09-21T03:46:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.042+0000 7fdfa3e00640 1 --2- 172.21.15.16:0/3223637073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa41034e0 0x7fdfa4076e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.043+0000 7fdfa3400640 1 --2- 172.21.15.16:0/3223637073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa4104560 0x7fdfa4075710 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000e9f0 tx=0x7fdf9000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.043+0000 7fdfa0c00640 1 -- 172.21.15.16:0/3223637073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9000cd80 con 0x7fdfa4104560 2024-09-21T03:46:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.043+0000 7fdfa0c00640 1 -- 172.21.15.16:0/3223637073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf900054d0 con 0x7fdfa4104560 2024-09-21T03:46:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.043+0000 7fdfa9a00640 1 -- 172.21.15.16:0/3223637073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa41a3e40 con 0x7fdfa4104560 2024-09-21T03:46:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.043+0000 7fdfa0c00640 1 -- 172.21.15.16:0/3223637073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90010690 con 0x7fdfa4104560 2024-09-21T03:46:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.043+0000 7fdfa9a00640 1 -- 172.21.15.16:0/3223637073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa41a4250 con 0x7fdfa4104560 2024-09-21T03:46:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.047+0000 7fdfb206b740 1 Processor -- start 2024-09-21T03:46:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.048+0000 7fdfb206b740 1 -- start start 2024-09-21T03:46:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.048+0000 7fdfb206b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc48c1b0 0x562abc8332b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.048+0000 7fdfb206b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc4632f0 0x562abc833920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.048+0000 7fdfb206b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc46b4d0 0x562abc834470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.048+0000 7fdfb206b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562abc7a66b0 con 0x562abc46b4d0 2024-09-21T03:46:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.048+0000 7fdfb206b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562abc795ae0 con 0x562abc4632f0 2024-09-21T03:46:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.048+0000 7fdfb206b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562abc7998f0 con 0x562abc48c1b0 2024-09-21T03:46:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.048+0000 7fdfa9000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc48c1b0 0x562abc8332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.048+0000 7fdfa3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc4632f0 0x562abc833920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.048+0000 7fdfa9000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc48c1b0 0x562abc8332b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:51148/0 (socket says 172.21.15.16:51148) 2024-09-21T03:46:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.048+0000 7fdfa9000640 1 -- 172.21.15.16:0/2545197229 learned_addr learned my addr 172.21.15.16:0/2545197229 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.049+0000 7fdfa9000640 1 -- 172.21.15.16:0/2545197229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc4632f0 msgr2=0x562abc833920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.049+0000 7fdfa9000640 1 --2- 172.21.15.16:0/2545197229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc4632f0 0x562abc833920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.049+0000 7fdfa9000640 1 -- 172.21.15.16:0/2545197229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc46b4d0 msgr2=0x562abc834470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:46:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.049+0000 7fdfa9000640 1 --2- 172.21.15.16:0/2545197229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc46b4d0 0x562abc834470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.049+0000 7fdfa0c00640 1 -- 172.21.15.16:0/3223637073 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fdf90004e90 con 0x7fdfa4104560 2024-09-21T03:46:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.049+0000 7fdfa9000640 1 -- 172.21.15.16:0/2545197229 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562abc762780 con 0x562abc48c1b0 2024-09-21T03:46:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.049+0000 7fdfa9000640 1 --2- 172.21.15.16:0/2545197229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc48c1b0 0x562abc8332b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009fd0 tx=0x7fdf9c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=203a24d191a67d6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.049+0000 7fdf99600640 1 -- 172.21.15.16:0/2545197229 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c004d20 con 0x562abc48c1b0 2024-09-21T03:46:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.050+0000 7fdf99600640 1 -- 172.21.15.16:0/2545197229 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf9c004050 con 0x562abc48c1b0 2024-09-21T03:46:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.050+0000 7fdf99600640 1 -- 172.21.15.16:0/2545197229 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c005280 con 0x562abc48c1b0 2024-09-21T03:46:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.051+0000 7fdfb206b740 1 -- 172.21.15.16:0/2545197229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc48c1b0 msgr2=0x562abc8332b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.051+0000 7fdfb206b740 1 --2- 172.21.15.16:0/2545197229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc48c1b0 0x562abc8332b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009fd0 tx=0x7fdf9c0118d0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.051+0000 7fdfb206b740 1 -- 172.21.15.16:0/2545197229 shutdown_connections 2024-09-21T03:46:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.051+0000 7fdfb206b740 1 --2- 172.21.15.16:0/2545197229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc46b4d0 0x562abc834470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.051+0000 7fdfb206b740 1 --2- 172.21.15.16:0/2545197229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc4632f0 0x562abc833920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.051+0000 7fdfb206b740 1 --2- 172.21.15.16:0/2545197229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc48c1b0 0x562abc8332b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.051+0000 7fdfb206b740 1 -- 172.21.15.16:0/2545197229 >> 172.21.15.16:0/2545197229 conn(0x562abc480300 msgr2=0x562abc82ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.051+0000 7fdfa0c00640 1 --2- 172.21.15.16:0/3223637073 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdf880b6510 0x7fdf880b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.051+0000 7fdfb206b740 1 -- 172.21.15.16:0/2545197229 shutdown_connections 2024-09-21T03:46:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.051+0000 7fdfb206b740 1 -- 172.21.15.16:0/2545197229 wait complete. 2024-09-21T03:46:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.051+0000 7fdfa0c00640 1 -- 172.21.15.16:0/3223637073 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fdf900e7b00 con 0x7fdfa4104560 2024-09-21T03:46:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.051+0000 7fdfa9000640 1 --2- 172.21.15.16:0/3223637073 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdf880b6510 0x7fdf880b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.052+0000 7fdfb206b740 1 -- start start 2024-09-21T03:46:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.052+0000 7fdfa9000640 1 --2- 172.21.15.16:0/3223637073 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdf880b6510 0x7fdf880b8a20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c005fd0 tx=0x7fdf9c004890 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.052+0000 7fdfb206b740 1 Processor -- start 2024-09-21T03:46:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.052+0000 7fdfb206b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc4632f0 0x562abc83c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.052+0000 7fdfb206b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc46b4d0 0x562abc83d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.053+0000 7fdfb206b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc48c1b0 0x562abc83dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.053+0000 7fdfb206b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562abc6cdab0 con 0x562abc48c1b0 2024-09-21T03:46:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.053+0000 7fdfb206b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562abc73c570 con 0x562abc4632f0 2024-09-21T03:46:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.053+0000 7fdfb206b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562abc834f90 con 0x562abc46b4d0 2024-09-21T03:46:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.053+0000 7fdfb206b740 10 client.0 _ll_register_callbacks cb 0x562abc74b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.053+0000 7fdfa3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc4632f0 0x562abc83c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.053+0000 7fdfa3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc4632f0 0x562abc83c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44290/0 (socket says 172.21.15.16:44290) 2024-09-21T03:46:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.053+0000 7fdfa3e00640 1 -- 172.21.15.16:0/639743454 learned_addr learned my addr 172.21.15.16:0/639743454 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.053+0000 7fdfa9000640 1 --2- 172.21.15.16:0/639743454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc48c1b0 0x562abc83dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.053+0000 7fdfa3400640 1 --2- 172.21.15.16:0/639743454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc46b4d0 0x562abc83d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.053+0000 7fdfa3e00640 1 -- 172.21.15.16:0/639743454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc46b4d0 msgr2=0x562abc83d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.053+0000 7fdfa3e00640 1 --2- 172.21.15.16:0/639743454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc46b4d0 0x562abc83d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.053+0000 7fdfa3e00640 1 -- 172.21.15.16:0/639743454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc48c1b0 msgr2=0x562abc83dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.053+0000 7fdfa3e00640 1 --2- 172.21.15.16:0/639743454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc48c1b0 0x562abc83dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.053+0000 7fdfa3e00640 1 -- 172.21.15.16:0/639743454 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562abc4f7790 con 0x562abc4632f0 2024-09-21T03:46:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.053+0000 7fdfa9000640 1 --2- 172.21.15.16:0/639743454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc48c1b0 0x562abc83dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.054+0000 7fdfa3e00640 1 --2- 172.21.15.16:0/639743454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc4632f0 0x562abc83c970 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdf94009230 tx=0x7fdf94003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e8fd714b0a4ae8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.054+0000 7fdf8e000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:46:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.054+0000 7fdf8e000640 1 -- 172.21.15.16:0/639743454 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94035070 con 0x562abc4632f0 2024-09-21T03:46:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.054+0000 7fdfb206b740 1 -- 172.21.15.16:0/639743454 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf9c004050 con 0x562abc4632f0 2024-09-21T03:46:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.054+0000 7fdf8e000640 1 -- 172.21.15.16:0/639743454 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fdf94003ba0 con 0x562abc4632f0 2024-09-21T03:46:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.054+0000 7fdf8e000640 1 -- 172.21.15.16:0/639743454 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdf94003da0 con 0x562abc4632f0 2024-09-21T03:46:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.054+0000 7fdfb206b740 10 client.25463 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.054+0000 7fdfb206b740 1 -- 172.21.15.16:0/639743454 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562abc570620 con 0x562abc4632f0 2024-09-21T03:46:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.054+0000 7fdf8e000640 1 -- 172.21.15.16:0/639743454 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf9c004050 con 0x562abc4632f0 2024-09-21T03:46:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.055+0000 7fdf8e000640 1 -- 172.21.15.16:0/639743454 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdf940466d0 con 0x562abc4632f0 2024-09-21T03:46:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.055+0000 7fdfb206b740 10 client.25463 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.055+0000 7fdfb206b740 10 client.25463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.055+0000 7fdfb206b740 20 client.25463 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.055+0000 7fdfb206b740 1 --2- 172.21.15.16:0/639743454 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562abc44ad80 0x562abc84a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.055+0000 7fdfb206b740 4 client.25463 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.055+0000 7fdfb206b740 1 -- 172.21.15.16:0/639743454 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562abc7a8600 con 0x562abc44ad80 2024-09-21T03:46:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.055+0000 7fdfa3400640 1 --2- 172.21.15.16:0/639743454 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562abc44ad80 0x562abc84a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.056+0000 7fdf8e000640 10 client.25463 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.056+0000 7fdfa3400640 1 --2- 172.21.15.16:0/639743454 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562abc44ad80 0x562abc84a160 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b31b5316c17646a server_cookie=f801f67031b09e8d in_seq=0 out_seq=0 2024-09-21T03:46:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.057+0000 7fdf8e000640 1 -- 172.21.15.16:0/639743454 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562abc7a8600 con 0x562abc44ad80 2024-09-21T03:46:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.057+0000 7fdf8e000640 10 client.25463 handle_command_reply: tid=0 2024-09-21T03:46:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.057+0000 7fdfb206b740 1 client.25463 shutdown 2024-09-21T03:46:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.057+0000 7fdfb206b740 1 -- 172.21.15.16:0/639743454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc4632f0 msgr2=0x562abc83c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.057+0000 7fdfb206b740 1 --2- 172.21.15.16:0/639743454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc4632f0 0x562abc83c970 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdf94009230 tx=0x7fdf94003820 comp rx=0 tx=0).stop 2024-09-21T03:46:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.057+0000 7fdfb206b740 1 -- 172.21.15.16:0/639743454 shutdown_connections 2024-09-21T03:46:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.057+0000 7fdfb206b740 1 --2- 172.21.15.16:0/639743454 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562abc44ad80 0x562abc84a160 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.057+0000 7fdfb206b740 1 --2- 172.21.15.16:0/639743454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc48c1b0 0x562abc83dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.057+0000 7fdfb206b740 1 --2- 172.21.15.16:0/639743454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc46b4d0 0x562abc83d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.057+0000 7fdf8e000640 0 client.25463 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.057+0000 7fdfb206b740 1 --2- 172.21.15.16:0/639743454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc4632f0 0x562abc83c970 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.057+0000 7fdfb206b740 1 -- 172.21.15.16:0/639743454 >> 172.21.15.16:0/639743454 conn(0x562abc480300 msgr2=0x562abc82d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.057+0000 7fdfb206b740 1 -- 172.21.15.16:0/639743454 shutdown_connections 2024-09-21T03:46:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.057+0000 7fdfb206b740 1 -- 172.21.15.16:0/639743454 wait complete. 2024-09-21T03:46:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.057+0000 7fdfb206b740 20 client.25463 trim_cache size 0 max 16384 2024-09-21T03:46:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.073+0000 7fdfb206b740 1 Processor -- start 2024-09-21T03:46:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.073+0000 7fdfb206b740 1 -- start start 2024-09-21T03:46:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.073+0000 7fdfb206b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc4632f0 0x562abc8335b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.073+0000 7fdfb206b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc46b4d0 0x562abc844a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.073+0000 7fdfb206b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc48c1b0 0x562abc8373c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.073+0000 7fdfb206b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41ae710 con 0x562abc4632f0 2024-09-21T03:46:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.073+0000 7fdfb206b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41b0250 con 0x562abc48c1b0 2024-09-21T03:46:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.073+0000 7fdfb206b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41aede0 con 0x562abc46b4d0 2024-09-21T03:46:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.074+0000 7fdfa3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc46b4d0 0x562abc844a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.074+0000 7fdfa3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc4632f0 0x562abc8335b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.074+0000 7fdfa3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc46b4d0 0x562abc844a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:51168/0 (socket says 172.21.15.16:51168) 2024-09-21T03:46:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.074+0000 7fdfa3400640 1 -- 172.21.15.16:0/1815040344 learned_addr learned my addr 172.21.15.16:0/1815040344 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.074+0000 7fdfa9000640 1 --2- 172.21.15.16:0/1815040344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc48c1b0 0x562abc8373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.074+0000 7fdfa3e00640 1 -- 172.21.15.16:0/1815040344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc46b4d0 msgr2=0x562abc844a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.074+0000 7fdfa3e00640 1 --2- 172.21.15.16:0/1815040344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc46b4d0 0x562abc844a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.074+0000 7fdfa3e00640 1 -- 172.21.15.16:0/1815040344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc48c1b0 msgr2=0x562abc8373c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.074+0000 7fdfa3e00640 1 --2- 172.21.15.16:0/1815040344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc48c1b0 0x562abc8373c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.074+0000 7fdfa3e00640 1 -- 172.21.15.16:0/1815040344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562abc83df10 con 0x562abc4632f0 2024-09-21T03:46:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.075+0000 7fdfa3e00640 1 --2- 172.21.15.16:0/1815040344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc4632f0 0x562abc8335b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdf940096c0 tx=0x7fdf9402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566dc48d275098a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.075+0000 7fdf98c00640 1 -- 172.21.15.16:0/1815040344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94035070 con 0x562abc4632f0 2024-09-21T03:46:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.075+0000 7fdf98c00640 1 -- 172.21.15.16:0/1815040344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf94003ba0 con 0x562abc4632f0 2024-09-21T03:46:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.075+0000 7fdf98c00640 1 -- 172.21.15.16:0/1815040344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94031d40 con 0x562abc4632f0 2024-09-21T03:46:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.076+0000 7fdfb206b740 1 -- 172.21.15.16:0/1815040344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc4632f0 msgr2=0x562abc8335b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.076+0000 7fdfb206b740 1 --2- 172.21.15.16:0/1815040344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc4632f0 0x562abc8335b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdf940096c0 tx=0x7fdf9402a410 comp rx=0 tx=0).stop 2024-09-21T03:46:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.076+0000 7fdfb206b740 1 -- 172.21.15.16:0/1815040344 shutdown_connections 2024-09-21T03:46:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.076+0000 7fdfb206b740 1 --2- 172.21.15.16:0/1815040344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc48c1b0 0x562abc8373c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.076+0000 7fdfb206b740 1 --2- 172.21.15.16:0/1815040344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc46b4d0 0x562abc844a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.076+0000 7fdfb206b740 1 --2- 172.21.15.16:0/1815040344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc4632f0 0x562abc8335b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.076+0000 7fdfb206b740 1 -- 172.21.15.16:0/1815040344 >> 172.21.15.16:0/1815040344 conn(0x562abc480300 msgr2=0x562abc832ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.076+0000 7fdfb206b740 1 -- 172.21.15.16:0/1815040344 shutdown_connections 2024-09-21T03:46:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.076+0000 7fdfb206b740 1 -- 172.21.15.16:0/1815040344 wait complete. 2024-09-21T03:46:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.076+0000 7fdfb206b740 1 -- start start 2024-09-21T03:46:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.077+0000 7fdfb206b740 1 Processor -- start 2024-09-21T03:46:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.077+0000 7fdfb206b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc4632f0 0x562abc844590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.077+0000 7fdfb206b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc46b4d0 0x562abc843250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.077+0000 7fdfb206b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc48c1b0 0x562abc843790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.077+0000 7fdfb206b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41afb80 con 0x562abc4632f0 2024-09-21T03:46:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.077+0000 7fdfb206b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41af4b0 con 0x562abc46b4d0 2024-09-21T03:46:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.077+0000 7fdfb206b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41ac6f0 con 0x562abc48c1b0 2024-09-21T03:46:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.077+0000 7fdfb206b740 10 client.0 _ll_register_callbacks cb 0x7fdf94009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.077+0000 7fdfa3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc4632f0 0x562abc844590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.077+0000 7fdfa3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc4632f0 0x562abc844590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55106/0 (socket says 172.21.15.16:55106) 2024-09-21T03:46:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.077+0000 7fdfa3e00640 1 -- 172.21.15.16:0/3561146327 learned_addr learned my addr 172.21.15.16:0/3561146327 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.077+0000 7fdfa3400640 1 --2- 172.21.15.16:0/3561146327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc46b4d0 0x562abc843250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.078+0000 7fdfa9000640 1 --2- 172.21.15.16:0/3561146327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc48c1b0 0x562abc843790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.078+0000 7fdfa3e00640 1 -- 172.21.15.16:0/3561146327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc48c1b0 msgr2=0x562abc843790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.078+0000 7fdfa3e00640 1 --2- 172.21.15.16:0/3561146327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc48c1b0 0x562abc843790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.078+0000 7fdfa3e00640 1 -- 172.21.15.16:0/3561146327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc46b4d0 msgr2=0x562abc843250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.078+0000 7fdfa3e00640 1 --2- 172.21.15.16:0/3561146327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc46b4d0 0x562abc843250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.078+0000 7fdfa3e00640 1 -- 172.21.15.16:0/3561146327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562abc843d50 con 0x562abc4632f0 2024-09-21T03:46:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.078+0000 7fdfa3e00640 1 --2- 172.21.15.16:0/3561146327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc4632f0 0x562abc844590 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdf94046c90 tx=0x7fdf94005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6d66f991521428 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.078+0000 7fdf8e000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:46:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.078+0000 7fdf8e000640 1 -- 172.21.15.16:0/3561146327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94035070 con 0x562abc4632f0 2024-09-21T03:46:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.078+0000 7fdf8e000640 1 -- 172.21.15.16:0/3561146327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fdf94046670 con 0x562abc4632f0 2024-09-21T03:46:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.078+0000 7fdfb206b740 1 -- 172.21.15.16:0/3561146327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf94003ba0 con 0x562abc4632f0 2024-09-21T03:46:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.079+0000 7fdf8e000640 1 -- 172.21.15.16:0/3561146327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdf940468b0 con 0x562abc4632f0 2024-09-21T03:46:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.079+0000 7fdfb206b740 10 client.15462 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.079+0000 7fdfb206b740 1 -- 172.21.15.16:0/3561146327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562abc842b70 con 0x562abc4632f0 2024-09-21T03:46:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.079+0000 7fdf8e000640 1 -- 172.21.15.16:0/3561146327 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf94003ba0 con 0x562abc4632f0 2024-09-21T03:46:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.079+0000 7fdf8e000640 1 -- 172.21.15.16:0/3561146327 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdf9405e4c0 con 0x562abc4632f0 2024-09-21T03:46:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.079+0000 7fdfb206b740 10 client.15462 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.079+0000 7fdfb206b740 10 client.15462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.079+0000 7fdfb206b740 20 client.15462 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.079+0000 7fdfb206b740 1 --2- 172.21.15.16:0/3561146327 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562abc4201c0 0x562abc8b7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.079+0000 7fdfb206b740 4 client.15462 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:46:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.079+0000 7fdfb206b740 1 -- 172.21.15.16:0/3561146327 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562abc7a40b0 con 0x562abc4201c0 2024-09-21T03:46:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.080+0000 7fdfa3400640 1 --2- 172.21.15.16:0/3561146327 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562abc4201c0 0x562abc8b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.080+0000 7fdf8e000640 10 client.15462 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.080+0000 7fdfa3400640 1 --2- 172.21.15.16:0/3561146327 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562abc4201c0 0x562abc8b7070 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18da7c66fcd147a server_cookie=f5008c8e1a19187e in_seq=0 out_seq=0 2024-09-21T03:46:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.094+0000 7fdf8e000640 1 -- 172.21.15.16:0/3561146327 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562abc7a40b0 con 0x562abc4201c0 2024-09-21T03:46:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.094+0000 7fdf8e000640 10 client.15462 handle_command_reply: tid=0 2024-09-21T03:46:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.094+0000 7fdfb206b740 1 client.15462 shutdown 2024-09-21T03:46:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.094+0000 7fdfb206b740 1 -- 172.21.15.16:0/3561146327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc4632f0 msgr2=0x562abc844590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.094+0000 7fdfb206b740 1 --2- 172.21.15.16:0/3561146327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc4632f0 0x562abc844590 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdf94046c90 tx=0x7fdf94005f30 comp rx=0 tx=0).stop 2024-09-21T03:46:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.094+0000 7fdfb206b740 1 -- 172.21.15.16:0/3561146327 shutdown_connections 2024-09-21T03:46:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.094+0000 7fdfb206b740 1 --2- 172.21.15.16:0/3561146327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abc48c1b0 0x562abc843790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.094+0000 7fdfb206b740 1 --2- 172.21.15.16:0/3561146327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562abc46b4d0 0x562abc843250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.094+0000 7fdfb206b740 1 --2- 172.21.15.16:0/3561146327 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562abc4201c0 0x562abc8b7070 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.094+0000 7fdfb206b740 1 --2- 172.21.15.16:0/3561146327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562abc4632f0 0x562abc844590 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.094+0000 7fdf8e000640 0 client.15462 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.094+0000 7fdfb206b740 1 -- 172.21.15.16:0/3561146327 >> 172.21.15.16:0/3561146327 conn(0x562abc480300 msgr2=0x562abc83b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.095+0000 7fdfb206b740 1 -- 172.21.15.16:0/3561146327 shutdown_connections 2024-09-21T03:46:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.095+0000 7fdfb206b740 1 -- 172.21.15.16:0/3561146327 wait complete. 2024-09-21T03:46:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.095+0000 7fdfb206b740 20 client.15462 trim_cache size 0 max 16384 2024-09-21T03:46:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:46:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:46:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:46:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:46:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.095+0000 7fdfa9a00640 1 -- 172.21.15.16:0/3223637073 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdf880b6510 msgr2=0x7fdf880b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.095+0000 7fdfa9a00640 1 --2- 172.21.15.16:0/3223637073 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdf880b6510 0x7fdf880b8a20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c005fd0 tx=0x7fdf9c004890 comp rx=0 tx=0).stop 2024-09-21T03:46:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.095+0000 7fdfa9a00640 1 -- 172.21.15.16:0/3223637073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa4104560 msgr2=0x7fdfa4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.095+0000 7fdfa9a00640 1 --2- 172.21.15.16:0/3223637073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa4104560 0x7fdfa4075710 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000e9f0 tx=0x7fdf9000edf0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.096+0000 7fdfa9a00640 1 -- 172.21.15.16:0/3223637073 shutdown_connections 2024-09-21T03:46:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.096+0000 7fdfa9a00640 1 --2- 172.21.15.16:0/3223637073 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fdf880b6510 0x7fdf880b8a20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.096+0000 7fdfa9a00640 1 --2- 172.21.15.16:0/3223637073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfa4104f00 0x7fdfa4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.096+0000 7fdfa9a00640 1 --2- 172.21.15.16:0/3223637073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa4104560 0x7fdfa4075710 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.096+0000 7fdfa9a00640 1 --2- 172.21.15.16:0/3223637073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfa41034e0 0x7fdfa4076e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.096+0000 7fdfa9a00640 1 -- 172.21.15.16:0/3223637073 >> 172.21.15.16:0/3223637073 conn(0x7fdfa40fee20 msgr2=0x7fdfa4107880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.096+0000 7fdfa9a00640 1 -- 172.21.15.16:0/3223637073 shutdown_connections 2024-09-21T03:46:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:28.096+0000 7fdfa9a00640 1 -- 172.21.15.16:0/3223637073 wait complete. 2024-09-21T03:46:28.899 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:28 smithi016 ceph-mon[30375]: pgmap v423: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T03:46:28.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:28 smithi123 ceph-mon[34673]: pgmap v423: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T03:46:28.939 INFO:teuthology.orchestra.run.smithi016.stdout:{"run_dir":"/var/run/ceph"} 2024-09-21T03:46:28.939 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-21T03:46:28.939 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 7864 2024-09-21T03:46:28.940 DEBUG:tasks.quiescer:Yielding 2024-09-21T03:46:28.940 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T03:46:28.953 INFO:tasks.workunit:Pulling workunits from ref 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:46:28.953 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T03:46:28.953 INFO:tasks.workunit:timeout=3h 2024-09-21T03:46:28.954 INFO:tasks.workunit:cleanup=True 2024-09-21T03:46:28.954 DEBUG:teuthology.orchestra.run.smithi016:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T03:46:28.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:46:28.990 INFO:teuthology.orchestra.run.smithi016.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T03:46:28.991 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-21T03:46:28.991 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 60h/96d Inode: 1099511628283 Links: 2 2024-09-21T03:46:28.991 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:46:28.991 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-21T03:46:28.991 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-21 03:44:17.197338832 +0000 2024-09-21T03:46:28.991 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-21 03:44:17.197338832 +0000 2024-09-21T03:46:28.991 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-21 03:46:12.641189900 +0000 2024-09-21T03:46:28.991 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-21 03:44:17.197338832 +0000 2024-09-21T03:46:28.991 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T03:46:28.991 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T03:46:29.101 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:46:29.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:28 smithi134 ceph-mon[35031]: pgmap v423: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T03:46:29.140 INFO:tasks.workunit.client.0.smithi016.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T03:46:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:30.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.012+0000 7f04ff200640 1 -- 172.21.15.16:0/1659952254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0500109c40 msgr2=0x7f0500110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:30.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.012+0000 7f04ff200640 1 --2- 172.21.15.16:0/1659952254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0500109c40 0x7f0500110470 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009ca0 tx=0x7f04f802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:46:30.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.013+0000 7f04ff200640 1 -- 172.21.15.16:0/1659952254 shutdown_connections 2024-09-21T03:46:30.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.013+0000 7f04ff200640 1 --2- 172.21.15.16:0/1659952254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0500109c40 0x7f0500110470 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:30.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.013+0000 7f04ff200640 1 --2- 172.21.15.16:0/1659952254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05001092f0 0x7f0500109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:30.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.013+0000 7f04ff200640 1 --2- 172.21.15.16:0/1659952254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05001039b0 0x7f0500103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:30.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.013+0000 7f04ff200640 1 -- 172.21.15.16:0/1659952254 >> 172.21.15.16:0/1659952254 conn(0x7f05000ff400 msgr2=0x7f0500101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:30.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.013+0000 7f04ff200640 1 -- 172.21.15.16:0/1659952254 shutdown_connections 2024-09-21T03:46:30.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.013+0000 7f04ff200640 1 -- 172.21.15.16:0/1659952254 wait complete. 2024-09-21T03:46:30.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.014+0000 7f04ff200640 1 Processor -- start 2024-09-21T03:46:30.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.014+0000 7f04ff200640 1 -- start start 2024-09-21T03:46:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.015+0000 7f04ff200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05001039b0 0x7f050019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.015+0000 7f04ff200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05001092f0 0x7f050019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.015+0000 7f04ff200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0500109c40 0x7f050019fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.015+0000 7f04ff200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0500113900 con 0x7f05001039b0 2024-09-21T03:46:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.015+0000 7f04ff200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0500113780 con 0x7f0500109c40 2024-09-21T03:46:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.015+0000 7f04ff200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0500113a80 con 0x7f05001092f0 2024-09-21T03:46:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.015+0000 7f04fde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05001039b0 0x7f050019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.015+0000 7f04fde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05001039b0 0x7f050019a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55126/0 (socket says 172.21.15.16:55126) 2024-09-21T03:46:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.015+0000 7f04fde00640 1 -- 172.21.15.16:0/2959150439 learned_addr learned my addr 172.21.15.16:0/2959150439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.015+0000 7f04fe800640 1 --2- 172.21.15.16:0/2959150439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0500109c40 0x7f050019fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.015+0000 7f04fd400640 1 --2- 172.21.15.16:0/2959150439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05001092f0 0x7f050019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.016+0000 7f04fde00640 1 -- 172.21.15.16:0/2959150439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05001092f0 msgr2=0x7f050019a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.016+0000 7f04fde00640 1 --2- 172.21.15.16:0/2959150439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05001092f0 0x7f050019a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.016+0000 7f04fde00640 1 -- 172.21.15.16:0/2959150439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0500109c40 msgr2=0x7f050019fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.016+0000 7f04fde00640 1 --2- 172.21.15.16:0/2959150439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0500109c40 0x7f050019fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.016+0000 7f04fde00640 1 -- 172.21.15.16:0/2959150439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05001a0280 con 0x7f05001039b0 2024-09-21T03:46:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.016+0000 7f04fe800640 1 --2- 172.21.15.16:0/2959150439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0500109c40 0x7f050019fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.016+0000 7f04fde00640 1 --2- 172.21.15.16:0/2959150439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05001039b0 0x7f050019a310 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f04f000e9f0 tx=0x7f04f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.016+0000 7f04f6a00640 1 -- 172.21.15.16:0/2959150439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f000cd80 con 0x7f05001039b0 2024-09-21T03:46:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.016+0000 7f04f6a00640 1 -- 172.21.15.16:0/2959150439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04f00054d0 con 0x7f05001039b0 2024-09-21T03:46:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.016+0000 7f04ff200640 1 -- 172.21.15.16:0/2959150439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05001a0570 con 0x7f05001039b0 2024-09-21T03:46:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.016+0000 7f04f6a00640 1 -- 172.21.15.16:0/2959150439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0010690 con 0x7f05001039b0 2024-09-21T03:46:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.017+0000 7f04ff200640 1 -- 172.21.15.16:0/2959150439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05001a6cd0 con 0x7f05001039b0 2024-09-21T03:46:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.018+0000 7f04ff200640 1 -- 172.21.15.16:0/2959150439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0500103dc0 con 0x7f05001039b0 2024-09-21T03:46:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.020+0000 7f04f6a00640 1 -- 172.21.15.16:0/2959150439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f04f0004e30 con 0x7f05001039b0 2024-09-21T03:46:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.022+0000 7f04f6a00640 1 --2- 172.21.15.16:0/2959150439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04e40b6590 0x7f04e40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.022+0000 7f04f6a00640 1 -- 172.21.15.16:0/2959150439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f04f00e7c20 con 0x7f05001039b0 2024-09-21T03:46:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.022+0000 7f04fd400640 1 --2- 172.21.15.16:0/2959150439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04e40b6590 0x7f04e40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.023+0000 7f04fd400640 1 --2- 172.21.15.16:0/2959150439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04e40b6590 0x7f04e40b8aa0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0500108c60 tx=0x7f04ec005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:30.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.024+0000 7f04f6a00640 1 -- 172.21.15.16:0/2959150439 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f04f00ae0a0 con 0x7f05001039b0 2024-09-21T03:46:30.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.266+0000 7f04ff200640 1 -- 172.21.15.16:0/2959150439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f050019b440 con 0x7f05001039b0 2024-09-21T03:46:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.267+0000 7f04f6a00640 1 -- 172.21.15.16:0/2959150439 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f04f00b2480 con 0x7f05001039b0 2024-09-21T03:46:30.358 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:46:30.359 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:46:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:46:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.269+0000 7f04ff200640 1 -- 172.21.15.16:0/2959150439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04e40b6590 msgr2=0x7f04e40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.269+0000 7f04ff200640 1 --2- 172.21.15.16:0/2959150439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04e40b6590 0x7f04e40b8aa0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0500108c60 tx=0x7f04ec005f90 comp rx=0 tx=0).stop 2024-09-21T03:46:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.269+0000 7f04ff200640 1 -- 172.21.15.16:0/2959150439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05001039b0 msgr2=0x7f050019a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.269+0000 7f04ff200640 1 --2- 172.21.15.16:0/2959150439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05001039b0 0x7f050019a310 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f04f000e9f0 tx=0x7f04f000edf0 comp rx=0 tx=0).stop 2024-09-21T03:46:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.269+0000 7f04ff200640 1 -- 172.21.15.16:0/2959150439 shutdown_connections 2024-09-21T03:46:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.269+0000 7f04ff200640 1 --2- 172.21.15.16:0/2959150439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04e40b6590 0x7f04e40b8aa0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.269+0000 7f04ff200640 1 --2- 172.21.15.16:0/2959150439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0500109c40 0x7f050019fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.269+0000 7f04ff200640 1 --2- 172.21.15.16:0/2959150439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05001092f0 0x7f050019a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.269+0000 7f04ff200640 1 --2- 172.21.15.16:0/2959150439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05001039b0 0x7f050019a310 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.270+0000 7f04ff200640 1 -- 172.21.15.16:0/2959150439 >> 172.21.15.16:0/2959150439 conn(0x7f05000ff400 msgr2=0x7f0500101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.270+0000 7f04ff200640 1 -- 172.21.15.16:0/2959150439 shutdown_connections 2024-09-21T03:46:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:30.270+0000 7f04ff200640 1 -- 172.21.15.16:0/2959150439 wait complete. 2024-09-21T03:46:30.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:30 smithi123 ceph-mon[34673]: pgmap v424: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:46:30.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:30 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2959150439' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:30.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:30 smithi016 ceph-mon[30375]: pgmap v424: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:46:30.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:30 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2959150439' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:31.062 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T03:43:09.332305+0000', 'modified': '2024-09-21T03:46:13.605821+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24731, 'mds_1': 24689, 'mds_2': 24682, 'mds_3': 24670, 'mds_4': 24668}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14700': {'gid': 14700, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6839/89613887', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 89613887}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 89613887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14712': {'gid': 14712, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6841/1546097467', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6840', 'nonce': 1546097467}, {'type': 'v1', 'addr': '172.21.15.16:6841', 'nonce': 1546097467}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24613': {'gid': 24613, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/2239407349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 2239407349}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 2239407349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24632': {'gid': 24632, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6833/3581513383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6832', 'nonce': 3581513383}, {'type': 'v1', 'addr': '172.21.15.123:6833', 'nonce': 3581513383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24658': {'gid': 24658, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/135285598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 135285598}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 135285598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24668': {'gid': 24668, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.123:6835/86959152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 86959152}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 86959152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24670': {'gid': 24670, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.134:6837/2624530453', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 2624530453}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 2624530453}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24682': {'gid': 24682, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.134:6839/606193011', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 606193011}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 606193011}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24689': {'gid': 24689, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.123:6837/2001424876', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 2001424876}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 2001424876}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24731': {'gid': 24731, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.123:6839/459522816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 459522816}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 459522816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24731, 'qdb_cluster': [24731, 24689, 24682, 24670, 24668]} 2024-09-21T03:46:31.063 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T03:46:31.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.e status 2024-09-21T03:46:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:30 smithi134 ceph-mon[35031]: pgmap v424: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:46:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:30 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2959150439' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.131+0000 7f0075200640 1 -- 172.21.15.16:0/1914610026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f007010dc20 msgr2=0x7f0070110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.131+0000 7f0075200640 1 --2- 172.21.15.16:0/1914610026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f007010dc20 0x7f0070110110 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0068009ca0 tx=0x7f006802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.131+0000 7f0075200640 1 -- 172.21.15.16:0/1914610026 shutdown_connections 2024-09-21T03:46:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.131+0000 7f0075200640 1 --2- 172.21.15.16:0/1914610026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f007010dc20 0x7f0070110110 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.131+0000 7f0075200640 1 --2- 172.21.15.16:0/1914610026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0070105110 0x7f007010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.131+0000 7f0075200640 1 --2- 172.21.15.16:0/1914610026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00701047c0 0x7f0070104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.131+0000 7f0075200640 1 -- 172.21.15.16:0/1914610026 >> 172.21.15.16:0/1914610026 conn(0x7f00700fe2c0 msgr2=0x7f0070100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:32.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.132+0000 7f0075200640 1 -- 172.21.15.16:0/1914610026 shutdown_connections 2024-09-21T03:46:32.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.132+0000 7f0075200640 1 -- 172.21.15.16:0/1914610026 wait complete. 2024-09-21T03:46:32.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.133+0000 7f0075200640 1 Processor -- start 2024-09-21T03:46:32.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.133+0000 7f0075200640 1 -- start start 2024-09-21T03:46:32.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.133+0000 7f0075200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00701047c0 0x7f0070102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.133+0000 7f0075200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0070105110 0x7f0070102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.133+0000 7f0075200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f007010dc20 0x7f0070102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.133+0000 7f0075200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00701138a0 con 0x7f00701047c0 2024-09-21T03:46:32.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.133+0000 7f0075200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0070113720 con 0x7f007010dc20 2024-09-21T03:46:32.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.133+0000 7f0075200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0070113a20 con 0x7f0070105110 2024-09-21T03:46:32.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.134+0000 7f006f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00701047c0 0x7f0070102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.134+0000 7f006f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00701047c0 0x7f0070102500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41686/0 (socket says 172.21.15.16:41686) 2024-09-21T03:46:32.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.134+0000 7f006f400640 1 -- 172.21.15.16:0/1612119754 learned_addr learned my addr 172.21.15.16:0/1612119754 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:32.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.134+0000 7f006ea00640 1 --2- 172.21.15.16:0/1612119754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0070105110 0x7f0070102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.134+0000 7f006fe00640 1 --2- 172.21.15.16:0/1612119754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f007010dc20 0x7f0070102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.134+0000 7f006f400640 1 -- 172.21.15.16:0/1612119754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0070105110 msgr2=0x7f0070102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:32.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.134+0000 7f006f400640 1 --2- 172.21.15.16:0/1612119754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0070105110 0x7f0070102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.134+0000 7f006f400640 1 -- 172.21.15.16:0/1612119754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f007010dc20 msgr2=0x7f0070102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:32.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.134+0000 7f006f400640 1 --2- 172.21.15.16:0/1612119754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f007010dc20 0x7f0070102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.134+0000 7f006f400640 1 -- 172.21.15.16:0/1612119754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00701a8510 con 0x7f00701047c0 2024-09-21T03:46:32.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.135+0000 7f006f400640 1 --2- 172.21.15.16:0/1612119754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00701047c0 0x7f0070102500 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f006000e9f0 tx=0x7f006000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:32.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.135+0000 7f0067e00640 1 -- 172.21.15.16:0/1612119754 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006000cd80 con 0x7f00701047c0 2024-09-21T03:46:32.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.135+0000 7f0067e00640 1 -- 172.21.15.16:0/1612119754 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00600108e0 con 0x7f00701047c0 2024-09-21T03:46:32.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.135+0000 7f0075200640 1 -- 172.21.15.16:0/1612119754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00701a8800 con 0x7f00701047c0 2024-09-21T03:46:32.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.135+0000 7f0067e00640 1 -- 172.21.15.16:0/1612119754 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060018980 con 0x7f00701047c0 2024-09-21T03:46:32.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.135+0000 7f0075200640 1 -- 172.21.15.16:0/1612119754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00701a8d90 con 0x7f00701047c0 2024-09-21T03:46:32.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.138+0000 7f007d491740 1 Processor -- start 2024-09-21T03:46:32.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.139+0000 7f007d491740 1 -- start start 2024-09-21T03:46:32.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.139+0000 7f0067e00640 1 -- 172.21.15.16:0/1612119754 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0060010430 con 0x7f00701047c0 2024-09-21T03:46:32.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.139+0000 7f007d491740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb948f60 0x55d4fbcef670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.139+0000 7f007d491740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb9200a0 0x55d4fbcefbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.139+0000 7f007d491740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb928280 0x55d4fbcf00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.139+0000 7f007d491740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbc66d50 con 0x55d4fb928280 2024-09-21T03:46:32.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.139+0000 7f007d491740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbc56180 con 0x55d4fb9200a0 2024-09-21T03:46:32.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.139+0000 7f007d491740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbc59f90 con 0x55d4fb948f60 2024-09-21T03:46:32.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.139+0000 7f006fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb948f60 0x55d4fbcef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.139+0000 7f006fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb948f60 0x55d4fbcef670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:51994/0 (socket says 172.21.15.16:51994) 2024-09-21T03:46:32.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.139+0000 7f006fe00640 1 -- 172.21.15.16:0/3524905262 learned_addr learned my addr 172.21.15.16:0/3524905262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:32.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.140+0000 7f006f400640 1 --2- 172.21.15.16:0/3524905262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb9200a0 0x55d4fbcefbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.140+0000 7f006ea00640 1 --2- 172.21.15.16:0/3524905262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb928280 0x55d4fbcf00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.140+0000 7f006fe00640 1 -- 172.21.15.16:0/3524905262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb9200a0 msgr2=0x55d4fbcefbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:32.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.140+0000 7f006fe00640 1 --2- 172.21.15.16:0/3524905262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb9200a0 0x55d4fbcefbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.140+0000 7f006fe00640 1 -- 172.21.15.16:0/3524905262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb928280 msgr2=0x55d4fbcf00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:32.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.140+0000 7f006fe00640 1 --2- 172.21.15.16:0/3524905262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb928280 0x55d4fbcf00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.140+0000 7f006fe00640 1 -- 172.21.15.16:0/3524905262 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4fbc20a70 con 0x55d4fb948f60 2024-09-21T03:46:32.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.140+0000 7f006fe00640 1 --2- 172.21.15.16:0/3524905262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb948f60 0x55d4fbcef670 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f006802f410 tx=0x7f006802f440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9587c177314077f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:32.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.141+0000 7f0064c00640 1 -- 172.21.15.16:0/3524905262 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068031cf0 con 0x55d4fb948f60 2024-09-21T03:46:32.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.141+0000 7f0064c00640 1 -- 172.21.15.16:0/3524905262 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0068036030 con 0x55d4fb948f60 2024-09-21T03:46:32.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.141+0000 7f0064c00640 1 -- 172.21.15.16:0/3524905262 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00680042a0 con 0x55d4fb948f60 2024-09-21T03:46:32.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.141+0000 7f0067e00640 1 --2- 172.21.15.16:0/1612119754 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f00540b6590 0x7f00540b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.142+0000 7f0067e00640 1 -- 172.21.15.16:0/1612119754 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f00600e79c0 con 0x7f00701047c0 2024-09-21T03:46:32.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.142+0000 7f006f400640 1 --2- 172.21.15.16:0/1612119754 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f00540b6590 0x7f00540b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.142+0000 7f007d491740 1 -- 172.21.15.16:0/3524905262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb948f60 msgr2=0x55d4fbcef670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:32.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.142+0000 7f007d491740 1 --2- 172.21.15.16:0/3524905262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb948f60 0x55d4fbcef670 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f006802f410 tx=0x7f006802f440 comp rx=0 tx=0).stop 2024-09-21T03:46:32.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.142+0000 7f007d491740 1 -- 172.21.15.16:0/3524905262 shutdown_connections 2024-09-21T03:46:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.142+0000 7f007d491740 1 --2- 172.21.15.16:0/3524905262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb928280 0x55d4fbcf00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.142+0000 7f007d491740 1 --2- 172.21.15.16:0/3524905262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb9200a0 0x55d4fbcefbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.142+0000 7f007d491740 1 --2- 172.21.15.16:0/3524905262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb948f60 0x55d4fbcef670 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.142+0000 7f007d491740 1 -- 172.21.15.16:0/3524905262 >> 172.21.15.16:0/3524905262 conn(0x55d4fb93d350 msgr2=0x55d4fbced560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.142+0000 7f007d491740 1 -- 172.21.15.16:0/3524905262 shutdown_connections 2024-09-21T03:46:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.142+0000 7f006f400640 1 --2- 172.21.15.16:0/1612119754 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f00540b6590 0x7f00540b8aa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f00600ad780 tx=0x7f00600ed530 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.142+0000 7f007d491740 1 -- 172.21.15.16:0/3524905262 wait complete. 2024-09-21T03:46:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.143+0000 7f007d491740 1 -- start start 2024-09-21T03:46:32.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.143+0000 7f007d491740 1 Processor -- start 2024-09-21T03:46:32.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.144+0000 7f007d491740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb9200a0 0x55d4fbcfb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.144+0000 7f007d491740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb928280 0x55d4fbcfb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.144+0000 7f007d491740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb948f60 0x55d4fbd06f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.144+0000 7f007d491740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbab75f0 con 0x55d4fb9200a0 2024-09-21T03:46:32.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.144+0000 7f007d491740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbbf1460 con 0x55d4fb928280 2024-09-21T03:46:32.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.144+0000 7f007d491740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbcfdc70 con 0x55d4fb948f60 2024-09-21T03:46:32.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.144+0000 7f007d491740 10 client.0 _ll_register_callbacks cb 0x55d4fbc01210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.144+0000 7f006fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb9200a0 0x55d4fbcfb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.144+0000 7f006fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb9200a0 0x55d4fbcfb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41698/0 (socket says 172.21.15.16:41698) 2024-09-21T03:46:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.144+0000 7f006fe00640 1 -- 172.21.15.16:0/3627674065 learned_addr learned my addr 172.21.15.16:0/3627674065 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.144+0000 7f006ea00640 1 --2- 172.21.15.16:0/3627674065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb928280 0x55d4fbcfb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.144+0000 7f006fe00640 1 --2- 172.21.15.16:0/3627674065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb948f60 0x55d4fbd06f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.145+0000 7f006fe00640 1 -- 172.21.15.16:0/3627674065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb948f60 msgr2=0x55d4fbd06f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.145+0000 7f006fe00640 1 --2- 172.21.15.16:0/3627674065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb948f60 0x55d4fbd06f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.145+0000 7f006fe00640 1 -- 172.21.15.16:0/3627674065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb928280 msgr2=0x55d4fbcfb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.145+0000 7f006fe00640 1 --2- 172.21.15.16:0/3627674065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb928280 0x55d4fbcfb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.145+0000 7f006fe00640 1 -- 172.21.15.16:0/3627674065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4fb9b4540 con 0x55d4fb9200a0 2024-09-21T03:46:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.145+0000 7f006ea00640 1 --2- 172.21.15.16:0/3627674065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb928280 0x55d4fbcfb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.145+0000 7f006fe00640 1 --2- 172.21.15.16:0/3627674065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb9200a0 0x55d4fbcfb1f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f00680096c0 tx=0x7f0068003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86646b143bd4c6e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.145+0000 7f0059600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:46:32.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.145+0000 7f0059600640 1 -- 172.21.15.16:0/3627674065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068058060 con 0x55d4fb9200a0 2024-09-21T03:46:32.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.146+0000 7f007d491740 1 -- 172.21.15.16:0/3627674065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d4fbbf3a70 con 0x55d4fb9200a0 2024-09-21T03:46:32.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.146+0000 7f0059600640 1 -- 172.21.15.16:0/3627674065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0068031660 con 0x55d4fb9200a0 2024-09-21T03:46:32.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.146+0000 7f0059600640 1 -- 172.21.15.16:0/3627674065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0068031860 con 0x55d4fb9200a0 2024-09-21T03:46:32.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.146+0000 7f007d491740 10 client.15480 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:32.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.146+0000 7f007d491740 1 -- 172.21.15.16:0/3627674065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d4fbc0e6f0 con 0x55d4fb9200a0 2024-09-21T03:46:32.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.146+0000 7f0059600640 1 -- 172.21.15.16:0/3627674065 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d4fbbf3a70 con 0x55d4fb9200a0 2024-09-21T03:46:32.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.146+0000 7f0059600640 1 -- 172.21.15.16:0/3627674065 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f006804d070 con 0x55d4fb9200a0 2024-09-21T03:46:32.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.146+0000 7f007d491740 10 client.15480 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:32.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.147+0000 7f007d491740 10 client.15480 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-21T03:46:32.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.147+0000 7f007d491740 20 client.15480 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:32.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.147+0000 7f007d491740 1 --2- 172.21.15.16:0/3627674065 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x55d4fb9078e0 0x55d4fbd0d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.147+0000 7f007d491740 4 client.15480 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:32.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.147+0000 7f007d491740 1 -- 172.21.15.16:0/3627674065 --> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4fbc68ca0 con 0x55d4fb9078e0 2024-09-21T03:46:32.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.147+0000 7f006ea00640 1 --2- 172.21.15.16:0/3627674065 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x55d4fb9078e0 0x55d4fbd0d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.148+0000 7f006ea00640 1 --2- 172.21.15.16:0/3627674065 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x55d4fb9078e0 0x55d4fbd0d150 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=541101ef00e938ea server_cookie=d1b85b427600d055 in_seq=0 out_seq=0 2024-09-21T03:46:32.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.148+0000 7f0059600640 10 client.15480 ms_handle_connect on v2:172.21.15.123:6834/86959152 2024-09-21T03:46:32.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.148+0000 7f0059600640 1 -- 172.21.15.16:0/3627674065 <== mds.4 v2:172.21.15.123:6834/86959152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d4fbc68ca0 con 0x55d4fb9078e0 2024-09-21T03:46:32.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.148+0000 7f0059600640 10 client.15480 handle_command_reply: tid=0 2024-09-21T03:46:32.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.149+0000 7f007d491740 1 client.15480 shutdown 2024-09-21T03:46:32.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.149+0000 7f007d491740 1 -- 172.21.15.16:0/3627674065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb9200a0 msgr2=0x55d4fbcfb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:32.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.149+0000 7f007d491740 1 --2- 172.21.15.16:0/3627674065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb9200a0 0x55d4fbcfb1f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f00680096c0 tx=0x7f0068003cc0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.149+0000 7f007d491740 1 -- 172.21.15.16:0/3627674065 shutdown_connections 2024-09-21T03:46:32.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.149+0000 7f007d491740 1 --2- 172.21.15.16:0/3627674065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb948f60 0x55d4fbd06f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.149+0000 7f007d491740 1 --2- 172.21.15.16:0/3627674065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb928280 0x55d4fbcfb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.149+0000 7f007d491740 1 --2- 172.21.15.16:0/3627674065 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x55d4fb9078e0 0x55d4fbd0d150 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.149+0000 7f007d491740 1 --2- 172.21.15.16:0/3627674065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb9200a0 0x55d4fbcfb1f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.149+0000 7f007d491740 1 -- 172.21.15.16:0/3627674065 >> 172.21.15.16:0/3627674065 conn(0x55d4fb93d350 msgr2=0x55d4fbcf2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:32.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.149+0000 7f0059600640 0 client.15480 ms_handle_reset on v2:172.21.15.123:6834/86959152 2024-09-21T03:46:32.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.149+0000 7f007d491740 1 -- 172.21.15.16:0/3627674065 shutdown_connections 2024-09-21T03:46:32.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.149+0000 7f007d491740 1 -- 172.21.15.16:0/3627674065 wait complete. 2024-09-21T03:46:32.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.149+0000 7f007d491740 20 client.15480 trim_cache size 0 max 16384 2024-09-21T03:46:32.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.168+0000 7f007d491740 1 Processor -- start 2024-09-21T03:46:32.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.168+0000 7f007d491740 1 -- start start 2024-09-21T03:46:32.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.169+0000 7f007d491740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb9200a0 0x55d4fbd06f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.169+0000 7f007d491740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb928280 0x55d4fbcf5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.169+0000 7f007d491740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb948f60 0x55d4fbcf6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.169+0000 7f007d491740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbd0b490 con 0x55d4fb928280 2024-09-21T03:46:32.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.169+0000 7f007d491740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbd0bb60 con 0x55d4fb948f60 2024-09-21T03:46:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.169+0000 7f007d491740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbd0c230 con 0x55d4fb9200a0 2024-09-21T03:46:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.169+0000 7f006ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb928280 0x55d4fbcf5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.169+0000 7f006ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb928280 0x55d4fbcf5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41710/0 (socket says 172.21.15.16:41710) 2024-09-21T03:46:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.169+0000 7f006fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb948f60 0x55d4fbcf6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.169+0000 7f006ea00640 1 -- 172.21.15.16:0/1316849341 learned_addr learned my addr 172.21.15.16:0/1316849341 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.169+0000 7f006fe00640 1 --2- 172.21.15.16:0/1316849341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb9200a0 0x55d4fbd06f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.170+0000 7f006ea00640 1 -- 172.21.15.16:0/1316849341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb9200a0 msgr2=0x55d4fbd06f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.170+0000 7f006ea00640 1 --2- 172.21.15.16:0/1316849341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb9200a0 0x55d4fbd06f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.170+0000 7f006ea00640 1 -- 172.21.15.16:0/1316849341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb948f60 msgr2=0x55d4fbcf6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.170+0000 7f006ea00640 1 --2- 172.21.15.16:0/1316849341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb948f60 0x55d4fbcf6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.170+0000 7f006ea00640 1 -- 172.21.15.16:0/1316849341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4fbcfc090 con 0x55d4fb928280 2024-09-21T03:46:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.170+0000 7f006fe00640 1 --2- 172.21.15.16:0/1316849341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb948f60 0x55d4fbcf6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.170+0000 7f006ea00640 1 --2- 172.21.15.16:0/1316849341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb928280 0x55d4fbcf5790 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f005c004700 tx=0x7f005c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11ce1e259f3aa5f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.170+0000 7f005be00640 1 -- 172.21.15.16:0/1316849341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c02a070 con 0x55d4fb928280 2024-09-21T03:46:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.170+0000 7f005be00640 1 -- 172.21.15.16:0/1316849341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f005c00fa10 con 0x55d4fb928280 2024-09-21T03:46:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.170+0000 7f005be00640 1 -- 172.21.15.16:0/1316849341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c025aa0 con 0x55d4fb928280 2024-09-21T03:46:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.171+0000 7f007d491740 1 -- 172.21.15.16:0/1316849341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb928280 msgr2=0x55d4fbcf5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.171+0000 7f007d491740 1 --2- 172.21.15.16:0/1316849341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb928280 0x55d4fbcf5790 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f005c004700 tx=0x7f005c005f90 comp rx=0 tx=0).stop 2024-09-21T03:46:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.171+0000 7f007d491740 1 -- 172.21.15.16:0/1316849341 shutdown_connections 2024-09-21T03:46:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.171+0000 7f007d491740 1 --2- 172.21.15.16:0/1316849341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb948f60 0x55d4fbcf6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.171+0000 7f007d491740 1 --2- 172.21.15.16:0/1316849341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb928280 0x55d4fbcf5790 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.171+0000 7f007d491740 1 --2- 172.21.15.16:0/1316849341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb9200a0 0x55d4fbd06f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.171+0000 7f007d491740 1 -- 172.21.15.16:0/1316849341 >> 172.21.15.16:0/1316849341 conn(0x55d4fb93d350 msgr2=0x55d4fbce82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.171+0000 7f007d491740 1 -- 172.21.15.16:0/1316849341 shutdown_connections 2024-09-21T03:46:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.171+0000 7f007d491740 1 -- 172.21.15.16:0/1316849341 wait complete. 2024-09-21T03:46:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.172+0000 7f007d491740 1 -- start start 2024-09-21T03:46:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.172+0000 7f007d491740 1 Processor -- start 2024-09-21T03:46:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.173+0000 7f007d491740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb9200a0 0x55d4fbd0e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.173+0000 7f007d491740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb928280 0x55d4fbd0ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.173+0000 7f007d491740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb948f60 0x55d4fbd0ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.173+0000 7f007d491740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbd0cfd0 con 0x55d4fb948f60 2024-09-21T03:46:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.173+0000 7f007d491740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbd0c900 con 0x55d4fb9200a0 2024-09-21T03:46:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.173+0000 7f007d491740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbd0d6a0 con 0x55d4fb928280 2024-09-21T03:46:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.173+0000 7f007d491740 10 client.0 _ll_register_callbacks cb 0x7f0068031ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.173+0000 7f006ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb928280 0x55d4fbd0ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.173+0000 7f006ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb928280 0x55d4fbd0ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52022/0 (socket says 172.21.15.16:52022) 2024-09-21T03:46:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.173+0000 7f006ea00640 1 -- 172.21.15.16:0/2217424959 learned_addr learned my addr 172.21.15.16:0/2217424959 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.173+0000 7f006fe00640 1 --2- 172.21.15.16:0/2217424959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb9200a0 0x55d4fbd0e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.173+0000 7f006fe00640 1 --2- 172.21.15.16:0/2217424959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb948f60 0x55d4fbd0ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.173+0000 7f006ea00640 1 -- 172.21.15.16:0/2217424959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb9200a0 msgr2=0x55d4fbd0e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.173+0000 7f006ea00640 1 --2- 172.21.15.16:0/2217424959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb9200a0 0x55d4fbd0e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.173+0000 7f006ea00640 1 -- 172.21.15.16:0/2217424959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb948f60 msgr2=0x55d4fbd0ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.173+0000 7f006ea00640 1 --2- 172.21.15.16:0/2217424959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb948f60 0x55d4fbd0ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.173+0000 7f006ea00640 1 -- 172.21.15.16:0/2217424959 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4fbd76610 con 0x55d4fb928280 2024-09-21T03:46:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.174+0000 7f006ea00640 1 --2- 172.21.15.16:0/2217424959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb928280 0x55d4fbd0ea20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f005c028040 tx=0x7f005c005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5a6ff6016f74cfb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.174+0000 7f0059600640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:46:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.174+0000 7f0059600640 1 -- 172.21.15.16:0/2217424959 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c02a040 con 0x55d4fb928280 2024-09-21T03:46:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.174+0000 7f007d491740 1 -- 172.21.15.16:0/2217424959 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f005c00fa10 con 0x55d4fb928280 2024-09-21T03:46:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.174+0000 7f0059600640 1 -- 172.21.15.16:0/2217424959 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f005c00fdc0 con 0x55d4fb928280 2024-09-21T03:46:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.175+0000 7f0059600640 1 -- 172.21.15.16:0/2217424959 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f005c00f560 con 0x55d4fb928280 2024-09-21T03:46:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.175+0000 7f007d491740 10 client.25477 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.175+0000 7f007d491740 1 -- 172.21.15.16:0/2217424959 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d4fbd77860 con 0x55d4fb928280 2024-09-21T03:46:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.175+0000 7f0059600640 1 -- 172.21.15.16:0/2217424959 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f005c00fa10 con 0x55d4fb928280 2024-09-21T03:46:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.175+0000 7f0059600640 1 -- 172.21.15.16:0/2217424959 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f005c0053e0 con 0x55d4fb928280 2024-09-21T03:46:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.175+0000 7f007d491740 10 client.25477 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.175+0000 7f007d491740 10 client.25477 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-21T03:46:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.175+0000 7f007d491740 20 client.25477 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.175+0000 7f007d491740 1 --2- 172.21.15.16:0/2217424959 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x55d4fb8dcc90 0x55d4fbd7afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.175+0000 7f007d491740 4 client.25477 mds_command: new command op to 24668 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:46:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.175+0000 7f007d491740 1 -- 172.21.15.16:0/2217424959 --> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] -- command(tid 0: {"prefix": "status"}) -- 0x55d4fbc64750 con 0x55d4fb8dcc90 2024-09-21T03:46:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.176+0000 7f006fe00640 1 --2- 172.21.15.16:0/2217424959 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x55d4fb8dcc90 0x55d4fbd7afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.176+0000 7f006fe00640 1 --2- 172.21.15.16:0/2217424959 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x55d4fb8dcc90 0x55d4fbd7afc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a93e12bdaf8f7a2 server_cookie=512d1ed58dd38414 in_seq=0 out_seq=0 2024-09-21T03:46:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.176+0000 7f0059600640 10 client.25477 ms_handle_connect on v2:172.21.15.123:6834/86959152 2024-09-21T03:46:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.176+0000 7f0059600640 1 -- 172.21.15.16:0/2217424959 <== mds.4 v2:172.21.15.123:6834/86959152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55d4fbc64750 con 0x55d4fb8dcc90 2024-09-21T03:46:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.176+0000 7f0059600640 10 client.25477 handle_command_reply: tid=0 2024-09-21T03:46:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.176+0000 7f007d491740 1 client.25477 shutdown 2024-09-21T03:46:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.177+0000 7f007d491740 1 -- 172.21.15.16:0/2217424959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb928280 msgr2=0x55d4fbd0ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.177+0000 7f007d491740 1 --2- 172.21.15.16:0/2217424959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb928280 0x55d4fbd0ea20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f005c028040 tx=0x7f005c005e70 comp rx=0 tx=0).stop 2024-09-21T03:46:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.177+0000 7f007d491740 1 -- 172.21.15.16:0/2217424959 shutdown_connections 2024-09-21T03:46:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.177+0000 7f007d491740 1 --2- 172.21.15.16:0/2217424959 >> [v2:172.21.15.123:6834/86959152,v1:172.21.15.123:6835/86959152] conn(0x55d4fb8dcc90 0x55d4fbd7afc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.177+0000 7f007d491740 1 --2- 172.21.15.16:0/2217424959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4fb948f60 0x55d4fbd0ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.177+0000 7f007d491740 1 --2- 172.21.15.16:0/2217424959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4fb928280 0x55d4fbd0ea20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.177+0000 7f007d491740 1 --2- 172.21.15.16:0/2217424959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4fb9200a0 0x55d4fbd0e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.177+0000 7f0059600640 0 client.25477 ms_handle_reset on v2:172.21.15.123:6834/86959152 2024-09-21T03:46:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.177+0000 7f007d491740 1 -- 172.21.15.16:0/2217424959 >> 172.21.15.16:0/2217424959 conn(0x55d4fb93d350 msgr2=0x55d4fbcf6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.177+0000 7f007d491740 1 -- 172.21.15.16:0/2217424959 shutdown_connections 2024-09-21T03:46:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.177+0000 7f007d491740 1 -- 172.21.15.16:0/2217424959 wait complete. 2024-09-21T03:46:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.177+0000 7f007d491740 20 client.25477 trim_cache size 0 max 16384 2024-09-21T03:46:32.183 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:46:32.183 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:46:32.183 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 4, 2024-09-21T03:46:32.183 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24668, 2024-09-21T03:46:32.183 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T03:46:32.183 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T03:46:32.183 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T03:46:32.183 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 146.645134799, 2024-09-21T03:46:32.183 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 30, 2024-09-21T03:46:32.183 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 86, 2024-09-21T03:46:32.183 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 86, 2024-09-21T03:46:32.184 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 239.925296989 2024-09-21T03:46:32.184 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:46:32.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.178+0000 7f0075200640 1 -- 172.21.15.16:0/1612119754 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f00540b6590 msgr2=0x7f00540b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:32.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.178+0000 7f0075200640 1 --2- 172.21.15.16:0/1612119754 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f00540b6590 0x7f00540b8aa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f00600ad780 tx=0x7f00600ed530 comp rx=0 tx=0).stop 2024-09-21T03:46:32.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.178+0000 7f0075200640 1 -- 172.21.15.16:0/1612119754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00701047c0 msgr2=0x7f0070102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:32.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.178+0000 7f0075200640 1 --2- 172.21.15.16:0/1612119754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00701047c0 0x7f0070102500 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f006000e9f0 tx=0x7f006000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.178+0000 7f0075200640 1 -- 172.21.15.16:0/1612119754 shutdown_connections 2024-09-21T03:46:32.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.178+0000 7f0075200640 1 --2- 172.21.15.16:0/1612119754 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f00540b6590 0x7f00540b8aa0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.178+0000 7f0075200640 1 --2- 172.21.15.16:0/1612119754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f007010dc20 0x7f0070102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.178+0000 7f0075200640 1 --2- 172.21.15.16:0/1612119754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0070105110 0x7f0070102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.178+0000 7f0075200640 1 --2- 172.21.15.16:0/1612119754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00701047c0 0x7f0070102500 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:32.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.178+0000 7f0075200640 1 -- 172.21.15.16:0/1612119754 >> 172.21.15.16:0/1612119754 conn(0x7f00700fe2c0 msgr2=0x7f0070100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:32.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.178+0000 7f0075200640 1 -- 172.21.15.16:0/1612119754 shutdown_connections 2024-09-21T03:46:32.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:32.178+0000 7f0075200640 1 -- 172.21.15.16:0/1612119754 wait complete. 2024-09-21T03:46:32.885 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:32 smithi016 ceph-mon[30375]: pgmap v425: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T03:46:32.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:32 smithi123 ceph-mon[34673]: pgmap v425: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T03:46:32.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T03:46:33.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:32 smithi134 ceph-mon[35031]: pgmap v425: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T03:46:33.307 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.944+0000 7f5bfee00640 1 -- 172.21.15.16:0/1364754066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf81092f0 msgr2=0x7f5bf8109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.944+0000 7f5bfee00640 1 --2- 172.21.15.16:0/1364754066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf81092f0 0x7f5bf8109700 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009900 tx=0x7f5be402a410 comp rx=0 tx=0).stop 2024-09-21T03:46:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.944+0000 7f5bfee00640 1 -- 172.21.15.16:0/1364754066 shutdown_connections 2024-09-21T03:46:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.944+0000 7f5bfee00640 1 --2- 172.21.15.16:0/1364754066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bf8109c40 0x7f5bf8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.944+0000 7f5bfee00640 1 --2- 172.21.15.16:0/1364754066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf81092f0 0x7f5bf8109700 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.944+0000 7f5bfee00640 1 --2- 172.21.15.16:0/1364754066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bf81039b0 0x7f5bf8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.944+0000 7f5bfee00640 1 -- 172.21.15.16:0/1364754066 >> 172.21.15.16:0/1364754066 conn(0x7f5bf80ff400 msgr2=0x7f5bf8101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.944+0000 7f5bfee00640 1 -- 172.21.15.16:0/1364754066 shutdown_connections 2024-09-21T03:46:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.945+0000 7f5bfee00640 1 -- 172.21.15.16:0/1364754066 wait complete. 2024-09-21T03:46:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.946+0000 7f5bfee00640 1 Processor -- start 2024-09-21T03:46:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.946+0000 7f5bfee00640 1 -- start start 2024-09-21T03:46:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.946+0000 7f5bfee00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bf81039b0 0x7f5bf8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.946+0000 7f5bfee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf81092f0 0x7f5bf8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.946+0000 7f5bfee00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bf8109c40 0x7f5bf8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.946+0000 7f5bfee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf8113950 con 0x7f5bf81092f0 2024-09-21T03:46:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.946+0000 7f5bfee00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf81137d0 con 0x7f5bf8109c40 2024-09-21T03:46:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.946+0000 7f5bfee00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf8113ad0 con 0x7f5bf81039b0 2024-09-21T03:46:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.947+0000 7f5bfda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bf81039b0 0x7f5bf8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.947+0000 7f5bfd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf81092f0 0x7f5bf8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.947+0000 7f5bfda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bf81039b0 0x7f5bf8075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52038/0 (socket says 172.21.15.16:52038) 2024-09-21T03:46:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.947+0000 7f5bfda00640 1 -- 172.21.15.16:0/3055508057 learned_addr learned my addr 172.21.15.16:0/3055508057 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.947+0000 7f5bfe400640 1 --2- 172.21.15.16:0/3055508057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bf8109c40 0x7f5bf8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.947+0000 7f5bfe400640 1 -- 172.21.15.16:0/3055508057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bf81039b0 msgr2=0x7f5bf8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.947+0000 7f5bfe400640 1 --2- 172.21.15.16:0/3055508057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bf81039b0 0x7f5bf8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.947+0000 7f5bfe400640 1 -- 172.21.15.16:0/3055508057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf81092f0 msgr2=0x7f5bf8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.947+0000 7f5bfe400640 1 --2- 172.21.15.16:0/3055508057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf81092f0 0x7f5bf8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.947+0000 7f5bfe400640 1 -- 172.21.15.16:0/3055508057 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf81a85a0 con 0x7f5bf8109c40 2024-09-21T03:46:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.947+0000 7f5bfd000640 1 --2- 172.21.15.16:0/3055508057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf81092f0 0x7f5bf8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.947+0000 7f5bfe400640 1 --2- 172.21.15.16:0/3055508057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bf8109c40 0x7f5bf8076190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009fd0 tx=0x7f5bf000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.948+0000 7f5bf6000640 1 -- 172.21.15.16:0/3055508057 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf000ce20 con 0x7f5bf8109c40 2024-09-21T03:46:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.948+0000 7f5bf6000640 1 -- 172.21.15.16:0/3055508057 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bf0005020 con 0x7f5bf8109c40 2024-09-21T03:46:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.948+0000 7f5bfee00640 1 -- 172.21.15.16:0/3055508057 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf81a8890 con 0x7f5bf8109c40 2024-09-21T03:46:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.948+0000 7f5bf6000640 1 -- 172.21.15.16:0/3055508057 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0010690 con 0x7f5bf8109c40 2024-09-21T03:46:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.948+0000 7f5bfee00640 1 -- 172.21.15.16:0/3055508057 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf81a8da0 con 0x7f5bf8109c40 2024-09-21T03:46:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.952+0000 7f5c0707b740 1 Processor -- start 2024-09-21T03:46:33.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.953+0000 7f5bf6000640 1 -- 172.21.15.16:0/3055508057 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5bf00054d0 con 0x7f5bf8109c40 2024-09-21T03:46:33.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.953+0000 7f5c0707b740 1 -- start start 2024-09-21T03:46:33.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.953+0000 7f5c0707b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f026477f60 0x55f02681e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.953+0000 7f5c0707b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f02644f0a0 0x55f02681ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.953+0000 7f5c0707b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026457280 0x55f02681f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.953+0000 7f5c0707b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f026795d50 con 0x55f02644f0a0 2024-09-21T03:46:33.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.953+0000 7f5c0707b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f026785180 con 0x55f026477f60 2024-09-21T03:46:33.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.953+0000 7f5c0707b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f026788f90 con 0x55f026457280 2024-09-21T03:46:33.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.954+0000 7f5bfda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f02644f0a0 0x55f02681ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.954+0000 7f5bfda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f02644f0a0 0x55f02681ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41734/0 (socket says 172.21.15.16:41734) 2024-09-21T03:46:33.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.954+0000 7f5bfda00640 1 -- 172.21.15.16:0/114085236 learned_addr learned my addr 172.21.15.16:0/114085236 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:33.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.954+0000 7f5bfd000640 1 --2- 172.21.15.16:0/114085236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f026477f60 0x55f02681e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.954+0000 7f5bfd000640 1 --2- 172.21.15.16:0/114085236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026457280 0x55f02681f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.954+0000 7f5bfda00640 1 -- 172.21.15.16:0/114085236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026457280 msgr2=0x55f02681f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:33.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.954+0000 7f5bfda00640 1 --2- 172.21.15.16:0/114085236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026457280 0x55f02681f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.954+0000 7f5bfda00640 1 -- 172.21.15.16:0/114085236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f026477f60 msgr2=0x55f02681e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:33.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.954+0000 7f5bfda00640 1 --2- 172.21.15.16:0/114085236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f026477f60 0x55f02681e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.954+0000 7f5bfda00640 1 -- 172.21.15.16:0/114085236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f02674fa70 con 0x55f02644f0a0 2024-09-21T03:46:33.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.955+0000 7f5bfda00640 1 --2- 172.21.15.16:0/114085236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f02644f0a0 0x55f02681ebb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009d70 tx=0x7f5bec011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28bdece2f7495e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.955+0000 7f5bf6000640 1 --2- 172.21.15.16:0/3055508057 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5be00b6510 0x7f5be00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.955+0000 7f5beaa00640 1 -- 172.21.15.16:0/114085236 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec01e070 con 0x55f02644f0a0 2024-09-21T03:46:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.955+0000 7f5beaa00640 1 -- 172.21.15.16:0/114085236 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bec019ae0 con 0x55f02644f0a0 2024-09-21T03:46:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.955+0000 7f5beaa00640 1 -- 172.21.15.16:0/114085236 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec00eab0 con 0x55f02644f0a0 2024-09-21T03:46:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.955+0000 7f5bf6000640 1 -- 172.21.15.16:0/3055508057 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f5bf00e7df0 con 0x7f5bf8109c40 2024-09-21T03:46:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.955+0000 7f5bfd000640 1 --2- 172.21.15.16:0/3055508057 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5be00b6510 0x7f5be00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.956+0000 7f5bfd000640 1 --2- 172.21.15.16:0/3055508057 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5be00b6510 0x7f5be00b8a20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5be402aa00 tx=0x7f5be4005780 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.956+0000 7f5c0707b740 1 -- 172.21.15.16:0/114085236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f02644f0a0 msgr2=0x55f02681ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.956+0000 7f5c0707b740 1 --2- 172.21.15.16:0/114085236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f02644f0a0 0x55f02681ebb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009d70 tx=0x7f5bec011810 comp rx=0 tx=0).stop 2024-09-21T03:46:33.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.956+0000 7f5c0707b740 1 -- 172.21.15.16:0/114085236 shutdown_connections 2024-09-21T03:46:33.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.956+0000 7f5c0707b740 1 --2- 172.21.15.16:0/114085236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026457280 0x55f02681f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.956+0000 7f5c0707b740 1 --2- 172.21.15.16:0/114085236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f02644f0a0 0x55f02681ebb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.956+0000 7f5c0707b740 1 --2- 172.21.15.16:0/114085236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f026477f60 0x55f02681e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.956+0000 7f5c0707b740 1 -- 172.21.15.16:0/114085236 >> 172.21.15.16:0/114085236 conn(0x55f02646c350 msgr2=0x55f02681c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:33.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.956+0000 7f5c0707b740 1 -- 172.21.15.16:0/114085236 shutdown_connections 2024-09-21T03:46:33.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.956+0000 7f5c0707b740 1 -- 172.21.15.16:0/114085236 wait complete. 2024-09-21T03:46:33.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.957+0000 7f5c0707b740 1 -- start start 2024-09-21T03:46:33.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.957+0000 7f5c0707b740 1 Processor -- start 2024-09-21T03:46:33.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5c0707b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02644f0a0 0x55f02682a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5c0707b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f026457280 0x55f02682a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5c0707b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f026477f60 0x55f026836170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5c0707b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f0265e65f0 con 0x55f026477f60 2024-09-21T03:46:33.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5c0707b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f026720460 con 0x55f026457280 2024-09-21T03:46:33.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5c0707b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f02682ce50 con 0x55f02644f0a0 2024-09-21T03:46:33.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5c0707b740 10 client.0 _ll_register_callbacks cb 0x55f026730210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:33.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5bfda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02644f0a0 0x55f02682a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5bfe400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f026477f60 0x55f026836170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5bfd000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f026457280 0x55f02682a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5bfda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02644f0a0 0x55f02682a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52044/0 (socket says 172.21.15.16:52044) 2024-09-21T03:46:33.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5bfda00640 1 -- 172.21.15.16:0/4234503762 learned_addr learned my addr 172.21.15.16:0/4234503762 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:33.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5bfda00640 1 -- 172.21.15.16:0/4234503762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f026457280 msgr2=0x55f02682a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:33.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5bfda00640 1 --2- 172.21.15.16:0/4234503762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f026457280 0x55f02682a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5bfda00640 1 -- 172.21.15.16:0/4234503762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f026477f60 msgr2=0x55f026836170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:33.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5bfda00640 1 --2- 172.21.15.16:0/4234503762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f026477f60 0x55f026836170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5bfda00640 1 -- 172.21.15.16:0/4234503762 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0264e3540 con 0x55f02644f0a0 2024-09-21T03:46:33.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.958+0000 7f5bfe400640 1 --2- 172.21.15.16:0/4234503762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f026477f60 0x55f026836170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:33.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.959+0000 7f5bfda00640 1 --2- 172.21.15.16:0/4234503762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02644f0a0 0x55f02682a1d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00e6d0 tx=0x7f5bec005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28abec3dbb966f4b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:33.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.959+0000 7f5bdf400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:46:33.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.959+0000 7f5bdf400640 1 -- 172.21.15.16:0/4234503762 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec027020 con 0x55f02644f0a0 2024-09-21T03:46:33.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.959+0000 7f5c0707b740 1 -- 172.21.15.16:0/4234503762 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f026722a70 con 0x55f02644f0a0 2024-09-21T03:46:33.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.959+0000 7f5bdf400640 1 -- 172.21.15.16:0/4234503762 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f5bec00ec40 con 0x55f02644f0a0 2024-09-21T03:46:33.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.959+0000 7f5bdf400640 1 -- 172.21.15.16:0/4234503762 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5bec032050 con 0x55f02644f0a0 2024-09-21T03:46:33.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.960+0000 7f5c0707b740 10 client.25489 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:33.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.960+0000 7f5c0707b740 1 -- 172.21.15.16:0/4234503762 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f02673d6f0 con 0x55f02644f0a0 2024-09-21T03:46:33.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.960+0000 7f5bdf400640 1 -- 172.21.15.16:0/4234503762 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f026722a70 con 0x55f02644f0a0 2024-09-21T03:46:33.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.960+0000 7f5bdf400640 1 -- 172.21.15.16:0/4234503762 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5bec019430 con 0x55f02644f0a0 2024-09-21T03:46:33.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.960+0000 7f5c0707b740 10 client.25489 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:33.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.960+0000 7f5c0707b740 10 client.25489 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:46:33.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.960+0000 7f5c0707b740 20 client.25489 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:33.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.960+0000 7f5c0707b740 1 --2- 172.21.15.16:0/4234503762 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x55f0264368e0 0x55f02683c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.960+0000 7f5c0707b740 4 client.25489 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:33.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.960+0000 7f5c0707b740 1 -- 172.21.15.16:0/4234503762 --> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f026797ca0 con 0x55f0264368e0 2024-09-21T03:46:33.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.961+0000 7f5bfd000640 1 --2- 172.21.15.16:0/4234503762 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x55f0264368e0 0x55f02683c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.961+0000 7f5bfd000640 1 --2- 172.21.15.16:0/4234503762 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x55f0264368e0 0x55f02683c440 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b4b8cbb0cdbc34c server_cookie=6d194fa27d12e4a0 in_seq=0 out_seq=0 2024-09-21T03:46:33.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.961+0000 7f5bdf400640 10 client.25489 ms_handle_connect on v2:172.21.15.134:6836/2624530453 2024-09-21T03:46:33.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.962+0000 7f5bdf400640 1 -- 172.21.15.16:0/4234503762 <== mds.3 v2:172.21.15.134:6836/2624530453 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f026797ca0 con 0x55f0264368e0 2024-09-21T03:46:33.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.962+0000 7f5bdf400640 10 client.25489 handle_command_reply: tid=0 2024-09-21T03:46:33.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.962+0000 7f5c0707b740 1 client.25489 shutdown 2024-09-21T03:46:33.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.963+0000 7f5c0707b740 1 -- 172.21.15.16:0/4234503762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02644f0a0 msgr2=0x55f02682a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:33.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.963+0000 7f5c0707b740 1 --2- 172.21.15.16:0/4234503762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02644f0a0 0x55f02682a1d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00e6d0 tx=0x7f5bec005d30 comp rx=0 tx=0).stop 2024-09-21T03:46:33.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.963+0000 7f5c0707b740 1 -- 172.21.15.16:0/4234503762 shutdown_connections 2024-09-21T03:46:33.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.963+0000 7f5c0707b740 1 --2- 172.21.15.16:0/4234503762 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x55f0264368e0 0x55f02683c440 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.963+0000 7f5c0707b740 1 --2- 172.21.15.16:0/4234503762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f026477f60 0x55f026836170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.963+0000 7f5c0707b740 1 --2- 172.21.15.16:0/4234503762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f026457280 0x55f02682a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.963+0000 7f5c0707b740 1 --2- 172.21.15.16:0/4234503762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02644f0a0 0x55f02682a1d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.963+0000 7f5bdf400640 0 client.25489 ms_handle_reset on v2:172.21.15.134:6836/2624530453 2024-09-21T03:46:33.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.963+0000 7f5c0707b740 1 -- 172.21.15.16:0/4234503762 >> 172.21.15.16:0/4234503762 conn(0x55f02646c350 msgr2=0x55f026821800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:33.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.963+0000 7f5c0707b740 1 -- 172.21.15.16:0/4234503762 shutdown_connections 2024-09-21T03:46:33.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.963+0000 7f5c0707b740 1 -- 172.21.15.16:0/4234503762 wait complete. 2024-09-21T03:46:33.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.963+0000 7f5c0707b740 20 client.25489 trim_cache size 0 max 16384 2024-09-21T03:46:33.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.982+0000 7f5c0707b740 1 Processor -- start 2024-09-21T03:46:33.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.982+0000 7f5c0707b740 1 -- start start 2024-09-21T03:46:33.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.982+0000 7f5c0707b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f02644f0a0 0x55f02683a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.982+0000 7f5c0707b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026457280 0x55f02683afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.982+0000 7f5c0707b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f026477f60 0x55f0268244a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.982+0000 7f5c0707b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f02683a780 con 0x55f02644f0a0 2024-09-21T03:46:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.982+0000 7f5c0707b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f02683ae50 con 0x55f026477f60 2024-09-21T03:46:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.982+0000 7f5c0707b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f02683b520 con 0x55f026457280 2024-09-21T03:46:33.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.983+0000 7f5bfda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f02644f0a0 0x55f02683a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.983+0000 7f5bfda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f02644f0a0 0x55f02683a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41758/0 (socket says 172.21.15.16:41758) 2024-09-21T03:46:33.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.983+0000 7f5bfda00640 1 -- 172.21.15.16:0/285299742 learned_addr learned my addr 172.21.15.16:0/285299742 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:33.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.983+0000 7f5bfd000640 1 --2- 172.21.15.16:0/285299742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026457280 0x55f02683afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.983+0000 7f5bfe400640 1 --2- 172.21.15.16:0/285299742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f026477f60 0x55f0268244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.983+0000 7f5bfda00640 1 -- 172.21.15.16:0/285299742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026457280 msgr2=0x55f02683afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:33.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.983+0000 7f5bfda00640 1 --2- 172.21.15.16:0/285299742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026457280 0x55f02683afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.983+0000 7f5bfda00640 1 -- 172.21.15.16:0/285299742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f026477f60 msgr2=0x55f0268244a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:33.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.983+0000 7f5bfda00640 1 --2- 172.21.15.16:0/285299742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f026477f60 0x55f0268244a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.983+0000 7f5bfda00640 1 -- 172.21.15.16:0/285299742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f026820960 con 0x55f02644f0a0 2024-09-21T03:46:33.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.983+0000 7f5bfda00640 1 --2- 172.21.15.16:0/285299742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f02644f0a0 0x55f02683a900 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00e700 tx=0x7f5bec009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146761fdcd07a401 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:33.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.984+0000 7f5bea000640 1 -- 172.21.15.16:0/285299742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec027040 con 0x55f02644f0a0 2024-09-21T03:46:33.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.984+0000 7f5bea000640 1 -- 172.21.15.16:0/285299742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bec01f040 con 0x55f02644f0a0 2024-09-21T03:46:33.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.984+0000 7f5bea000640 1 -- 172.21.15.16:0/285299742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec01e070 con 0x55f02644f0a0 2024-09-21T03:46:33.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.985+0000 7f5c0707b740 1 -- 172.21.15.16:0/285299742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f02644f0a0 msgr2=0x55f02683a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:33.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.985+0000 7f5c0707b740 1 --2- 172.21.15.16:0/285299742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f02644f0a0 0x55f02683a900 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00e700 tx=0x7f5bec009700 comp rx=0 tx=0).stop 2024-09-21T03:46:33.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.985+0000 7f5c0707b740 1 -- 172.21.15.16:0/285299742 shutdown_connections 2024-09-21T03:46:33.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.985+0000 7f5c0707b740 1 --2- 172.21.15.16:0/285299742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f026477f60 0x55f0268244a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.985+0000 7f5c0707b740 1 --2- 172.21.15.16:0/285299742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026457280 0x55f02683afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.985+0000 7f5c0707b740 1 --2- 172.21.15.16:0/285299742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f02644f0a0 0x55f02683a900 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.985+0000 7f5c0707b740 1 -- 172.21.15.16:0/285299742 >> 172.21.15.16:0/285299742 conn(0x55f02646c350 msgr2=0x55f0268362f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:33.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.985+0000 7f5c0707b740 1 -- 172.21.15.16:0/285299742 shutdown_connections 2024-09-21T03:46:33.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.985+0000 7f5c0707b740 1 -- 172.21.15.16:0/285299742 wait complete. 2024-09-21T03:46:33.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.985+0000 7f5c0707b740 1 -- start start 2024-09-21T03:46:33.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.985+0000 7f5c0707b740 1 Processor -- start 2024-09-21T03:46:33.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.986+0000 7f5c0707b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f02644f0a0 0x55f02683de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.986+0000 7f5c0707b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026457280 0x55f02683e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.986+0000 7f5c0707b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f026477f60 0x55f026843410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.986+0000 7f5c0707b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f02683c2c0 con 0x55f026477f60 2024-09-21T03:46:33.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.986+0000 7f5c0707b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f02683bbf0 con 0x55f02644f0a0 2024-09-21T03:46:33.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.986+0000 7f5c0707b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f02683c990 con 0x55f026457280 2024-09-21T03:46:33.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.986+0000 7f5c0707b740 10 client.0 _ll_register_callbacks cb 0x7f5bf00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:33.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.986+0000 7f5bfda00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f02644f0a0 0x55f02683de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.986+0000 7f5bfe400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f026477f60 0x55f026843410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.986+0000 7f5bfda00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f02644f0a0 0x55f02683de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:50646/0 (socket says 172.21.15.16:50646) 2024-09-21T03:46:33.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.986+0000 7f5bfda00640 1 -- 172.21.15.16:0/1499036458 learned_addr learned my addr 172.21.15.16:0/1499036458 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:33.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.986+0000 7f5bfd000640 1 --2- 172.21.15.16:0/1499036458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026457280 0x55f02683e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.987+0000 7f5bfda00640 1 -- 172.21.15.16:0/1499036458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026457280 msgr2=0x55f02683e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:33.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.987+0000 7f5bfda00640 1 --2- 172.21.15.16:0/1499036458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026457280 0x55f02683e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.987+0000 7f5bfda00640 1 -- 172.21.15.16:0/1499036458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f026477f60 msgr2=0x55f026843410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:33.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.987+0000 7f5bfda00640 1 --2- 172.21.15.16:0/1499036458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f026477f60 0x55f026843410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.987+0000 7f5bfda00640 1 -- 172.21.15.16:0/1499036458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f026843a80 con 0x55f02644f0a0 2024-09-21T03:46:33.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.987+0000 7f5bfd000640 1 --2- 172.21.15.16:0/1499036458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026457280 0x55f02683e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:33.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.987+0000 7f5bfda00640 1 --2- 172.21.15.16:0/1499036458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f02644f0a0 0x55f02683de10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5bec0197d0 tx=0x7f5bec009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d687a5606cd99bf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:33.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.987+0000 7f5bdf400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:46:33.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.987+0000 7f5bdf400640 1 -- 172.21.15.16:0/1499036458 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec01e070 con 0x55f02644f0a0 2024-09-21T03:46:33.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.987+0000 7f5c0707b740 1 -- 172.21.15.16:0/1499036458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bec01f040 con 0x55f02644f0a0 2024-09-21T03:46:33.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.987+0000 7f5bdf400640 1 -- 172.21.15.16:0/1499036458 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f5bec0053d0 con 0x55f02644f0a0 2024-09-21T03:46:33.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.988+0000 7f5bdf400640 1 -- 172.21.15.16:0/1499036458 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5bec04c050 con 0x55f02644f0a0 2024-09-21T03:46:33.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.988+0000 7f5c0707b740 10 client.25511 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:33.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.988+0000 7f5c0707b740 1 -- 172.21.15.16:0/1499036458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f026844e40 con 0x55f02644f0a0 2024-09-21T03:46:33.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.988+0000 7f5bdf400640 1 -- 172.21.15.16:0/1499036458 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bec01f040 con 0x55f02644f0a0 2024-09-21T03:46:33.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.988+0000 7f5bdf400640 1 -- 172.21.15.16:0/1499036458 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5bec047030 con 0x55f02644f0a0 2024-09-21T03:46:33.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.988+0000 7f5c0707b740 10 client.25511 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:33.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.988+0000 7f5c0707b740 10 client.25511 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:46:33.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.988+0000 7f5c0707b740 20 client.25511 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:33.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.988+0000 7f5c0707b740 1 --2- 172.21.15.16:0/1499036458 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x55f02640bc90 0x55f0268aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:33.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.988+0000 7f5c0707b740 4 client.25511 mds_command: new command op to 24670 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:46:33.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.989+0000 7f5c0707b740 1 -- 172.21.15.16:0/1499036458 --> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] -- command(tid 0: {"prefix": "status"}) -- 0x55f026793750 con 0x55f02640bc90 2024-09-21T03:46:33.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.989+0000 7f5bfd000640 1 --2- 172.21.15.16:0/1499036458 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x55f02640bc90 0x55f0268aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:33.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.989+0000 7f5bfd000640 1 --2- 172.21.15.16:0/1499036458 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x55f02640bc90 0x55f0268aa760 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9707be249e8c0a83 server_cookie=9d59e8177dc38a3d in_seq=0 out_seq=0 2024-09-21T03:46:33.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.989+0000 7f5bdf400640 10 client.25511 ms_handle_connect on v2:172.21.15.134:6836/2624530453 2024-09-21T03:46:33.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.990+0000 7f5bdf400640 1 -- 172.21.15.16:0/1499036458 <== mds.3 v2:172.21.15.134:6836/2624530453 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f026793750 con 0x55f02640bc90 2024-09-21T03:46:33.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.990+0000 7f5bdf400640 10 client.25511 handle_command_reply: tid=0 2024-09-21T03:46:33.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.990+0000 7f5c0707b740 1 client.25511 shutdown 2024-09-21T03:46:33.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.990+0000 7f5c0707b740 1 -- 172.21.15.16:0/1499036458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f02644f0a0 msgr2=0x55f02683de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:33.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.990+0000 7f5c0707b740 1 --2- 172.21.15.16:0/1499036458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f02644f0a0 0x55f02683de10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5bec0197d0 tx=0x7f5bec009fd0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.990+0000 7f5c0707b740 1 -- 172.21.15.16:0/1499036458 shutdown_connections 2024-09-21T03:46:33.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.990+0000 7f5c0707b740 1 --2- 172.21.15.16:0/1499036458 >> [v2:172.21.15.134:6836/2624530453,v1:172.21.15.134:6837/2624530453] conn(0x55f02640bc90 0x55f0268aa760 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.990+0000 7f5c0707b740 1 --2- 172.21.15.16:0/1499036458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f026477f60 0x55f026843410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.990+0000 7f5c0707b740 1 --2- 172.21.15.16:0/1499036458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026457280 0x55f02683e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.990+0000 7f5c0707b740 1 --2- 172.21.15.16:0/1499036458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f02644f0a0 0x55f02683de10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:33.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.990+0000 7f5bdf400640 0 client.25511 ms_handle_reset on v2:172.21.15.134:6836/2624530453 2024-09-21T03:46:33.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.990+0000 7f5c0707b740 1 -- 172.21.15.16:0/1499036458 >> 172.21.15.16:0/1499036458 conn(0x55f02646c350 msgr2=0x55f026825180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:33.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.990+0000 7f5c0707b740 1 -- 172.21.15.16:0/1499036458 shutdown_connections 2024-09-21T03:46:33.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.990+0000 7f5c0707b740 1 -- 172.21.15.16:0/1499036458 wait complete. 2024-09-21T03:46:33.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.990+0000 7f5c0707b740 20 client.25511 trim_cache size 0 max 16384 2024-09-21T03:46:33.997 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:46:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:46:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 3, 2024-09-21T03:46:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24670, 2024-09-21T03:46:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T03:46:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T03:46:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T03:46:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 149.46790500899999, 2024-09-21T03:46:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 30, 2024-09-21T03:46:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 86, 2024-09-21T03:46:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 86, 2024-09-21T03:46:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 236.635307165 2024-09-21T03:46:33.998 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:46:33.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.991+0000 7f5bfee00640 1 -- 172.21.15.16:0/3055508057 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5be00b6510 msgr2=0x7f5be00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:33.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.991+0000 7f5bfee00640 1 --2- 172.21.15.16:0/3055508057 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5be00b6510 0x7f5be00b8a20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5be402aa00 tx=0x7f5be4005780 comp rx=0 tx=0).stop 2024-09-21T03:46:33.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.991+0000 7f5bfee00640 1 -- 172.21.15.16:0/3055508057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bf8109c40 msgr2=0x7f5bf8076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:33.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.991+0000 7f5bfee00640 1 --2- 172.21.15.16:0/3055508057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bf8109c40 0x7f5bf8076190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009fd0 tx=0x7f5bf000ef10 comp rx=0 tx=0).stop 2024-09-21T03:46:33.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.991+0000 7f5bfee00640 1 -- 172.21.15.16:0/3055508057 shutdown_connections 2024-09-21T03:46:33.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.991+0000 7f5bfee00640 1 --2- 172.21.15.16:0/3055508057 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5be00b6510 0x7f5be00b8a20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:34.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.991+0000 7f5bfee00640 1 --2- 172.21.15.16:0/3055508057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bf8109c40 0x7f5bf8076190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:34.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.991+0000 7f5bfee00640 1 --2- 172.21.15.16:0/3055508057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bf81092f0 0x7f5bf8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:34.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.991+0000 7f5bfee00640 1 --2- 172.21.15.16:0/3055508057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bf81039b0 0x7f5bf8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:34.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.991+0000 7f5bfee00640 1 -- 172.21.15.16:0/3055508057 >> 172.21.15.16:0/3055508057 conn(0x7f5bf80ff400 msgr2=0x7f5bf8101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:34.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.991+0000 7f5bfee00640 1 -- 172.21.15.16:0/3055508057 shutdown_connections 2024-09-21T03:46:34.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:33.991+0000 7f5bfee00640 1 -- 172.21.15.16:0/3055508057 wait complete. 2024-09-21T03:46:34.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:34 smithi016 ceph-mon[30375]: pgmap v426: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-21T03:46:34.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T03:46:34.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:34 smithi123 ceph-mon[34673]: pgmap v426: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-21T03:46:35.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:34 smithi134 ceph-mon[35031]: pgmap v426: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-21T03:46:35.282 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:35.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.914+0000 7fcbe7400640 1 -- 172.21.15.16:0/1208728480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe0105d80 msgr2=0x7fcbe0106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:35.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.914+0000 7fcbe7400640 1 --2- 172.21.15.16:0/1208728480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe0105d80 0x7fcbe0106190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009900 tx=0x7fcbd402a410 comp rx=0 tx=0).stop 2024-09-21T03:46:35.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.914+0000 7fcbe7400640 1 -- 172.21.15.16:0/1208728480 shutdown_connections 2024-09-21T03:46:35.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.914+0000 7fcbe7400640 1 --2- 172.21.15.16:0/1208728480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbe010c010 0x7fcbe0112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.914+0000 7fcbe7400640 1 --2- 172.21.15.16:0/1208728480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbe010b6c0 0x7fcbe010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.914+0000 7fcbe7400640 1 --2- 172.21.15.16:0/1208728480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe0105d80 0x7fcbe0106190 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.914+0000 7fcbe7400640 1 -- 172.21.15.16:0/1208728480 >> 172.21.15.16:0/1208728480 conn(0x7fcbe0101810 msgr2=0x7fcbe0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.914+0000 7fcbe7400640 1 -- 172.21.15.16:0/1208728480 shutdown_connections 2024-09-21T03:46:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.914+0000 7fcbe7400640 1 -- 172.21.15.16:0/1208728480 wait complete. 2024-09-21T03:46:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.915+0000 7fcbe7400640 1 Processor -- start 2024-09-21T03:46:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.915+0000 7fcbe7400640 1 -- start start 2024-09-21T03:46:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.916+0000 7fcbe7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbe0105d80 0x7fcbe019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.916+0000 7fcbe7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe010b6c0 0x7fcbe019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.916+0000 7fcbe7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbe010c010 0x7fcbe01a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.916+0000 7fcbe7400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0115d20 con 0x7fcbe010b6c0 2024-09-21T03:46:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.916+0000 7fcbe7400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0115ba0 con 0x7fcbe0105d80 2024-09-21T03:46:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.916+0000 7fcbe7400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0115ea0 con 0x7fcbe010c010 2024-09-21T03:46:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.916+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe010b6c0 0x7fcbe019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.916+0000 7fcbe4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbe0105d80 0x7fcbe019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.916+0000 7fcbe4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbe0105d80 0x7fcbe019c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:50664/0 (socket says 172.21.15.16:50664) 2024-09-21T03:46:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.916+0000 7fcbe5400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbe010c010 0x7fcbe01a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.916+0000 7fcbe4a00640 1 -- 172.21.15.16:0/2239999055 learned_addr learned my addr 172.21.15.16:0/2239999055 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.917+0000 7fcbdfe00640 1 -- 172.21.15.16:0/2239999055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbe010c010 msgr2=0x7fcbe01a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.917+0000 7fcbdfe00640 1 --2- 172.21.15.16:0/2239999055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbe010c010 0x7fcbe01a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.917+0000 7fcbdfe00640 1 -- 172.21.15.16:0/2239999055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbe0105d80 msgr2=0x7fcbe019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.917+0000 7fcbdfe00640 1 --2- 172.21.15.16:0/2239999055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbe0105d80 0x7fcbe019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.917+0000 7fcbdfe00640 1 -- 172.21.15.16:0/2239999055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe01a2640 con 0x7fcbe010b6c0 2024-09-21T03:46:35.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.917+0000 7fcbdfe00640 1 --2- 172.21.15.16:0/2239999055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe010b6c0 0x7fcbe019cc70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcbe006caa0 tx=0x7fcbcc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:35.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.917+0000 7fcbdd600640 1 -- 172.21.15.16:0/2239999055 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc00cc40 con 0x7fcbe010b6c0 2024-09-21T03:46:35.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.917+0000 7fcbdd600640 1 -- 172.21.15.16:0/2239999055 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbcc00cde0 con 0x7fcbe010b6c0 2024-09-21T03:46:35.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.917+0000 7fcbe7400640 1 -- 172.21.15.16:0/2239999055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe01a2930 con 0x7fcbe010b6c0 2024-09-21T03:46:35.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.917+0000 7fcbdd600640 1 -- 172.21.15.16:0/2239999055 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc018780 con 0x7fcbe010b6c0 2024-09-21T03:46:35.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.917+0000 7fcbe7400640 1 -- 172.21.15.16:0/2239999055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe01a9090 con 0x7fcbe010b6c0 2024-09-21T03:46:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.921+0000 7fcbee039740 1 Processor -- start 2024-09-21T03:46:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.921+0000 7fcbee039740 1 -- start start 2024-09-21T03:46:35.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.921+0000 7fcbdd600640 1 -- 172.21.15.16:0/2239999055 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fcbcc010a90 con 0x7fcbe010b6c0 2024-09-21T03:46:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.921+0000 7fcbee039740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ceacf60 0x556a7d253490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.921+0000 7fcbee039740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 0x556a7d2539d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.921+0000 7fcbee039740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ce8c280 0x556a7d253f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.921+0000 7fcbee039740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a7d1cad50 con 0x556a7ceacf60 2024-09-21T03:46:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.921+0000 7fcbee039740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556a7d1ba180 con 0x556a7ce840a0 2024-09-21T03:46:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.921+0000 7fcbee039740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556a7d1bdf90 con 0x556a7ce8c280 2024-09-21T03:46:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.922+0000 7fcbe5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ceacf60 0x556a7d253490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.922+0000 7fcbe4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 0x556a7d2539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.922+0000 7fcbe4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 0x556a7d2539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:50680/0 (socket says 172.21.15.16:50680) 2024-09-21T03:46:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.922+0000 7fcbe4a00640 1 -- 172.21.15.16:0/1229247370 learned_addr learned my addr 172.21.15.16:0/1229247370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.922+0000 7fcbdfe00640 1 --2- 172.21.15.16:0/1229247370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ce8c280 0x556a7d253f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.922+0000 7fcbe5400640 1 -- 172.21.15.16:0/1229247370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ce8c280 msgr2=0x556a7d253f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.922+0000 7fcbe5400640 1 --2- 172.21.15.16:0/1229247370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ce8c280 0x556a7d253f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.922+0000 7fcbe5400640 1 -- 172.21.15.16:0/1229247370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 msgr2=0x556a7d2539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.922+0000 7fcbe5400640 1 --2- 172.21.15.16:0/1229247370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 0x556a7d2539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.922+0000 7fcbe5400640 1 -- 172.21.15.16:0/1229247370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a7d184a70 con 0x556a7ceacf60 2024-09-21T03:46:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.923+0000 7fcbe5400640 1 --2- 172.21.15.16:0/1229247370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ceacf60 0x556a7d253490 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8009f00 tx=0x7fcbd8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae1386a64fc2ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.923+0000 7fcbd1c00640 1 -- 172.21.15.16:0/1229247370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8004d20 con 0x556a7ceacf60 2024-09-21T03:46:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.923+0000 7fcbd1c00640 1 -- 172.21.15.16:0/1229247370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbd8004030 con 0x556a7ceacf60 2024-09-21T03:46:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.923+0000 7fcbd1c00640 1 -- 172.21.15.16:0/1229247370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8005260 con 0x556a7ceacf60 2024-09-21T03:46:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.924+0000 7fcbee039740 1 -- 172.21.15.16:0/1229247370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ceacf60 msgr2=0x556a7d253490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.924+0000 7fcbee039740 1 --2- 172.21.15.16:0/1229247370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ceacf60 0x556a7d253490 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8009f00 tx=0x7fcbd8011810 comp rx=0 tx=0).stop 2024-09-21T03:46:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.924+0000 7fcbdd600640 1 --2- 172.21.15.16:0/2239999055 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcbc80b6510 0x7fcbc80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.924+0000 7fcbee039740 1 -- 172.21.15.16:0/1229247370 shutdown_connections 2024-09-21T03:46:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.924+0000 7fcbee039740 1 --2- 172.21.15.16:0/1229247370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ce8c280 0x556a7d253f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.924+0000 7fcbee039740 1 --2- 172.21.15.16:0/1229247370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 0x556a7d2539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.924+0000 7fcbee039740 1 --2- 172.21.15.16:0/1229247370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ceacf60 0x556a7d253490 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.924+0000 7fcbdd600640 1 -- 172.21.15.16:0/2239999055 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fcbcc0e7950 con 0x7fcbe010b6c0 2024-09-21T03:46:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.924+0000 7fcbee039740 1 -- 172.21.15.16:0/1229247370 >> 172.21.15.16:0/1229247370 conn(0x556a7cea1350 msgr2=0x556a7d251380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.924+0000 7fcbe5400640 1 --2- 172.21.15.16:0/2239999055 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcbc80b6510 0x7fcbc80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.925+0000 7fcbee039740 1 -- 172.21.15.16:0/1229247370 shutdown_connections 2024-09-21T03:46:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.925+0000 7fcbee039740 1 -- 172.21.15.16:0/1229247370 wait complete. 2024-09-21T03:46:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.925+0000 7fcbee039740 1 -- start start 2024-09-21T03:46:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.925+0000 7fcbe5400640 1 --2- 172.21.15.16:0/2239999055 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcbc80b6510 0x7fcbc80b8a20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8011d10 tx=0x7fcbd8004910 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.926+0000 7fcbee039740 1 Processor -- start 2024-09-21T03:46:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.926+0000 7fcbee039740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 0x556a7d25f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.926+0000 7fcbee039740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 0x556a7d25f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.926+0000 7fcbee039740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ceacf60 0x556a7d26af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.926+0000 7fcbee039740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a7d01b5f0 con 0x556a7ce8c280 2024-09-21T03:46:35.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.926+0000 7fcbee039740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556a7d155460 con 0x556a7ce840a0 2024-09-21T03:46:35.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.926+0000 7fcbee039740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556a7d261c70 con 0x556a7ceacf60 2024-09-21T03:46:35.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.926+0000 7fcbee039740 10 client.0 _ll_register_callbacks cb 0x556a7d165210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:35.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.926+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 0x556a7d25f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.926+0000 7fcbe4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 0x556a7d25f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.926+0000 7fcbe4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 0x556a7d25f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:50686/0 (socket says 172.21.15.16:50686) 2024-09-21T03:46:35.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.927+0000 7fcbe4a00640 1 -- 172.21.15.16:0/3243116922 learned_addr learned my addr 172.21.15.16:0/3243116922 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:35.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.927+0000 7fcbe5400640 1 --2- 172.21.15.16:0/3243116922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ceacf60 0x556a7d26af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.927+0000 7fcbe4a00640 1 -- 172.21.15.16:0/3243116922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ceacf60 msgr2=0x556a7d26af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.927+0000 7fcbe4a00640 1 --2- 172.21.15.16:0/3243116922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ceacf60 0x556a7d26af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.927+0000 7fcbe4a00640 1 -- 172.21.15.16:0/3243116922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 msgr2=0x556a7d25f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.927+0000 7fcbe4a00640 1 --2- 172.21.15.16:0/3243116922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 0x556a7d25f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.927+0000 7fcbe4a00640 1 -- 172.21.15.16:0/3243116922 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a7cf18540 con 0x556a7ce840a0 2024-09-21T03:46:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.927+0000 7fcbdfe00640 1 --2- 172.21.15.16:0/3243116922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 0x556a7d25f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.927+0000 7fcbe4a00640 1 --2- 172.21.15.16:0/3243116922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 0x556a7d25f000 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4000c00 tx=0x7fcbd40043f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1796f0b30f02189 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.927+0000 7fcbc6000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:46:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.928+0000 7fcbc6000640 1 -- 172.21.15.16:0/3243116922 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4035040 con 0x556a7ce840a0 2024-09-21T03:46:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.928+0000 7fcbee039740 1 -- 172.21.15.16:0/3243116922 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556a7d157a70 con 0x556a7ce840a0 2024-09-21T03:46:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.928+0000 7fcbc6000640 1 -- 172.21.15.16:0/3243116922 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fcbd4003ac0 con 0x556a7ce840a0 2024-09-21T03:46:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.928+0000 7fcbc6000640 1 -- 172.21.15.16:0/3243116922 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcbd4003cc0 con 0x556a7ce840a0 2024-09-21T03:46:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.928+0000 7fcbee039740 10 client.25523 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.928+0000 7fcbee039740 1 -- 172.21.15.16:0/3243116922 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556a7d1726f0 con 0x556a7ce840a0 2024-09-21T03:46:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.928+0000 7fcbc6000640 1 -- 172.21.15.16:0/3243116922 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556a7d157a70 con 0x556a7ce840a0 2024-09-21T03:46:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.929+0000 7fcbc6000640 1 -- 172.21.15.16:0/3243116922 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcbd403c440 con 0x556a7ce840a0 2024-09-21T03:46:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.929+0000 7fcbee039740 10 client.25523 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.929+0000 7fcbee039740 10 client.25523 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:46:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.929+0000 7fcbee039740 20 client.25523 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.929+0000 7fcbee039740 1 --2- 172.21.15.16:0/3243116922 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x556a7ce6b8e0 0x556a7d2714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.929+0000 7fcbee039740 4 client.25523 mds_command: new command op to 24682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.929+0000 7fcbee039740 1 -- 172.21.15.16:0/3243116922 --> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a7d1ccca0 con 0x556a7ce6b8e0 2024-09-21T03:46:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.929+0000 7fcbdfe00640 1 --2- 172.21.15.16:0/3243116922 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x556a7ce6b8e0 0x556a7d2714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.930+0000 7fcbdfe00640 1 --2- 172.21.15.16:0/3243116922 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x556a7ce6b8e0 0x556a7d2714b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7af093b2a55ede6 server_cookie=88e81f9e5b4955a5 in_seq=0 out_seq=0 2024-09-21T03:46:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.930+0000 7fcbc6000640 10 client.25523 ms_handle_connect on v2:172.21.15.134:6838/606193011 2024-09-21T03:46:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.930+0000 7fcbc6000640 1 -- 172.21.15.16:0/3243116922 <== mds.2 v2:172.21.15.134:6838/606193011 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556a7d1ccca0 con 0x556a7ce6b8e0 2024-09-21T03:46:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.930+0000 7fcbc6000640 10 client.25523 handle_command_reply: tid=0 2024-09-21T03:46:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.930+0000 7fcbee039740 1 client.25523 shutdown 2024-09-21T03:46:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.931+0000 7fcbee039740 1 -- 172.21.15.16:0/3243116922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 msgr2=0x556a7d25f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.931+0000 7fcbee039740 1 --2- 172.21.15.16:0/3243116922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 0x556a7d25f000 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4000c00 tx=0x7fcbd40043f0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.931+0000 7fcbee039740 1 -- 172.21.15.16:0/3243116922 shutdown_connections 2024-09-21T03:46:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.931+0000 7fcbee039740 1 --2- 172.21.15.16:0/3243116922 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x556a7ce6b8e0 0x556a7d2714b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.931+0000 7fcbee039740 1 --2- 172.21.15.16:0/3243116922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ceacf60 0x556a7d26af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.931+0000 7fcbee039740 1 --2- 172.21.15.16:0/3243116922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 0x556a7d25f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.931+0000 7fcbee039740 1 --2- 172.21.15.16:0/3243116922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 0x556a7d25f000 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.931+0000 7fcbc6000640 0 client.25523 ms_handle_reset on v2:172.21.15.134:6838/606193011 2024-09-21T03:46:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.931+0000 7fcbee039740 1 -- 172.21.15.16:0/3243116922 >> 172.21.15.16:0/3243116922 conn(0x556a7cea1350 msgr2=0x556a7d256670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.931+0000 7fcbee039740 1 -- 172.21.15.16:0/3243116922 shutdown_connections 2024-09-21T03:46:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.931+0000 7fcbee039740 1 -- 172.21.15.16:0/3243116922 wait complete. 2024-09-21T03:46:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.931+0000 7fcbee039740 20 client.25523 trim_cache size 0 max 16384 2024-09-21T03:46:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.950+0000 7fcbee039740 1 Processor -- start 2024-09-21T03:46:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.950+0000 7fcbee039740 1 -- start start 2024-09-21T03:46:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.950+0000 7fcbee039740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 0x556a7d2714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.950+0000 7fcbee039740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 0x556a7d24c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.950+0000 7fcbee039740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ceacf60 0x556a7d26aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.950+0000 7fcbee039740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a7d26f7f0 con 0x556a7ce8c280 2024-09-21T03:46:35.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.950+0000 7fcbee039740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556a7d26fec0 con 0x556a7ce840a0 2024-09-21T03:46:35.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.950+0000 7fcbee039740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556a7d270590 con 0x556a7ceacf60 2024-09-21T03:46:35.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.951+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 0x556a7d24c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.951+0000 7fcbe4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 0x556a7d2714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.951+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 0x556a7d24c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41810/0 (socket says 172.21.15.16:41810) 2024-09-21T03:46:35.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.951+0000 7fcbdfe00640 1 -- 172.21.15.16:0/3739956931 learned_addr learned my addr 172.21.15.16:0/3739956931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:35.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.951+0000 7fcbe5400640 1 --2- 172.21.15.16:0/3739956931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ceacf60 0x556a7d26aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.951+0000 7fcbe4a00640 1 -- 172.21.15.16:0/3739956931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ceacf60 msgr2=0x556a7d26aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:35.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.951+0000 7fcbe4a00640 1 --2- 172.21.15.16:0/3739956931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ceacf60 0x556a7d26aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.951+0000 7fcbe4a00640 1 -- 172.21.15.16:0/3739956931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 msgr2=0x556a7d24c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:35.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.951+0000 7fcbe4a00640 1 --2- 172.21.15.16:0/3739956931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 0x556a7d24c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.951+0000 7fcbe4a00640 1 -- 172.21.15.16:0/3739956931 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a7d25a990 con 0x556a7ce840a0 2024-09-21T03:46:35.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.952+0000 7fcbe4a00640 1 --2- 172.21.15.16:0/3739956931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 0x556a7d2714b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcbd403c860 tx=0x7fcbd402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97b02c827fb60c5c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:35.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.952+0000 7fcbd1200640 1 -- 172.21.15.16:0/3739956931 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4035030 con 0x556a7ce840a0 2024-09-21T03:46:35.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.952+0000 7fcbd1200640 1 -- 172.21.15.16:0/3739956931 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbd4003e60 con 0x556a7ce840a0 2024-09-21T03:46:35.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.952+0000 7fcbd1200640 1 -- 172.21.15.16:0/3739956931 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd40038c0 con 0x556a7ce840a0 2024-09-21T03:46:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.953+0000 7fcbee039740 1 -- 172.21.15.16:0/3739956931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 msgr2=0x556a7d2714b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.953+0000 7fcbee039740 1 --2- 172.21.15.16:0/3739956931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 0x556a7d2714b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcbd403c860 tx=0x7fcbd402a410 comp rx=0 tx=0).stop 2024-09-21T03:46:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.953+0000 7fcbee039740 1 -- 172.21.15.16:0/3739956931 shutdown_connections 2024-09-21T03:46:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.953+0000 7fcbee039740 1 --2- 172.21.15.16:0/3739956931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ceacf60 0x556a7d26aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.953+0000 7fcbee039740 1 --2- 172.21.15.16:0/3739956931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 0x556a7d24c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.953+0000 7fcbee039740 1 --2- 172.21.15.16:0/3739956931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ce840a0 0x556a7d2714b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.953+0000 7fcbee039740 1 -- 172.21.15.16:0/3739956931 >> 172.21.15.16:0/3739956931 conn(0x556a7cea1350 msgr2=0x556a7d25efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:35.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.953+0000 7fcbee039740 1 -- 172.21.15.16:0/3739956931 shutdown_connections 2024-09-21T03:46:35.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.953+0000 7fcbee039740 1 -- 172.21.15.16:0/3739956931 wait complete. 2024-09-21T03:46:35.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.953+0000 7fcbee039740 1 -- start start 2024-09-21T03:46:35.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.954+0000 7fcbee039740 1 Processor -- start 2024-09-21T03:46:35.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.954+0000 7fcbee039740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ce840a0 0x556a7d279f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.954+0000 7fcbee039740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 0x556a7d273410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.954+0000 7fcbee039740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ceacf60 0x556a7d273970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.954+0000 7fcbee039740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a7d271330 con 0x556a7ce8c280 2024-09-21T03:46:35.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.954+0000 7fcbee039740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556a7d270c60 con 0x556a7ceacf60 2024-09-21T03:46:35.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.954+0000 7fcbee039740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556a7d271a00 con 0x556a7ce840a0 2024-09-21T03:46:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.954+0000 7fcbee039740 10 client.0 _ll_register_callbacks cb 0x7fcbd4004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.954+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 0x556a7d273410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.955+0000 7fcbe4a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ce840a0 0x556a7d279f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.955+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 0x556a7d273410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41816/0 (socket says 172.21.15.16:41816) 2024-09-21T03:46:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.955+0000 7fcbe4a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ce840a0 0x556a7d279f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52126/0 (socket says 172.21.15.16:52126) 2024-09-21T03:46:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.955+0000 7fcbdfe00640 1 -- 172.21.15.16:0/1142770783 learned_addr learned my addr 172.21.15.16:0/1142770783 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.955+0000 7fcbe5400640 1 --2- 172.21.15.16:0/1142770783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ceacf60 0x556a7d273970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.955+0000 7fcbe4a00640 1 -- 172.21.15.16:0/1142770783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ceacf60 msgr2=0x556a7d273970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.955+0000 7fcbe4a00640 1 --2- 172.21.15.16:0/1142770783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ceacf60 0x556a7d273970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.955+0000 7fcbe4a00640 1 -- 172.21.15.16:0/1142770783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 msgr2=0x556a7d273410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.955+0000 7fcbe4a00640 1 --2- 172.21.15.16:0/1142770783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 0x556a7d273410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.955+0000 7fcbe4a00640 1 -- 172.21.15.16:0/1142770783 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a7d274100 con 0x556a7ce840a0 2024-09-21T03:46:35.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.955+0000 7fcbdfe00640 1 --2- 172.21.15.16:0/1142770783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 0x556a7d273410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:35.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.955+0000 7fcbe4a00640 1 --2- 172.21.15.16:0/1142770783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ce840a0 0x556a7d279f60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4005db0 tx=0x7fcbd403c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9be78df00e9342d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:35.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.956+0000 7fcbc6000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:46:35.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.956+0000 7fcbc6000640 1 -- 172.21.15.16:0/1142770783 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4035040 con 0x556a7ce840a0 2024-09-21T03:46:35.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.956+0000 7fcbee039740 1 -- 172.21.15.16:0/1142770783 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbd4003e60 con 0x556a7ce840a0 2024-09-21T03:46:35.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.956+0000 7fcbc6000640 1 -- 172.21.15.16:0/1142770783 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fcbd40316a0 con 0x556a7ce840a0 2024-09-21T03:46:35.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.956+0000 7fcbc6000640 1 -- 172.21.15.16:0/1142770783 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcbd4003ad0 con 0x556a7ce840a0 2024-09-21T03:46:35.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.956+0000 7fcbee039740 10 client.25513 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:35.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.956+0000 7fcbee039740 1 -- 172.21.15.16:0/1142770783 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556a7d2dc980 con 0x556a7ce840a0 2024-09-21T03:46:35.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.957+0000 7fcbc6000640 1 -- 172.21.15.16:0/1142770783 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbd4003e60 con 0x556a7ce840a0 2024-09-21T03:46:35.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.957+0000 7fcbc6000640 1 -- 172.21.15.16:0/1142770783 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcbd4052890 con 0x556a7ce840a0 2024-09-21T03:46:35.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.957+0000 7fcbee039740 10 client.25513 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:35.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.957+0000 7fcbee039740 10 client.25513 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:46:35.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.957+0000 7fcbee039740 20 client.25513 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:35.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.957+0000 7fcbee039740 1 --2- 172.21.15.16:0/1142770783 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x556a7ce40c90 0x556a7d2dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:35.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.957+0000 7fcbee039740 4 client.25513 mds_command: new command op to 24682 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:46:35.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.957+0000 7fcbee039740 1 -- 172.21.15.16:0/1142770783 --> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] -- command(tid 0: {"prefix": "status"}) -- 0x556a7d1c8750 con 0x556a7ce40c90 2024-09-21T03:46:35.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.957+0000 7fcbdfe00640 1 --2- 172.21.15.16:0/1142770783 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x556a7ce40c90 0x556a7d2dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:35.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.958+0000 7fcbdfe00640 1 --2- 172.21.15.16:0/1142770783 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x556a7ce40c90 0x556a7d2dfd30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18fed17e1edca55c server_cookie=55e85e59bd74b2bc in_seq=0 out_seq=0 2024-09-21T03:46:35.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.958+0000 7fcbc6000640 10 client.25513 ms_handle_connect on v2:172.21.15.134:6838/606193011 2024-09-21T03:46:35.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.958+0000 7fcbc6000640 1 -- 172.21.15.16:0/1142770783 <== mds.2 v2:172.21.15.134:6838/606193011 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556a7d1c8750 con 0x556a7ce40c90 2024-09-21T03:46:35.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.958+0000 7fcbc6000640 10 client.25513 handle_command_reply: tid=0 2024-09-21T03:46:35.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.958+0000 7fcbee039740 1 client.25513 shutdown 2024-09-21T03:46:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.958+0000 7fcbee039740 1 -- 172.21.15.16:0/1142770783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ce840a0 msgr2=0x556a7d279f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.958+0000 7fcbee039740 1 --2- 172.21.15.16:0/1142770783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ce840a0 0x556a7d279f60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4005db0 tx=0x7fcbd403c720 comp rx=0 tx=0).stop 2024-09-21T03:46:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.958+0000 7fcbee039740 1 -- 172.21.15.16:0/1142770783 shutdown_connections 2024-09-21T03:46:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.958+0000 7fcbee039740 1 --2- 172.21.15.16:0/1142770783 >> [v2:172.21.15.134:6838/606193011,v1:172.21.15.134:6839/606193011] conn(0x556a7ce40c90 0x556a7d2dfd30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.958+0000 7fcbee039740 1 --2- 172.21.15.16:0/1142770783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a7ceacf60 0x556a7d273970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.958+0000 7fcbee039740 1 --2- 172.21.15.16:0/1142770783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a7ce8c280 0x556a7d273410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.958+0000 7fcbc6000640 0 client.25513 ms_handle_reset on v2:172.21.15.134:6838/606193011 2024-09-21T03:46:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.958+0000 7fcbee039740 1 --2- 172.21.15.16:0/1142770783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7ce840a0 0x556a7d279f60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.958+0000 7fcbee039740 1 -- 172.21.15.16:0/1142770783 >> 172.21.15.16:0/1142770783 conn(0x556a7cea1350 msgr2=0x556a7d2544c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.959+0000 7fcbee039740 1 -- 172.21.15.16:0/1142770783 shutdown_connections 2024-09-21T03:46:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.959+0000 7fcbee039740 1 -- 172.21.15.16:0/1142770783 wait complete. 2024-09-21T03:46:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.959+0000 7fcbee039740 20 client.25513 trim_cache size 0 max 16384 2024-09-21T03:46:35.966 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:46:35.966 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:46:35.966 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-09-21T03:46:35.966 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24682, 2024-09-21T03:46:35.966 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T03:46:35.966 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T03:46:35.966 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T03:46:35.966 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 152.441949548, 2024-09-21T03:46:35.966 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 30, 2024-09-21T03:46:35.966 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 86, 2024-09-21T03:46:35.966 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 86, 2024-09-21T03:46:35.966 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 233.35520812300001 2024-09-21T03:46:35.966 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:46:35.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.960+0000 7fcbe7400640 1 -- 172.21.15.16:0/2239999055 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcbc80b6510 msgr2=0x7fcbc80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:35.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.960+0000 7fcbe7400640 1 --2- 172.21.15.16:0/2239999055 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcbc80b6510 0x7fcbc80b8a20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8011d10 tx=0x7fcbd8004910 comp rx=0 tx=0).stop 2024-09-21T03:46:35.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.960+0000 7fcbe7400640 1 -- 172.21.15.16:0/2239999055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe010b6c0 msgr2=0x7fcbe019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:35.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.960+0000 7fcbe7400640 1 --2- 172.21.15.16:0/2239999055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe010b6c0 0x7fcbe019cc70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcbe006caa0 tx=0x7fcbcc00ee50 comp rx=0 tx=0).stop 2024-09-21T03:46:35.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.960+0000 7fcbe7400640 1 -- 172.21.15.16:0/2239999055 shutdown_connections 2024-09-21T03:46:35.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.960+0000 7fcbe7400640 1 --2- 172.21.15.16:0/2239999055 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcbc80b6510 0x7fcbc80b8a20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.960+0000 7fcbe7400640 1 --2- 172.21.15.16:0/2239999055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbe010c010 0x7fcbe01a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.960+0000 7fcbe7400640 1 --2- 172.21.15.16:0/2239999055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbe010b6c0 0x7fcbe019cc70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.960+0000 7fcbe7400640 1 --2- 172.21.15.16:0/2239999055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbe0105d80 0x7fcbe019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:35.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.960+0000 7fcbe7400640 1 -- 172.21.15.16:0/2239999055 >> 172.21.15.16:0/2239999055 conn(0x7fcbe0101810 msgr2=0x7fcbe010ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:35.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.960+0000 7fcbe7400640 1 -- 172.21.15.16:0/2239999055 shutdown_connections 2024-09-21T03:46:35.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:35.960+0000 7fcbe7400640 1 -- 172.21.15.16:0/2239999055 wait complete. 2024-09-21T03:46:36.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T03:46:36.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:36 smithi123 ceph-mon[34673]: pgmap v427: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-21T03:46:36.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:36 smithi016 ceph-mon[30375]: pgmap v427: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-21T03:46:37.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:36 smithi134 ceph-mon[35031]: pgmap v427: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-21T03:46:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:37.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.781+0000 7fe37be00640 1 -- 172.21.15.16:0/3151614326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c1092f0 msgr2=0x7fe37c109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:37.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.781+0000 7fe37be00640 1 --2- 172.21.15.16:0/3151614326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c1092f0 0x7fe37c109700 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe368009900 tx=0x7fe36802a410 comp rx=0 tx=0).stop 2024-09-21T03:46:37.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.781+0000 7fe37be00640 1 -- 172.21.15.16:0/3151614326 shutdown_connections 2024-09-21T03:46:37.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.781+0000 7fe37be00640 1 --2- 172.21.15.16:0/3151614326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe37c109c40 0x7fe37c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.781+0000 7fe37be00640 1 --2- 172.21.15.16:0/3151614326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c1092f0 0x7fe37c109700 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.781+0000 7fe37be00640 1 --2- 172.21.15.16:0/3151614326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe37c1039b0 0x7fe37c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.781+0000 7fe37be00640 1 -- 172.21.15.16:0/3151614326 >> 172.21.15.16:0/3151614326 conn(0x7fe37c0ff440 msgr2=0x7fe37c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:37.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.781+0000 7fe37be00640 1 -- 172.21.15.16:0/3151614326 shutdown_connections 2024-09-21T03:46:37.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.781+0000 7fe37be00640 1 -- 172.21.15.16:0/3151614326 wait complete. 2024-09-21T03:46:37.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.782+0000 7fe37be00640 1 Processor -- start 2024-09-21T03:46:37.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.783+0000 7fe37be00640 1 -- start start 2024-09-21T03:46:37.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.783+0000 7fe37be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c1039b0 0x7fe37c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.783+0000 7fe37be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe37c1092f0 0x7fe37c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.783+0000 7fe37be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe37c109c40 0x7fe37c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.783+0000 7fe37be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c113950 con 0x7fe37c1039b0 2024-09-21T03:46:37.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.783+0000 7fe37be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1137d0 con 0x7fe37c1092f0 2024-09-21T03:46:37.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.783+0000 7fe37be00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c113ad0 con 0x7fe37c109c40 2024-09-21T03:46:37.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.783+0000 7fe37aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c1039b0 0x7fe37c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.783+0000 7fe37aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c1039b0 0x7fe37c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41824/0 (socket says 172.21.15.16:41824) 2024-09-21T03:46:37.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.783+0000 7fe37aa00640 1 -- 172.21.15.16:0/544030559 learned_addr learned my addr 172.21.15.16:0/544030559 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:37.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.783+0000 7fe37b400640 1 --2- 172.21.15.16:0/544030559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe37c109c40 0x7fe37c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.783+0000 7fe37a000640 1 --2- 172.21.15.16:0/544030559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe37c1092f0 0x7fe37c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.784+0000 7fe37aa00640 1 -- 172.21.15.16:0/544030559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe37c109c40 msgr2=0x7fe37c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:37.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.784+0000 7fe37aa00640 1 --2- 172.21.15.16:0/544030559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe37c109c40 0x7fe37c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.784+0000 7fe37aa00640 1 -- 172.21.15.16:0/544030559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe37c1092f0 msgr2=0x7fe37c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:37.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.784+0000 7fe37aa00640 1 --2- 172.21.15.16:0/544030559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe37c1092f0 0x7fe37c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.784+0000 7fe37aa00640 1 -- 172.21.15.16:0/544030559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c1a85c0 con 0x7fe37c1039b0 2024-09-21T03:46:37.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.784+0000 7fe37b400640 1 --2- 172.21.15.16:0/544030559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe37c109c40 0x7fe37c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:37.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.784+0000 7fe37aa00640 1 --2- 172.21.15.16:0/544030559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c1039b0 0x7fe37c075710 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00e9f0 tx=0x7fe36c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:37.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.784+0000 7fe373400640 1 -- 172.21.15.16:0/544030559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c00cd80 con 0x7fe37c1039b0 2024-09-21T03:46:37.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.785+0000 7fe373400640 1 -- 172.21.15.16:0/544030559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe36c0054d0 con 0x7fe37c1039b0 2024-09-21T03:46:37.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.785+0000 7fe373400640 1 -- 172.21.15.16:0/544030559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c010690 con 0x7fe37c1039b0 2024-09-21T03:46:37.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.785+0000 7fe37be00640 1 -- 172.21.15.16:0/544030559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c1a88b0 con 0x7fe37c1039b0 2024-09-21T03:46:37.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.785+0000 7fe37be00640 1 -- 172.21.15.16:0/544030559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c1a8d70 con 0x7fe37c1039b0 2024-09-21T03:46:37.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.788+0000 7fe388840740 1 Processor -- start 2024-09-21T03:46:37.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.788+0000 7fe388840740 1 -- start start 2024-09-21T03:46:37.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.789+0000 7fe388840740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5b0f60 0x55d7dd957670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.789+0000 7fe373400640 1 -- 172.21.15.16:0/544030559 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe36c004e90 con 0x7fe37c1039b0 2024-09-21T03:46:37.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.789+0000 7fe388840740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd5880a0 0x55d7dd957bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.789+0000 7fe388840740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd590280 0x55d7dd9580f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.789+0000 7fe388840740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d7dd8ced50 con 0x55d7dd5880a0 2024-09-21T03:46:37.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.789+0000 7fe388840740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d7dd8be180 con 0x55d7dd5b0f60 2024-09-21T03:46:37.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.789+0000 7fe388840740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d7dd8c1f90 con 0x55d7dd590280 2024-09-21T03:46:37.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.789+0000 7fe37b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5b0f60 0x55d7dd957670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.789+0000 7fe37aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd5880a0 0x55d7dd957bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.789+0000 7fe37b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5b0f60 0x55d7dd957670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:50738/0 (socket says 172.21.15.16:50738) 2024-09-21T03:46:37.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.789+0000 7fe37b400640 1 -- 172.21.15.16:0/3886447006 learned_addr learned my addr 172.21.15.16:0/3886447006 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:37.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.789+0000 7fe37a000640 1 --2- 172.21.15.16:0/3886447006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd590280 0x55d7dd9580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.790+0000 7fe37b400640 1 -- 172.21.15.16:0/3886447006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd590280 msgr2=0x55d7dd9580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:37.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.790+0000 7fe37b400640 1 --2- 172.21.15.16:0/3886447006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd590280 0x55d7dd9580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.790+0000 7fe37b400640 1 -- 172.21.15.16:0/3886447006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd5880a0 msgr2=0x55d7dd957bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:37.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.790+0000 7fe37b400640 1 --2- 172.21.15.16:0/3886447006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd5880a0 0x55d7dd957bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.790+0000 7fe37b400640 1 -- 172.21.15.16:0/3886447006 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7dd888a70 con 0x55d7dd5b0f60 2024-09-21T03:46:37.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.790+0000 7fe37b400640 1 --2- 172.21.15.16:0/3886447006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5b0f60 0x55d7dd957670 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe374010750 tx=0x7fe374010c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=167573fe984f1bc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:37.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.790+0000 7fe367e00640 1 -- 172.21.15.16:0/3886447006 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37400ead0 con 0x55d7dd5b0f60 2024-09-21T03:46:37.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.790+0000 7fe367e00640 1 -- 172.21.15.16:0/3886447006 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe374010e60 con 0x55d7dd5b0f60 2024-09-21T03:46:37.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.790+0000 7fe367e00640 1 -- 172.21.15.16:0/3886447006 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3740113e0 con 0x55d7dd5b0f60 2024-09-21T03:46:37.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.791+0000 7fe373400640 1 --2- 172.21.15.16:0/544030559 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe3600b6590 0x7fe3600b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.791+0000 7fe388840740 1 -- 172.21.15.16:0/3886447006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5b0f60 msgr2=0x55d7dd957670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:37.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.791+0000 7fe388840740 1 --2- 172.21.15.16:0/3886447006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5b0f60 0x55d7dd957670 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe374010750 tx=0x7fe374010c10 comp rx=0 tx=0).stop 2024-09-21T03:46:37.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.791+0000 7fe373400640 1 -- 172.21.15.16:0/544030559 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fe36c0e7aa0 con 0x7fe37c1039b0 2024-09-21T03:46:37.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.791+0000 7fe37aa00640 1 --2- 172.21.15.16:0/544030559 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe3600b6590 0x7fe3600b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.791+0000 7fe388840740 1 -- 172.21.15.16:0/3886447006 shutdown_connections 2024-09-21T03:46:37.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.791+0000 7fe388840740 1 --2- 172.21.15.16:0/3886447006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd590280 0x55d7dd9580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.791+0000 7fe388840740 1 --2- 172.21.15.16:0/3886447006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd5880a0 0x55d7dd957bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.792+0000 7fe388840740 1 --2- 172.21.15.16:0/3886447006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5b0f60 0x55d7dd957670 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.792+0000 7fe388840740 1 -- 172.21.15.16:0/3886447006 >> 172.21.15.16:0/3886447006 conn(0x55d7dd5a5350 msgr2=0x55d7dd955560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:37.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.792+0000 7fe388840740 1 -- 172.21.15.16:0/3886447006 shutdown_connections 2024-09-21T03:46:37.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.792+0000 7fe388840740 1 -- 172.21.15.16:0/3886447006 wait complete. 2024-09-21T03:46:37.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.792+0000 7fe37aa00640 1 --2- 172.21.15.16:0/544030559 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe3600b6590 0x7fe3600b8aa0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0ad520 tx=0x7fe36c0a9040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:37.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.792+0000 7fe388840740 1 -- start start 2024-09-21T03:46:37.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.793+0000 7fe388840740 1 Processor -- start 2024-09-21T03:46:37.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.793+0000 7fe388840740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5880a0 0x55d7dd9632b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.793+0000 7fe388840740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd590280 0x55d7dd9637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.793+0000 7fe388840740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd5b0f60 0x55d7dd96f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.793+0000 7fe388840740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d7dd71f5f0 con 0x55d7dd590280 2024-09-21T03:46:37.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.793+0000 7fe388840740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d7dd859460 con 0x55d7dd5880a0 2024-09-21T03:46:37.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.793+0000 7fe388840740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d7dd965e50 con 0x55d7dd5b0f60 2024-09-21T03:46:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.793+0000 7fe388840740 10 client.0 _ll_register_callbacks cb 0x55d7dd869210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.794+0000 7fe37a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd590280 0x55d7dd9637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.794+0000 7fe37b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd5b0f60 0x55d7dd96f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.794+0000 7fe37b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd5b0f60 0x55d7dd96f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52148/0 (socket says 172.21.15.16:52148) 2024-09-21T03:46:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.794+0000 7fe37b400640 1 -- 172.21.15.16:0/156480862 learned_addr learned my addr 172.21.15.16:0/156480862 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.794+0000 7fe37b400640 1 --2- 172.21.15.16:0/156480862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5880a0 0x55d7dd9632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.794+0000 7fe37a000640 1 -- 172.21.15.16:0/156480862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd5b0f60 msgr2=0x55d7dd96f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.794+0000 7fe37a000640 1 --2- 172.21.15.16:0/156480862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd5b0f60 0x55d7dd96f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.794+0000 7fe37a000640 1 -- 172.21.15.16:0/156480862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5880a0 msgr2=0x55d7dd9632b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.794+0000 7fe37a000640 1 --2- 172.21.15.16:0/156480862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5880a0 0x55d7dd9632b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.794+0000 7fe37a000640 1 -- 172.21.15.16:0/156480862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7dd61c540 con 0x55d7dd590280 2024-09-21T03:46:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.795+0000 7fe37a000640 1 --2- 172.21.15.16:0/156480862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd590280 0x55d7dd9637f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe368009230 tx=0x7fe3680042c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f40a1eb1ba3509 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.795+0000 7fe364c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:46:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.795+0000 7fe364c00640 1 -- 172.21.15.16:0/156480862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368035040 con 0x55d7dd590280 2024-09-21T03:46:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.795+0000 7fe364c00640 1 -- 172.21.15.16:0/156480862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fe368005450 con 0x55d7dd590280 2024-09-21T03:46:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.795+0000 7fe388840740 1 -- 172.21.15.16:0/156480862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d7dd85ba70 con 0x55d7dd590280 2024-09-21T03:46:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.795+0000 7fe364c00640 1 -- 172.21.15.16:0/156480862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe368005650 con 0x55d7dd590280 2024-09-21T03:46:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.795+0000 7fe388840740 10 client.15534 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.795+0000 7fe388840740 1 -- 172.21.15.16:0/156480862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d7dd8766f0 con 0x55d7dd590280 2024-09-21T03:46:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.795+0000 7fe364c00640 1 -- 172.21.15.16:0/156480862 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d7dd85ba70 con 0x55d7dd590280 2024-09-21T03:46:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.796+0000 7fe364c00640 1 -- 172.21.15.16:0/156480862 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe368003860 con 0x55d7dd590280 2024-09-21T03:46:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.796+0000 7fe388840740 10 client.15534 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.796+0000 7fe388840740 10 client.15534 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:46:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.796+0000 7fe388840740 20 client.15534 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.796+0000 7fe388840740 1 --2- 172.21.15.16:0/156480862 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x55d7dd56f8e0 0x55d7dd975720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.796+0000 7fe388840740 4 client.15534 mds_command: new command op to 24689 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.796+0000 7fe388840740 1 -- 172.21.15.16:0/156480862 --> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7dd8d0ca0 con 0x55d7dd56f8e0 2024-09-21T03:46:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.797+0000 7fe37b400640 1 --2- 172.21.15.16:0/156480862 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x55d7dd56f8e0 0x55d7dd975720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.797+0000 7fe37b400640 1 --2- 172.21.15.16:0/156480862 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x55d7dd56f8e0 0x55d7dd975720 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8557c1ecf39ca9d4 server_cookie=e1d1a4448dac4312 in_seq=0 out_seq=0 2024-09-21T03:46:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.797+0000 7fe364c00640 10 client.15534 ms_handle_connect on v2:172.21.15.123:6836/2001424876 2024-09-21T03:46:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.798+0000 7fe364c00640 1 -- 172.21.15.16:0/156480862 <== mds.1 v2:172.21.15.123:6836/2001424876 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d7dd8d0ca0 con 0x55d7dd56f8e0 2024-09-21T03:46:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.798+0000 7fe364c00640 10 client.15534 handle_command_reply: tid=0 2024-09-21T03:46:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.798+0000 7fe388840740 1 client.15534 shutdown 2024-09-21T03:46:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.798+0000 7fe388840740 1 -- 172.21.15.16:0/156480862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd590280 msgr2=0x55d7dd9637f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.798+0000 7fe388840740 1 --2- 172.21.15.16:0/156480862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd590280 0x55d7dd9637f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe368009230 tx=0x7fe3680042c0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.798+0000 7fe388840740 1 -- 172.21.15.16:0/156480862 shutdown_connections 2024-09-21T03:46:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.798+0000 7fe388840740 1 --2- 172.21.15.16:0/156480862 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x55d7dd56f8e0 0x55d7dd975720 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.799+0000 7fe388840740 1 --2- 172.21.15.16:0/156480862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd5b0f60 0x55d7dd96f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.799+0000 7fe388840740 1 --2- 172.21.15.16:0/156480862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd590280 0x55d7dd9637f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.799+0000 7fe388840740 1 --2- 172.21.15.16:0/156480862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5880a0 0x55d7dd9632b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.799+0000 7fe388840740 1 -- 172.21.15.16:0/156480862 >> 172.21.15.16:0/156480862 conn(0x55d7dd5a5350 msgr2=0x55d7dd95a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.799+0000 7fe364c00640 0 client.15534 ms_handle_reset on v2:172.21.15.123:6836/2001424876 2024-09-21T03:46:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.799+0000 7fe388840740 1 -- 172.21.15.16:0/156480862 shutdown_connections 2024-09-21T03:46:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.799+0000 7fe388840740 1 -- 172.21.15.16:0/156480862 wait complete. 2024-09-21T03:46:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.799+0000 7fe388840740 20 client.15534 trim_cache size 0 max 16384 2024-09-21T03:46:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.819+0000 7fe388840740 1 Processor -- start 2024-09-21T03:46:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.819+0000 7fe388840740 1 -- start start 2024-09-21T03:46:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.819+0000 7fe388840740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5880a0 0x55d7dd963290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.819+0000 7fe388840740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd590280 0x55d7dd9637d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.819+0000 7fe388840740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd5b0f60 0x55d7dd96f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.819+0000 7fe388840740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d7dd973a60 con 0x55d7dd5b0f60 2024-09-21T03:46:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.819+0000 7fe388840740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d7dd974130 con 0x55d7dd5880a0 2024-09-21T03:46:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.819+0000 7fe388840740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d7dd974800 con 0x55d7dd590280 2024-09-21T03:46:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.819+0000 7fe37b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5880a0 0x55d7dd963290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.819+0000 7fe37b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd5b0f60 0x55d7dd96f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.819+0000 7fe37b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5880a0 0x55d7dd963290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:50766/0 (socket says 172.21.15.16:50766) 2024-09-21T03:46:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.819+0000 7fe37a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd590280 0x55d7dd9637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.819+0000 7fe37b400640 1 -- 172.21.15.16:0/3516839252 learned_addr learned my addr 172.21.15.16:0/3516839252 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.820+0000 7fe37a000640 1 -- 172.21.15.16:0/3516839252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5880a0 msgr2=0x55d7dd963290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.820+0000 7fe37a000640 1 --2- 172.21.15.16:0/3516839252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5880a0 0x55d7dd963290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.820+0000 7fe37a000640 1 -- 172.21.15.16:0/3516839252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd5b0f60 msgr2=0x55d7dd96f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.820+0000 7fe37a000640 1 --2- 172.21.15.16:0/3516839252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd5b0f60 0x55d7dd96f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.820+0000 7fe37a000640 1 -- 172.21.15.16:0/3516839252 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7dd973870 con 0x55d7dd590280 2024-09-21T03:46:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.820+0000 7fe37b400640 1 --2- 172.21.15.16:0/3516839252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5880a0 0x55d7dd963290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.820+0000 7fe37a000640 1 --2- 172.21.15.16:0/3516839252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd590280 0x55d7dd9637d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe368001eb0 tx=0x7fe368005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8580a4d7eb8c82ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.820+0000 7fe367400640 1 -- 172.21.15.16:0/3516839252 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368000e60 con 0x55d7dd590280 2024-09-21T03:46:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.821+0000 7fe367400640 1 -- 172.21.15.16:0/3516839252 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe368040070 con 0x55d7dd590280 2024-09-21T03:46:37.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.821+0000 7fe367400640 1 -- 172.21.15.16:0/3516839252 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368035070 con 0x55d7dd590280 2024-09-21T03:46:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.821+0000 7fe388840740 1 -- 172.21.15.16:0/3516839252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd590280 msgr2=0x55d7dd9637d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.821+0000 7fe388840740 1 --2- 172.21.15.16:0/3516839252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd590280 0x55d7dd9637d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe368001eb0 tx=0x7fe368005970 comp rx=0 tx=0).stop 2024-09-21T03:46:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.821+0000 7fe388840740 1 -- 172.21.15.16:0/3516839252 shutdown_connections 2024-09-21T03:46:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.821+0000 7fe388840740 1 --2- 172.21.15.16:0/3516839252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd5b0f60 0x55d7dd96f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.821+0000 7fe388840740 1 --2- 172.21.15.16:0/3516839252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd590280 0x55d7dd9637d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.821+0000 7fe388840740 1 --2- 172.21.15.16:0/3516839252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5880a0 0x55d7dd963290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.821+0000 7fe388840740 1 -- 172.21.15.16:0/3516839252 >> 172.21.15.16:0/3516839252 conn(0x55d7dd5a5350 msgr2=0x55d7dd973160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.822+0000 7fe388840740 1 -- 172.21.15.16:0/3516839252 shutdown_connections 2024-09-21T03:46:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.822+0000 7fe388840740 1 -- 172.21.15.16:0/3516839252 wait complete. 2024-09-21T03:46:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.822+0000 7fe388840740 1 -- start start 2024-09-21T03:46:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.822+0000 7fe388840740 1 Processor -- start 2024-09-21T03:46:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.823+0000 7fe388840740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd5880a0 0x55d7dd97dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.823+0000 7fe388840740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd590280 0x55d7dd9774a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.823+0000 7fe388840740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5b0f60 0x55d7dd9779e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.823+0000 7fe388840740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d7dd9755a0 con 0x55d7dd590280 2024-09-21T03:46:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.823+0000 7fe388840740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d7dd974ed0 con 0x55d7dd5b0f60 2024-09-21T03:46:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.823+0000 7fe388840740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d7dd975c70 con 0x55d7dd5880a0 2024-09-21T03:46:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.823+0000 7fe388840740 10 client.0 _ll_register_callbacks cb 0x7fe368005eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.823+0000 7fe37b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd5880a0 0x55d7dd97dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.823+0000 7fe37a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd590280 0x55d7dd9774a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.823+0000 7fe37b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd5880a0 0x55d7dd97dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52156/0 (socket says 172.21.15.16:52156) 2024-09-21T03:46:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.823+0000 7fe37b400640 1 -- 172.21.15.16:0/3825784692 learned_addr learned my addr 172.21.15.16:0/3825784692 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.823+0000 7fe37b400640 1 --2- 172.21.15.16:0/3825784692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5b0f60 0x55d7dd9779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.824+0000 7fe37b400640 1 -- 172.21.15.16:0/3825784692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5b0f60 msgr2=0x55d7dd9779e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.824+0000 7fe37b400640 1 --2- 172.21.15.16:0/3825784692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5b0f60 0x55d7dd9779e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.824+0000 7fe37b400640 1 -- 172.21.15.16:0/3825784692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd590280 msgr2=0x55d7dd9774a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.824+0000 7fe37b400640 1 --2- 172.21.15.16:0/3825784692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd590280 0x55d7dd9774a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.824+0000 7fe37b400640 1 -- 172.21.15.16:0/3825784692 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7dd977fd0 con 0x55d7dd5880a0 2024-09-21T03:46:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.824+0000 7fe37b400640 1 --2- 172.21.15.16:0/3825784692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd5880a0 0x55d7dd97dff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe374010750 tx=0x7fe374010c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb032a68ada0ca5f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.824+0000 7fe364c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:46:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.824+0000 7fe364c00640 1 -- 172.21.15.16:0/3825784692 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374025040 con 0x55d7dd5880a0 2024-09-21T03:46:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.824+0000 7fe388840740 1 -- 172.21.15.16:0/3825784692 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe368040070 con 0x55d7dd5880a0 2024-09-21T03:46:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.824+0000 7fe364c00640 1 -- 172.21.15.16:0/3825784692 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fe37400ebd0 con 0x55d7dd5880a0 2024-09-21T03:46:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.825+0000 7fe364c00640 1 -- 172.21.15.16:0/3825784692 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe37400edd0 con 0x55d7dd5880a0 2024-09-21T03:46:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.825+0000 7fe388840740 10 client.25531 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.825+0000 7fe388840740 1 -- 172.21.15.16:0/3825784692 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d7dd97da00 con 0x55d7dd5880a0 2024-09-21T03:46:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.825+0000 7fe364c00640 1 -- 172.21.15.16:0/3825784692 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe368040070 con 0x55d7dd5880a0 2024-09-21T03:46:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.825+0000 7fe364c00640 1 -- 172.21.15.16:0/3825784692 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe37400f040 con 0x55d7dd5880a0 2024-09-21T03:46:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.825+0000 7fe388840740 10 client.25531 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.825+0000 7fe388840740 10 client.25531 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:46:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.825+0000 7fe388840740 20 client.25531 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.825+0000 7fe388840740 1 --2- 172.21.15.16:0/3825784692 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x55d7dd544c90 0x55d7dd9e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.825+0000 7fe388840740 4 client.25531 mds_command: new command op to 24689 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:46:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.825+0000 7fe388840740 1 -- 172.21.15.16:0/3825784692 --> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] -- command(tid 0: {"prefix": "status"}) -- 0x55d7dd8cc750 con 0x55d7dd544c90 2024-09-21T03:46:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.826+0000 7fe37a000640 1 --2- 172.21.15.16:0/3825784692 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x55d7dd544c90 0x55d7dd9e40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.826+0000 7fe37a000640 1 --2- 172.21.15.16:0/3825784692 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x55d7dd544c90 0x55d7dd9e40a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=954400bef20c58dd server_cookie=e09d40956584a29a in_seq=0 out_seq=0 2024-09-21T03:46:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.826+0000 7fe364c00640 10 client.25531 ms_handle_connect on v2:172.21.15.123:6836/2001424876 2024-09-21T03:46:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.826+0000 7fe364c00640 1 -- 172.21.15.16:0/3825784692 <== mds.1 v2:172.21.15.123:6836/2001424876 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d7dd8cc750 con 0x55d7dd544c90 2024-09-21T03:46:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.826+0000 7fe364c00640 10 client.25531 handle_command_reply: tid=0 2024-09-21T03:46:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.826+0000 7fe388840740 1 client.25531 shutdown 2024-09-21T03:46:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.827+0000 7fe388840740 1 -- 172.21.15.16:0/3825784692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd5880a0 msgr2=0x55d7dd97dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.827+0000 7fe388840740 1 --2- 172.21.15.16:0/3825784692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd5880a0 0x55d7dd97dff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe374010750 tx=0x7fe374010c10 comp rx=0 tx=0).stop 2024-09-21T03:46:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.827+0000 7fe388840740 1 -- 172.21.15.16:0/3825784692 shutdown_connections 2024-09-21T03:46:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.827+0000 7fe388840740 1 --2- 172.21.15.16:0/3825784692 >> [v2:172.21.15.123:6836/2001424876,v1:172.21.15.123:6837/2001424876] conn(0x55d7dd544c90 0x55d7dd9e40a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.827+0000 7fe388840740 1 --2- 172.21.15.16:0/3825784692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7dd5b0f60 0x55d7dd9779e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.827+0000 7fe388840740 1 --2- 172.21.15.16:0/3825784692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d7dd590280 0x55d7dd9774a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.827+0000 7fe388840740 1 --2- 172.21.15.16:0/3825784692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dd5880a0 0x55d7dd97dff0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.827+0000 7fe364c00640 0 client.25531 ms_handle_reset on v2:172.21.15.123:6836/2001424876 2024-09-21T03:46:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.827+0000 7fe388840740 1 -- 172.21.15.16:0/3825784692 >> 172.21.15.16:0/3825784692 conn(0x55d7dd5a5350 msgr2=0x55d7dd973160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.827+0000 7fe388840740 1 -- 172.21.15.16:0/3825784692 shutdown_connections 2024-09-21T03:46:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.827+0000 7fe388840740 1 -- 172.21.15.16:0/3825784692 wait complete. 2024-09-21T03:46:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.827+0000 7fe388840740 20 client.25531 trim_cache size 0 max 16384 2024-09-21T03:46:37.835 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:46:37.835 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:46:37.835 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-09-21T03:46:37.835 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24689, 2024-09-21T03:46:37.835 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T03:46:37.835 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T03:46:37.835 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T03:46:37.835 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 155.325416311, 2024-09-21T03:46:37.835 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 30, 2024-09-21T03:46:37.835 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 86, 2024-09-21T03:46:37.835 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 86, 2024-09-21T03:46:37.835 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 237.92918185799999 2024-09-21T03:46:37.836 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:46:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.828+0000 7fe37be00640 1 -- 172.21.15.16:0/544030559 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe3600b6590 msgr2=0x7fe3600b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.828+0000 7fe37be00640 1 --2- 172.21.15.16:0/544030559 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe3600b6590 0x7fe3600b8aa0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0ad520 tx=0x7fe36c0a9040 comp rx=0 tx=0).stop 2024-09-21T03:46:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.828+0000 7fe37be00640 1 -- 172.21.15.16:0/544030559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c1039b0 msgr2=0x7fe37c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.828+0000 7fe37be00640 1 --2- 172.21.15.16:0/544030559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c1039b0 0x7fe37c075710 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00e9f0 tx=0x7fe36c00edf0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.828+0000 7fe37be00640 1 -- 172.21.15.16:0/544030559 shutdown_connections 2024-09-21T03:46:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.828+0000 7fe37be00640 1 --2- 172.21.15.16:0/544030559 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe3600b6590 0x7fe3600b8aa0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.828+0000 7fe37be00640 1 --2- 172.21.15.16:0/544030559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe37c109c40 0x7fe37c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.828+0000 7fe37be00640 1 --2- 172.21.15.16:0/544030559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe37c1092f0 0x7fe37c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.828+0000 7fe37be00640 1 --2- 172.21.15.16:0/544030559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c1039b0 0x7fe37c075710 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.828+0000 7fe37be00640 1 -- 172.21.15.16:0/544030559 >> 172.21.15.16:0/544030559 conn(0x7fe37c0ff440 msgr2=0x7fe37c101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.828+0000 7fe37be00640 1 -- 172.21.15.16:0/544030559 shutdown_connections 2024-09-21T03:46:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:37.828+0000 7fe37be00640 1 -- 172.21.15.16:0/544030559 wait complete. 2024-09-21T03:46:38.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:46:38.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:38 smithi123 ceph-mon[34673]: pgmap v428: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-21T03:46:38.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:38 smithi016 ceph-mon[30375]: pgmap v428: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-21T03:46:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:38 smithi134 ceph-mon[35031]: pgmap v428: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-21T03:46:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:39.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.761+0000 7f5d41e00640 1 -- 172.21.15.16:0/814925163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3c109c20 msgr2=0x7f5d3c110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:39.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.761+0000 7f5d41e00640 1 --2- 172.21.15.16:0/814925163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3c109c20 0x7f5d3c110450 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009ca0 tx=0x7f5d3402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.761+0000 7f5d41e00640 1 -- 172.21.15.16:0/814925163 shutdown_connections 2024-09-21T03:46:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.761+0000 7f5d41e00640 1 --2- 172.21.15.16:0/814925163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3c109c20 0x7f5d3c110450 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.761+0000 7f5d41e00640 1 --2- 172.21.15.16:0/814925163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d3c1092d0 0x7f5d3c1096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.761+0000 7f5d41e00640 1 --2- 172.21.15.16:0/814925163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d3c103990 0x7f5d3c103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.761+0000 7f5d41e00640 1 -- 172.21.15.16:0/814925163 >> 172.21.15.16:0/814925163 conn(0x7f5d3c0ff3e0 msgr2=0x7f5d3c101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.761+0000 7f5d41e00640 1 -- 172.21.15.16:0/814925163 shutdown_connections 2024-09-21T03:46:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.761+0000 7f5d41e00640 1 -- 172.21.15.16:0/814925163 wait complete. 2024-09-21T03:46:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.762+0000 7f5d41e00640 1 Processor -- start 2024-09-21T03:46:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.762+0000 7f5d41e00640 1 -- start start 2024-09-21T03:46:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.763+0000 7f5d41e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d3c103990 0x7f5d3c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.763+0000 7f5d41e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d3c1092d0 0x7f5d3c19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.763+0000 7f5d41e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3c109c20 0x7f5d3c19fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.763+0000 7f5d41e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1138e0 con 0x7f5d3c109c20 2024-09-21T03:46:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.763+0000 7f5d41e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c113760 con 0x7f5d3c103990 2024-09-21T03:46:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.763+0000 7f5d41e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c113a60 con 0x7f5d3c1092d0 2024-09-21T03:46:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.763+0000 7f5d41400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3c109c20 0x7f5d3c19fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.763+0000 7f5d3be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d3c1092d0 0x7f5d3c19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.763+0000 7f5d40a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d3c103990 0x7f5d3c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.763+0000 7f5d3be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d3c1092d0 0x7f5d3c19a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52174/0 (socket says 172.21.15.16:52174) 2024-09-21T03:46:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.763+0000 7f5d3be00640 1 -- 172.21.15.16:0/709593844 learned_addr learned my addr 172.21.15.16:0/709593844 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:39.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.764+0000 7f5d3be00640 1 -- 172.21.15.16:0/709593844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d3c103990 msgr2=0x7f5d3c19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:39.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.764+0000 7f5d3be00640 1 --2- 172.21.15.16:0/709593844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d3c103990 0x7f5d3c19a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.764+0000 7f5d3be00640 1 -- 172.21.15.16:0/709593844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3c109c20 msgr2=0x7f5d3c19fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:39.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.764+0000 7f5d3be00640 1 --2- 172.21.15.16:0/709593844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3c109c20 0x7f5d3c19fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.764+0000 7f5d3be00640 1 -- 172.21.15.16:0/709593844 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c1a01e0 con 0x7f5d3c1092d0 2024-09-21T03:46:39.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.764+0000 7f5d41400640 1 --2- 172.21.15.16:0/709593844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3c109c20 0x7f5d3c19fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:39.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.764+0000 7f5d3be00640 1 --2- 172.21.15.16:0/709593844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d3c1092d0 0x7f5d3c19a810 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800e9f0 tx=0x7f5d2800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:39.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.765+0000 7f5d39600640 1 -- 172.21.15.16:0/709593844 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2800ccc0 con 0x7f5d3c1092d0 2024-09-21T03:46:39.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.765+0000 7f5d41e00640 1 -- 172.21.15.16:0/709593844 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c1a04d0 con 0x7f5d3c1092d0 2024-09-21T03:46:39.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.765+0000 7f5d39600640 1 -- 172.21.15.16:0/709593844 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d280108e0 con 0x7f5d3c1092d0 2024-09-21T03:46:39.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.765+0000 7f5d39600640 1 -- 172.21.15.16:0/709593844 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28018860 con 0x7f5d3c1092d0 2024-09-21T03:46:39.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.765+0000 7f5d41e00640 1 -- 172.21.15.16:0/709593844 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c1a6c30 con 0x7f5d3c1092d0 2024-09-21T03:46:39.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.768+0000 7f5d4a09c740 1 Processor -- start 2024-09-21T03:46:39.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.768+0000 7f5d4a09c740 1 -- start start 2024-09-21T03:46:39.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.768+0000 7f5d4a09c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cfdf60 0x55c4f10a4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.768+0000 7f5d4a09c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 0x55c4f10a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.768+0000 7f5d4a09c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 0x55c4f10a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.768+0000 7f5d4a09c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c4f101bd50 con 0x55c4f0cdd280 2024-09-21T03:46:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.768+0000 7f5d4a09c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c4f100b180 con 0x55c4f0cd50a0 2024-09-21T03:46:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.768+0000 7f5d4a09c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c4f100ef90 con 0x55c4f0cfdf60 2024-09-21T03:46:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.769+0000 7f5d39600640 1 -- 172.21.15.16:0/709593844 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5d28010430 con 0x7f5d3c1092d0 2024-09-21T03:46:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.769+0000 7f5d40a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 0x55c4f10a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.769+0000 7f5d41400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cfdf60 0x55c4f10a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.769+0000 7f5d40a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 0x55c4f10a4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:50806/0 (socket says 172.21.15.16:50806) 2024-09-21T03:46:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.769+0000 7f5d40a00640 1 -- 172.21.15.16:0/2605218818 learned_addr learned my addr 172.21.15.16:0/2605218818 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.769+0000 7f5d3be00640 1 --2- 172.21.15.16:0/2605218818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 0x55c4f10a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.769+0000 7f5d40a00640 1 -- 172.21.15.16:0/2605218818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cfdf60 msgr2=0x55c4f10a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:39.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.769+0000 7f5d40a00640 1 --2- 172.21.15.16:0/2605218818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cfdf60 0x55c4f10a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.769+0000 7f5d40a00640 1 -- 172.21.15.16:0/2605218818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 msgr2=0x55c4f10a50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:39.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.769+0000 7f5d40a00640 1 --2- 172.21.15.16:0/2605218818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 0x55c4f10a50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.769+0000 7f5d40a00640 1 -- 172.21.15.16:0/2605218818 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4f0fd5a70 con 0x55c4f0cd50a0 2024-09-21T03:46:39.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.769+0000 7f5d41400640 1 --2- 172.21.15.16:0/2605218818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cfdf60 0x55c4f10a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:39.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.770+0000 7f5d40a00640 1 --2- 172.21.15.16:0/2605218818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 0x55c4f10a4bb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5d30009cf0 tx=0x7f5d30011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c99b6932cdd0299 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:39.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.770+0000 7f5d2dc00640 1 -- 172.21.15.16:0/2605218818 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30004230 con 0x55c4f0cd50a0 2024-09-21T03:46:39.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.770+0000 7f5d2dc00640 1 -- 172.21.15.16:0/2605218818 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d300043d0 con 0x55c4f0cd50a0 2024-09-21T03:46:39.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.770+0000 7f5d2dc00640 1 -- 172.21.15.16:0/2605218818 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30019880 con 0x55c4f0cd50a0 2024-09-21T03:46:39.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.771+0000 7f5d39600640 1 --2- 172.21.15.16:0/709593844 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d240b6510 0x7f5d240b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.771+0000 7f5d39600640 1 -- 172.21.15.16:0/709593844 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f5d280e7970 con 0x7f5d3c1092d0 2024-09-21T03:46:39.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.771+0000 7f5d4a09c740 1 -- 172.21.15.16:0/2605218818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 msgr2=0x55c4f10a4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:39.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.771+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/2605218818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 0x55c4f10a4bb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5d30009cf0 tx=0x7f5d30011810 comp rx=0 tx=0).stop 2024-09-21T03:46:39.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.771+0000 7f5d41400640 1 --2- 172.21.15.16:0/709593844 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d240b6510 0x7f5d240b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.771+0000 7f5d4a09c740 1 -- 172.21.15.16:0/2605218818 shutdown_connections 2024-09-21T03:46:39.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.771+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/2605218818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 0x55c4f10a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.771+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/2605218818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 0x55c4f10a4bb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.771+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/2605218818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cfdf60 0x55c4f10a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.771+0000 7f5d4a09c740 1 -- 172.21.15.16:0/2605218818 >> 172.21.15.16:0/2605218818 conn(0x55c4f0cf2350 msgr2=0x55c4f10a2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:39.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.771+0000 7f5d4a09c740 1 -- 172.21.15.16:0/2605218818 shutdown_connections 2024-09-21T03:46:39.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.771+0000 7f5d4a09c740 1 -- 172.21.15.16:0/2605218818 wait complete. 2024-09-21T03:46:39.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.772+0000 7f5d41400640 1 --2- 172.21.15.16:0/709593844 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d240b6510 0x7f5d240b8a20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5d34006010 tx=0x7f5d34032eb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:39.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.772+0000 7f5d4a09c740 1 -- start start 2024-09-21T03:46:39.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.772+0000 7f5d4a09c740 1 Processor -- start 2024-09-21T03:46:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.773+0000 7f5d4a09c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 0x55c4f10b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.773+0000 7f5d4a09c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cdd280 0x55c4f10b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.773+0000 7f5d4a09c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cfdf60 0x55c4f10bc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.773+0000 7f5d4a09c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c4f0e6c5f0 con 0x55c4f0cfdf60 2024-09-21T03:46:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.773+0000 7f5d4a09c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c4f0fa6460 con 0x55c4f0cd50a0 2024-09-21T03:46:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.773+0000 7f5d4a09c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c4f10b2e50 con 0x55c4f0cdd280 2024-09-21T03:46:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.773+0000 7f5d4a09c740 10 client.0 _ll_register_callbacks cb 0x55c4f0fb6210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.773+0000 7f5d41400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cfdf60 0x55c4f10bc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.773+0000 7f5d3be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cdd280 0x55c4f10b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.773+0000 7f5d40a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 0x55c4f10b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.773+0000 7f5d3be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cdd280 0x55c4f10b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52188/0 (socket says 172.21.15.16:52188) 2024-09-21T03:46:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.773+0000 7f5d3be00640 1 -- 172.21.15.16:0/1733879175 learned_addr learned my addr 172.21.15.16:0/1733879175 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.774+0000 7f5d3be00640 1 -- 172.21.15.16:0/1733879175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 msgr2=0x55c4f10b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.774+0000 7f5d3be00640 1 --2- 172.21.15.16:0/1733879175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 0x55c4f10b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.774+0000 7f5d3be00640 1 -- 172.21.15.16:0/1733879175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cfdf60 msgr2=0x55c4f10bc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.774+0000 7f5d3be00640 1 --2- 172.21.15.16:0/1733879175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cfdf60 0x55c4f10bc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.774+0000 7f5d3be00640 1 -- 172.21.15.16:0/1733879175 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4f0d69540 con 0x55c4f0cdd280 2024-09-21T03:46:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.774+0000 7f5d40a00640 1 --2- 172.21.15.16:0/1733879175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 0x55c4f10b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.774+0000 7f5d3be00640 1 --2- 172.21.15.16:0/1733879175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cdd280 0x55c4f10b06d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5d280ad730 tx=0x7f5d280096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b170dfb70d1e1319 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.774+0000 7f5d22000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:46:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.774+0000 7f5d22000640 1 -- 172.21.15.16:0/1733879175 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d280ad0f0 con 0x55c4f0cdd280 2024-09-21T03:46:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.774+0000 7f5d4a09c740 1 -- 172.21.15.16:0/1733879175 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c4f0fa8a70 con 0x55c4f0cdd280 2024-09-21T03:46:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.774+0000 7f5d22000640 1 -- 172.21.15.16:0/1733879175 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f5d280ac470 con 0x55c4f0cdd280 2024-09-21T03:46:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.774+0000 7f5d22000640 1 -- 172.21.15.16:0/1733879175 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d280d4020 con 0x55c4f0cdd280 2024-09-21T03:46:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.775+0000 7f5d4a09c740 10 client.25543 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.775+0000 7f5d4a09c740 1 -- 172.21.15.16:0/1733879175 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c4f0fc36f0 con 0x55c4f0cdd280 2024-09-21T03:46:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.775+0000 7f5d22000640 1 -- 172.21.15.16:0/1733879175 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c4f0fa8a70 con 0x55c4f0cdd280 2024-09-21T03:46:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.775+0000 7f5d22000640 1 -- 172.21.15.16:0/1733879175 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5d280ac670 con 0x55c4f0cdd280 2024-09-21T03:46:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.775+0000 7f5d4a09c740 10 client.25543 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.775+0000 7f5d4a09c740 10 client.25543 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:46:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.775+0000 7f5d4a09c740 20 client.25543 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.775+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/1733879175 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c4f0cbc8e0 0x55c4f10c2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.775+0000 7f5d4a09c740 4 client.25543 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.775+0000 7f5d4a09c740 1 -- 172.21.15.16:0/1733879175 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4f101dca0 con 0x55c4f0cbc8e0 2024-09-21T03:46:39.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.776+0000 7f5d40a00640 1 --2- 172.21.15.16:0/1733879175 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c4f0cbc8e0 0x55c4f10c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.776+0000 7f5d40a00640 1 --2- 172.21.15.16:0/1733879175 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c4f0cbc8e0 0x55c4f10c2700 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d050f7b7262da611 server_cookie=927abe1426c18f93 in_seq=0 out_seq=0 2024-09-21T03:46:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.776+0000 7f5d22000640 10 client.25543 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.777+0000 7f5d22000640 1 -- 172.21.15.16:0/1733879175 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c4f101dca0 con 0x55c4f0cbc8e0 2024-09-21T03:46:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.777+0000 7f5d22000640 10 client.25543 handle_command_reply: tid=0 2024-09-21T03:46:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.777+0000 7f5d4a09c740 1 client.25543 shutdown 2024-09-21T03:46:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.777+0000 7f5d4a09c740 1 -- 172.21.15.16:0/1733879175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cdd280 msgr2=0x55c4f10b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.777+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/1733879175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cdd280 0x55c4f10b06d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5d280ad730 tx=0x7f5d280096a0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.778+0000 7f5d4a09c740 1 -- 172.21.15.16:0/1733879175 shutdown_connections 2024-09-21T03:46:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.778+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/1733879175 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c4f0cbc8e0 0x55c4f10c2700 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.778+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/1733879175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cfdf60 0x55c4f10bc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.778+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/1733879175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cdd280 0x55c4f10b06d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.778+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/1733879175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 0x55c4f10b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.778+0000 7f5d22000640 0 client.25543 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.778+0000 7f5d4a09c740 1 -- 172.21.15.16:0/1733879175 >> 172.21.15.16:0/1733879175 conn(0x55c4f0cf2350 msgr2=0x55c4f10a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.778+0000 7f5d4a09c740 1 -- 172.21.15.16:0/1733879175 shutdown_connections 2024-09-21T03:46:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.778+0000 7f5d4a09c740 1 -- 172.21.15.16:0/1733879175 wait complete. 2024-09-21T03:46:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.778+0000 7f5d4a09c740 20 client.25543 trim_cache size 0 max 16384 2024-09-21T03:46:39.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.797+0000 7f5d4a09c740 1 Processor -- start 2024-09-21T03:46:39.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.797+0000 7f5d4a09c740 1 -- start start 2024-09-21T03:46:39.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.798+0000 7f5d4a09c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 0x55c4f10aa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.798+0000 7f5d4a09c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 0x55c4f10c0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.798+0000 7f5d4a09c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cfdf60 0x55c4f10bc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.798+0000 7f5d4a09c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c4f10c0a40 con 0x55c4f0cdd280 2024-09-21T03:46:39.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.798+0000 7f5d4a09c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c4f10c1110 con 0x55c4f0cd50a0 2024-09-21T03:46:39.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.798+0000 7f5d4a09c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c4f10c17e0 con 0x55c4f0cfdf60 2024-09-21T03:46:39.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.798+0000 7f5d3be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 0x55c4f10c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.798+0000 7f5d40a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 0x55c4f10aa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.798+0000 7f5d3be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 0x55c4f10c0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41914/0 (socket says 172.21.15.16:41914) 2024-09-21T03:46:39.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.798+0000 7f5d3be00640 1 -- 172.21.15.16:0/872560060 learned_addr learned my addr 172.21.15.16:0/872560060 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:39.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.798+0000 7f5d41400640 1 --2- 172.21.15.16:0/872560060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cfdf60 0x55c4f10bc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.799+0000 7f5d3be00640 1 -- 172.21.15.16:0/872560060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cfdf60 msgr2=0x55c4f10bc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:39.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.799+0000 7f5d3be00640 1 --2- 172.21.15.16:0/872560060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cfdf60 0x55c4f10bc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.799+0000 7f5d3be00640 1 -- 172.21.15.16:0/872560060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 msgr2=0x55c4f10aa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:39.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.799+0000 7f5d3be00640 1 --2- 172.21.15.16:0/872560060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 0x55c4f10aa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.799+0000 7f5d3be00640 1 -- 172.21.15.16:0/872560060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4f10a6960 con 0x55c4f0cdd280 2024-09-21T03:46:39.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.799+0000 7f5d3be00640 1 --2- 172.21.15.16:0/872560060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 0x55c4f10c0140 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5d280057a0 tx=0x7f5d280b61b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f0399658ec16fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:39.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.799+0000 7f5d2d200640 1 -- 172.21.15.16:0/872560060 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d280acf10 con 0x55c4f0cdd280 2024-09-21T03:46:39.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.799+0000 7f5d2d200640 1 -- 172.21.15.16:0/872560060 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d280b6430 con 0x55c4f0cdd280 2024-09-21T03:46:39.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.799+0000 7f5d2d200640 1 -- 172.21.15.16:0/872560060 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d280ac6b0 con 0x55c4f0cdd280 2024-09-21T03:46:39.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.800+0000 7f5d4a09c740 1 -- 172.21.15.16:0/872560060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 msgr2=0x55c4f10c0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:39.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.800+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/872560060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 0x55c4f10c0140 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5d280057a0 tx=0x7f5d280b61b0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.800+0000 7f5d4a09c740 1 -- 172.21.15.16:0/872560060 shutdown_connections 2024-09-21T03:46:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.800+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/872560060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cfdf60 0x55c4f10bc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.800+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/872560060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 0x55c4f10c0140 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.800+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/872560060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cd50a0 0x55c4f10aa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.800+0000 7f5d4a09c740 1 -- 172.21.15.16:0/872560060 >> 172.21.15.16:0/872560060 conn(0x55c4f0cf2350 msgr2=0x55c4f10ab140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.800+0000 7f5d4a09c740 1 -- 172.21.15.16:0/872560060 shutdown_connections 2024-09-21T03:46:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.800+0000 7f5d4a09c740 1 -- 172.21.15.16:0/872560060 wait complete. 2024-09-21T03:46:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.801+0000 7f5d4a09c740 1 -- start start 2024-09-21T03:46:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.801+0000 7f5d4a09c740 1 Processor -- start 2024-09-21T03:46:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.801+0000 7f5d4a09c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cd50a0 0x55c4f10cac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.801+0000 7f5d4a09c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 0x55c4f10cb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.802+0000 7f5d4a09c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cfdf60 0x55c4f10c4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.802+0000 7f5d4a09c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c4f10c2580 con 0x55c4f0cdd280 2024-09-21T03:46:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.802+0000 7f5d4a09c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c4f10c1eb0 con 0x55c4f0cfdf60 2024-09-21T03:46:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.802+0000 7f5d4a09c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c4f10c2c50 con 0x55c4f0cd50a0 2024-09-21T03:46:39.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.802+0000 7f5d4a09c740 10 client.0 _ll_register_callbacks cb 0x7f5d280ed220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:39.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.802+0000 7f5d3be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 0x55c4f10cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.802+0000 7f5d3be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 0x55c4f10cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41922/0 (socket says 172.21.15.16:41922) 2024-09-21T03:46:39.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.802+0000 7f5d3be00640 1 -- 172.21.15.16:0/3634278955 learned_addr learned my addr 172.21.15.16:0/3634278955 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:39.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.802+0000 7f5d40a00640 1 --2- 172.21.15.16:0/3634278955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cd50a0 0x55c4f10cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.802+0000 7f5d41400640 1 --2- 172.21.15.16:0/3634278955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cfdf60 0x55c4f10c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.802+0000 7f5d3be00640 1 -- 172.21.15.16:0/3634278955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cd50a0 msgr2=0x55c4f10cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:39.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.802+0000 7f5d3be00640 1 --2- 172.21.15.16:0/3634278955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cd50a0 0x55c4f10cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.802+0000 7f5d3be00640 1 -- 172.21.15.16:0/3634278955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cfdf60 msgr2=0x55c4f10c4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:39.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.802+0000 7f5d3be00640 1 --2- 172.21.15.16:0/3634278955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cfdf60 0x55c4f10c4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.802+0000 7f5d3be00640 1 -- 172.21.15.16:0/3634278955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4f10c48d0 con 0x55c4f0cdd280 2024-09-21T03:46:39.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.802+0000 7f5d40a00640 1 --2- 172.21.15.16:0/3634278955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cd50a0 0x55c4f10cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:39.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.803+0000 7f5d3be00640 1 --2- 172.21.15.16:0/3634278955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 0x55c4f10cb250 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5d280aaa10 tx=0x7f5d2800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa88968c50bd8d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:39.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.803+0000 7f5d22000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:46:39.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.803+0000 7f5d22000640 1 -- 172.21.15.16:0/3634278955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d280acf10 con 0x55c4f0cdd280 2024-09-21T03:46:39.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.803+0000 7f5d4a09c740 1 -- 172.21.15.16:0/3634278955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d280b6430 con 0x55c4f0cdd280 2024-09-21T03:46:39.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.803+0000 7f5d22000640 1 -- 172.21.15.16:0/3634278955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f5d280ac680 con 0x55c4f0cdd280 2024-09-21T03:46:39.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.803+0000 7f5d22000640 1 -- 172.21.15.16:0/3634278955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d28010720 con 0x55c4f0cdd280 2024-09-21T03:46:39.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.803+0000 7f5d4a09c740 10 client.15558 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.803+0000 7f5d4a09c740 1 -- 172.21.15.16:0/3634278955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c4f10c48d0 con 0x55c4f0cdd280 2024-09-21T03:46:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.804+0000 7f5d22000640 1 -- 172.21.15.16:0/3634278955 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d280b6430 con 0x55c4f0cdd280 2024-09-21T03:46:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.804+0000 7f5d22000640 1 -- 172.21.15.16:0/3634278955 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5d280b66e0 con 0x55c4f0cdd280 2024-09-21T03:46:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.804+0000 7f5d4a09c740 10 client.15558 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.804+0000 7f5d4a09c740 10 client.15558 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:46:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.804+0000 7f5d4a09c740 20 client.15558 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.804+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/3634278955 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c4f0c91c90 0x55c4f1130630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.804+0000 7f5d4a09c740 4 client.15558 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:46:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.804+0000 7f5d4a09c740 1 -- 172.21.15.16:0/3634278955 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "status"}) -- 0x55c4f1019750 con 0x55c4f0c91c90 2024-09-21T03:46:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.804+0000 7f5d40a00640 1 --2- 172.21.15.16:0/3634278955 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c4f0c91c90 0x55c4f1130630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.805+0000 7f5d40a00640 1 --2- 172.21.15.16:0/3634278955 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c4f0c91c90 0x55c4f1130630 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7cbf634fff5140a server_cookie=c33dabc07eba7fe7 in_seq=0 out_seq=0 2024-09-21T03:46:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.805+0000 7f5d22000640 10 client.15558 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.805+0000 7f5d22000640 1 -- 172.21.15.16:0/3634278955 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c4f1019750 con 0x55c4f0c91c90 2024-09-21T03:46:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.805+0000 7f5d22000640 10 client.15558 handle_command_reply: tid=0 2024-09-21T03:46:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.805+0000 7f5d4a09c740 1 client.15558 shutdown 2024-09-21T03:46:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.806+0000 7f5d4a09c740 1 -- 172.21.15.16:0/3634278955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 msgr2=0x55c4f10cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.806+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/3634278955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 0x55c4f10cb250 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5d280aaa10 tx=0x7f5d2800ce60 comp rx=0 tx=0).stop 2024-09-21T03:46:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.806+0000 7f5d4a09c740 1 -- 172.21.15.16:0/3634278955 shutdown_connections 2024-09-21T03:46:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.806+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/3634278955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4f0cfdf60 0x55c4f10c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.806+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/3634278955 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c4f0c91c90 0x55c4f1130630 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.806+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/3634278955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f0cdd280 0x55c4f10cb250 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.806+0000 7f5d4a09c740 1 --2- 172.21.15.16:0/3634278955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4f0cd50a0 0x55c4f10cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.806+0000 7f5d4a09c740 1 -- 172.21.15.16:0/3634278955 >> 172.21.15.16:0/3634278955 conn(0x55c4f0cf2350 msgr2=0x55c4f10c0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.806+0000 7f5d22000640 0 client.15558 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.806+0000 7f5d4a09c740 1 -- 172.21.15.16:0/3634278955 shutdown_connections 2024-09-21T03:46:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.806+0000 7f5d4a09c740 1 -- 172.21.15.16:0/3634278955 wait complete. 2024-09-21T03:46:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.806+0000 7f5d4a09c740 20 client.15558 trim_cache size 0 max 16384 2024-09-21T03:46:39.812 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:46:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "2b324840-77ca-11ef-baf5-efdc52797490", 2024-09-21T03:46:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-21T03:46:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24731, 2024-09-21T03:46:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T03:46:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T03:46:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T03:46:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 209.80026148600001, 2024-09-21T03:46:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 30, 2024-09-21T03:46:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 86, 2024-09-21T03:46:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 86, 2024-09-21T03:46:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 226.20282810500001 2024-09-21T03:46:39.813 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:46:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.807+0000 7f5d41e00640 1 -- 172.21.15.16:0/709593844 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d240b6510 msgr2=0x7f5d240b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.807+0000 7f5d41e00640 1 --2- 172.21.15.16:0/709593844 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d240b6510 0x7f5d240b8a20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5d34006010 tx=0x7f5d34032eb0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.807+0000 7f5d41e00640 1 -- 172.21.15.16:0/709593844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d3c1092d0 msgr2=0x7f5d3c19a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.807+0000 7f5d41e00640 1 --2- 172.21.15.16:0/709593844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d3c1092d0 0x7f5d3c19a810 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800e9f0 tx=0x7f5d2800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.807+0000 7f5d41e00640 1 -- 172.21.15.16:0/709593844 shutdown_connections 2024-09-21T03:46:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.807+0000 7f5d41e00640 1 --2- 172.21.15.16:0/709593844 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d240b6510 0x7f5d240b8a20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.807+0000 7f5d41e00640 1 --2- 172.21.15.16:0/709593844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3c109c20 0x7f5d3c19fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.807+0000 7f5d41e00640 1 --2- 172.21.15.16:0/709593844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d3c1092d0 0x7f5d3c19a810 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.807+0000 7f5d41e00640 1 --2- 172.21.15.16:0/709593844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d3c103990 0x7f5d3c19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.807+0000 7f5d41e00640 1 -- 172.21.15.16:0/709593844 >> 172.21.15.16:0/709593844 conn(0x7f5d3c0ff3e0 msgr2=0x7f5d3c101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.807+0000 7f5d41e00640 1 -- 172.21.15.16:0/709593844 shutdown_connections 2024-09-21T03:46:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:39.807+0000 7f5d41e00640 1 -- 172.21.15.16:0/709593844 wait complete. 2024-09-21T03:46:40.718 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-21T03:46:40.904 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:40 smithi134 ceph-mon[35031]: pgmap v429: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-21T03:46:40.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:40 smithi123 ceph-mon[34673]: pgmap v429: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-21T03:46:40.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:40 smithi016 ceph-mon[30375]: pgmap v429: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-21T03:46:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.288+0000 7f64f6c00640 1 -- 172.21.15.16:0/1421783104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64f01034e0 msgr2=0x7f64f0103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.288+0000 7f64f6c00640 1 --2- 172.21.15.16:0/1421783104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64f01034e0 0x7f64f0103950 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009860 tx=0x7f64e402a340 comp rx=0 tx=0).stop 2024-09-21T03:46:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.288+0000 7f64f6c00640 1 -- 172.21.15.16:0/1421783104 shutdown_connections 2024-09-21T03:46:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.288+0000 7f64f6c00640 1 --2- 172.21.15.16:0/1421783104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f0104f00 0x7f64f010b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.288+0000 7f64f6c00640 1 --2- 172.21.15.16:0/1421783104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f0104560 0x7f64f0104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.288+0000 7f64f6c00640 1 --2- 172.21.15.16:0/1421783104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64f01034e0 0x7f64f0103950 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.288+0000 7f64f6c00640 1 -- 172.21.15.16:0/1421783104 >> 172.21.15.16:0/1421783104 conn(0x7f64f00fee20 msgr2=0x7f64f01012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.289+0000 7f64f6c00640 1 -- 172.21.15.16:0/1421783104 shutdown_connections 2024-09-21T03:46:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.289+0000 7f64f6c00640 1 -- 172.21.15.16:0/1421783104 wait complete. 2024-09-21T03:46:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.290+0000 7f64f6c00640 1 Processor -- start 2024-09-21T03:46:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.290+0000 7f64f6c00640 1 -- start start 2024-09-21T03:46:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.290+0000 7f64f6c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64f01034e0 0x7f64f019bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.290+0000 7f64f6c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f0104560 0x7f64f019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.290+0000 7f64f6c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f0104f00 0x7f64f019d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.290+0000 7f64f6c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64f010f120 con 0x7f64f0104560 2024-09-21T03:46:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.290+0000 7f64f6c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64f010efa0 con 0x7f64f0104f00 2024-09-21T03:46:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.290+0000 7f64f6c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64f010f2a0 con 0x7f64f01034e0 2024-09-21T03:46:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.291+0000 7f64f4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f0104560 0x7f64f019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.291+0000 7f64f5800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64f01034e0 0x7f64f019bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.291+0000 7f64f4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f0104560 0x7f64f019c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41952/0 (socket says 172.21.15.16:41952) 2024-09-21T03:46:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.291+0000 7f64f4e00640 1 -- 172.21.15.16:0/4033326031 learned_addr learned my addr 172.21.15.16:0/4033326031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.291+0000 7f64f6200640 1 --2- 172.21.15.16:0/4033326031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f0104f00 0x7f64f019d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.291+0000 7f64f4e00640 1 -- 172.21.15.16:0/4033326031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64f01034e0 msgr2=0x7f64f019bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.291+0000 7f64f4e00640 1 --2- 172.21.15.16:0/4033326031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64f01034e0 0x7f64f019bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.291+0000 7f64f4e00640 1 -- 172.21.15.16:0/4033326031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f0104f00 msgr2=0x7f64f019d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.291+0000 7f64f4e00640 1 --2- 172.21.15.16:0/4033326031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f0104f00 0x7f64f019d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.291+0000 7f64f4e00640 1 -- 172.21.15.16:0/4033326031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f019da10 con 0x7f64f0104560 2024-09-21T03:46:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.291+0000 7f64f4e00640 1 --2- 172.21.15.16:0/4033326031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f0104560 0x7f64f019c640 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00e9f0 tx=0x7f64dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.292+0000 7f64ee000640 1 -- 172.21.15.16:0/4033326031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc00cd80 con 0x7f64f0104560 2024-09-21T03:46:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.292+0000 7f64f6c00640 1 -- 172.21.15.16:0/4033326031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f019dd00 con 0x7f64f0104560 2024-09-21T03:46:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.292+0000 7f64ee000640 1 -- 172.21.15.16:0/4033326031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64dc0054d0 con 0x7f64f0104560 2024-09-21T03:46:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.292+0000 7f64ee000640 1 -- 172.21.15.16:0/4033326031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc010690 con 0x7f64f0104560 2024-09-21T03:46:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.292+0000 7f64f6c00640 1 -- 172.21.15.16:0/4033326031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f01a43f0 con 0x7f64f0104560 2024-09-21T03:46:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.295+0000 7f64ff25e740 1 Processor -- start 2024-09-21T03:46:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.295+0000 7f64ff25e740 1 -- start start 2024-09-21T03:46:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.296+0000 7f64ee000640 1 -- 172.21.15.16:0/4033326031 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f64dc004e90 con 0x7f64f0104560 2024-09-21T03:46:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.296+0000 7f64ff25e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401681b0 0x55dd4050f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.296+0000 7f64ff25e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 0x55dd4050f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.296+0000 7f64ff25e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401474d0 0x55dd40510470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.296+0000 7f64ff25e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dd404826b0 con 0x55dd401681b0 2024-09-21T03:46:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.296+0000 7f64ff25e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dd40471ae0 con 0x55dd401474d0 2024-09-21T03:46:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.296+0000 7f64ff25e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dd404758f0 con 0x55dd4013f2f0 2024-09-21T03:46:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.296+0000 7f64f6200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401681b0 0x55dd4050f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.296+0000 7f64f4e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401474d0 0x55dd40510470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.296+0000 7f64f5800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 0x55dd4050f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.296+0000 7f64f5800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 0x55dd4050f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52210/0 (socket says 172.21.15.16:52210) 2024-09-21T03:46:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.296+0000 7f64f5800640 1 -- 172.21.15.16:0/2933649481 learned_addr learned my addr 172.21.15.16:0/2933649481 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.296+0000 7f64f5800640 1 -- 172.21.15.16:0/2933649481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401474d0 msgr2=0x55dd40510470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.296+0000 7f64f5800640 1 --2- 172.21.15.16:0/2933649481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401474d0 0x55dd40510470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.296+0000 7f64f5800640 1 -- 172.21.15.16:0/2933649481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401681b0 msgr2=0x55dd4050f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.296+0000 7f64f5800640 1 --2- 172.21.15.16:0/2933649481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401681b0 0x55dd4050f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.296+0000 7f64f5800640 1 -- 172.21.15.16:0/2933649481 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd4043e780 con 0x55dd4013f2f0 2024-09-21T03:46:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.297+0000 7f64f5800640 1 --2- 172.21.15.16:0/2933649481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 0x55dd4050f920 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009860 tx=0x7f64e4005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=313eaed94f988d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.297+0000 7f64e2a00640 1 -- 172.21.15.16:0/2933649481 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4037040 con 0x55dd4013f2f0 2024-09-21T03:46:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.297+0000 7f64e2a00640 1 -- 172.21.15.16:0/2933649481 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64e4030810 con 0x55dd4013f2f0 2024-09-21T03:46:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.297+0000 7f64e2a00640 1 -- 172.21.15.16:0/2933649481 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4003990 con 0x55dd4013f2f0 2024-09-21T03:46:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.298+0000 7f64ee000640 1 --2- 172.21.15.16:0/4033326031 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64d80b6590 0x7f64d80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.298+0000 7f64ee000640 1 -- 172.21.15.16:0/4033326031 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f64dc0e7aa0 con 0x7f64f0104560 2024-09-21T03:46:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.298+0000 7f64f6200640 1 --2- 172.21.15.16:0/4033326031 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64d80b6590 0x7f64d80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.298+0000 7f64ff25e740 1 -- 172.21.15.16:0/2933649481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 msgr2=0x55dd4050f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.298+0000 7f64ff25e740 1 --2- 172.21.15.16:0/2933649481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 0x55dd4050f920 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009860 tx=0x7f64e4005cf0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.298+0000 7f64ff25e740 1 -- 172.21.15.16:0/2933649481 shutdown_connections 2024-09-21T03:46:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.298+0000 7f64ff25e740 1 --2- 172.21.15.16:0/2933649481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401474d0 0x55dd40510470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.298+0000 7f64ff25e740 1 --2- 172.21.15.16:0/2933649481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 0x55dd4050f920 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.298+0000 7f64ff25e740 1 --2- 172.21.15.16:0/2933649481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401681b0 0x55dd4050f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.298+0000 7f64ff25e740 1 -- 172.21.15.16:0/2933649481 >> 172.21.15.16:0/2933649481 conn(0x55dd4015c300 msgr2=0x55dd4050ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.299+0000 7f64ff25e740 1 -- 172.21.15.16:0/2933649481 shutdown_connections 2024-09-21T03:46:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.299+0000 7f64f6200640 1 --2- 172.21.15.16:0/4033326031 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64d80b6590 0x7f64d80b8aa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f64e8005fd0 tx=0x7f64e8010040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.299+0000 7f64ff25e740 1 -- 172.21.15.16:0/2933649481 wait complete. 2024-09-21T03:46:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.299+0000 7f64ff25e740 1 -- start start 2024-09-21T03:46:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.299+0000 7f64ff25e740 1 Processor -- start 2024-09-21T03:46:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.300+0000 7f64ff25e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd4013f2f0 0x55dd40518850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.300+0000 7f64ff25e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd401474d0 0x55dd40518f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.300+0000 7f64ff25e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401681b0 0x55dd40519a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.300+0000 7f64ff25e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dd403a9ab0 con 0x55dd4013f2f0 2024-09-21T03:46:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.300+0000 7f64ff25e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dd40418570 con 0x55dd401681b0 2024-09-21T03:46:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.300+0000 7f64ff25e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dd40510f30 con 0x55dd401474d0 2024-09-21T03:46:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.300+0000 7f64ff25e740 10 client.0 _ll_register_callbacks cb 0x55dd40427100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.300+0000 7f64f5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd4013f2f0 0x55dd40518850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.300+0000 7f64f4e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd401474d0 0x55dd40518f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.300+0000 7f64f5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd4013f2f0 0x55dd40518850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41968/0 (socket says 172.21.15.16:41968) 2024-09-21T03:46:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.300+0000 7f64f5800640 1 -- 172.21.15.16:0/2118789981 learned_addr learned my addr 172.21.15.16:0/2118789981 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.300+0000 7f64f6200640 1 --2- 172.21.15.16:0/2118789981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401681b0 0x55dd40519a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.300+0000 7f64f5800640 1 -- 172.21.15.16:0/2118789981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd401474d0 msgr2=0x55dd40518f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.301+0000 7f64f5800640 1 --2- 172.21.15.16:0/2118789981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd401474d0 0x55dd40518f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.301+0000 7f64f5800640 1 -- 172.21.15.16:0/2118789981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401681b0 msgr2=0x55dd40519a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.301+0000 7f64f5800640 1 --2- 172.21.15.16:0/2118789981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401681b0 0x55dd40519a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.301+0000 7f64f5800640 1 -- 172.21.15.16:0/2118789981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd401d3790 con 0x55dd4013f2f0 2024-09-21T03:46:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.301+0000 7f64f4e00640 1 --2- 172.21.15.16:0/2118789981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd401474d0 0x55dd40518f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.301+0000 7f64f6200640 1 --2- 172.21.15.16:0/2118789981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401681b0 0x55dd40519a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.301+0000 7f64f5800640 1 --2- 172.21.15.16:0/2118789981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd4013f2f0 0x55dd40518850 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f64e402a860 tx=0x7f64e4030f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c60508abd489bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.301+0000 7f64d7400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:46:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.301+0000 7f64d7400640 1 -- 172.21.15.16:0/2118789981 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4037030 con 0x55dd4013f2f0 2024-09-21T03:46:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.301+0000 7f64d7400640 1 -- 172.21.15.16:0/2118789981 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f64e403d890 con 0x55dd4013f2f0 2024-09-21T03:46:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.301+0000 7f64ff25e740 1 -- 172.21.15.16:0/2118789981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64e4030810 con 0x55dd4013f2f0 2024-09-21T03:46:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.301+0000 7f64d7400640 1 -- 172.21.15.16:0/2118789981 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f64e4058050 con 0x55dd4013f2f0 2024-09-21T03:46:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.301+0000 7f64ff25e740 10 client.15570 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.301+0000 7f64ff25e740 1 -- 172.21.15.16:0/2118789981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dd4024c620 con 0x55dd4013f2f0 2024-09-21T03:46:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.302+0000 7f64d7400640 1 -- 172.21.15.16:0/2118789981 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64e4030810 con 0x55dd4013f2f0 2024-09-21T03:46:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.302+0000 7f64d7400640 1 -- 172.21.15.16:0/2118789981 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f64e4046b30 con 0x55dd4013f2f0 2024-09-21T03:46:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.302+0000 7f64ff25e740 10 client.15570 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.302+0000 7f64ff25e740 10 client.15570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.302+0000 7f64ff25e740 20 client.15570 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.302+0000 7f64ff25e740 1 --2- 172.21.15.16:0/2118789981 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55dd40126d80 0x55dd40525b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.302+0000 7f64ff25e740 4 client.15570 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.302+0000 7f64ff25e740 1 -- 172.21.15.16:0/2118789981 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd40484600 con 0x55dd40126d80 2024-09-21T03:46:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.302+0000 7f64f4e00640 1 --2- 172.21.15.16:0/2118789981 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55dd40126d80 0x55dd40525b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.303+0000 7f64f4e00640 1 --2- 172.21.15.16:0/2118789981 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55dd40126d80 0x55dd40525b80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7aaec7bee6e8905 server_cookie=5c64370d7d2537c5 in_seq=0 out_seq=0 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.303+0000 7f64d7400640 10 client.15570 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.304+0000 7f64d7400640 1 -- 172.21.15.16:0/2118789981 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dd40484600 con 0x55dd40126d80 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.304+0000 7f64d7400640 10 client.15570 handle_command_reply: tid=0 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.304+0000 7f64ff25e740 1 client.15570 shutdown 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.304+0000 7f64ff25e740 1 -- 172.21.15.16:0/2118789981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd4013f2f0 msgr2=0x55dd40518850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.304+0000 7f64ff25e740 1 --2- 172.21.15.16:0/2118789981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd4013f2f0 0x55dd40518850 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f64e402a860 tx=0x7f64e4030f70 comp rx=0 tx=0).stop 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.304+0000 7f64ff25e740 1 -- 172.21.15.16:0/2118789981 shutdown_connections 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.304+0000 7f64ff25e740 1 --2- 172.21.15.16:0/2118789981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401681b0 0x55dd40519a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.304+0000 7f64ff25e740 1 --2- 172.21.15.16:0/2118789981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd401474d0 0x55dd40518f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.304+0000 7f64ff25e740 1 --2- 172.21.15.16:0/2118789981 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55dd40126d80 0x55dd40525b80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.304+0000 7f64ff25e740 1 --2- 172.21.15.16:0/2118789981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd4013f2f0 0x55dd40518850 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.304+0000 7f64d7400640 0 client.15570 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.304+0000 7f64ff25e740 1 -- 172.21.15.16:0/2118789981 >> 172.21.15.16:0/2118789981 conn(0x55dd4015c300 msgr2=0x55dd40509360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.304+0000 7f64ff25e740 1 -- 172.21.15.16:0/2118789981 shutdown_connections 2024-09-21T03:46:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.304+0000 7f64ff25e740 1 -- 172.21.15.16:0/2118789981 wait complete. 2024-09-21T03:46:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.304+0000 7f64ff25e740 20 client.15570 trim_cache size 0 max 16384 2024-09-21T03:46:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.320+0000 7f64ff25e740 1 Processor -- start 2024-09-21T03:46:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.320+0000 7f64ff25e740 1 -- start start 2024-09-21T03:46:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.320+0000 7f64ff25e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 0x55dd4050c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.320+0000 7f64ff25e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401474d0 0x55dd40520960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.320+0000 7f64ff25e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401681b0 0x55dd40517a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.320+0000 7f64ff25e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64f01ae790 con 0x55dd401474d0 2024-09-21T03:46:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.320+0000 7f64ff25e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64f01b02d0 con 0x55dd401681b0 2024-09-21T03:46:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.320+0000 7f64ff25e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64f01aee60 con 0x55dd4013f2f0 2024-09-21T03:46:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.320+0000 7f64f4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401474d0 0x55dd40520960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.320+0000 7f64f4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401474d0 0x55dd40520960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41974/0 (socket says 172.21.15.16:41974) 2024-09-21T03:46:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.320+0000 7f64f4e00640 1 -- 172.21.15.16:0/1982199346 learned_addr learned my addr 172.21.15.16:0/1982199346 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.320+0000 7f64f5800640 1 --2- 172.21.15.16:0/1982199346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 0x55dd4050c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.320+0000 7f64f6200640 1 --2- 172.21.15.16:0/1982199346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401681b0 0x55dd40517a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.321+0000 7f64f4e00640 1 -- 172.21.15.16:0/1982199346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 msgr2=0x55dd4050c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.321+0000 7f64f4e00640 1 --2- 172.21.15.16:0/1982199346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 0x55dd4050c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.321+0000 7f64f4e00640 1 -- 172.21.15.16:0/1982199346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401681b0 msgr2=0x55dd40517a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.321+0000 7f64f4e00640 1 --2- 172.21.15.16:0/1982199346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401681b0 0x55dd40517a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.321+0000 7f64f4e00640 1 -- 172.21.15.16:0/1982199346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd40516a60 con 0x55dd401474d0 2024-09-21T03:46:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.321+0000 7f64f5800640 1 --2- 172.21.15.16:0/1982199346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 0x55dd4050c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.321+0000 7f64f4e00640 1 --2- 172.21.15.16:0/1982199346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401474d0 0x55dd40520960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00c6a0 tx=0x7f64dc0eb4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa4224cf103247d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.321+0000 7f64e2000640 1 -- 172.21.15.16:0/1982199346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0a96c0 con 0x55dd401474d0 2024-09-21T03:46:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.321+0000 7f64e2000640 1 -- 172.21.15.16:0/1982199346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64dc0ad330 con 0x55dd401474d0 2024-09-21T03:46:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.321+0000 7f64e2000640 1 -- 172.21.15.16:0/1982199346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc005180 con 0x55dd401474d0 2024-09-21T03:46:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.322+0000 7f64ff25e740 1 -- 172.21.15.16:0/1982199346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401474d0 msgr2=0x55dd40520960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.322+0000 7f64ff25e740 1 --2- 172.21.15.16:0/1982199346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401474d0 0x55dd40520960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00c6a0 tx=0x7f64dc0eb4e0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.322+0000 7f64ff25e740 1 -- 172.21.15.16:0/1982199346 shutdown_connections 2024-09-21T03:46:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.322+0000 7f64ff25e740 1 --2- 172.21.15.16:0/1982199346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401681b0 0x55dd40517a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.322+0000 7f64ff25e740 1 --2- 172.21.15.16:0/1982199346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401474d0 0x55dd40520960 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.322+0000 7f64ff25e740 1 --2- 172.21.15.16:0/1982199346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 0x55dd4050c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.322+0000 7f64ff25e740 1 -- 172.21.15.16:0/1982199346 >> 172.21.15.16:0/1982199346 conn(0x55dd4015c300 msgr2=0x55dd40516de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.322+0000 7f64ff25e740 1 -- 172.21.15.16:0/1982199346 shutdown_connections 2024-09-21T03:46:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.322+0000 7f64ff25e740 1 -- 172.21.15.16:0/1982199346 wait complete. 2024-09-21T03:46:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.323+0000 7f64ff25e740 1 -- start start 2024-09-21T03:46:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.323+0000 7f64ff25e740 1 Processor -- start 2024-09-21T03:46:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.323+0000 7f64ff25e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 0x55dd4051da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.323+0000 7f64ff25e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401474d0 0x55dd4051e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.323+0000 7f64ff25e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401681b0 0x55dd4051ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.323+0000 7f64ff25e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64f01afc00 con 0x55dd401474d0 2024-09-21T03:46:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.323+0000 7f64ff25e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64f01af530 con 0x55dd401681b0 2024-09-21T03:46:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.323+0000 7f64ff25e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64f01ac770 con 0x55dd4013f2f0 2024-09-21T03:46:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.323+0000 7f64ff25e740 10 client.0 _ll_register_callbacks cb 0x7f64e8009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.323+0000 7f64f4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401474d0 0x55dd4051e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.324+0000 7f64f4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401474d0 0x55dd4051e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41986/0 (socket says 172.21.15.16:41986) 2024-09-21T03:46:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.324+0000 7f64f4e00640 1 -- 172.21.15.16:0/2051882431 learned_addr learned my addr 172.21.15.16:0/2051882431 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.324+0000 7f64f5800640 1 --2- 172.21.15.16:0/2051882431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 0x55dd4051da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.324+0000 7f64f6200640 1 --2- 172.21.15.16:0/2051882431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401681b0 0x55dd4051ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.324+0000 7f64f4e00640 1 -- 172.21.15.16:0/2051882431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 msgr2=0x55dd4051da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.324+0000 7f64f4e00640 1 --2- 172.21.15.16:0/2051882431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 0x55dd4051da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.324+0000 7f64f4e00640 1 -- 172.21.15.16:0/2051882431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401681b0 msgr2=0x55dd4051ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.324+0000 7f64f4e00640 1 --2- 172.21.15.16:0/2051882431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401681b0 0x55dd4051ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.324+0000 7f64f4e00640 1 -- 172.21.15.16:0/2051882431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd4051f410 con 0x55dd401474d0 2024-09-21T03:46:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.324+0000 7f64f4e00640 1 --2- 172.21.15.16:0/2051882431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401474d0 0x55dd4051e140 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0a9da0 tx=0x7f64dc0b21c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a045650a56c6047a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.324+0000 7f64d7400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:46:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.324+0000 7f64d7400640 1 -- 172.21.15.16:0/2051882431 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0acf90 con 0x55dd401474d0 2024-09-21T03:46:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.324+0000 7f64d7400640 1 -- 172.21.15.16:0/2051882431 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f64dc0aed50 con 0x55dd401474d0 2024-09-21T03:46:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.324+0000 7f64ff25e740 1 -- 172.21.15.16:0/2051882431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64dc0ad330 con 0x55dd401474d0 2024-09-21T03:46:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.325+0000 7f64d7400640 1 -- 172.21.15.16:0/2051882431 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f64dc0ad170 con 0x55dd401474d0 2024-09-21T03:46:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.325+0000 7f64ff25e740 10 client.15576 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.325+0000 7f64ff25e740 1 -- 172.21.15.16:0/2051882431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dd40520780 con 0x55dd401474d0 2024-09-21T03:46:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.325+0000 7f64d7400640 1 -- 172.21.15.16:0/2051882431 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64dc0ad330 con 0x55dd401474d0 2024-09-21T03:46:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.325+0000 7f64d7400640 1 -- 172.21.15.16:0/2051882431 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f64dc0ac4f0 con 0x55dd401474d0 2024-09-21T03:46:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.325+0000 7f64ff25e740 10 client.15576 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.325+0000 7f64ff25e740 10 client.15576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.325+0000 7f64ff25e740 20 client.15576 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.325+0000 7f64ff25e740 1 --2- 172.21.15.16:0/2051882431 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55dd400fc1c0 0x55dd40592800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.325+0000 7f64ff25e740 4 client.15576 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:46:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.325+0000 7f64ff25e740 1 -- 172.21.15.16:0/2051882431 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd404800b0 con 0x55dd400fc1c0 2024-09-21T03:46:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.326+0000 7f64f5800640 1 --2- 172.21.15.16:0/2051882431 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55dd400fc1c0 0x55dd40592800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.326+0000 7f64f5800640 1 --2- 172.21.15.16:0/2051882431 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55dd400fc1c0 0x55dd40592800 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab35629fcee8dd69 server_cookie=7fbf43aa4ceb9a11 in_seq=0 out_seq=0 2024-09-21T03:46:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.326+0000 7f64d7400640 10 client.15576 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.334+0000 7f64d7400640 1 -- 172.21.15.16:0/2051882431 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd404800b0 con 0x55dd400fc1c0 2024-09-21T03:46:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.334+0000 7f64d7400640 10 client.15576 handle_command_reply: tid=0 2024-09-21T03:46:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.334+0000 7f64ff25e740 1 client.15576 shutdown 2024-09-21T03:46:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.334+0000 7f64ff25e740 1 -- 172.21.15.16:0/2051882431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401474d0 msgr2=0x55dd4051e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.334+0000 7f64ff25e740 1 --2- 172.21.15.16:0/2051882431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401474d0 0x55dd4051e140 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0a9da0 tx=0x7f64dc0b21c0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.334+0000 7f64ff25e740 1 -- 172.21.15.16:0/2051882431 shutdown_connections 2024-09-21T03:46:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.334+0000 7f64ff25e740 1 --2- 172.21.15.16:0/2051882431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd401681b0 0x55dd4051ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.334+0000 7f64ff25e740 1 --2- 172.21.15.16:0/2051882431 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55dd400fc1c0 0x55dd40592800 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.334+0000 7f64ff25e740 1 --2- 172.21.15.16:0/2051882431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd401474d0 0x55dd4051e140 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.334+0000 7f64ff25e740 1 --2- 172.21.15.16:0/2051882431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd4013f2f0 0x55dd4051da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.334+0000 7f64d7400640 0 client.15576 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.334+0000 7f64ff25e740 1 -- 172.21.15.16:0/2051882431 >> 172.21.15.16:0/2051882431 conn(0x55dd4015c300 msgr2=0x55dd40513310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.334+0000 7f64ff25e740 1 -- 172.21.15.16:0/2051882431 shutdown_connections 2024-09-21T03:46:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.334+0000 7f64ff25e740 1 -- 172.21.15.16:0/2051882431 wait complete. 2024-09-21T03:46:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.335+0000 7f64ff25e740 20 client.15576 trim_cache size 0 max 16384 2024-09-21T03:46:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:46:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:46:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:46:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:46:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.335+0000 7f64f6c00640 1 -- 172.21.15.16:0/4033326031 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64d80b6590 msgr2=0x7f64d80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.335+0000 7f64f6c00640 1 --2- 172.21.15.16:0/4033326031 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64d80b6590 0x7f64d80b8aa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f64e8005fd0 tx=0x7f64e8010040 comp rx=0 tx=0).stop 2024-09-21T03:46:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.335+0000 7f64f6c00640 1 -- 172.21.15.16:0/4033326031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f0104560 msgr2=0x7f64f019c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.335+0000 7f64f6c00640 1 --2- 172.21.15.16:0/4033326031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f0104560 0x7f64f019c640 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00e9f0 tx=0x7f64dc00edf0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.335+0000 7f64f6c00640 1 -- 172.21.15.16:0/4033326031 shutdown_connections 2024-09-21T03:46:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.335+0000 7f64f6c00640 1 --2- 172.21.15.16:0/4033326031 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64d80b6590 0x7f64d80b8aa0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.335+0000 7f64f6c00640 1 --2- 172.21.15.16:0/4033326031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f0104f00 0x7f64f019d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.335+0000 7f64f6c00640 1 --2- 172.21.15.16:0/4033326031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f0104560 0x7f64f019c640 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.335+0000 7f64f6c00640 1 --2- 172.21.15.16:0/4033326031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64f01034e0 0x7f64f019bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.335+0000 7f64f6c00640 1 -- 172.21.15.16:0/4033326031 >> 172.21.15.16:0/4033326031 conn(0x7f64f00fee20 msgr2=0x7f64f00755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.336+0000 7f64f6c00640 1 -- 172.21.15.16:0/4033326031 shutdown_connections 2024-09-21T03:46:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:41.336+0000 7f64f6c00640 1 -- 172.21.15.16:0/4033326031 wait complete. 2024-09-21T03:46:42.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:42 smithi123 ceph-mon[34673]: pgmap v430: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-09-21T03:46:42.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:42 smithi016 ceph-mon[30375]: pgmap v430: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-09-21T03:46:43.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:42 smithi134 ceph-mon[35031]: pgmap v430: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-09-21T03:46:44.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:44 smithi123 ceph-mon[34673]: pgmap v431: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-09-21T03:46:44.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:44 smithi016 ceph-mon[30375]: pgmap v431: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-09-21T03:46:45.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:44 smithi134 ceph-mon[35031]: pgmap v431: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-09-21T03:46:46.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:46 smithi123 ceph-mon[34673]: pgmap v432: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:46:46.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:46 smithi016 ceph-mon[30375]: pgmap v432: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:46:47.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:46 smithi134 ceph-mon[35031]: pgmap v432: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:46:48.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:48 smithi123 ceph-mon[34673]: pgmap v433: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-21T03:46:48.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:48 smithi016 ceph-mon[30375]: pgmap v433: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-21T03:46:49.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:48 smithi134 ceph-mon[35031]: pgmap v433: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-21T03:46:50.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:50 smithi016 ceph-mon[30375]: pgmap v434: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T03:46:51.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:50 smithi134 ceph-mon[35031]: pgmap v434: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T03:46:51.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:50 smithi123 ceph-mon[34673]: pgmap v434: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T03:46:52.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:52 smithi016 ceph-mon[30375]: pgmap v435: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-21T03:46:53.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:52 smithi134 ceph-mon[35031]: pgmap v435: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-21T03:46:53.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:52 smithi123 ceph-mon[34673]: pgmap v435: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-21T03:46:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.536+0000 7fccbda00640 1 -- 172.21.15.16:0/1134062078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb8104f00 msgr2=0x7fccb810b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.536+0000 7fccbda00640 1 --2- 172.21.15.16:0/1134062078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb8104f00 0x7fccb810b8a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009be0 tx=0x7fccb002a6c0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.536+0000 7fccbda00640 1 -- 172.21.15.16:0/1134062078 shutdown_connections 2024-09-21T03:46:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.536+0000 7fccbda00640 1 --2- 172.21.15.16:0/1134062078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb8104f00 0x7fccb810b8a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.536+0000 7fccbda00640 1 --2- 172.21.15.16:0/1134062078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb8104560 0x7fccb8104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.536+0000 7fccbda00640 1 --2- 172.21.15.16:0/1134062078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccb81034e0 0x7fccb8103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.536+0000 7fccbda00640 1 -- 172.21.15.16:0/1134062078 >> 172.21.15.16:0/1134062078 conn(0x7fccb80fee20 msgr2=0x7fccb81012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.536+0000 7fccbda00640 1 -- 172.21.15.16:0/1134062078 shutdown_connections 2024-09-21T03:46:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.536+0000 7fccbda00640 1 -- 172.21.15.16:0/1134062078 wait complete. 2024-09-21T03:46:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.538+0000 7fccbda00640 1 Processor -- start 2024-09-21T03:46:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.538+0000 7fccbda00640 1 -- start start 2024-09-21T03:46:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.538+0000 7fccbda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccb81034e0 0x7fccb819bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.538+0000 7fccbda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb8104560 0x7fccb819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.538+0000 7fccbda00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb8104f00 0x7fccb819d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.538+0000 7fccbda00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fccb810f100 con 0x7fccb8104560 2024-09-21T03:46:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.538+0000 7fccbda00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fccb810ef80 con 0x7fccb8104f00 2024-09-21T03:46:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.538+0000 7fccbda00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fccb810f280 con 0x7fccb81034e0 2024-09-21T03:46:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.539+0000 7fccb7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccb81034e0 0x7fccb819bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.539+0000 7fccb7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb8104560 0x7fccb819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.539+0000 7fccbd000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb8104f00 0x7fccb819d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.539+0000 7fccb7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccb81034e0 0x7fccb819bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:44704/0 (socket says 172.21.15.16:44704) 2024-09-21T03:46:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.539+0000 7fccbd000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb8104f00 0x7fccb819d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37374/0 (socket says 172.21.15.16:37374) 2024-09-21T03:46:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.539+0000 7fccb7e00640 1 -- 172.21.15.16:0/3226119751 learned_addr learned my addr 172.21.15.16:0/3226119751 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.539+0000 7fccbd000640 1 -- 172.21.15.16:0/3226119751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccb81034e0 msgr2=0x7fccb819bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.539+0000 7fccbd000640 1 --2- 172.21.15.16:0/3226119751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccb81034e0 0x7fccb819bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.539+0000 7fccbd000640 1 -- 172.21.15.16:0/3226119751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb8104560 msgr2=0x7fccb819c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.539+0000 7fccbd000640 1 --2- 172.21.15.16:0/3226119751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb8104560 0x7fccb819c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.539+0000 7fccbd000640 1 -- 172.21.15.16:0/3226119751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb819d9c0 con 0x7fccb8104f00 2024-09-21T03:46:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.539+0000 7fccb7e00640 1 --2- 172.21.15.16:0/3226119751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccb81034e0 0x7fccb819bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.539+0000 7fccb7400640 1 --2- 172.21.15.16:0/3226119751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb8104560 0x7fccb819c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.539+0000 7fccbd000640 1 --2- 172.21.15.16:0/3226119751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb8104f00 0x7fccb819d1a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fccb002abe0 tx=0x7fccb0030d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.540+0000 7fccb4c00640 1 -- 172.21.15.16:0/3226119751 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0041020 con 0x7fccb8104f00 2024-09-21T03:46:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.540+0000 7fccb4c00640 1 -- 172.21.15.16:0/3226119751 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccb00318f0 con 0x7fccb8104f00 2024-09-21T03:46:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.540+0000 7fccb4c00640 1 -- 172.21.15.16:0/3226119751 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0036040 con 0x7fccb8104f00 2024-09-21T03:46:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.540+0000 7fccbda00640 1 -- 172.21.15.16:0/3226119751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb819dc50 con 0x7fccb8104f00 2024-09-21T03:46:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.540+0000 7fccbda00640 1 -- 172.21.15.16:0/3226119751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb81a41c0 con 0x7fccb8104f00 2024-09-21T03:46:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.544+0000 7fccc61e8740 1 Processor -- start 2024-09-21T03:46:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.545+0000 7fccb4c00640 1 -- 172.21.15.16:0/3226119751 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fccb0044020 con 0x7fccb8104f00 2024-09-21T03:46:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.545+0000 7fccc61e8740 1 -- start start 2024-09-21T03:46:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.545+0000 7fccc61e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef599771b0 0x55ef59d1e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.545+0000 7fccc61e8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef5994e2f0 0x55ef59d1e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.545+0000 7fccc61e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599564d0 0x55ef59d1f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.545+0000 7fccc61e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ef59c916b0 con 0x55ef599564d0 2024-09-21T03:46:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.545+0000 7fccc61e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef59c80ae0 con 0x55ef599771b0 2024-09-21T03:46:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.545+0000 7fccc61e8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ef59c848f0 con 0x55ef5994e2f0 2024-09-21T03:46:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.546+0000 7fccb7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef5994e2f0 0x55ef59d1e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.546+0000 7fccb7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef599771b0 0x55ef59d1e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.546+0000 7fccb7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef5994e2f0 0x55ef59d1e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:44712/0 (socket says 172.21.15.16:44712) 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.546+0000 7fccb7e00640 1 -- 172.21.15.16:0/824727536 learned_addr learned my addr 172.21.15.16:0/824727536 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.546+0000 7fccb7400640 1 --2- 172.21.15.16:0/824727536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599564d0 0x55ef59d1f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.546+0000 7fccb7e00640 1 -- 172.21.15.16:0/824727536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef599771b0 msgr2=0x55ef59d1e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.546+0000 7fccb7e00640 1 --2- 172.21.15.16:0/824727536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef599771b0 0x55ef59d1e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.546+0000 7fccb7e00640 1 -- 172.21.15.16:0/824727536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599564d0 msgr2=0x55ef59d1f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.546+0000 7fccb7e00640 1 --2- 172.21.15.16:0/824727536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599564d0 0x55ef59d1f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.546+0000 7fccb7e00640 1 -- 172.21.15.16:0/824727536 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef59c4d780 con 0x55ef5994e2f0 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.546+0000 7fccb7400640 1 --2- 172.21.15.16:0/824727536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599564d0 0x55ef59d1f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.547+0000 7fccb4c00640 1 --2- 172.21.15.16:0/3226119751 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcc9c0b6510 0x7fcc9c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.547+0000 7fccb7e00640 1 --2- 172.21.15.16:0/824727536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef5994e2f0 0x55ef59d1e920 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcca8011650 tx=0x7fcca8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f679ddd59bc22f2c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.547+0000 7fccb4c00640 1 -- 172.21.15.16:0/3226119751 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fccb01082d0 con 0x7fccb8104f00 2024-09-21T03:46:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.547+0000 7fccad600640 1 -- 172.21.15.16:0/824727536 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca801e070 con 0x55ef5994e2f0 2024-09-21T03:46:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.547+0000 7fccb7400640 1 --2- 172.21.15.16:0/3226119751 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcc9c0b6510 0x7fcc9c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.547+0000 7fccad600640 1 -- 172.21.15.16:0/824727536 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca800fb70 con 0x55ef5994e2f0 2024-09-21T03:46:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.547+0000 7fccad600640 1 -- 172.21.15.16:0/824727536 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8019c10 con 0x55ef5994e2f0 2024-09-21T03:46:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.548+0000 7fccc61e8740 1 -- 172.21.15.16:0/824727536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef5994e2f0 msgr2=0x55ef59d1e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.548+0000 7fccc61e8740 1 --2- 172.21.15.16:0/824727536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef5994e2f0 0x55ef59d1e920 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcca8011650 tx=0x7fcca8009f90 comp rx=0 tx=0).stop 2024-09-21T03:46:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.548+0000 7fccb7400640 1 --2- 172.21.15.16:0/3226119751 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcc9c0b6510 0x7fcc9c0b8a20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcca40096c0 tx=0x7fcca4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.548+0000 7fccc61e8740 1 -- 172.21.15.16:0/824727536 shutdown_connections 2024-09-21T03:46:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.548+0000 7fccc61e8740 1 --2- 172.21.15.16:0/824727536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599564d0 0x55ef59d1f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.548+0000 7fccc61e8740 1 --2- 172.21.15.16:0/824727536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef5994e2f0 0x55ef59d1e920 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.548+0000 7fccc61e8740 1 --2- 172.21.15.16:0/824727536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef599771b0 0x55ef59d1e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.548+0000 7fccc61e8740 1 -- 172.21.15.16:0/824727536 >> 172.21.15.16:0/824727536 conn(0x55ef5996b300 msgr2=0x55ef59d19d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.548+0000 7fccc61e8740 1 -- 172.21.15.16:0/824727536 shutdown_connections 2024-09-21T03:46:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.548+0000 7fccc61e8740 1 -- 172.21.15.16:0/824727536 wait complete. 2024-09-21T03:46:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.549+0000 7fccc61e8740 1 -- start start 2024-09-21T03:46:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.549+0000 7fccc61e8740 1 Processor -- start 2024-09-21T03:46:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.549+0000 7fccc61e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef5994e2f0 0x55ef59d27950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.549+0000 7fccc61e8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef599564d0 0x55ef59d28010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.549+0000 7fccc61e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef599771b0 0x55ef59d28b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.550+0000 7fccc61e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ef59bb8ab0 con 0x55ef5994e2f0 2024-09-21T03:46:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.550+0000 7fccc61e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef59c27570 con 0x55ef599771b0 2024-09-21T03:46:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.550+0000 7fccc61e8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ef59d1ff90 con 0x55ef599564d0 2024-09-21T03:46:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.550+0000 7fccc61e8740 10 client.0 _ll_register_callbacks cb 0x55ef59c36100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.550+0000 7fccb7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef5994e2f0 0x55ef59d27950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.550+0000 7fccb7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef5994e2f0 0x55ef59d27950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38222/0 (socket says 172.21.15.16:38222) 2024-09-21T03:46:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.550+0000 7fccb7e00640 1 -- 172.21.15.16:0/1588121451 learned_addr learned my addr 172.21.15.16:0/1588121451 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.550+0000 7fccb7400640 1 --2- 172.21.15.16:0/1588121451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef599564d0 0x55ef59d28010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.550+0000 7fccbd000640 1 --2- 172.21.15.16:0/1588121451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef599771b0 0x55ef59d28b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.550+0000 7fccbd000640 1 -- 172.21.15.16:0/1588121451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef599564d0 msgr2=0x55ef59d28010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.550+0000 7fccbd000640 1 --2- 172.21.15.16:0/1588121451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef599564d0 0x55ef59d28010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.550+0000 7fccbd000640 1 -- 172.21.15.16:0/1588121451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef5994e2f0 msgr2=0x55ef59d27950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.550+0000 7fccbd000640 1 --2- 172.21.15.16:0/1588121451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef5994e2f0 0x55ef59d27950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.550+0000 7fccbd000640 1 -- 172.21.15.16:0/1588121451 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef599e2790 con 0x55ef599771b0 2024-09-21T03:46:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.550+0000 7fccb7e00640 1 --2- 172.21.15.16:0/1588121451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef5994e2f0 0x55ef59d27950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.551+0000 7fccbd000640 1 --2- 172.21.15.16:0/1588121451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef599771b0 0x55ef59d28b80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fccb00cb2c0 tx=0x7fccb00cde20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a76e0bf1b5f6311 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.551+0000 7fcca2000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:46:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.551+0000 7fcca2000640 1 -- 172.21.15.16:0/1588121451 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb00cdb60 con 0x55ef599771b0 2024-09-21T03:46:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.551+0000 7fcca2000640 1 -- 172.21.15.16:0/1588121451 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fccb0004b40 con 0x55ef599771b0 2024-09-21T03:46:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.551+0000 7fccc61e8740 1 -- 172.21.15.16:0/1588121451 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcca800fb70 con 0x55ef599771b0 2024-09-21T03:46:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.551+0000 7fcca2000640 1 -- 172.21.15.16:0/1588121451 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fccb00ce750 con 0x55ef599771b0 2024-09-21T03:46:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.551+0000 7fccc61e8740 10 client.25595 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.551+0000 7fccc61e8740 1 -- 172.21.15.16:0/1588121451 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ef59a5b620 con 0x55ef599771b0 2024-09-21T03:46:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.552+0000 7fcca2000640 1 -- 172.21.15.16:0/1588121451 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca800fb70 con 0x55ef599771b0 2024-09-21T03:46:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.552+0000 7fcca2000640 1 -- 172.21.15.16:0/1588121451 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fccb00b91d0 con 0x55ef599771b0 2024-09-21T03:46:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.552+0000 7fccc61e8740 10 client.25595 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.552+0000 7fccc61e8740 10 client.25595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.552+0000 7fccc61e8740 20 client.25595 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.552+0000 7fccc61e8740 1 --2- 172.21.15.16:0/1588121451 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55ef59935d80 0x55ef59d35140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.552+0000 7fccc61e8740 4 client.25595 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.552+0000 7fccc61e8740 1 -- 172.21.15.16:0/1588121451 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef59c93600 con 0x55ef59935d80 2024-09-21T03:46:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.552+0000 7fccb7e00640 1 --2- 172.21.15.16:0/1588121451 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55ef59935d80 0x55ef59d35140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.553+0000 7fcca2000640 10 client.25595 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.553+0000 7fccb7e00640 1 --2- 172.21.15.16:0/1588121451 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55ef59935d80 0x55ef59d35140 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f33bac97d8957792 server_cookie=2bca5c5ab5c1b1e0 in_seq=0 out_seq=0 2024-09-21T03:46:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.554+0000 7fcca2000640 1 -- 172.21.15.16:0/1588121451 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ef59c93600 con 0x55ef59935d80 2024-09-21T03:46:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.554+0000 7fcca2000640 10 client.25595 handle_command_reply: tid=0 2024-09-21T03:46:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.554+0000 7fccc61e8740 1 client.25595 shutdown 2024-09-21T03:46:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.554+0000 7fccc61e8740 1 -- 172.21.15.16:0/1588121451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef599771b0 msgr2=0x55ef59d28b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.554+0000 7fccc61e8740 1 --2- 172.21.15.16:0/1588121451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef599771b0 0x55ef59d28b80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fccb00cb2c0 tx=0x7fccb00cde20 comp rx=0 tx=0).stop 2024-09-21T03:46:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.554+0000 7fccc61e8740 1 -- 172.21.15.16:0/1588121451 shutdown_connections 2024-09-21T03:46:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.554+0000 7fccc61e8740 1 --2- 172.21.15.16:0/1588121451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef599771b0 0x55ef59d28b80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.554+0000 7fccc61e8740 1 --2- 172.21.15.16:0/1588121451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef599564d0 0x55ef59d28010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.554+0000 7fccc61e8740 1 --2- 172.21.15.16:0/1588121451 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55ef59935d80 0x55ef59d35140 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.554+0000 7fccc61e8740 1 --2- 172.21.15.16:0/1588121451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef5994e2f0 0x55ef59d27950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.554+0000 7fccc61e8740 1 -- 172.21.15.16:0/1588121451 >> 172.21.15.16:0/1588121451 conn(0x55ef5996b300 msgr2=0x55ef59d18360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.554+0000 7fcca2000640 0 client.25595 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.554+0000 7fccc61e8740 1 -- 172.21.15.16:0/1588121451 shutdown_connections 2024-09-21T03:46:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.554+0000 7fccc61e8740 1 -- 172.21.15.16:0/1588121451 wait complete. 2024-09-21T03:46:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.554+0000 7fccc61e8740 20 client.25595 trim_cache size 0 max 16384 2024-09-21T03:46:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.573+0000 7fccc61e8740 1 Processor -- start 2024-09-21T03:46:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.573+0000 7fccc61e8740 1 -- start start 2024-09-21T03:46:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.573+0000 7fccc61e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef5994e2f0 0x55ef59d14ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.573+0000 7fccc61e8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef599564d0 0x55ef59d1dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.573+0000 7fccc61e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599771b0 0x55ef59d1e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.573+0000 7fccc61e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81ae770 con 0x55ef599771b0 2024-09-21T03:46:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.573+0000 7fccc61e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81b02b0 con 0x55ef5994e2f0 2024-09-21T03:46:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.573+0000 7fccc61e8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81aee40 con 0x55ef599564d0 2024-09-21T03:46:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.573+0000 7fccb7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef5994e2f0 0x55ef59d14ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.573+0000 7fccbd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599771b0 0x55ef59d1e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.573+0000 7fccb7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef5994e2f0 0x55ef59d14ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37406/0 (socket says 172.21.15.16:37406) 2024-09-21T03:46:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.573+0000 7fccb7e00640 1 -- 172.21.15.16:0/848052052 learned_addr learned my addr 172.21.15.16:0/848052052 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.573+0000 7fccb7400640 1 --2- 172.21.15.16:0/848052052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef599564d0 0x55ef59d1dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.573+0000 7fccbd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599771b0 0x55ef59d1e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38238/0 (socket says 172.21.15.16:38238) 2024-09-21T03:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.574+0000 7fccb7e00640 1 -- 172.21.15.16:0/848052052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef599564d0 msgr2=0x55ef59d1dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.574+0000 7fccb7e00640 1 --2- 172.21.15.16:0/848052052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef599564d0 0x55ef59d1dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.574+0000 7fccb7e00640 1 -- 172.21.15.16:0/848052052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599771b0 msgr2=0x55ef59d1e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.574+0000 7fccb7e00640 1 --2- 172.21.15.16:0/848052052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599771b0 0x55ef59d1e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.574+0000 7fccb7e00640 1 -- 172.21.15.16:0/848052052 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef59d25ae0 con 0x55ef5994e2f0 2024-09-21T03:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.574+0000 7fccb7e00640 1 --2- 172.21.15.16:0/848052052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef5994e2f0 0x55ef59d14ae0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcca8000bc0 tx=0x7fcca80056f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52900e5614fb8225 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.574+0000 7fccacc00640 1 -- 172.21.15.16:0/848052052 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca801e040 con 0x55ef5994e2f0 2024-09-21T03:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.574+0000 7fccacc00640 1 -- 172.21.15.16:0/848052052 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca8029030 con 0x55ef5994e2f0 2024-09-21T03:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.574+0000 7fccacc00640 1 -- 172.21.15.16:0/848052052 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8019aa0 con 0x55ef5994e2f0 2024-09-21T03:46:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.575+0000 7fccc61e8740 1 -- 172.21.15.16:0/848052052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef5994e2f0 msgr2=0x55ef59d14ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.576+0000 7fccc61e8740 1 --2- 172.21.15.16:0/848052052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef5994e2f0 0x55ef59d14ae0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcca8000bc0 tx=0x7fcca80056f0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.576+0000 7fccc61e8740 1 -- 172.21.15.16:0/848052052 shutdown_connections 2024-09-21T03:46:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.576+0000 7fccc61e8740 1 --2- 172.21.15.16:0/848052052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599771b0 0x55ef59d1e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.576+0000 7fccc61e8740 1 --2- 172.21.15.16:0/848052052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef599564d0 0x55ef59d1dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.576+0000 7fccc61e8740 1 --2- 172.21.15.16:0/848052052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef5994e2f0 0x55ef59d14ae0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.576+0000 7fccc61e8740 1 -- 172.21.15.16:0/848052052 >> 172.21.15.16:0/848052052 conn(0x55ef5996b300 msgr2=0x55ef59d26b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.576+0000 7fccc61e8740 1 -- 172.21.15.16:0/848052052 shutdown_connections 2024-09-21T03:46:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.576+0000 7fccc61e8740 1 -- 172.21.15.16:0/848052052 wait complete. 2024-09-21T03:46:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.576+0000 7fccc61e8740 1 -- start start 2024-09-21T03:46:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.576+0000 7fccc61e8740 1 Processor -- start 2024-09-21T03:46:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.577+0000 7fccc61e8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef5994e2f0 0x55ef59d2f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.577+0000 7fccc61e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef599564d0 0x55ef59d2e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.577+0000 7fccc61e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599771b0 0x55ef59d2e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.577+0000 7fccc61e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81afbe0 con 0x55ef599771b0 2024-09-21T03:46:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.577+0000 7fccc61e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81af510 con 0x55ef599564d0 2024-09-21T03:46:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.577+0000 7fccc61e8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81ac750 con 0x55ef5994e2f0 2024-09-21T03:46:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.577+0000 7fccc61e8740 10 client.0 _ll_register_callbacks cb 0x7fccb00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.577+0000 7fccbd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599771b0 0x55ef59d2e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.577+0000 7fccb7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef599564d0 0x55ef59d2e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.577+0000 7fccbd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599771b0 0x55ef59d2e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38244/0 (socket says 172.21.15.16:38244) 2024-09-21T03:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.577+0000 7fccb7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef599564d0 0x55ef59d2e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37418/0 (socket says 172.21.15.16:37418) 2024-09-21T03:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.577+0000 7fccbd000640 1 -- 172.21.15.16:0/931075587 learned_addr learned my addr 172.21.15.16:0/931075587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.577+0000 7fccb7e00640 1 --2- 172.21.15.16:0/931075587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef5994e2f0 0x55ef59d2f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.578+0000 7fccbd000640 1 -- 172.21.15.16:0/931075587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef5994e2f0 msgr2=0x55ef59d2f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.578+0000 7fccbd000640 1 --2- 172.21.15.16:0/931075587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef5994e2f0 0x55ef59d2f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.578+0000 7fccbd000640 1 -- 172.21.15.16:0/931075587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef599564d0 msgr2=0x55ef59d2e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.578+0000 7fccbd000640 1 --2- 172.21.15.16:0/931075587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef599564d0 0x55ef59d2e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.578+0000 7fccbd000640 1 -- 172.21.15.16:0/931075587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef59d2f120 con 0x55ef599771b0 2024-09-21T03:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.578+0000 7fccbd000640 1 --2- 172.21.15.16:0/931075587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599771b0 0x55ef59d2e990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fccb003c040 tx=0x7fccb0003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd073169f267945 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.578+0000 7fcca2000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.578+0000 7fcca2000640 1 -- 172.21.15.16:0/931075587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb00cd9a0 con 0x55ef599771b0 2024-09-21T03:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.578+0000 7fcca2000640 1 -- 172.21.15.16:0/931075587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fccb0004510 con 0x55ef599771b0 2024-09-21T03:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.578+0000 7fccc61e8740 1 -- 172.21.15.16:0/931075587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcca8029030 con 0x55ef599771b0 2024-09-21T03:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.578+0000 7fcca2000640 1 -- 172.21.15.16:0/931075587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fccb00cdb60 con 0x55ef599771b0 2024-09-21T03:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.579+0000 7fccc61e8740 10 client.15594 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.579+0000 7fccc61e8740 1 -- 172.21.15.16:0/931075587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ef59d9fbb0 con 0x55ef599771b0 2024-09-21T03:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.579+0000 7fcca2000640 1 -- 172.21.15.16:0/931075587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca8029030 con 0x55ef599771b0 2024-09-21T03:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.579+0000 7fcca2000640 1 -- 172.21.15.16:0/931075587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fccb0031ac0 con 0x55ef599771b0 2024-09-21T03:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.579+0000 7fccc61e8740 10 client.15594 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.579+0000 7fccc61e8740 10 client.15594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.579+0000 7fccc61e8740 20 client.15594 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.579+0000 7fccc61e8740 1 --2- 172.21.15.16:0/931075587 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55ef5990b1c0 0x55ef59da2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.579+0000 7fccc61e8740 4 client.15594 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.579+0000 7fccc61e8740 1 -- 172.21.15.16:0/931075587 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef59d151c0 con 0x55ef5990b1c0 2024-09-21T03:46:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.580+0000 7fccb7e00640 1 --2- 172.21.15.16:0/931075587 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55ef5990b1c0 0x55ef59da2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.580+0000 7fccb7e00640 1 --2- 172.21.15.16:0/931075587 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55ef5990b1c0 0x55ef59da2440 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4e613bb069e3265 server_cookie=b14df45e249a941 in_seq=0 out_seq=0 2024-09-21T03:46:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.580+0000 7fcca2000640 10 client.15594 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.586+0000 7fcca2000640 1 -- 172.21.15.16:0/931075587 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef59d151c0 con 0x55ef5990b1c0 2024-09-21T03:46:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.586+0000 7fcca2000640 10 client.15594 handle_command_reply: tid=0 2024-09-21T03:46:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.586+0000 7fccc61e8740 1 client.15594 shutdown 2024-09-21T03:46:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.586+0000 7fccc61e8740 1 -- 172.21.15.16:0/931075587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599771b0 msgr2=0x55ef59d2e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.586+0000 7fccc61e8740 1 --2- 172.21.15.16:0/931075587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599771b0 0x55ef59d2e990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fccb003c040 tx=0x7fccb0003b70 comp rx=0 tx=0).stop 2024-09-21T03:46:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.586+0000 7fccc61e8740 1 -- 172.21.15.16:0/931075587 shutdown_connections 2024-09-21T03:46:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.586+0000 7fccc61e8740 1 --2- 172.21.15.16:0/931075587 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55ef5990b1c0 0x55ef59da2440 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.586+0000 7fccc61e8740 1 --2- 172.21.15.16:0/931075587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef599771b0 0x55ef59d2e990 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.586+0000 7fccc61e8740 1 --2- 172.21.15.16:0/931075587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef599564d0 0x55ef59d2e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.586+0000 7fccc61e8740 1 --2- 172.21.15.16:0/931075587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef5994e2f0 0x55ef59d2f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.586+0000 7fccc61e8740 1 -- 172.21.15.16:0/931075587 >> 172.21.15.16:0/931075587 conn(0x55ef5996b300 msgr2=0x55ef59d26b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.586+0000 7fcca2000640 0 client.15594 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.586+0000 7fccc61e8740 1 -- 172.21.15.16:0/931075587 shutdown_connections 2024-09-21T03:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.586+0000 7fccc61e8740 1 -- 172.21.15.16:0/931075587 wait complete. 2024-09-21T03:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.586+0000 7fccc61e8740 20 client.15594 trim_cache size 0 max 16384 2024-09-21T03:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.587+0000 7fccbda00640 1 -- 172.21.15.16:0/3226119751 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcc9c0b6510 msgr2=0x7fcc9c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.587+0000 7fccbda00640 1 --2- 172.21.15.16:0/3226119751 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcc9c0b6510 0x7fcc9c0b8a20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcca40096c0 tx=0x7fcca4009260 comp rx=0 tx=0).stop 2024-09-21T03:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.587+0000 7fccbda00640 1 -- 172.21.15.16:0/3226119751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb8104f00 msgr2=0x7fccb819d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.587+0000 7fccbda00640 1 --2- 172.21.15.16:0/3226119751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb8104f00 0x7fccb819d1a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fccb002abe0 tx=0x7fccb0030d20 comp rx=0 tx=0).stop 2024-09-21T03:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.587+0000 7fccbda00640 1 -- 172.21.15.16:0/3226119751 shutdown_connections 2024-09-21T03:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.587+0000 7fccbda00640 1 --2- 172.21.15.16:0/3226119751 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcc9c0b6510 0x7fcc9c0b8a20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.587+0000 7fccbda00640 1 --2- 172.21.15.16:0/3226119751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb8104f00 0x7fccb819d1a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.587+0000 7fccbda00640 1 --2- 172.21.15.16:0/3226119751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb8104560 0x7fccb819c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.587+0000 7fccbda00640 1 --2- 172.21.15.16:0/3226119751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccb81034e0 0x7fccb819bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.587+0000 7fccbda00640 1 -- 172.21.15.16:0/3226119751 >> 172.21.15.16:0/3226119751 conn(0x7fccb80fee20 msgr2=0x7fccb80755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.587+0000 7fccbda00640 1 -- 172.21.15.16:0/3226119751 shutdown_connections 2024-09-21T03:46:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:46:54.587+0000 7fccbda00640 1 -- 172.21.15.16:0/3226119751 wait complete. 2024-09-21T03:46:54.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:54 smithi016 ceph-mon[30375]: pgmap v436: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T03:46:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:54 smithi134 ceph-mon[35031]: pgmap v436: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T03:46:55.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:54 smithi123 ceph-mon[34673]: pgmap v436: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T03:46:55.719 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 14.5 2024-09-21T03:46:55.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:46:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:56.935 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:56 smithi016 ceph-mon[30375]: pgmap v437: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-21T03:46:57.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:56 smithi134 ceph-mon[35031]: pgmap v437: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-21T03:46:57.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.134+0000 7f63c5000640 1 -- 172.21.15.16:0/3854864326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63c010b500 msgr2=0x7f63c010b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:57.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.134+0000 7f63c5000640 1 --2- 172.21.15.16:0/3854864326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63c010b500 0x7f63c010b910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009900 tx=0x7f63ac02a410 comp rx=0 tx=0).stop 2024-09-21T03:46:57.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.134+0000 7f63c5000640 1 -- 172.21.15.16:0/3854864326 shutdown_connections 2024-09-21T03:46:57.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.134+0000 7f63c5000640 1 --2- 172.21.15.16:0/3854864326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c010be50 0x7f63c0112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.134+0000 7f63c5000640 1 --2- 172.21.15.16:0/3854864326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63c010b500 0x7f63c010b910 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.134+0000 7f63c5000640 1 --2- 172.21.15.16:0/3854864326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63c0105bc0 0x7f63c0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.134+0000 7f63c5000640 1 -- 172.21.15.16:0/3854864326 >> 172.21.15.16:0/3854864326 conn(0x7f63c0101630 msgr2=0x7f63c0103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:57.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.135+0000 7f63c5000640 1 -- 172.21.15.16:0/3854864326 shutdown_connections 2024-09-21T03:46:57.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.135+0000 7f63c5000640 1 -- 172.21.15.16:0/3854864326 wait complete. 2024-09-21T03:46:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.136+0000 7f63c5000640 1 Processor -- start 2024-09-21T03:46:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.136+0000 7f63c5000640 1 -- start start 2024-09-21T03:46:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.136+0000 7f63c5000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63c0105bc0 0x7f63c019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.136+0000 7f63c5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c010b500 0x7f63c019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.137+0000 7f63c5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63c010be50 0x7f63c01a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.137+0000 7f63c5000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f63c0115b10 con 0x7f63c010b500 2024-09-21T03:46:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.137+0000 7f63c5000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63c0115990 con 0x7f63c010be50 2024-09-21T03:46:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.137+0000 7f63c5000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f63c0115c90 con 0x7f63c0105bc0 2024-09-21T03:46:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.137+0000 7f63bf400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63c0105bc0 0x7f63c019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.137+0000 7f63bf400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63c0105bc0 0x7f63c019c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:44768/0 (socket says 172.21.15.16:44768) 2024-09-21T03:46:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.137+0000 7f63bf400640 1 -- 172.21.15.16:0/3862364522 learned_addr learned my addr 172.21.15.16:0/3862364522 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.137+0000 7f63bfe00640 1 --2- 172.21.15.16:0/3862364522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63c010be50 0x7f63c01a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.137+0000 7f63bf400640 1 -- 172.21.15.16:0/3862364522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63c010be50 msgr2=0x7f63c01a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.137+0000 7f63bf400640 1 --2- 172.21.15.16:0/3862364522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63c010be50 0x7f63c01a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.137+0000 7f63bf400640 1 -- 172.21.15.16:0/3862364522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c010b500 msgr2=0x7f63c019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.137+0000 7f63bf400640 1 --2- 172.21.15.16:0/3862364522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c010b500 0x7f63c019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.137+0000 7f63bf400640 1 -- 172.21.15.16:0/3862364522 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c01a2450 con 0x7f63c0105bc0 2024-09-21T03:46:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.138+0000 7f63bf400640 1 --2- 172.21.15.16:0/3862364522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63c0105bc0 0x7f63c019c4e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f63b000e9f0 tx=0x7f63b000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.138+0000 7f63b7e00640 1 -- 172.21.15.16:0/3862364522 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b000cd80 con 0x7f63c0105bc0 2024-09-21T03:46:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.138+0000 7f63c5000640 1 -- 172.21.15.16:0/3862364522 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c01a2740 con 0x7f63c0105bc0 2024-09-21T03:46:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.138+0000 7f63b7e00640 1 -- 172.21.15.16:0/3862364522 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63b00054d0 con 0x7f63c0105bc0 2024-09-21T03:46:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.138+0000 7f63c5000640 1 -- 172.21.15.16:0/3862364522 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c01a8ea0 con 0x7f63c0105bc0 2024-09-21T03:46:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.138+0000 7f63b7e00640 1 -- 172.21.15.16:0/3862364522 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0010690 con 0x7f63c0105bc0 2024-09-21T03:46:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.139+0000 7f63c5000640 1 -- 172.21.15.16:0/3862364522 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c0105fd0 con 0x7f63c0105bc0 2024-09-21T03:46:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.142+0000 7f63b7e00640 1 -- 172.21.15.16:0/3862364522 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f63b0004e30 con 0x7f63c0105bc0 2024-09-21T03:46:57.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.143+0000 7f63b7e00640 1 --2- 172.21.15.16:0/3862364522 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f63a40b6590 0x7f63a40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:57.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.144+0000 7f63b7e00640 1 -- 172.21.15.16:0/3862364522 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f63b00e7da0 con 0x7f63c0105bc0 2024-09-21T03:46:57.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.144+0000 7f63bea00640 1 --2- 172.21.15.16:0/3862364522 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f63a40b6590 0x7f63a40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:57.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.145+0000 7f63b7e00640 1 -- 172.21.15.16:0/3862364522 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f63b0056710 con 0x7f63c0105bc0 2024-09-21T03:46:57.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.145+0000 7f63bea00640 1 --2- 172.21.15.16:0/3862364522 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f63a40b6590 0x7f63a40b8aa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f63ac02a930 tx=0x7f63ac031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:57.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:56 smithi123 ceph-mon[34673]: pgmap v437: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-21T03:46:57.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.452+0000 7f63c5000640 1 -- 172.21.15.16:0/3862364522 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63c019d610 con 0x7f63c0105bc0 2024-09-21T03:46:57.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.453+0000 7f63b7e00640 1 -- 172.21.15.16:0/3862364522 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f63b00ae220 con 0x7f63c0105bc0 2024-09-21T03:46:57.455 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:46:57.455 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:46:57.456 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:46:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.456+0000 7f63c5000640 1 -- 172.21.15.16:0/3862364522 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f63a40b6590 msgr2=0x7f63a40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.456+0000 7f63c5000640 1 --2- 172.21.15.16:0/3862364522 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f63a40b6590 0x7f63a40b8aa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f63ac02a930 tx=0x7f63ac031400 comp rx=0 tx=0).stop 2024-09-21T03:46:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.456+0000 7f63c5000640 1 -- 172.21.15.16:0/3862364522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63c0105bc0 msgr2=0x7f63c019c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.456+0000 7f63c5000640 1 --2- 172.21.15.16:0/3862364522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63c0105bc0 0x7f63c019c4e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f63b000e9f0 tx=0x7f63b000edf0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.456+0000 7f63c5000640 1 -- 172.21.15.16:0/3862364522 shutdown_connections 2024-09-21T03:46:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.456+0000 7f63c5000640 1 --2- 172.21.15.16:0/3862364522 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f63a40b6590 0x7f63a40b8aa0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.456+0000 7f63c5000640 1 --2- 172.21.15.16:0/3862364522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63c010be50 0x7f63c01a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.456+0000 7f63c5000640 1 --2- 172.21.15.16:0/3862364522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c010b500 0x7f63c019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.456+0000 7f63c5000640 1 --2- 172.21.15.16:0/3862364522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63c0105bc0 0x7f63c019c4e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.456+0000 7f63c5000640 1 -- 172.21.15.16:0/3862364522 >> 172.21.15.16:0/3862364522 conn(0x7f63c0101630 msgr2=0x7f63c0103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.456+0000 7f63c5000640 1 -- 172.21.15.16:0/3862364522 shutdown_connections 2024-09-21T03:46:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:57.456+0000 7f63c5000640 1 -- 172.21.15.16:0/3862364522 wait complete. 2024-09-21T03:46:57.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:57 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3862364522' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:58.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:57 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3862364522' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:58.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:57 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3862364522' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:46:58.282 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0' 2024-09-21T03:46:58.282 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0 2024-09-21T03:46:58.726 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:46:59.033 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:46:58 smithi016 ceph-mon[30375]: pgmap v438: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:46:59.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:46:58 smithi134 ceph-mon[35031]: pgmap v438: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:46:59.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:46:58 smithi123 ceph-mon[34673]: pgmap v438: 129 pgs: 129 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:46:59.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.451+0000 7f1d8d400640 1 -- 172.21.15.16:0/2962021872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d88105bf0 msgr2=0x7f1d88106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.451+0000 7f1d8d400640 1 --2- 172.21.15.16:0/2962021872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d88105bf0 0x7f1d88106000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009900 tx=0x7f1d7802a400 comp rx=0 tx=0).stop 2024-09-21T03:46:59.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.451+0000 7f1d8d400640 1 -- 172.21.15.16:0/2962021872 shutdown_connections 2024-09-21T03:46:59.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.451+0000 7f1d8d400640 1 --2- 172.21.15.16:0/2962021872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8810be80 0x7f1d881126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.451+0000 7f1d8d400640 1 --2- 172.21.15.16:0/2962021872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d8810b530 0x7f1d8810b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.451+0000 7f1d8d400640 1 --2- 172.21.15.16:0/2962021872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d88105bf0 0x7f1d88106000 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.451+0000 7f1d8d400640 1 -- 172.21.15.16:0/2962021872 >> 172.21.15.16:0/2962021872 conn(0x7f1d88101680 msgr2=0x7f1d88103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:59.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.451+0000 7f1d8d400640 1 -- 172.21.15.16:0/2962021872 shutdown_connections 2024-09-21T03:46:59.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.451+0000 7f1d8d400640 1 -- 172.21.15.16:0/2962021872 wait complete. 2024-09-21T03:46:59.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.453+0000 7f1d8d400640 1 Processor -- start 2024-09-21T03:46:59.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.453+0000 7f1d8d400640 1 -- start start 2024-09-21T03:46:59.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.454+0000 7f1d8d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d88105bf0 0x7f1d8819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.454+0000 7f1d8d400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d8810b530 0x7f1d8819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.454+0000 7f1d8d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8810be80 0x7f1d881a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.454+0000 7f1d8d400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1d88115b90 con 0x7f1d8810be80 2024-09-21T03:46:59.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.454+0000 7f1d8d400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d88115a10 con 0x7f1d88105bf0 2024-09-21T03:46:59.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.454+0000 7f1d8d400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1d88115d10 con 0x7f1d8810b530 2024-09-21T03:46:59.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.454+0000 7f1d87e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d88105bf0 0x7f1d8819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.454+0000 7f1d87e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d88105bf0 0x7f1d8819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37466/0 (socket says 172.21.15.16:37466) 2024-09-21T03:46:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.454+0000 7f1d87e00640 1 -- 172.21.15.16:0/2974785087 learned_addr learned my addr 172.21.15.16:0/2974785087 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.454+0000 7f1d87400640 1 --2- 172.21.15.16:0/2974785087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d8810b530 0x7f1d8819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.455+0000 7f1d87e00640 1 -- 172.21.15.16:0/2974785087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d8810b530 msgr2=0x7f1d8819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.455+0000 7f1d87e00640 1 --2- 172.21.15.16:0/2974785087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d8810b530 0x7f1d8819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.455+0000 7f1d87e00640 1 -- 172.21.15.16:0/2974785087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8810be80 msgr2=0x7f1d881a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.455+0000 7f1d87e00640 1 --2- 172.21.15.16:0/2974785087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8810be80 0x7f1d881a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.455+0000 7f1d87e00640 1 -- 172.21.15.16:0/2974785087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d881a24f0 con 0x7f1d88105bf0 2024-09-21T03:46:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.455+0000 7f1d87e00640 1 --2- 172.21.15.16:0/2974785087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d88105bf0 0x7f1d8819c580 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1d78005ab0 tx=0x7f1d78003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.455+0000 7f1d84c00640 1 -- 172.21.15.16:0/2974785087 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7803f070 con 0x7f1d88105bf0 2024-09-21T03:46:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.456+0000 7f1d8d400640 1 -- 172.21.15.16:0/2974785087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d881a2780 con 0x7f1d88105bf0 2024-09-21T03:46:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.456+0000 7f1d84c00640 1 -- 172.21.15.16:0/2974785087 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d78031b60 con 0x7f1d88105bf0 2024-09-21T03:46:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.456+0000 7f1d84c00640 1 -- 172.21.15.16:0/2974785087 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d78035030 con 0x7f1d88105bf0 2024-09-21T03:46:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.456+0000 7f1d8d400640 1 -- 172.21.15.16:0/2974785087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d881a8f40 con 0x7f1d88105bf0 2024-09-21T03:46:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.460+0000 7f1d84c00640 1 -- 172.21.15.16:0/2974785087 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1d78042070 con 0x7f1d88105bf0 2024-09-21T03:46:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.460+0000 7f1d956d2740 1 Processor -- start 2024-09-21T03:46:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.461+0000 7f1d956d2740 1 -- start start 2024-09-21T03:46:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.461+0000 7f1d956d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f98840 0x55e7b3340510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.461+0000 7f1d956d2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f6f980 0x55e7b3340a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.461+0000 7f1d956d2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f77b60 0x55e7b3340f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.461+0000 7f1d956d2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e7b32b5ca0 con 0x55e7b2f77b60 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.461+0000 7f1d956d2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e7b32a50d0 con 0x55e7b2f6f980 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.461+0000 7f1d956d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e7b32a8ee0 con 0x55e7b2f98840 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.462+0000 7f1d8ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f98840 0x55e7b3340510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.462+0000 7f1d8ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f98840 0x55e7b3340510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:44780/0 (socket says 172.21.15.16:44780) 2024-09-21T03:46:59.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.462+0000 7f1d8ca00640 1 -- 172.21.15.16:0/3316675008 learned_addr learned my addr 172.21.15.16:0/3316675008 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:59.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.462+0000 7f1d87e00640 1 --2- 172.21.15.16:0/3316675008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f6f980 0x55e7b3340a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.463+0000 7f1d84c00640 1 --2- 172.21.15.16:0/2974785087 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1d6c0b6500 0x7f1d6c0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.463+0000 7f1d87400640 1 --2- 172.21.15.16:0/3316675008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f77b60 0x55e7b3340f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.463+0000 7f1d8ca00640 1 -- 172.21.15.16:0/3316675008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f6f980 msgr2=0x55e7b3340a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:59.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.463+0000 7f1d8ca00640 1 --2- 172.21.15.16:0/3316675008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f6f980 0x55e7b3340a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.463+0000 7f1d8ca00640 1 -- 172.21.15.16:0/3316675008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f77b60 msgr2=0x55e7b3340f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:59.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.463+0000 7f1d84c00640 1 -- 172.21.15.16:0/2974785087 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f1d78107af0 con 0x7f1d88105bf0 2024-09-21T03:46:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.463+0000 7f1d8ca00640 1 --2- 172.21.15.16:0/3316675008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f77b60 0x55e7b3340f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.463+0000 7f1d8ca00640 1 -- 172.21.15.16:0/3316675008 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7b3257400 con 0x55e7b2f98840 2024-09-21T03:46:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.463+0000 7f1d87400640 1 --2- 172.21.15.16:0/3316675008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f77b60 0x55e7b3340f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:46:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.464+0000 7f1d8ca00640 1 --2- 172.21.15.16:0/2974785087 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1d6c0b6500 0x7f1d6c0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.465+0000 7f1d8ca00640 1 --2- 172.21.15.16:0/3316675008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f98840 0x55e7b3340510 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1d80005c40 tx=0x7f1d80009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd8ecdaa7e891620 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.465+0000 7f1d7d200640 1 -- 172.21.15.16:0/3316675008 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80021030 con 0x55e7b2f98840 2024-09-21T03:46:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.465+0000 7f1d8ca00640 1 --2- 172.21.15.16:0/2974785087 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1d6c0b6500 0x7f1d6c0b8a10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1d80019000 tx=0x7f1d8001b000 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.465+0000 7f1d7d200640 1 -- 172.21.15.16:0/3316675008 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d80010bc0 con 0x55e7b2f98840 2024-09-21T03:46:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.465+0000 7f1d7d200640 1 -- 172.21.15.16:0/3316675008 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8000f400 con 0x55e7b2f98840 2024-09-21T03:46:59.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.466+0000 7f1d956d2740 1 -- 172.21.15.16:0/3316675008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f98840 msgr2=0x55e7b3340510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.466+0000 7f1d956d2740 1 --2- 172.21.15.16:0/3316675008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f98840 0x55e7b3340510 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1d80005c40 tx=0x7f1d80009f90 comp rx=0 tx=0).stop 2024-09-21T03:46:59.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.467+0000 7f1d956d2740 1 -- 172.21.15.16:0/3316675008 shutdown_connections 2024-09-21T03:46:59.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.467+0000 7f1d956d2740 1 --2- 172.21.15.16:0/3316675008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f77b60 0x55e7b3340f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.467+0000 7f1d956d2740 1 --2- 172.21.15.16:0/3316675008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f6f980 0x55e7b3340a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.467+0000 7f1d956d2740 1 --2- 172.21.15.16:0/3316675008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f98840 0x55e7b3340510 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.467+0000 7f1d956d2740 1 -- 172.21.15.16:0/3316675008 >> 172.21.15.16:0/3316675008 conn(0x55e7b2f8c990 msgr2=0x55e7b333e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:59.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.467+0000 7f1d956d2740 1 -- 172.21.15.16:0/3316675008 shutdown_connections 2024-09-21T03:46:59.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.467+0000 7f1d956d2740 1 -- 172.21.15.16:0/3316675008 wait complete. 2024-09-21T03:46:59.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.468+0000 7f1d956d2740 1 -- start start 2024-09-21T03:46:59.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.468+0000 7f1d956d2740 1 Processor -- start 2024-09-21T03:46:59.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.468+0000 7f1d956d2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 0x55e7b334bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.468+0000 7f1d956d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 0x55e7b334c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.469+0000 7f1d956d2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f98840 0x55e7b3358060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.469+0000 7f1d956d2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e7b31067a0 con 0x55e7b2f6f980 2024-09-21T03:46:59.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.469+0000 7f1d956d2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e7b324aa10 con 0x55e7b2f98840 2024-09-21T03:46:59.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.469+0000 7f1d956d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e7b334ecf0 con 0x55e7b2f77b60 2024-09-21T03:46:59.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.469+0000 7f1d956d2740 10 client.0 _ll_register_callbacks cb 0x55e7b325abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:59.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.469+0000 7f1d87e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 0x55e7b334bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.469+0000 7f1d87e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 0x55e7b334bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38296/0 (socket says 172.21.15.16:38296) 2024-09-21T03:46:59.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.469+0000 7f1d87e00640 1 -- 172.21.15.16:0/1127093398 learned_addr learned my addr 172.21.15.16:0/1127093398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:59.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.469+0000 7f1d87400640 1 --2- 172.21.15.16:0/1127093398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 0x55e7b334c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.469+0000 7f1d8ca00640 1 --2- 172.21.15.16:0/1127093398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f98840 0x55e7b3358060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.470+0000 7f1d87400640 1 -- 172.21.15.16:0/1127093398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f98840 msgr2=0x55e7b3358060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:59.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.470+0000 7f1d87400640 1 --2- 172.21.15.16:0/1127093398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f98840 0x55e7b3358060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.470+0000 7f1d87400640 1 -- 172.21.15.16:0/1127093398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 msgr2=0x55e7b334bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:59.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.470+0000 7f1d87400640 1 --2- 172.21.15.16:0/1127093398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 0x55e7b334bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.470+0000 7f1d87400640 1 -- 172.21.15.16:0/1127093398 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7b3012b90 con 0x55e7b2f77b60 2024-09-21T03:46:59.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.470+0000 7f1d87e00640 1 --2- 172.21.15.16:0/1127093398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 0x55e7b334bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:46:59.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.470+0000 7f1d87400640 1 --2- 172.21.15.16:0/1127093398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 0x55e7b334c0f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d8810afb0 tx=0x7f1d74013b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63461b48923baa88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:59.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.470+0000 7f1d71600640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:46:59.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.471+0000 7f1d71600640 1 -- 172.21.15.16:0/1127093398 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74004440 con 0x55e7b2f77b60 2024-09-21T03:46:59.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.471+0000 7f1d71600640 1 -- 172.21.15.16:0/1127093398 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f1d74011990 con 0x55e7b2f77b60 2024-09-21T03:46:59.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.471+0000 7f1d956d2740 1 -- 172.21.15.16:0/1127093398 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d80010bc0 con 0x55e7b2f77b60 2024-09-21T03:46:59.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.471+0000 7f1d71600640 1 -- 172.21.15.16:0/1127093398 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d74011b90 con 0x55e7b2f77b60 2024-09-21T03:46:59.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.471+0000 7f1d956d2740 10 client.25603 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:59.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.471+0000 7f1d956d2740 1 -- 172.21.15.16:0/1127093398 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e7b335aa30 con 0x55e7b2f77b60 2024-09-21T03:46:59.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.472+0000 7f1d71600640 1 -- 172.21.15.16:0/1127093398 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d80010bc0 con 0x55e7b2f77b60 2024-09-21T03:46:59.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.472+0000 7f1d71600640 1 -- 172.21.15.16:0/1127093398 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1d7401a070 con 0x55e7b2f77b60 2024-09-21T03:46:59.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.472+0000 7f1d956d2740 10 client.25603 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:59.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.472+0000 7f1d956d2740 10 client.25603 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:46:59.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.472+0000 7f1d956d2740 20 client.25603 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:59.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.472+0000 7f1d956d2740 1 --2- 172.21.15.16:0/1127093398 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e7b2f571c0 0x55e7b335def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.472+0000 7f1d956d2740 4 client.25603 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:46:59.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.472+0000 7f1d956d2740 1 -- 172.21.15.16:0/1127093398 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7b32b7bf0 con 0x55e7b2f571c0 2024-09-21T03:46:59.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.473+0000 7f1d87e00640 1 --2- 172.21.15.16:0/1127093398 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e7b2f571c0 0x55e7b335def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.473+0000 7f1d87e00640 1 --2- 172.21.15.16:0/1127093398 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e7b2f571c0 0x55e7b335def0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28b381f09f624e11 server_cookie=c89aabc4a29e928f in_seq=0 out_seq=0 2024-09-21T03:46:59.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.473+0000 7f1d71600640 10 client.25603 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:59.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.474+0000 7f1d71600640 1 -- 172.21.15.16:0/1127093398 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e7b32b7bf0 con 0x55e7b2f571c0 2024-09-21T03:46:59.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.474+0000 7f1d71600640 10 client.25603 handle_command_reply: tid=0 2024-09-21T03:46:59.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.474+0000 7f1d956d2740 1 client.25603 shutdown 2024-09-21T03:46:59.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.474+0000 7f1d956d2740 1 -- 172.21.15.16:0/1127093398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 msgr2=0x55e7b334c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.474+0000 7f1d956d2740 1 --2- 172.21.15.16:0/1127093398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 0x55e7b334c0f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d8810afb0 tx=0x7f1d74013b70 comp rx=0 tx=0).stop 2024-09-21T03:46:59.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.475+0000 7f1d956d2740 1 -- 172.21.15.16:0/1127093398 shutdown_connections 2024-09-21T03:46:59.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.475+0000 7f1d956d2740 1 --2- 172.21.15.16:0/1127093398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f98840 0x55e7b3358060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.475+0000 7f1d956d2740 1 --2- 172.21.15.16:0/1127093398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 0x55e7b334c0f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.475+0000 7f1d956d2740 1 --2- 172.21.15.16:0/1127093398 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e7b2f571c0 0x55e7b335def0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.475+0000 7f1d956d2740 1 --2- 172.21.15.16:0/1127093398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 0x55e7b334bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.475+0000 7f1d71600640 0 client.25603 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:59.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.475+0000 7f1d956d2740 1 -- 172.21.15.16:0/1127093398 >> 172.21.15.16:0/1127093398 conn(0x55e7b2f8c990 msgr2=0x55e7b33436a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:59.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.475+0000 7f1d956d2740 1 -- 172.21.15.16:0/1127093398 shutdown_connections 2024-09-21T03:46:59.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.475+0000 7f1d956d2740 1 -- 172.21.15.16:0/1127093398 wait complete. 2024-09-21T03:46:59.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.475+0000 7f1d956d2740 20 client.25603 trim_cache size 0 max 16384 2024-09-21T03:46:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.493+0000 7f1d956d2740 1 Processor -- start 2024-09-21T03:46:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.493+0000 7f1d956d2740 1 -- start start 2024-09-21T03:46:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.494+0000 7f1d956d2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 0x55e7b334da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.494+0000 7f1d956d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 0x55e7b3341c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.494+0000 7f1d956d2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f98840 0x55e7b3357f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.494+0000 7f1d956d2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1d881b2cc0 con 0x55e7b2f6f980 2024-09-21T03:46:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.494+0000 7f1d956d2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d881b4800 con 0x55e7b2f98840 2024-09-21T03:46:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.494+0000 7f1d956d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1d881b3390 con 0x55e7b2f77b60 2024-09-21T03:46:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.494+0000 7f1d87400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 0x55e7b3341c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.494+0000 7f1d87400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 0x55e7b3341c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:44802/0 (socket says 172.21.15.16:44802) 2024-09-21T03:46:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.494+0000 7f1d87400640 1 -- 172.21.15.16:0/542673053 learned_addr learned my addr 172.21.15.16:0/542673053 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.494+0000 7f1d8ca00640 1 --2- 172.21.15.16:0/542673053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f98840 0x55e7b3357f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.495+0000 7f1d87e00640 1 --2- 172.21.15.16:0/542673053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 0x55e7b334da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.495+0000 7f1d87400640 1 -- 172.21.15.16:0/542673053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f98840 msgr2=0x55e7b3357f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.495+0000 7f1d87400640 1 --2- 172.21.15.16:0/542673053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f98840 0x55e7b3357f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.495+0000 7f1d87400640 1 -- 172.21.15.16:0/542673053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 msgr2=0x55e7b334da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.495+0000 7f1d87400640 1 --2- 172.21.15.16:0/542673053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 0x55e7b334da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.495+0000 7f1d87400640 1 -- 172.21.15.16:0/542673053 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7b3342a50 con 0x55e7b2f77b60 2024-09-21T03:46:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.495+0000 7f1d87400640 1 --2- 172.21.15.16:0/542673053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 0x55e7b3341c50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d7401c780 tx=0x7f1d74009df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af81d7832cd31615 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.496+0000 7f1d73e00640 1 -- 172.21.15.16:0/542673053 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7401c800 con 0x55e7b2f77b60 2024-09-21T03:46:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.496+0000 7f1d73e00640 1 -- 172.21.15.16:0/542673053 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d7402f070 con 0x55e7b2f77b60 2024-09-21T03:46:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.496+0000 7f1d73e00640 1 -- 172.21.15.16:0/542673053 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74003850 con 0x55e7b2f77b60 2024-09-21T03:46:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.497+0000 7f1d956d2740 1 -- 172.21.15.16:0/542673053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 msgr2=0x55e7b3341c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.497+0000 7f1d956d2740 1 --2- 172.21.15.16:0/542673053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 0x55e7b3341c50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d7401c780 tx=0x7f1d74009df0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.497+0000 7f1d956d2740 1 -- 172.21.15.16:0/542673053 shutdown_connections 2024-09-21T03:46:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.497+0000 7f1d956d2740 1 --2- 172.21.15.16:0/542673053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f98840 0x55e7b3357f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.497+0000 7f1d956d2740 1 --2- 172.21.15.16:0/542673053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 0x55e7b3341c50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.497+0000 7f1d956d2740 1 --2- 172.21.15.16:0/542673053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 0x55e7b334da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.497+0000 7f1d956d2740 1 -- 172.21.15.16:0/542673053 >> 172.21.15.16:0/542673053 conn(0x55e7b2f8c990 msgr2=0x55e7b3341540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.497+0000 7f1d956d2740 1 -- 172.21.15.16:0/542673053 shutdown_connections 2024-09-21T03:46:59.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.497+0000 7f1d956d2740 1 -- 172.21.15.16:0/542673053 wait complete. 2024-09-21T03:46:59.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.498+0000 7f1d956d2740 1 -- start start 2024-09-21T03:46:59.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.498+0000 7f1d956d2740 1 Processor -- start 2024-09-21T03:46:59.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.499+0000 7f1d956d2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 0x55e7b3352400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.499+0000 7f1d956d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 0x55e7b33529c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.499+0000 7f1d956d2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f98840 0x55e7b33d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.499+0000 7f1d956d2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1d881b4130 con 0x55e7b2f6f980 2024-09-21T03:46:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.499+0000 7f1d956d2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d881b3a60 con 0x55e7b2f98840 2024-09-21T03:46:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.499+0000 7f1d956d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1d881b0ca0 con 0x55e7b2f77b60 2024-09-21T03:46:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.499+0000 7f1d956d2740 10 client.0 _ll_register_callbacks cb 0x7f1d740097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:46:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.499+0000 7f1d87e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 0x55e7b3352400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.499+0000 7f1d87e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 0x55e7b3352400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38320/0 (socket says 172.21.15.16:38320) 2024-09-21T03:46:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.499+0000 7f1d87400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 0x55e7b33529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.499+0000 7f1d87e00640 1 -- 172.21.15.16:0/3719592983 learned_addr learned my addr 172.21.15.16:0/3719592983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:46:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.499+0000 7f1d8ca00640 1 --2- 172.21.15.16:0/3719592983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f98840 0x55e7b33d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.500+0000 7f1d87400640 1 -- 172.21.15.16:0/3719592983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f98840 msgr2=0x55e7b33d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.500+0000 7f1d87400640 1 --2- 172.21.15.16:0/3719592983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f98840 0x55e7b33d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.500+0000 7f1d87400640 1 -- 172.21.15.16:0/3719592983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 msgr2=0x55e7b3352400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:46:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.500+0000 7f1d87400640 1 --2- 172.21.15.16:0/3719592983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 0x55e7b3352400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.500+0000 7f1d87400640 1 -- 172.21.15.16:0/3719592983 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7b33d05e0 con 0x55e7b2f77b60 2024-09-21T03:46:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.501+0000 7f1d87400640 1 --2- 172.21.15.16:0/3719592983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 0x55e7b33529c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1d7402c040 tx=0x7f1d74009cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e69124a7ebba4f34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:46:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.501+0000 7f1d71600640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:46:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.501+0000 7f1d71600640 1 -- 172.21.15.16:0/3719592983 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7403c070 con 0x55e7b2f77b60 2024-09-21T03:46:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.501+0000 7f1d956d2740 1 -- 172.21.15.16:0/3719592983 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d7402f070 con 0x55e7b2f77b60 2024-09-21T03:46:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.501+0000 7f1d71600640 1 -- 172.21.15.16:0/3719592983 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f1d7401cbc0 con 0x55e7b2f77b60 2024-09-21T03:46:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.501+0000 7f1d71600640 1 -- 172.21.15.16:0/3719592983 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d740035f0 con 0x55e7b2f77b60 2024-09-21T03:46:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.502+0000 7f1d956d2740 10 client.25609 fetch_fsmap learned FSMap version 30 2024-09-21T03:46:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.502+0000 7f1d956d2740 1 -- 172.21.15.16:0/3719592983 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e7b33d1750 con 0x55e7b2f77b60 2024-09-21T03:46:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.502+0000 7f1d71600640 1 -- 172.21.15.16:0/3719592983 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d7402f070 con 0x55e7b2f77b60 2024-09-21T03:46:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.502+0000 7f1d71600640 1 -- 172.21.15.16:0/3719592983 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1d7403f030 con 0x55e7b2f77b60 2024-09-21T03:46:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.502+0000 7f1d956d2740 10 client.25609 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:46:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.502+0000 7f1d956d2740 10 client.25609 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:46:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.502+0000 7f1d956d2740 20 client.25609 populate_metadata read hostname 'smithi016' 2024-09-21T03:46:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.502+0000 7f1d956d2740 1 --2- 172.21.15.16:0/3719592983 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e7b2f2c570 0x55e7b33d3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:46:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.503+0000 7f1d956d2740 4 client.25609 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}] 2024-09-21T03:46:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.503+0000 7f1d956d2740 1 -- 172.21.15.16:0/3719592983 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}) -- 0x55e7b32b36a0 con 0x55e7b2f2c570 2024-09-21T03:46:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.503+0000 7f1d87e00640 1 --2- 172.21.15.16:0/3719592983 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e7b2f2c570 0x55e7b33d3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:46:59.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.504+0000 7f1d87e00640 1 --2- 172.21.15.16:0/3719592983 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e7b2f2c570 0x55e7b33d3c50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6dbcafad469e650 server_cookie=eb255ca4173876be in_seq=0 out_seq=0 2024-09-21T03:46:59.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.504+0000 7f1d71600640 10 client.25609 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.515+0000 7f1d71600640 1 -- 172.21.15.16:0/3719592983 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55e7b32b36a0 con 0x55e7b2f2c570 2024-09-21T03:46:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.515+0000 7f1d71600640 10 client.25609 handle_command_reply: tid=0 2024-09-21T03:46:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.515+0000 7f1d956d2740 1 client.25609 shutdown 2024-09-21T03:46:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.515+0000 7f1d956d2740 1 -- 172.21.15.16:0/3719592983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 msgr2=0x55e7b33529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.515+0000 7f1d956d2740 1 --2- 172.21.15.16:0/3719592983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 0x55e7b33529c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1d7402c040 tx=0x7f1d74009cb0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.515+0000 7f1d956d2740 1 -- 172.21.15.16:0/3719592983 shutdown_connections 2024-09-21T03:46:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.515+0000 7f1d956d2740 1 --2- 172.21.15.16:0/3719592983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7b2f98840 0x55e7b33d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.515+0000 7f1d956d2740 1 --2- 172.21.15.16:0/3719592983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b2f77b60 0x55e7b33529c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.515+0000 7f1d956d2740 1 --2- 172.21.15.16:0/3719592983 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e7b2f2c570 0x55e7b33d3c50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.516+0000 7f1d956d2740 1 --2- 172.21.15.16:0/3719592983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7b2f6f980 0x55e7b3352400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.516+0000 7f1d956d2740 1 -- 172.21.15.16:0/3719592983 >> 172.21.15.16:0/3719592983 conn(0x55e7b2f8c990 msgr2=0x55e7b334d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:59.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.516+0000 7f1d71600640 0 client.25609 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:46:59.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.516+0000 7f1d956d2740 1 -- 172.21.15.16:0/3719592983 shutdown_connections 2024-09-21T03:46:59.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.516+0000 7f1d956d2740 1 -- 172.21.15.16:0/3719592983 wait complete. 2024-09-21T03:46:59.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.516+0000 7f1d956d2740 20 client.25609 trim_cache size 0 max 16384 2024-09-21T03:46:59.519 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:46:59.519 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:46:59.519 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:46:59.519 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 2, 2024-09-21T03:46:59.519 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:46:59.520 INFO:teuthology.orchestra.run.smithi016.stdout: "e97f8336": { 2024-09-21T03:46:59.520 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 2, 2024-09-21T03:46:59.520 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:46:59.520 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:46:59.520 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:46:59.520 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:46:59.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:46:59.520 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:46:59.520 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 134.5, 2024-09-21T03:46:59.520 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:46:59.520 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:46:59.520 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:46:59.520 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:46:59.520 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:46:59.521 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:46:59.521 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:46:59.521 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:46:59.521 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:46:59.521 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:46:59.521 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:46:59.521 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:46:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.517+0000 7f1d8d400640 1 -- 172.21.15.16:0/2974785087 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1d6c0b6500 msgr2=0x7f1d6c0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.517+0000 7f1d8d400640 1 --2- 172.21.15.16:0/2974785087 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1d6c0b6500 0x7f1d6c0b8a10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1d80019000 tx=0x7f1d8001b000 comp rx=0 tx=0).stop 2024-09-21T03:46:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.517+0000 7f1d8d400640 1 -- 172.21.15.16:0/2974785087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d88105bf0 msgr2=0x7f1d8819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:46:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.517+0000 7f1d8d400640 1 --2- 172.21.15.16:0/2974785087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d88105bf0 0x7f1d8819c580 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1d78005ab0 tx=0x7f1d78003c10 comp rx=0 tx=0).stop 2024-09-21T03:46:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.518+0000 7f1d8d400640 1 -- 172.21.15.16:0/2974785087 shutdown_connections 2024-09-21T03:46:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.518+0000 7f1d8d400640 1 --2- 172.21.15.16:0/2974785087 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1d6c0b6500 0x7f1d6c0b8a10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.518+0000 7f1d8d400640 1 --2- 172.21.15.16:0/2974785087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8810be80 0x7f1d881a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.518+0000 7f1d8d400640 1 --2- 172.21.15.16:0/2974785087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d8810b530 0x7f1d8819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.518+0000 7f1d8d400640 1 --2- 172.21.15.16:0/2974785087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d88105bf0 0x7f1d8819c580 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:46:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.518+0000 7f1d8d400640 1 -- 172.21.15.16:0/2974785087 >> 172.21.15.16:0/2974785087 conn(0x7f1d88101680 msgr2=0x7f1d88103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:46:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.520+0000 7f1d8d400640 1 -- 172.21.15.16:0/2974785087 shutdown_connections 2024-09-21T03:46:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:46:59.520+0000 7f1d8d400640 1 -- 172.21.15.16:0/2974785087 wait complete. 2024-09-21T03:47:00.469 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e97f8336', quiesce took 4.7 seconds. Will release after: 9.8 2024-09-21T03:47:01.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:00 smithi134 ceph-mon[35031]: pgmap v439: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:47:01.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:00 smithi123 ceph-mon[34673]: pgmap v439: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:47:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:00 smithi016 ceph-mon[30375]: pgmap v439: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:47:03.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:02 smithi134 ceph-mon[35031]: pgmap v440: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:03.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:02 smithi123 ceph-mon[34673]: pgmap v440: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:03.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:02 smithi016 ceph-mon[30375]: pgmap v440: 129 pgs: 129 active+clean; 667 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:05.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:04 smithi134 ceph-mon[35031]: pgmap v441: 129 pgs: 129 active+clean; 672 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:47:05.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:04 smithi123 ceph-mon[34673]: pgmap v441: 129 pgs: 129 active+clean; 672 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:47:05.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:04 smithi016 ceph-mon[30375]: pgmap v441: 129 pgs: 129 active+clean; 672 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:47:07.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:06 smithi134 ceph-mon[35031]: pgmap v442: 129 pgs: 129 active+clean; 672 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T03:47:07.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:06 smithi123 ceph-mon[34673]: pgmap v442: 129 pgs: 129 active+clean; 672 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T03:47:07.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:06 smithi016 ceph-mon[30375]: pgmap v442: 129 pgs: 129 active+clean; 672 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T03:47:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.870+0000 7fda8b200640 1 -- 172.21.15.16:0/702882183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c1066c0 msgr2=0x7fda8c108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.870+0000 7fda8b200640 1 --2- 172.21.15.16:0/702882183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c1066c0 0x7fda8c108b40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fda78009900 tx=0x7fda7802a400 comp rx=0 tx=0).stop 2024-09-21T03:47:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.870+0000 7fda8b200640 1 -- 172.21.15.16:0/702882183 shutdown_connections 2024-09-21T03:47:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.870+0000 7fda8b200640 1 --2- 172.21.15.16:0/702882183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda8c109280 0x7fda8c10b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.870+0000 7fda8b200640 1 --2- 172.21.15.16:0/702882183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c1066c0 0x7fda8c108b40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.870+0000 7fda8b200640 1 --2- 172.21.15.16:0/702882183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c06b710 0x7fda8c106180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.870+0000 7fda8b200640 1 -- 172.21.15.16:0/702882183 >> 172.21.15.16:0/702882183 conn(0x7fda8c0fdd80 msgr2=0x7fda8c100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.870+0000 7fda8b200640 1 -- 172.21.15.16:0/702882183 shutdown_connections 2024-09-21T03:47:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.870+0000 7fda8b200640 1 -- 172.21.15.16:0/702882183 wait complete. 2024-09-21T03:47:07.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.871+0000 7fda8b200640 1 Processor -- start 2024-09-21T03:47:07.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.871+0000 7fda8b200640 1 -- start start 2024-09-21T03:47:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.872+0000 7fda8b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda8c06b710 0x7fda8c19bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.872+0000 7fda8b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c1066c0 0x7fda8c19c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.872+0000 7fda8b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c109280 0x7fda8c19d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.872+0000 7fda8b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c10f110 con 0x7fda8c109280 2024-09-21T03:47:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.872+0000 7fda8b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c10ef90 con 0x7fda8c06b710 2024-09-21T03:47:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.872+0000 7fda8b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c10f290 con 0x7fda8c1066c0 2024-09-21T03:47:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.872+0000 7fda8a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c109280 0x7fda8c19d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.872+0000 7fda89e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda8c06b710 0x7fda8c19bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.872+0000 7fda8a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c109280 0x7fda8c19d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52284/0 (socket says 172.21.15.16:52284) 2024-09-21T03:47:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.872+0000 7fda8a800640 1 -- 172.21.15.16:0/3902474669 learned_addr learned my addr 172.21.15.16:0/3902474669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.872+0000 7fda89400640 1 --2- 172.21.15.16:0/3902474669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c1066c0 0x7fda8c19c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.873+0000 7fda8a800640 1 -- 172.21.15.16:0/3902474669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c1066c0 msgr2=0x7fda8c19c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.873+0000 7fda8a800640 1 --2- 172.21.15.16:0/3902474669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c1066c0 0x7fda8c19c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.873+0000 7fda8a800640 1 -- 172.21.15.16:0/3902474669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda8c06b710 msgr2=0x7fda8c19bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.873+0000 7fda8a800640 1 --2- 172.21.15.16:0/3902474669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda8c06b710 0x7fda8c19bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.873+0000 7fda8a800640 1 -- 172.21.15.16:0/3902474669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8c19d9b0 con 0x7fda8c109280 2024-09-21T03:47:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.873+0000 7fda89400640 1 --2- 172.21.15.16:0/3902474669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c1066c0 0x7fda8c19c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.873+0000 7fda89e00640 1 --2- 172.21.15.16:0/3902474669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda8c06b710 0x7fda8c19bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.874+0000 7fda8a800640 1 --2- 172.21.15.16:0/3902474669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c109280 0x7fda8c19d190 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fda8400eb10 tx=0x7fda8400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.874+0000 7fda82a00640 1 -- 172.21.15.16:0/3902474669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84004b20 con 0x7fda8c109280 2024-09-21T03:47:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.874+0000 7fda8b200640 1 -- 172.21.15.16:0/3902474669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8c19dca0 con 0x7fda8c109280 2024-09-21T03:47:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.874+0000 7fda82a00640 1 -- 172.21.15.16:0/3902474669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda84004cc0 con 0x7fda8c109280 2024-09-21T03:47:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.874+0000 7fda82a00640 1 -- 172.21.15.16:0/3902474669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84010670 con 0x7fda8c109280 2024-09-21T03:47:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.874+0000 7fda8b200640 1 -- 172.21.15.16:0/3902474669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8c1a4390 con 0x7fda8c109280 2024-09-21T03:47:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.877+0000 7fda97fcc740 1 Processor -- start 2024-09-21T03:47:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.880+0000 7fda82a00640 1 -- 172.21.15.16:0/3902474669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fda84005020 con 0x7fda8c109280 2024-09-21T03:47:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.880+0000 7fda97fcc740 1 -- start start 2024-09-21T03:47:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.880+0000 7fda97fcc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e69191b1b0 0x55e691cc22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.880+0000 7fda97fcc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918f22f0 0x55e691cc2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.881+0000 7fda97fcc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6918fa4d0 0x55e691cc3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.881+0000 7fda97fcc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e691c356b0 con 0x55e6918fa4d0 2024-09-21T03:47:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.881+0000 7fda97fcc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e691c24ae0 con 0x55e69191b1b0 2024-09-21T03:47:07.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.881+0000 7fda97fcc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e691c288f0 con 0x55e6918f22f0 2024-09-21T03:47:07.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.881+0000 7fda89400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e69191b1b0 0x55e691cc22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.881+0000 7fda89400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e69191b1b0 0x55e691cc22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34742/0 (socket says 172.21.15.16:34742) 2024-09-21T03:47:07.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.881+0000 7fda89400640 1 -- 172.21.15.16:0/355018284 learned_addr learned my addr 172.21.15.16:0/355018284 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:07.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.881+0000 7fda89e00640 1 --2- 172.21.15.16:0/355018284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918f22f0 0x55e691cc2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.881+0000 7fda89400640 1 --2- 172.21.15.16:0/355018284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6918fa4d0 0x55e691cc3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.882+0000 7fda89e00640 1 -- 172.21.15.16:0/355018284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e69191b1b0 msgr2=0x55e691cc22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.882+0000 7fda89e00640 1 --2- 172.21.15.16:0/355018284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e69191b1b0 0x55e691cc22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.882+0000 7fda89e00640 1 -- 172.21.15.16:0/355018284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6918fa4d0 msgr2=0x55e691cc3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.882+0000 7fda89e00640 1 --2- 172.21.15.16:0/355018284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6918fa4d0 0x55e691cc3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.882+0000 7fda89e00640 1 -- 172.21.15.16:0/355018284 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e691bf1780 con 0x55e6918f22f0 2024-09-21T03:47:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.882+0000 7fda89e00640 1 --2- 172.21.15.16:0/355018284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918f22f0 0x55e691cc2920 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009fd0 tx=0x7fda7c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d5f7a761945d29a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.883+0000 7fda82a00640 1 --2- 172.21.15.16:0/3902474669 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fda700b6510 0x7fda700b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.883+0000 7fda77400640 1 -- 172.21.15.16:0/355018284 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c004030 con 0x55e6918f22f0 2024-09-21T03:47:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.883+0000 7fda77400640 1 -- 172.21.15.16:0/355018284 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda7c0054d0 con 0x55e6918f22f0 2024-09-21T03:47:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.883+0000 7fda77400640 1 -- 172.21.15.16:0/355018284 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c004c20 con 0x55e6918f22f0 2024-09-21T03:47:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.883+0000 7fda82a00640 1 -- 172.21.15.16:0/3902474669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fda840e7c60 con 0x7fda8c109280 2024-09-21T03:47:07.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.883+0000 7fda89400640 1 --2- 172.21.15.16:0/3902474669 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fda700b6510 0x7fda700b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.884+0000 7fda97fcc740 1 -- 172.21.15.16:0/355018284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918f22f0 msgr2=0x55e691cc2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.884+0000 7fda97fcc740 1 --2- 172.21.15.16:0/355018284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918f22f0 0x55e691cc2920 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009fd0 tx=0x7fda7c0118d0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.884+0000 7fda89400640 1 --2- 172.21.15.16:0/3902474669 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fda700b6510 0x7fda700b8a20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fda780096c0 tx=0x7fda78031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.885+0000 7fda97fcc740 1 -- 172.21.15.16:0/355018284 shutdown_connections 2024-09-21T03:47:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.885+0000 7fda97fcc740 1 --2- 172.21.15.16:0/355018284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6918fa4d0 0x55e691cc3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.885+0000 7fda97fcc740 1 --2- 172.21.15.16:0/355018284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918f22f0 0x55e691cc2920 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.885+0000 7fda97fcc740 1 --2- 172.21.15.16:0/355018284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e69191b1b0 0x55e691cc22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.885+0000 7fda97fcc740 1 -- 172.21.15.16:0/355018284 >> 172.21.15.16:0/355018284 conn(0x55e69190f300 msgr2=0x55e691cbdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.885+0000 7fda97fcc740 1 -- 172.21.15.16:0/355018284 shutdown_connections 2024-09-21T03:47:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.885+0000 7fda97fcc740 1 -- 172.21.15.16:0/355018284 wait complete. 2024-09-21T03:47:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.885+0000 7fda97fcc740 1 -- start start 2024-09-21T03:47:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.886+0000 7fda97fcc740 1 Processor -- start 2024-09-21T03:47:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.886+0000 7fda97fcc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 0x55e691ccb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.886+0000 7fda97fcc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918fa4d0 0x55e691ccbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.886+0000 7fda97fcc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69191b1b0 0x55e691cccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.887+0000 7fda97fcc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e691b5cab0 con 0x55e69191b1b0 2024-09-21T03:47:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.887+0000 7fda97fcc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e691bcb570 con 0x55e6918f22f0 2024-09-21T03:47:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.887+0000 7fda97fcc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e691cc3f90 con 0x55e6918fa4d0 2024-09-21T03:47:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.887+0000 7fda97fcc740 10 client.0 _ll_register_callbacks cb 0x55e691bda100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.887+0000 7fda89e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 0x55e691ccb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.887+0000 7fda89e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 0x55e691ccb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34752/0 (socket says 172.21.15.16:34752) 2024-09-21T03:47:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.887+0000 7fda89e00640 1 -- 172.21.15.16:0/667641048 learned_addr learned my addr 172.21.15.16:0/667641048 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.887+0000 7fda89400640 1 --2- 172.21.15.16:0/667641048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918fa4d0 0x55e691ccbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.887+0000 7fda8a800640 1 --2- 172.21.15.16:0/667641048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69191b1b0 0x55e691cccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.887+0000 7fda89400640 1 -- 172.21.15.16:0/667641048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 msgr2=0x55e691ccb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.887+0000 7fda89400640 1 --2- 172.21.15.16:0/667641048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 0x55e691ccb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.888+0000 7fda89400640 1 -- 172.21.15.16:0/667641048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69191b1b0 msgr2=0x55e691cccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.888+0000 7fda89400640 1 --2- 172.21.15.16:0/667641048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69191b1b0 0x55e691cccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.888+0000 7fda89400640 1 -- 172.21.15.16:0/667641048 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e691986790 con 0x55e6918fa4d0 2024-09-21T03:47:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.888+0000 7fda89400640 1 --2- 172.21.15.16:0/667641048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918fa4d0 0x55e691ccbfb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda78032fd0 tx=0x7fda7802a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edeeef7f8ba33f8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.888+0000 7fda6be00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:47:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.888+0000 7fda6be00640 1 -- 172.21.15.16:0/667641048 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78035040 con 0x55e6918fa4d0 2024-09-21T03:47:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.888+0000 7fda97fcc740 1 -- 172.21.15.16:0/667641048 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda7c0054d0 con 0x55e6918fa4d0 2024-09-21T03:47:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.888+0000 7fda6be00640 1 -- 172.21.15.16:0/667641048 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fda78003980 con 0x55e6918fa4d0 2024-09-21T03:47:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.889+0000 7fda6be00640 1 -- 172.21.15.16:0/667641048 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fda78003b80 con 0x55e6918fa4d0 2024-09-21T03:47:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.889+0000 7fda97fcc740 10 client.25621 fetch_fsmap learned FSMap version 30 2024-09-21T03:47:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.889+0000 7fda97fcc740 1 -- 172.21.15.16:0/667641048 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e6919ff620 con 0x55e6918fa4d0 2024-09-21T03:47:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.889+0000 7fda6be00640 1 -- 172.21.15.16:0/667641048 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda7c0054d0 con 0x55e6918fa4d0 2024-09-21T03:47:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.889+0000 7fda6be00640 1 -- 172.21.15.16:0/667641048 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fda78053070 con 0x55e6918fa4d0 2024-09-21T03:47:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.889+0000 7fda97fcc740 10 client.25621 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:47:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.889+0000 7fda97fcc740 10 client.25621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.890+0000 7fda97fcc740 20 client.25621 populate_metadata read hostname 'smithi016' 2024-09-21T03:47:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.890+0000 7fda97fcc740 1 --2- 172.21.15.16:0/667641048 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e6918d9d80 0x55e691cd90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.890+0000 7fda97fcc740 4 client.25621 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.890+0000 7fda97fcc740 1 -- 172.21.15.16:0/667641048 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e691c37600 con 0x55e6918d9d80 2024-09-21T03:47:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.890+0000 7fda89e00640 1 --2- 172.21.15.16:0/667641048 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e6918d9d80 0x55e691cd90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.891+0000 7fda6be00640 10 client.25621 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.891+0000 7fda89e00640 1 --2- 172.21.15.16:0/667641048 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e6918d9d80 0x55e691cd90f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=372c5215a0d1769b server_cookie=2bc56020273ed972 in_seq=0 out_seq=0 2024-09-21T03:47:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.892+0000 7fda6be00640 1 -- 172.21.15.16:0/667641048 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e691c37600 con 0x55e6918d9d80 2024-09-21T03:47:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.892+0000 7fda6be00640 10 client.25621 handle_command_reply: tid=0 2024-09-21T03:47:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.892+0000 7fda97fcc740 1 client.25621 shutdown 2024-09-21T03:47:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.892+0000 7fda97fcc740 1 -- 172.21.15.16:0/667641048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918fa4d0 msgr2=0x55e691ccbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.892+0000 7fda97fcc740 1 --2- 172.21.15.16:0/667641048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918fa4d0 0x55e691ccbfb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda78032fd0 tx=0x7fda7802a400 comp rx=0 tx=0).stop 2024-09-21T03:47:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.892+0000 7fda97fcc740 1 -- 172.21.15.16:0/667641048 shutdown_connections 2024-09-21T03:47:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.892+0000 7fda97fcc740 1 --2- 172.21.15.16:0/667641048 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e6918d9d80 0x55e691cd90f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.892+0000 7fda97fcc740 1 --2- 172.21.15.16:0/667641048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69191b1b0 0x55e691cccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.892+0000 7fda97fcc740 1 --2- 172.21.15.16:0/667641048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918fa4d0 0x55e691ccbfb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.892+0000 7fda6be00640 0 client.25621 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.892+0000 7fda97fcc740 1 --2- 172.21.15.16:0/667641048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 0x55e691ccb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.892+0000 7fda97fcc740 1 -- 172.21.15.16:0/667641048 >> 172.21.15.16:0/667641048 conn(0x55e69190f300 msgr2=0x55e691cbc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.892+0000 7fda97fcc740 1 -- 172.21.15.16:0/667641048 shutdown_connections 2024-09-21T03:47:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.892+0000 7fda97fcc740 1 -- 172.21.15.16:0/667641048 wait complete. 2024-09-21T03:47:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.893+0000 7fda97fcc740 20 client.25621 trim_cache size 0 max 16384 2024-09-21T03:47:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.908+0000 7fda97fcc740 1 Processor -- start 2024-09-21T03:47:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.908+0000 7fda97fcc740 1 -- start start 2024-09-21T03:47:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.908+0000 7fda97fcc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 0x55e691cc6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.908+0000 7fda97fcc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918fa4d0 0x55e691cc1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.908+0000 7fda97fcc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69191b1b0 0x55e691cc23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.908+0000 7fda97fcc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c1ae830 con 0x55e69191b1b0 2024-09-21T03:47:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.908+0000 7fda97fcc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c1b0370 con 0x55e6918f22f0 2024-09-21T03:47:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.908+0000 7fda97fcc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c1aef00 con 0x55e6918fa4d0 2024-09-21T03:47:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.909+0000 7fda89e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 0x55e691cc6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.909+0000 7fda89e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 0x55e691cc6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34768/0 (socket says 172.21.15.16:34768) 2024-09-21T03:47:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.909+0000 7fda89e00640 1 -- 172.21.15.16:0/1367891750 learned_addr learned my addr 172.21.15.16:0/1367891750 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.909+0000 7fda8a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69191b1b0 0x55e691cc23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.909+0000 7fda89400640 1 --2- 172.21.15.16:0/1367891750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918fa4d0 0x55e691cc1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.909+0000 7fda89e00640 1 -- 172.21.15.16:0/1367891750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918fa4d0 msgr2=0x55e691cc1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.909+0000 7fda89e00640 1 --2- 172.21.15.16:0/1367891750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918fa4d0 0x55e691cc1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.909+0000 7fda89e00640 1 -- 172.21.15.16:0/1367891750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69191b1b0 msgr2=0x55e691cc23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.909+0000 7fda89e00640 1 --2- 172.21.15.16:0/1367891750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69191b1b0 0x55e691cc23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.909+0000 7fda89e00640 1 -- 172.21.15.16:0/1367891750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e691cc9a80 con 0x55e6918f22f0 2024-09-21T03:47:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.910+0000 7fda89e00640 1 --2- 172.21.15.16:0/1367891750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 0x55e691cc6a80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009230 tx=0x7fda7c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=722f0e9f161aa9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.910+0000 7fda76a00640 1 -- 172.21.15.16:0/1367891750 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c004250 con 0x55e6918f22f0 2024-09-21T03:47:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.910+0000 7fda76a00640 1 -- 172.21.15.16:0/1367891750 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda7c0043f0 con 0x55e6918f22f0 2024-09-21T03:47:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.910+0000 7fda76a00640 1 -- 172.21.15.16:0/1367891750 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c003a50 con 0x55e6918f22f0 2024-09-21T03:47:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.911+0000 7fda97fcc740 1 -- 172.21.15.16:0/1367891750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 msgr2=0x55e691cc6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.911+0000 7fda97fcc740 1 --2- 172.21.15.16:0/1367891750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 0x55e691cc6a80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009230 tx=0x7fda7c005850 comp rx=0 tx=0).stop 2024-09-21T03:47:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.911+0000 7fda97fcc740 1 -- 172.21.15.16:0/1367891750 shutdown_connections 2024-09-21T03:47:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.911+0000 7fda97fcc740 1 --2- 172.21.15.16:0/1367891750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69191b1b0 0x55e691cc23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.911+0000 7fda97fcc740 1 --2- 172.21.15.16:0/1367891750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918fa4d0 0x55e691cc1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.911+0000 7fda97fcc740 1 --2- 172.21.15.16:0/1367891750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 0x55e691cc6a80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.911+0000 7fda97fcc740 1 -- 172.21.15.16:0/1367891750 >> 172.21.15.16:0/1367891750 conn(0x55e69190f300 msgr2=0x55e691cc6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.911+0000 7fda97fcc740 1 -- 172.21.15.16:0/1367891750 shutdown_connections 2024-09-21T03:47:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.911+0000 7fda97fcc740 1 -- 172.21.15.16:0/1367891750 wait complete. 2024-09-21T03:47:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.912+0000 7fda97fcc740 1 -- start start 2024-09-21T03:47:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.912+0000 7fda97fcc740 1 Processor -- start 2024-09-21T03:47:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.913+0000 7fda97fcc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 0x55e691cd32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.913+0000 7fda97fcc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918fa4d0 0x55e691cd3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.913+0000 7fda97fcc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69191b1b0 0x55e691cd2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.913+0000 7fda97fcc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c1afca0 con 0x55e69191b1b0 2024-09-21T03:47:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.913+0000 7fda97fcc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c1af5d0 con 0x55e6918f22f0 2024-09-21T03:47:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.913+0000 7fda97fcc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c1ac810 con 0x55e6918fa4d0 2024-09-21T03:47:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.913+0000 7fda97fcc740 10 client.0 _ll_register_callbacks cb 0x7fda78004130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.913+0000 7fda89e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 0x55e691cd32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.913+0000 7fda89e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 0x55e691cd32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34784/0 (socket says 172.21.15.16:34784) 2024-09-21T03:47:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.913+0000 7fda89400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918fa4d0 0x55e691cd3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.913+0000 7fda89e00640 1 -- 172.21.15.16:0/4145984813 learned_addr learned my addr 172.21.15.16:0/4145984813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.913+0000 7fda8a800640 1 --2- 172.21.15.16:0/4145984813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69191b1b0 0x55e691cd2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.914+0000 7fda89400640 1 -- 172.21.15.16:0/4145984813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 msgr2=0x55e691cd32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.914+0000 7fda89400640 1 --2- 172.21.15.16:0/4145984813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 0x55e691cd32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.914+0000 7fda89400640 1 -- 172.21.15.16:0/4145984813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69191b1b0 msgr2=0x55e691cd2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.914+0000 7fda89400640 1 --2- 172.21.15.16:0/4145984813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69191b1b0 0x55e691cd2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.914+0000 7fda89400640 1 -- 172.21.15.16:0/4145984813 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e691cd27f0 con 0x55e6918fa4d0 2024-09-21T03:47:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.914+0000 7fda8a800640 1 --2- 172.21.15.16:0/4145984813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69191b1b0 0x55e691cd2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.914+0000 7fda89400640 1 --2- 172.21.15.16:0/4145984813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918fa4d0 0x55e691cd3970 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda78006010 tx=0x7fda7804aab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e61af3224e38adb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.914+0000 7fda6be00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:47:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.914+0000 7fda6be00640 1 -- 172.21.15.16:0/4145984813 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78035040 con 0x55e6918fa4d0 2024-09-21T03:47:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.914+0000 7fda97fcc740 1 -- 172.21.15.16:0/4145984813 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda7c0043f0 con 0x55e6918fa4d0 2024-09-21T03:47:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.914+0000 7fda6be00640 1 -- 172.21.15.16:0/4145984813 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fda78003980 con 0x55e6918fa4d0 2024-09-21T03:47:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.915+0000 7fda6be00640 1 -- 172.21.15.16:0/4145984813 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fda780315a0 con 0x55e6918fa4d0 2024-09-21T03:47:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.915+0000 7fda97fcc740 10 client.25627 fetch_fsmap learned FSMap version 30 2024-09-21T03:47:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.915+0000 7fda97fcc740 1 -- 172.21.15.16:0/4145984813 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e691cd16b0 con 0x55e6918fa4d0 2024-09-21T03:47:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.915+0000 7fda6be00640 1 -- 172.21.15.16:0/4145984813 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda7c0043f0 con 0x55e6918fa4d0 2024-09-21T03:47:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.915+0000 7fda6be00640 1 -- 172.21.15.16:0/4145984813 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fda78053040 con 0x55e6918fa4d0 2024-09-21T03:47:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.916+0000 7fda97fcc740 10 client.25627 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:47:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.916+0000 7fda97fcc740 10 client.25627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.916+0000 7fda97fcc740 20 client.25627 populate_metadata read hostname 'smithi016' 2024-09-21T03:47:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.916+0000 7fda97fcc740 1 --2- 172.21.15.16:0/4145984813 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e6918af1c0 0x55e691d46020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.916+0000 7fda97fcc740 4 client.25627 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:47:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.916+0000 7fda97fcc740 1 -- 172.21.15.16:0/4145984813 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e691c330b0 con 0x55e6918af1c0 2024-09-21T03:47:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.916+0000 7fda89e00640 1 --2- 172.21.15.16:0/4145984813 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e6918af1c0 0x55e691d46020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.917+0000 7fda89e00640 1 --2- 172.21.15.16:0/4145984813 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e6918af1c0 0x55e691d46020 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2182eebc3003dc04 server_cookie=af8e7f57aa5d570f in_seq=0 out_seq=0 2024-09-21T03:47:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.917+0000 7fda6be00640 10 client.25627 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.925+0000 7fda6be00640 1 -- 172.21.15.16:0/4145984813 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e691c330b0 con 0x55e6918af1c0 2024-09-21T03:47:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.925+0000 7fda6be00640 10 client.25627 handle_command_reply: tid=0 2024-09-21T03:47:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.925+0000 7fda97fcc740 1 client.25627 shutdown 2024-09-21T03:47:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.925+0000 7fda97fcc740 1 -- 172.21.15.16:0/4145984813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918fa4d0 msgr2=0x55e691cd3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.925+0000 7fda97fcc740 1 --2- 172.21.15.16:0/4145984813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918fa4d0 0x55e691cd3970 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda78006010 tx=0x7fda7804aab0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.925+0000 7fda97fcc740 1 -- 172.21.15.16:0/4145984813 shutdown_connections 2024-09-21T03:47:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.925+0000 7fda97fcc740 1 --2- 172.21.15.16:0/4145984813 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e6918af1c0 0x55e691d46020 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.925+0000 7fda97fcc740 1 --2- 172.21.15.16:0/4145984813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69191b1b0 0x55e691cd2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.925+0000 7fda97fcc740 1 --2- 172.21.15.16:0/4145984813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6918fa4d0 0x55e691cd3970 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.925+0000 7fda97fcc740 1 --2- 172.21.15.16:0/4145984813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6918f22f0 0x55e691cd32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.925+0000 7fda97fcc740 1 -- 172.21.15.16:0/4145984813 >> 172.21.15.16:0/4145984813 conn(0x55e69190f300 msgr2=0x55e691cc1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.926+0000 7fda6be00640 0 client.25627 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.926+0000 7fda97fcc740 1 -- 172.21.15.16:0/4145984813 shutdown_connections 2024-09-21T03:47:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.926+0000 7fda97fcc740 1 -- 172.21.15.16:0/4145984813 wait complete. 2024-09-21T03:47:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.926+0000 7fda97fcc740 20 client.25627 trim_cache size 0 max 16384 2024-09-21T03:47:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:47:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:47:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:47:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:47:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.927+0000 7fda8b200640 1 -- 172.21.15.16:0/3902474669 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fda700b6510 msgr2=0x7fda700b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.927+0000 7fda8b200640 1 --2- 172.21.15.16:0/3902474669 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fda700b6510 0x7fda700b8a20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fda780096c0 tx=0x7fda78031400 comp rx=0 tx=0).stop 2024-09-21T03:47:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.927+0000 7fda8b200640 1 -- 172.21.15.16:0/3902474669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c109280 msgr2=0x7fda8c19d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.927+0000 7fda8b200640 1 --2- 172.21.15.16:0/3902474669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c109280 0x7fda8c19d190 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fda8400eb10 tx=0x7fda8400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.927+0000 7fda8b200640 1 -- 172.21.15.16:0/3902474669 shutdown_connections 2024-09-21T03:47:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.927+0000 7fda8b200640 1 --2- 172.21.15.16:0/3902474669 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fda700b6510 0x7fda700b8a20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.927+0000 7fda8b200640 1 --2- 172.21.15.16:0/3902474669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c109280 0x7fda8c19d190 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.927+0000 7fda8b200640 1 --2- 172.21.15.16:0/3902474669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda8c1066c0 0x7fda8c19c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.927+0000 7fda8b200640 1 --2- 172.21.15.16:0/3902474669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda8c06b710 0x7fda8c19bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.927+0000 7fda8b200640 1 -- 172.21.15.16:0/3902474669 >> 172.21.15.16:0/3902474669 conn(0x7fda8c0fdd80 msgr2=0x7fda8c1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.927+0000 7fda8b200640 1 -- 172.21.15.16:0/3902474669 shutdown_connections 2024-09-21T03:47:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:07.927+0000 7fda8b200640 1 -- 172.21.15.16:0/3902474669 wait complete. 2024-09-21T03:47:08.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:08 smithi016 ceph-mon[30375]: pgmap v443: 129 pgs: 129 active+clean; 672 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T03:47:09.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:08 smithi134 ceph-mon[35031]: pgmap v443: 129 pgs: 129 active+clean; 672 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T03:47:09.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:08 smithi123 ceph-mon[34673]: pgmap v443: 129 pgs: 129 active+clean; 672 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T03:47:10.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:47:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:11.055 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:10 smithi016 ceph-mon[30375]: pgmap v444: 129 pgs: 129 active+clean; 672 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:47:11.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:10 smithi134 ceph-mon[35031]: pgmap v444: 129 pgs: 129 active+clean; 672 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:47:11.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:10 smithi123 ceph-mon[34673]: pgmap v444: 129 pgs: 129 active+clean; 672 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T03:47:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.574+0000 7f3ceaa00640 1 -- 172.21.15.16:0/324734697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ce4109c40 msgr2=0x7f3ce4110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.574+0000 7f3ceaa00640 1 --2- 172.21.15.16:0/324734697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ce4109c40 0x7f3ce4110470 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009ca0 tx=0x7f3cdc02a8b0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.574+0000 7f3ceaa00640 1 -- 172.21.15.16:0/324734697 shutdown_connections 2024-09-21T03:47:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.574+0000 7f3ceaa00640 1 --2- 172.21.15.16:0/324734697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ce4109c40 0x7f3ce4110470 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.574+0000 7f3ceaa00640 1 --2- 172.21.15.16:0/324734697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce41092f0 0x7f3ce4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.574+0000 7f3ceaa00640 1 --2- 172.21.15.16:0/324734697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ce41039b0 0x7f3ce4103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.574+0000 7f3ceaa00640 1 -- 172.21.15.16:0/324734697 >> 172.21.15.16:0/324734697 conn(0x7f3ce40ff440 msgr2=0x7f3ce4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.575+0000 7f3ceaa00640 1 -- 172.21.15.16:0/324734697 shutdown_connections 2024-09-21T03:47:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.575+0000 7f3ceaa00640 1 -- 172.21.15.16:0/324734697 wait complete. 2024-09-21T03:47:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.577+0000 7f3ceaa00640 1 Processor -- start 2024-09-21T03:47:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.577+0000 7f3ceaa00640 1 -- start start 2024-09-21T03:47:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.577+0000 7f3ceaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce41039b0 0x7f3ce410d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.578+0000 7f3ceaa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ce41092f0 0x7f3ce410dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.578+0000 7f3ceaa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ce4109c40 0x7f3ce410e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.578+0000 7f3ceaa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce4077040 con 0x7f3ce41039b0 2024-09-21T03:47:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.578+0000 7f3ceaa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce4076ec0 con 0x7f3ce4109c40 2024-09-21T03:47:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.578+0000 7f3ceaa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce40771c0 con 0x7f3ce41092f0 2024-09-21T03:47:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.578+0000 7f3ce8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ce41092f0 0x7f3ce410dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.578+0000 7f3ce8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ce41092f0 0x7f3ce410dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:48516/0 (socket says 172.21.15.16:48516) 2024-09-21T03:47:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.578+0000 7f3ce8c00640 1 -- 172.21.15.16:0/3337497952 learned_addr learned my addr 172.21.15.16:0/3337497952 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.578+0000 7f3ce9600640 1 --2- 172.21.15.16:0/3337497952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce41039b0 0x7f3ce410d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.579+0000 7f3ce8c00640 1 -- 172.21.15.16:0/3337497952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ce4109c40 msgr2=0x7f3ce410e2d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:47:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.579+0000 7f3ce8c00640 1 --2- 172.21.15.16:0/3337497952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ce4109c40 0x7f3ce410e2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.579+0000 7f3ce8c00640 1 -- 172.21.15.16:0/3337497952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce41039b0 msgr2=0x7f3ce410d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.579+0000 7f3ce8c00640 1 --2- 172.21.15.16:0/3337497952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce41039b0 0x7f3ce410d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.579+0000 7f3ce8c00640 1 -- 172.21.15.16:0/3337497952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce41ac940 con 0x7f3ce41092f0 2024-09-21T03:47:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.579+0000 7f3ce8c00640 1 --2- 172.21.15.16:0/3337497952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ce41092f0 0x7f3ce410dd90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000e9f0 tx=0x7f3cd000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.580+0000 7f3ce2000640 1 -- 172.21.15.16:0/3337497952 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd000ccc0 con 0x7f3ce41092f0 2024-09-21T03:47:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.580+0000 7f3ce2000640 1 -- 172.21.15.16:0/3337497952 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cd00108e0 con 0x7f3ce41092f0 2024-09-21T03:47:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.580+0000 7f3ce2000640 1 -- 172.21.15.16:0/3337497952 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0018860 con 0x7f3ce41092f0 2024-09-21T03:47:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.580+0000 7f3ceaa00640 1 -- 172.21.15.16:0/3337497952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce41acc30 con 0x7f3ce41092f0 2024-09-21T03:47:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.580+0000 7f3ceaa00640 1 -- 172.21.15.16:0/3337497952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce41ad120 con 0x7f3ce41092f0 2024-09-21T03:47:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.582+0000 7f3ceaa00640 1 -- 172.21.15.16:0/3337497952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce4103dc0 con 0x7f3ce41092f0 2024-09-21T03:47:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.587+0000 7f3ce2000640 1 -- 172.21.15.16:0/3337497952 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3cd0010430 con 0x7f3ce41092f0 2024-09-21T03:47:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.590+0000 7f3ce2000640 1 --2- 172.21.15.16:0/3337497952 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3ccc0b6590 0x7f3ccc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.591+0000 7f3ce2000640 1 -- 172.21.15.16:0/3337497952 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f3cd00e7ea0 con 0x7f3ce41092f0 2024-09-21T03:47:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.591+0000 7f3ce9600640 1 --2- 172.21.15.16:0/3337497952 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3ccc0b6590 0x7f3ccc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.591+0000 7f3ce2000640 1 -- 172.21.15.16:0/3337497952 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3cd00e8300 con 0x7f3ce41092f0 2024-09-21T03:47:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.592+0000 7f3ce9600640 1 --2- 172.21.15.16:0/3337497952 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3ccc0b6590 0x7f3ccc0b8aa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3ce41102b0 tx=0x7f3cd8005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:11.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.856+0000 7f3ceaa00640 1 -- 172.21.15.16:0/3337497952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ce402e640 con 0x7f3ce41092f0 2024-09-21T03:47:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.857+0000 7f3ce2000640 1 -- 172.21.15.16:0/3337497952 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f3cd00ae320 con 0x7f3ce41092f0 2024-09-21T03:47:11.859 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:47:11.859 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:47:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:47:11.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.859+0000 7f3ceaa00640 1 -- 172.21.15.16:0/3337497952 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3ccc0b6590 msgr2=0x7f3ccc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:11.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.859+0000 7f3ceaa00640 1 --2- 172.21.15.16:0/3337497952 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3ccc0b6590 0x7f3ccc0b8aa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3ce41102b0 tx=0x7f3cd8005f90 comp rx=0 tx=0).stop 2024-09-21T03:47:11.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.859+0000 7f3ceaa00640 1 -- 172.21.15.16:0/3337497952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ce41092f0 msgr2=0x7f3ce410dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:11.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.859+0000 7f3ceaa00640 1 --2- 172.21.15.16:0/3337497952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ce41092f0 0x7f3ce410dd90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000e9f0 tx=0x7f3cd000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.859+0000 7f3ceaa00640 1 -- 172.21.15.16:0/3337497952 shutdown_connections 2024-09-21T03:47:11.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.860+0000 7f3ceaa00640 1 --2- 172.21.15.16:0/3337497952 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3ccc0b6590 0x7f3ccc0b8aa0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.860+0000 7f3ceaa00640 1 --2- 172.21.15.16:0/3337497952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ce4109c40 0x7f3ce410e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.860+0000 7f3ceaa00640 1 --2- 172.21.15.16:0/3337497952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ce41092f0 0x7f3ce410dd90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.860+0000 7f3ceaa00640 1 --2- 172.21.15.16:0/3337497952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce41039b0 0x7f3ce410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:11.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.860+0000 7f3ceaa00640 1 -- 172.21.15.16:0/3337497952 >> 172.21.15.16:0/3337497952 conn(0x7f3ce40ff440 msgr2=0x7f3ce4101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:11.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.860+0000 7f3ceaa00640 1 -- 172.21.15.16:0/3337497952 shutdown_connections 2024-09-21T03:47:11.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:11.860+0000 7f3ceaa00640 1 -- 172.21.15.16:0/3337497952 wait complete. 2024-09-21T03:47:12.756 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db --set-id e97f8336 --release --await' 2024-09-21T03:47:12.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db --set-id e97f8336 --release --await 2024-09-21T03:47:13.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:12 smithi134 ceph-mon[35031]: pgmap v445: 129 pgs: 129 active+clean; 672 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-21T03:47:13.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:12 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3337497952' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:13.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:12 smithi123 ceph-mon[34673]: pgmap v445: 129 pgs: 129 active+clean; 672 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-21T03:47:13.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:12 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3337497952' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:13.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:12 smithi016 ceph-mon[30375]: pgmap v445: 129 pgs: 129 active+clean; 672 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-21T03:47:13.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:12 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3337497952' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:13.304 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:14.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.178+0000 7f8f38c00640 1 -- 172.21.15.16:0/2539569746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3410b500 msgr2=0x7f8f3410b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.178+0000 7f8f38c00640 1 --2- 172.21.15.16:0/2539569746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3410b500 0x7f8f3410b910 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009900 tx=0x7f8f2002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.178+0000 7f8f38c00640 1 -- 172.21.15.16:0/2539569746 shutdown_connections 2024-09-21T03:47:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.178+0000 7f8f38c00640 1 --2- 172.21.15.16:0/2539569746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f3410be50 0x7f8f34112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.179+0000 7f8f38c00640 1 --2- 172.21.15.16:0/2539569746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3410b500 0x7f8f3410b910 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.179+0000 7f8f38c00640 1 --2- 172.21.15.16:0/2539569746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f34105bc0 0x7f8f34105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.179+0000 7f8f38c00640 1 -- 172.21.15.16:0/2539569746 >> 172.21.15.16:0/2539569746 conn(0x7f8f34101630 msgr2=0x7f8f34103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.179+0000 7f8f38c00640 1 -- 172.21.15.16:0/2539569746 shutdown_connections 2024-09-21T03:47:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.179+0000 7f8f38c00640 1 -- 172.21.15.16:0/2539569746 wait complete. 2024-09-21T03:47:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.182+0000 7f8f38c00640 1 Processor -- start 2024-09-21T03:47:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.182+0000 7f8f38c00640 1 -- start start 2024-09-21T03:47:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.183+0000 7f8f38c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f34105bc0 0x7f8f3410fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.183+0000 7f8f38c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3410b500 0x7f8f34110060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.183+0000 7f8f38c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f3410be50 0x7f8f341105a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.183+0000 7f8f38c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34115b60 con 0x7f8f34105bc0 2024-09-21T03:47:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.183+0000 7f8f38c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341159e0 con 0x7f8f3410b500 2024-09-21T03:47:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.183+0000 7f8f38c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34115ce0 con 0x7f8f3410be50 2024-09-21T03:47:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.183+0000 7f8f33400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f34105bc0 0x7f8f3410fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.183+0000 7f8f33400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f34105bc0 0x7f8f3410fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59536/0 (socket says 172.21.15.16:59536) 2024-09-21T03:47:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.183+0000 7f8f33400640 1 -- 172.21.15.16:0/2722297346 learned_addr learned my addr 172.21.15.16:0/2722297346 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.184+0000 7f8f32a00640 1 --2- 172.21.15.16:0/2722297346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3410b500 0x7f8f34110060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.184+0000 7f8f33400640 1 -- 172.21.15.16:0/2722297346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f3410be50 msgr2=0x7f8f341105a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.184+0000 7f8f33e00640 1 --2- 172.21.15.16:0/2722297346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f3410be50 0x7f8f341105a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.184+0000 7f8f33400640 1 --2- 172.21.15.16:0/2722297346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f3410be50 0x7f8f341105a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.184+0000 7f8f33400640 1 -- 172.21.15.16:0/2722297346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3410b500 msgr2=0x7f8f34110060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.184+0000 7f8f33400640 1 --2- 172.21.15.16:0/2722297346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3410b500 0x7f8f34110060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.184+0000 7f8f33400640 1 -- 172.21.15.16:0/2722297346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f341a2210 con 0x7f8f34105bc0 2024-09-21T03:47:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.185+0000 7f8f33400640 1 --2- 172.21.15.16:0/2722297346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f34105bc0 0x7f8f3410fb20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400d950 tx=0x7f8f2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.185+0000 7f8f2be00640 1 -- 172.21.15.16:0/2722297346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24017070 con 0x7f8f34105bc0 2024-09-21T03:47:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.186+0000 7f8f38c00640 1 -- 172.21.15.16:0/2722297346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f341a2500 con 0x7f8f34105bc0 2024-09-21T03:47:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.186+0000 7f8f38c00640 1 -- 172.21.15.16:0/2722297346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f341a2a60 con 0x7f8f34105bc0 2024-09-21T03:47:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.187+0000 7f8f2be00640 1 -- 172.21.15.16:0/2722297346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f24009c30 con 0x7f8f34105bc0 2024-09-21T03:47:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.187+0000 7f8f2be00640 1 -- 172.21.15.16:0/2722297346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24020060 con 0x7f8f34105bc0 2024-09-21T03:47:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.190+0000 7f8f40dc8740 1 Processor -- start 2024-09-21T03:47:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.191+0000 7f8f40dc8740 1 -- start start 2024-09-21T03:47:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.191+0000 7f8f40dc8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699c08520 0x562699faf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.191+0000 7f8f40dc8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699bdf660 0x562699faf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.191+0000 7f8f40dc8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699be7840 0x562699fafdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.191+0000 7f8f40dc8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562699f26290 con 0x562699be7840 2024-09-21T03:47:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.191+0000 7f8f40dc8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562699f156c0 con 0x562699bdf660 2024-09-21T03:47:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.191+0000 7f8f40dc8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562699f194d0 con 0x562699c08520 2024-09-21T03:47:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.192+0000 7f8f32a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699be7840 0x562699fafdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.192+0000 7f8f32a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699be7840 0x562699fafdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59544/0 (socket says 172.21.15.16:59544) 2024-09-21T03:47:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.192+0000 7f8f32a00640 1 -- 172.21.15.16:0/2842719192 learned_addr learned my addr 172.21.15.16:0/2842719192 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.193+0000 7f8f32a00640 1 -- 172.21.15.16:0/2842719192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699c08520 msgr2=0x562699faf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.193+0000 7f8f33e00640 1 --2- 172.21.15.16:0/2842719192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699c08520 0x562699faf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.193+0000 7f8f32a00640 1 --2- 172.21.15.16:0/2842719192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699c08520 0x562699faf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.193+0000 7f8f32a00640 1 -- 172.21.15.16:0/2842719192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699bdf660 msgr2=0x562699faf880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:47:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.193+0000 7f8f32a00640 1 --2- 172.21.15.16:0/2842719192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699bdf660 0x562699faf880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.193+0000 7f8f32a00640 1 -- 172.21.15.16:0/2842719192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562699ea2460 con 0x562699be7840 2024-09-21T03:47:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.193+0000 7f8f32a00640 1 --2- 172.21.15.16:0/2842719192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699be7840 0x562699fafdc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8f20006010 tx=0x7f8f200040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81953a97d220c6c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.194+0000 7f8f1fe00640 1 -- 172.21.15.16:0/2842719192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20035030 con 0x562699be7840 2024-09-21T03:47:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.194+0000 7f8f1fe00640 1 -- 172.21.15.16:0/2842719192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f200044c0 con 0x562699be7840 2024-09-21T03:47:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.195+0000 7f8f2be00640 1 -- 172.21.15.16:0/2722297346 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8f2400f620 con 0x7f8f34105bc0 2024-09-21T03:47:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.195+0000 7f8f40dc8740 1 -- 172.21.15.16:0/2842719192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699be7840 msgr2=0x562699fafdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.195+0000 7f8f1fe00640 1 -- 172.21.15.16:0/2842719192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20035030 con 0x562699be7840 2024-09-21T03:47:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.195+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/2842719192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699be7840 0x562699fafdc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8f20006010 tx=0x7f8f200040d0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.196+0000 7f8f40dc8740 1 -- 172.21.15.16:0/2842719192 shutdown_connections 2024-09-21T03:47:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.196+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/2842719192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699be7840 0x562699fafdc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.196+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/2842719192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699bdf660 0x562699faf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.196+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/2842719192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699c08520 0x562699faf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.196+0000 7f8f40dc8740 1 -- 172.21.15.16:0/2842719192 >> 172.21.15.16:0/2842719192 conn(0x562699bfc910 msgr2=0x562699fad230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.197+0000 7f8f2be00640 1 --2- 172.21.15.16:0/2722297346 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8f180b6590 0x7f8f180b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.197+0000 7f8f2be00640 1 -- 172.21.15.16:0/2722297346 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f8f240e71b0 con 0x7f8f34105bc0 2024-09-21T03:47:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.198+0000 7f8f40dc8740 1 -- 172.21.15.16:0/2842719192 shutdown_connections 2024-09-21T03:47:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.198+0000 7f8f40dc8740 1 -- 172.21.15.16:0/2842719192 wait complete. 2024-09-21T03:47:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.199+0000 7f8f40dc8740 1 -- start start 2024-09-21T03:47:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.199+0000 7f8f32a00640 1 --2- 172.21.15.16:0/2722297346 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8f180b6590 0x7f8f180b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.199+0000 7f8f40dc8740 1 Processor -- start 2024-09-21T03:47:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.200+0000 7f8f32a00640 1 --2- 172.21.15.16:0/2722297346 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8f180b6590 0x7f8f180b8aa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8f20005ef0 tx=0x7f8f20009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.201+0000 7f8f40dc8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bdf660 0x562699fbab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.201+0000 7f8f40dc8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699be7840 0x562699fbb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.201+0000 7f8f40dc8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699c08520 0x562699fc6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.201+0000 7f8f40dc8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562699d76bb0 con 0x562699bdf660 2024-09-21T03:47:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.201+0000 7f8f40dc8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562699eb06c0 con 0x562699be7840 2024-09-21T03:47:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.201+0000 7f8f40dc8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562699fbdb10 con 0x562699c08520 2024-09-21T03:47:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.201+0000 7f8f40dc8740 10 client.0 _ll_register_callbacks cb 0x562699ecb490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.201+0000 7f8f32a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699be7840 0x562699fbb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.201+0000 7f8f32a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699be7840 0x562699fbb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:47020/0 (socket says 172.21.15.16:47020) 2024-09-21T03:47:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.201+0000 7f8f32a00640 1 -- 172.21.15.16:0/2284668738 learned_addr learned my addr 172.21.15.16:0/2284668738 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.201+0000 7f8f33e00640 1 --2- 172.21.15.16:0/2284668738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699c08520 0x562699fc6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.201+0000 7f8f33400640 1 --2- 172.21.15.16:0/2284668738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bdf660 0x562699fbab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.202+0000 7f8f32a00640 1 -- 172.21.15.16:0/2284668738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699c08520 msgr2=0x562699fc6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.202+0000 7f8f32a00640 1 --2- 172.21.15.16:0/2284668738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699c08520 0x562699fc6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.202+0000 7f8f32a00640 1 -- 172.21.15.16:0/2284668738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bdf660 msgr2=0x562699fbab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:14.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.202+0000 7f8f32a00640 1 --2- 172.21.15.16:0/2284668738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bdf660 0x562699fbab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.202+0000 7f8f32a00640 1 -- 172.21.15.16:0/2284668738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562699c73b00 con 0x562699be7840 2024-09-21T03:47:14.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.202+0000 7f8f33400640 1 --2- 172.21.15.16:0/2284668738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bdf660 0x562699fbab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:14.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.202+0000 7f8f32a00640 1 --2- 172.21.15.16:0/2284668738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699be7840 0x562699fbb120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8f2003f000 tx=0x7f8f200040d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=807f5c615118a663 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:14.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.202+0000 7f8f1cc00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:47:14.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.204+0000 7f8f1cc00640 1 -- 172.21.15.16:0/2284668738 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2005e060 con 0x562699be7840 2024-09-21T03:47:14.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.204+0000 7f8f1cc00640 1 -- 172.21.15.16:0/2284668738 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f8f200316b0 con 0x562699be7840 2024-09-21T03:47:14.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.204+0000 7f8f40dc8740 1 -- 172.21.15.16:0/2284668738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562699f281e0 con 0x562699be7840 2024-09-21T03:47:14.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.205+0000 7f8f1cc00640 1 -- 172.21.15.16:0/2284668738 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8f20035040 con 0x562699be7840 2024-09-21T03:47:14.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.205+0000 7f8f40dc8740 10 client.25655 fetch_fsmap learned FSMap version 30 2024-09-21T03:47:14.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.205+0000 7f8f40dc8740 1 -- 172.21.15.16:0/2284668738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562699cea960 con 0x562699be7840 2024-09-21T03:47:14.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.205+0000 7f8f1cc00640 1 -- 172.21.15.16:0/2284668738 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562699f281e0 con 0x562699be7840 2024-09-21T03:47:14.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.205+0000 7f8f1cc00640 1 -- 172.21.15.16:0/2284668738 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8f20059a10 con 0x562699be7840 2024-09-21T03:47:14.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.205+0000 7f8f40dc8740 10 client.25655 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:47:14.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.205+0000 7f8f40dc8740 10 client.25655 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:47:14.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.206+0000 7f8f40dc8740 20 client.25655 populate_metadata read hostname 'smithi016' 2024-09-21T03:47:14.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.206+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/2284668738 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562699bc6ea0 0x562699fccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.206+0000 7f8f40dc8740 4 client.25655 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:14.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.206+0000 7f8f40dc8740 1 -- 172.21.15.16:0/2284668738 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562699f23c90 con 0x562699bc6ea0 2024-09-21T03:47:14.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.206+0000 7f8f33400640 1 --2- 172.21.15.16:0/2284668738 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562699bc6ea0 0x562699fccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.207+0000 7f8f33400640 1 --2- 172.21.15.16:0/2284668738 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562699bc6ea0 0x562699fccc10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d336df582243222 server_cookie=6c6ab8dcf3743e56 in_seq=0 out_seq=0 2024-09-21T03:47:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.207+0000 7f8f1cc00640 10 client.25655 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.208+0000 7f8f1cc00640 1 -- 172.21.15.16:0/2284668738 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562699f23c90 con 0x562699bc6ea0 2024-09-21T03:47:14.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.208+0000 7f8f1cc00640 10 client.25655 handle_command_reply: tid=0 2024-09-21T03:47:14.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.208+0000 7f8f40dc8740 1 client.25655 shutdown 2024-09-21T03:47:14.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.211+0000 7f8f40dc8740 1 -- 172.21.15.16:0/2284668738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699be7840 msgr2=0x562699fbb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.211+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/2284668738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699be7840 0x562699fbb120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8f2003f000 tx=0x7f8f200040d0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.211+0000 7f8f40dc8740 1 -- 172.21.15.16:0/2284668738 shutdown_connections 2024-09-21T03:47:14.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.211+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/2284668738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699c08520 0x562699fc6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.211+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/2284668738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699be7840 0x562699fbb120 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.211+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/2284668738 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562699bc6ea0 0x562699fccc10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.211+0000 7f8f1cc00640 0 client.25655 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:14.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.211+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/2284668738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bdf660 0x562699fbab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.211+0000 7f8f40dc8740 1 -- 172.21.15.16:0/2284668738 >> 172.21.15.16:0/2284668738 conn(0x562699bfc910 msgr2=0x562699fb24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:14.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.212+0000 7f8f40dc8740 1 -- 172.21.15.16:0/2284668738 shutdown_connections 2024-09-21T03:47:14.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.212+0000 7f8f40dc8740 1 -- 172.21.15.16:0/2284668738 wait complete. 2024-09-21T03:47:14.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.212+0000 7f8f40dc8740 20 client.25655 trim_cache size 0 max 16384 2024-09-21T03:47:14.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.248+0000 7f8f40dc8740 1 Processor -- start 2024-09-21T03:47:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.249+0000 7f8f40dc8740 1 -- start start 2024-09-21T03:47:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.249+0000 7f8f40dc8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699bc6ea0 0x562699fbb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.249+0000 7f8f40dc8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bdf660 0x562699fb5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.249+0000 7f8f40dc8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699be7840 0x562699fb7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.249+0000 7f8f40dc8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1400fc40 con 0x562699bdf660 2024-09-21T03:47:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.249+0000 7f8f40dc8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8f14011780 con 0x562699be7840 2024-09-21T03:47:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.249+0000 7f8f40dc8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f14010310 con 0x562699bc6ea0 2024-09-21T03:47:14.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.250+0000 7f8f33400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699bc6ea0 0x562699fbb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.250+0000 7f8f33400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699bc6ea0 0x562699fbb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:50086/0 (socket says 172.21.15.16:50086) 2024-09-21T03:47:14.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.250+0000 7f8f33400640 1 -- 172.21.15.16:0/803586898 learned_addr learned my addr 172.21.15.16:0/803586898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:14.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.250+0000 7f8f33e00640 1 --2- 172.21.15.16:0/803586898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699be7840 0x562699fb7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.250+0000 7f8f32a00640 1 --2- 172.21.15.16:0/803586898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bdf660 0x562699fb5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.250+0000 7f8f33e00640 1 -- 172.21.15.16:0/803586898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699bc6ea0 msgr2=0x562699fbb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.250+0000 7f8f33e00640 1 --2- 172.21.15.16:0/803586898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699bc6ea0 0x562699fbb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.250+0000 7f8f33e00640 1 -- 172.21.15.16:0/803586898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bdf660 msgr2=0x562699fb5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.251+0000 7f8f33e00640 1 --2- 172.21.15.16:0/803586898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bdf660 0x562699fb5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.251+0000 7f8f33e00640 1 -- 172.21.15.16:0/803586898 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562699fb1b50 con 0x562699be7840 2024-09-21T03:47:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.251+0000 7f8f32a00640 1 --2- 172.21.15.16:0/803586898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bdf660 0x562699fb5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.251+0000 7f8f33e00640 1 --2- 172.21.15.16:0/803586898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699be7840 0x562699fb7db0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009eb0 tx=0x7f8f2c013830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6d228ac2393328f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:14.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.252+0000 7f8f1f400640 1 -- 172.21.15.16:0/803586898 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c00e690 con 0x562699be7840 2024-09-21T03:47:14.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.252+0000 7f8f1f400640 1 -- 172.21.15.16:0/803586898 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f2c00ece0 con 0x562699be7840 2024-09-21T03:47:14.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.252+0000 7f8f1f400640 1 -- 172.21.15.16:0/803586898 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c01bce0 con 0x562699be7840 2024-09-21T03:47:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.253+0000 7f8f40dc8740 1 -- 172.21.15.16:0/803586898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699be7840 msgr2=0x562699fb7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.253+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/803586898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699be7840 0x562699fb7db0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009eb0 tx=0x7f8f2c013830 comp rx=0 tx=0).stop 2024-09-21T03:47:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.253+0000 7f8f40dc8740 1 -- 172.21.15.16:0/803586898 shutdown_connections 2024-09-21T03:47:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.253+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/803586898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699be7840 0x562699fb7db0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.253+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/803586898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bdf660 0x562699fb5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.253+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/803586898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699bc6ea0 0x562699fbb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.253+0000 7f8f40dc8740 1 -- 172.21.15.16:0/803586898 >> 172.21.15.16:0/803586898 conn(0x562699bfc910 msgr2=0x562699fba950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.253+0000 7f8f40dc8740 1 -- 172.21.15.16:0/803586898 shutdown_connections 2024-09-21T03:47:14.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.254+0000 7f8f40dc8740 1 -- 172.21.15.16:0/803586898 wait complete. 2024-09-21T03:47:14.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.254+0000 7f8f40dc8740 1 -- start start 2024-09-21T03:47:14.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.254+0000 7f8f40dc8740 1 Processor -- start 2024-09-21T03:47:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.255+0000 7f8f40dc8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bc6ea0 0x562699fbf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.255+0000 7f8f40dc8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699bdf660 0x562699fbfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.255+0000 7f8f40dc8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699be7840 0x56269a030410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.255+0000 7f8f40dc8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f140110b0 con 0x562699bc6ea0 2024-09-21T03:47:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.255+0000 7f8f40dc8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8f140109e0 con 0x562699be7840 2024-09-21T03:47:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.255+0000 7f8f40dc8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1400dc20 con 0x562699bdf660 2024-09-21T03:47:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.255+0000 7f8f40dc8740 10 client.0 _ll_register_callbacks cb 0x7f8f20009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.255+0000 7f8f33400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bc6ea0 0x562699fbf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.256+0000 7f8f33400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bc6ea0 0x562699fbf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59574/0 (socket says 172.21.15.16:59574) 2024-09-21T03:47:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.256+0000 7f8f33400640 1 -- 172.21.15.16:0/3143457917 learned_addr learned my addr 172.21.15.16:0/3143457917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.256+0000 7f8f32a00640 1 --2- 172.21.15.16:0/3143457917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699bdf660 0x562699fbfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.256+0000 7f8f33400640 1 -- 172.21.15.16:0/3143457917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699bdf660 msgr2=0x562699fbfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.256+0000 7f8f33400640 1 --2- 172.21.15.16:0/3143457917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699bdf660 0x562699fbfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.256+0000 7f8f33400640 1 -- 172.21.15.16:0/3143457917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699be7840 msgr2=0x56269a030410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.256+0000 7f8f33400640 1 --2- 172.21.15.16:0/3143457917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699be7840 0x56269a030410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.256+0000 7f8f33400640 1 -- 172.21.15.16:0/3143457917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56269a030a80 con 0x562699bc6ea0 2024-09-21T03:47:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.257+0000 7f8f33400640 1 --2- 172.21.15.16:0/3143457917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bc6ea0 0x562699fbf8b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009610 tx=0x7f8f240a8ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0d0e35a00c2b6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.257+0000 7f8f1cc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:47:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.257+0000 7f8f1cc00640 1 -- 172.21.15.16:0/3143457917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f240ad630 con 0x562699bc6ea0 2024-09-21T03:47:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.257+0000 7f8f1cc00640 1 -- 172.21.15.16:0/3143457917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f8f240d4180 con 0x562699bc6ea0 2024-09-21T03:47:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.257+0000 7f8f40dc8740 1 -- 172.21.15.16:0/3143457917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f2c00ece0 con 0x562699bc6ea0 2024-09-21T03:47:14.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.258+0000 7f8f1cc00640 1 -- 172.21.15.16:0/3143457917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8f240b1a10 con 0x562699bc6ea0 2024-09-21T03:47:14.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.258+0000 7f8f40dc8740 10 client.15645 fetch_fsmap learned FSMap version 30 2024-09-21T03:47:14.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.258+0000 7f8f40dc8740 1 -- 172.21.15.16:0/3143457917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56269a037eb0 con 0x562699bc6ea0 2024-09-21T03:47:14.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.258+0000 7f8f1cc00640 1 -- 172.21.15.16:0/3143457917 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f2c00ece0 con 0x562699bc6ea0 2024-09-21T03:47:14.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.259+0000 7f8f1cc00640 1 -- 172.21.15.16:0/3143457917 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8f240abc00 con 0x562699bc6ea0 2024-09-21T03:47:14.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.259+0000 7f8f40dc8740 10 client.15645 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:47:14.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.259+0000 7f8f40dc8740 10 client.15645 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:47:14.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.259+0000 7f8f40dc8740 20 client.15645 populate_metadata read hostname 'smithi016' 2024-09-21T03:47:14.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.259+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/3143457917 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562699c08520 0x56269a03a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:14.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.259+0000 7f8f40dc8740 4 client.15645 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e97f8336", "release": true, "await": true}] 2024-09-21T03:47:14.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.259+0000 7f8f40dc8740 1 -- 172.21.15.16:0/3143457917 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e97f8336", "release": true, "await": true}) -- 0x562699f210c0 con 0x562699c08520 2024-09-21T03:47:14.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.259+0000 7f8f32a00640 1 --2- 172.21.15.16:0/3143457917 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562699c08520 0x56269a03a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:14.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.260+0000 7f8f32a00640 1 --2- 172.21.15.16:0/3143457917 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562699c08520 0x56269a03a6a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcab814d8cc17bbe server_cookie=645cfd14186412f9 in_seq=0 out_seq=0 2024-09-21T03:47:14.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.260+0000 7f8f1cc00640 10 client.15645 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:14.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.262+0000 7f8f1cc00640 1 -- 172.21.15.16:0/3143457917 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x562699f210c0 con 0x562699c08520 2024-09-21T03:47:14.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.262+0000 7f8f1cc00640 10 client.15645 handle_command_reply: tid=0 2024-09-21T03:47:14.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.262+0000 7f8f40dc8740 1 client.15645 shutdown 2024-09-21T03:47:14.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.263+0000 7f8f40dc8740 1 -- 172.21.15.16:0/3143457917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bc6ea0 msgr2=0x562699fbf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.263+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/3143457917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bc6ea0 0x562699fbf8b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009610 tx=0x7f8f240a8ef0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.263+0000 7f8f40dc8740 1 -- 172.21.15.16:0/3143457917 shutdown_connections 2024-09-21T03:47:14.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.263+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/3143457917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562699be7840 0x56269a030410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.263+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/3143457917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562699bdf660 0x562699fbfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.263+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/3143457917 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562699c08520 0x56269a03a6a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.263+0000 7f8f40dc8740 1 --2- 172.21.15.16:0/3143457917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562699bc6ea0 0x562699fbf8b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.263+0000 7f8f1cc00640 0 client.15645 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:14.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.263+0000 7f8f40dc8740 1 -- 172.21.15.16:0/3143457917 >> 172.21.15.16:0/3143457917 conn(0x562699bfc910 msgr2=0x562699fb0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:14.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.264+0000 7f8f40dc8740 1 -- 172.21.15.16:0/3143457917 shutdown_connections 2024-09-21T03:47:14.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.264+0000 7f8f40dc8740 1 -- 172.21.15.16:0/3143457917 wait complete. 2024-09-21T03:47:14.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.264+0000 7f8f40dc8740 20 client.15645 trim_cache size 0 max 16384 2024-09-21T03:47:14.267 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:47:14.267 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:47:14.267 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:47:14.267 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 4, 2024-09-21T03:47:14.267 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:47:14.267 INFO:teuthology.orchestra.run.smithi016.stdout: "e97f8336": { 2024-09-21T03:47:14.267 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 4, 2024-09-21T03:47:14.267 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:47:14.267 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:47:14.267 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:47:14.267 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:47:14.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:47:14.268 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:47:14.268 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 134.5, 2024-09-21T03:47:14.268 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:47:14.268 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:47:14.268 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:47:14.268 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:47:14.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:47:14.268 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:47:14.268 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:47:14.268 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:47:14.268 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:47:14.268 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:47:14.268 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:47:14.268 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:47:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.265+0000 7f8f38c00640 1 -- 172.21.15.16:0/2722297346 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8f180b6590 msgr2=0x7f8f180b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.265+0000 7f8f38c00640 1 --2- 172.21.15.16:0/2722297346 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8f180b6590 0x7f8f180b8aa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8f20005ef0 tx=0x7f8f20009260 comp rx=0 tx=0).stop 2024-09-21T03:47:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.265+0000 7f8f38c00640 1 -- 172.21.15.16:0/2722297346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f34105bc0 msgr2=0x7f8f3410fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.265+0000 7f8f38c00640 1 --2- 172.21.15.16:0/2722297346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f34105bc0 0x7f8f3410fb20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400d950 tx=0x7f8f2400de10 comp rx=0 tx=0).stop 2024-09-21T03:47:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.265+0000 7f8f38c00640 1 -- 172.21.15.16:0/2722297346 shutdown_connections 2024-09-21T03:47:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.265+0000 7f8f38c00640 1 --2- 172.21.15.16:0/2722297346 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8f180b6590 0x7f8f180b8aa0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.265+0000 7f8f38c00640 1 --2- 172.21.15.16:0/2722297346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f3410be50 0x7f8f341105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.265+0000 7f8f38c00640 1 --2- 172.21.15.16:0/2722297346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3410b500 0x7f8f34110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.265+0000 7f8f38c00640 1 --2- 172.21.15.16:0/2722297346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f34105bc0 0x7f8f3410fb20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.265+0000 7f8f38c00640 1 -- 172.21.15.16:0/2722297346 >> 172.21.15.16:0/2722297346 conn(0x7f8f34101630 msgr2=0x7f8f34103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:14.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.266+0000 7f8f38c00640 1 -- 172.21.15.16:0/2722297346 shutdown_connections 2024-09-21T03:47:14.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:14.266+0000 7f8f38c00640 1 -- 172.21.15.16:0/2722297346 wait complete. 2024-09-21T03:47:14.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:14 smithi016 ceph-mon[30375]: pgmap v446: 129 pgs: 129 active+clean; 673 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-21T03:47:14.958 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:14 smithi123 ceph-mon[34673]: pgmap v446: 129 pgs: 129 active+clean; 673 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-21T03:47:15.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:14 smithi134 ceph-mon[35031]: pgmap v446: 129 pgs: 129 active+clean; 673 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-21T03:47:15.205 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e97f8336', total seconds elapsed: 19.5 2024-09-21T03:47:16.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:15 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:47:16.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:15 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:47:16.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:15 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:47:17.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:16 smithi134 ceph-mon[35031]: pgmap v447: 129 pgs: 129 active+clean; 673 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T03:47:17.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:47:17.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:47:17.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:47:17.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:16 smithi123 ceph-mon[34673]: pgmap v447: 129 pgs: 129 active+clean; 673 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T03:47:17.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:47:17.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:47:17.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:47:17.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:16 smithi016 ceph-mon[30375]: pgmap v447: 129 pgs: 129 active+clean; 673 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T03:47:17.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:47:17.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:47:17.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:47:19.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:18 smithi134 ceph-mon[35031]: pgmap v448: 129 pgs: 129 active+clean; 673 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:47:19.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:18 smithi123 ceph-mon[34673]: pgmap v448: 129 pgs: 129 active+clean; 673 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:47:19.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:18 smithi016 ceph-mon[30375]: pgmap v448: 129 pgs: 129 active+clean; 673 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T03:47:21.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:20 smithi134 ceph-mon[35031]: pgmap v449: 129 pgs: 129 active+clean; 673 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T03:47:21.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:20 smithi123 ceph-mon[34673]: pgmap v449: 129 pgs: 129 active+clean; 673 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T03:47:21.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:20 smithi016 ceph-mon[30375]: pgmap v449: 129 pgs: 129 active+clean; 673 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T03:47:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.239+0000 7fa57a200640 1 -- 172.21.15.16:0/449316324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa57406b710 msgr2=0x7fa574106180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.239+0000 7fa57a200640 1 --2- 172.21.15.16:0/449316324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa57406b710 0x7fa574106180 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa568009900 tx=0x7fa56802a410 comp rx=0 tx=0).stop 2024-09-21T03:47:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.239+0000 7fa57a200640 1 -- 172.21.15.16:0/449316324 shutdown_connections 2024-09-21T03:47:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.239+0000 7fa57a200640 1 --2- 172.21.15.16:0/449316324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa574109280 0x7fa57410b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.239+0000 7fa57a200640 1 --2- 172.21.15.16:0/449316324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5741066c0 0x7fa574108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.239+0000 7fa57a200640 1 --2- 172.21.15.16:0/449316324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa57406b710 0x7fa574106180 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.239+0000 7fa57a200640 1 -- 172.21.15.16:0/449316324 >> 172.21.15.16:0/449316324 conn(0x7fa5740fdd80 msgr2=0x7fa574100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.239+0000 7fa57a200640 1 -- 172.21.15.16:0/449316324 shutdown_connections 2024-09-21T03:47:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.239+0000 7fa57a200640 1 -- 172.21.15.16:0/449316324 wait complete. 2024-09-21T03:47:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.241+0000 7fa57a200640 1 Processor -- start 2024-09-21T03:47:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.241+0000 7fa57a200640 1 -- start start 2024-09-21T03:47:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.241+0000 7fa57a200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa57406b710 0x7fa5741037c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.241+0000 7fa57a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5741066c0 0x7fa574102080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.241+0000 7fa57a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa574109280 0x7fa5741025c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.241+0000 7fa57a200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa57410f190 con 0x7fa574109280 2024-09-21T03:47:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.241+0000 7fa57a200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa57410f010 con 0x7fa5741066c0 2024-09-21T03:47:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.241+0000 7fa57a200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa57410f310 con 0x7fa57406b710 2024-09-21T03:47:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.242+0000 7fa573e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5741066c0 0x7fa574102080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.242+0000 7fa578e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa57406b710 0x7fa5741037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.242+0000 7fa579800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa574109280 0x7fa5741025c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.242+0000 7fa573e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5741066c0 0x7fa574102080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:47058/0 (socket says 172.21.15.16:47058) 2024-09-21T03:47:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.242+0000 7fa573e00640 1 -- 172.21.15.16:0/3734937931 learned_addr learned my addr 172.21.15.16:0/3734937931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.242+0000 7fa578e00640 1 -- 172.21.15.16:0/3734937931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5741066c0 msgr2=0x7fa574102080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.242+0000 7fa578e00640 1 --2- 172.21.15.16:0/3734937931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5741066c0 0x7fa574102080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.242+0000 7fa578e00640 1 -- 172.21.15.16:0/3734937931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa574109280 msgr2=0x7fa5741025c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.242+0000 7fa578e00640 1 --2- 172.21.15.16:0/3734937931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa574109280 0x7fa5741025c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.242+0000 7fa578e00640 1 -- 172.21.15.16:0/3734937931 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa574102cf0 con 0x7fa57406b710 2024-09-21T03:47:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.243+0000 7fa578e00640 1 --2- 172.21.15.16:0/3734937931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa57406b710 0x7fa5741037c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa56802aa00 tx=0x7fa568003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.243+0000 7fa571600640 1 -- 172.21.15.16:0/3734937931 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56803f070 con 0x7fa57406b710 2024-09-21T03:47:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.243+0000 7fa571600640 1 -- 172.21.15.16:0/3734937931 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa568031c50 con 0x7fa57406b710 2024-09-21T03:47:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.243+0000 7fa57a200640 1 -- 172.21.15.16:0/3734937931 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5741a3e90 con 0x7fa57406b710 2024-09-21T03:47:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.243+0000 7fa571600640 1 -- 172.21.15.16:0/3734937931 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568035030 con 0x7fa57406b710 2024-09-21T03:47:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.243+0000 7fa57a200640 1 -- 172.21.15.16:0/3734937931 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5741a4240 con 0x7fa57406b710 2024-09-21T03:47:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.247+0000 7fa5828a7740 1 Processor -- start 2024-09-21T03:47:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.247+0000 7fa5828a7740 1 -- start start 2024-09-21T03:47:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.247+0000 7fa5828a7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241e31b0 0x557f2458a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.247+0000 7fa5828a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241ba2f0 0x557f2458a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.247+0000 7fa5828a7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 0x557f2458b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.247+0000 7fa5828a7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557f244fd6b0 con 0x557f241c24d0 2024-09-21T03:47:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.247+0000 7fa5828a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557f244ecae0 con 0x557f241ba2f0 2024-09-21T03:47:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.247+0000 7fa5828a7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557f244f08f0 con 0x557f241e31b0 2024-09-21T03:47:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.247+0000 7fa571600640 1 -- 172.21.15.16:0/3734937931 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa568042070 con 0x7fa57406b710 2024-09-21T03:47:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.248+0000 7fa579800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241e31b0 0x557f2458a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.248+0000 7fa579800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241e31b0 0x557f2458a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:50128/0 (socket says 172.21.15.16:50128) 2024-09-21T03:47:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.248+0000 7fa579800640 1 -- 172.21.15.16:0/2230941131 learned_addr learned my addr 172.21.15.16:0/2230941131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.248+0000 7fa578e00640 1 --2- 172.21.15.16:0/2230941131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241ba2f0 0x557f2458a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.248+0000 7fa573e00640 1 --2- 172.21.15.16:0/2230941131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 0x557f2458b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.248+0000 7fa579800640 1 -- 172.21.15.16:0/2230941131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241ba2f0 msgr2=0x557f2458a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.248+0000 7fa579800640 1 --2- 172.21.15.16:0/2230941131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241ba2f0 0x557f2458a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.248+0000 7fa579800640 1 -- 172.21.15.16:0/2230941131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 msgr2=0x557f2458b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.249+0000 7fa579800640 1 --2- 172.21.15.16:0/2230941131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 0x557f2458b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.249+0000 7fa579800640 1 -- 172.21.15.16:0/2230941131 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f244b9780 con 0x557f241e31b0 2024-09-21T03:47:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.249+0000 7fa573e00640 1 --2- 172.21.15.16:0/2230941131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 0x557f2458b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.249+0000 7fa579800640 1 --2- 172.21.15.16:0/2230941131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241e31b0 0x557f2458a2b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0039f0 tx=0x7fa56c003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f220bd6ebe8dfec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.249+0000 7fa566000640 1 -- 172.21.15.16:0/2230941131 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c01a470 con 0x557f241e31b0 2024-09-21T03:47:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.249+0000 7fa566000640 1 -- 172.21.15.16:0/2230941131 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa56c015070 con 0x557f241e31b0 2024-09-21T03:47:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.249+0000 7fa566000640 1 -- 172.21.15.16:0/2230941131 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c0107e0 con 0x557f241e31b0 2024-09-21T03:47:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.250+0000 7fa571600640 1 --2- 172.21.15.16:0/3734937931 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa55c0b6510 0x7fa55c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.250+0000 7fa571600640 1 -- 172.21.15.16:0/3734937931 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fa568107e70 con 0x7fa57406b710 2024-09-21T03:47:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.250+0000 7fa578e00640 1 --2- 172.21.15.16:0/3734937931 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa55c0b6510 0x7fa55c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.251+0000 7fa5828a7740 1 -- 172.21.15.16:0/2230941131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241e31b0 msgr2=0x557f2458a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.251+0000 7fa5828a7740 1 --2- 172.21.15.16:0/2230941131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241e31b0 0x557f2458a2b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0039f0 tx=0x7fa56c003eb0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.251+0000 7fa5828a7740 1 -- 172.21.15.16:0/2230941131 shutdown_connections 2024-09-21T03:47:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.251+0000 7fa5828a7740 1 --2- 172.21.15.16:0/2230941131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 0x557f2458b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.251+0000 7fa5828a7740 1 --2- 172.21.15.16:0/2230941131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241ba2f0 0x557f2458a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.251+0000 7fa5828a7740 1 --2- 172.21.15.16:0/2230941131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241e31b0 0x557f2458a2b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.251+0000 7fa5828a7740 1 -- 172.21.15.16:0/2230941131 >> 172.21.15.16:0/2230941131 conn(0x557f241d7300 msgr2=0x557f24585d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.251+0000 7fa5828a7740 1 -- 172.21.15.16:0/2230941131 shutdown_connections 2024-09-21T03:47:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.251+0000 7fa578e00640 1 --2- 172.21.15.16:0/3734937931 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa55c0b6510 0x7fa55c0b8a20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa5680cd9a0 tx=0x7fa56810b8b0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.251+0000 7fa5828a7740 1 -- 172.21.15.16:0/2230941131 wait complete. 2024-09-21T03:47:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.252+0000 7fa5828a7740 1 -- start start 2024-09-21T03:47:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.252+0000 7fa5828a7740 1 Processor -- start 2024-09-21T03:47:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.252+0000 7fa5828a7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241ba2f0 0x557f245939d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.252+0000 7fa5828a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241c24d0 0x557f24594090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.253+0000 7fa5828a7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241e31b0 0x557f24594c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.253+0000 7fa5828a7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557f24424ab0 con 0x557f241e31b0 2024-09-21T03:47:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.253+0000 7fa5828a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557f24493570 con 0x557f241c24d0 2024-09-21T03:47:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.253+0000 7fa5828a7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557f2458bf90 con 0x557f241ba2f0 2024-09-21T03:47:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.253+0000 7fa5828a7740 10 client.0 _ll_register_callbacks cb 0x557f244a2100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.253+0000 7fa579800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241e31b0 0x557f24594c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.253+0000 7fa579800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241e31b0 0x557f24594c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59618/0 (socket says 172.21.15.16:59618) 2024-09-21T03:47:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.253+0000 7fa579800640 1 -- 172.21.15.16:0/3813390413 learned_addr learned my addr 172.21.15.16:0/3813390413 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.253+0000 7fa573e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241c24d0 0x557f24594090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.253+0000 7fa579800640 1 --2- 172.21.15.16:0/3813390413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241ba2f0 0x557f245939d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.253+0000 7fa573e00640 1 -- 172.21.15.16:0/3813390413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241ba2f0 msgr2=0x557f245939d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.253+0000 7fa573e00640 1 --2- 172.21.15.16:0/3813390413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241ba2f0 0x557f245939d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.253+0000 7fa573e00640 1 -- 172.21.15.16:0/3813390413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241e31b0 msgr2=0x557f24594c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.253+0000 7fa573e00640 1 --2- 172.21.15.16:0/3813390413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241e31b0 0x557f24594c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.253+0000 7fa573e00640 1 -- 172.21.15.16:0/3813390413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f2424e790 con 0x557f241c24d0 2024-09-21T03:47:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.254+0000 7fa573e00640 1 --2- 172.21.15.16:0/3813390413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241c24d0 0x557f24594090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa560012700 tx=0x7fa560012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7641ff2174a3a5bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.254+0000 7fa55aa00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:47:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.254+0000 7fa55aa00640 1 -- 172.21.15.16:0/3813390413 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56000f940 con 0x557f241c24d0 2024-09-21T03:47:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.254+0000 7fa55aa00640 1 -- 172.21.15.16:0/3813390413 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fa56000fae0 con 0x557f241c24d0 2024-09-21T03:47:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.254+0000 7fa5828a7740 1 -- 172.21.15.16:0/3813390413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa56c015070 con 0x557f241c24d0 2024-09-21T03:47:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.255+0000 7fa55aa00640 1 -- 172.21.15.16:0/3813390413 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa560025050 con 0x557f241c24d0 2024-09-21T03:47:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.255+0000 7fa5828a7740 10 client.25670 fetch_fsmap learned FSMap version 30 2024-09-21T03:47:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.255+0000 7fa5828a7740 1 -- 172.21.15.16:0/3813390413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f242c7620 con 0x557f241c24d0 2024-09-21T03:47:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.255+0000 7fa55aa00640 1 -- 172.21.15.16:0/3813390413 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa56c015070 con 0x557f241c24d0 2024-09-21T03:47:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.255+0000 7fa55aa00640 1 -- 172.21.15.16:0/3813390413 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa56001cc90 con 0x557f241c24d0 2024-09-21T03:47:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.255+0000 7fa5828a7740 10 client.25670 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:47:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.255+0000 7fa5828a7740 10 client.25670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.255+0000 7fa5828a7740 20 client.25670 populate_metadata read hostname 'smithi016' 2024-09-21T03:47:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.255+0000 7fa5828a7740 1 --2- 172.21.15.16:0/3813390413 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557f241a1d80 0x557f245a1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.255+0000 7fa5828a7740 4 client.25670 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.255+0000 7fa5828a7740 1 -- 172.21.15.16:0/3813390413 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f244ff600 con 0x557f241a1d80 2024-09-21T03:47:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.256+0000 7fa579800640 1 --2- 172.21.15.16:0/3813390413 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557f241a1d80 0x557f245a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.256+0000 7fa579800640 1 --2- 172.21.15.16:0/3813390413 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557f241a1d80 0x557f245a1180 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3996ed09e0ed6143 server_cookie=73c8e85946fab562 in_seq=0 out_seq=0 2024-09-21T03:47:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.256+0000 7fa55aa00640 10 client.25670 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.257+0000 7fa55aa00640 1 -- 172.21.15.16:0/3813390413 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557f244ff600 con 0x557f241a1d80 2024-09-21T03:47:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.257+0000 7fa55aa00640 10 client.25670 handle_command_reply: tid=0 2024-09-21T03:47:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.257+0000 7fa5828a7740 1 client.25670 shutdown 2024-09-21T03:47:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.257+0000 7fa5828a7740 1 -- 172.21.15.16:0/3813390413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241c24d0 msgr2=0x557f24594090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.257+0000 7fa5828a7740 1 --2- 172.21.15.16:0/3813390413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241c24d0 0x557f24594090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa560012700 tx=0x7fa560012bc0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.257+0000 7fa5828a7740 1 -- 172.21.15.16:0/3813390413 shutdown_connections 2024-09-21T03:47:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.257+0000 7fa5828a7740 1 --2- 172.21.15.16:0/3813390413 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557f241a1d80 0x557f245a1180 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.257+0000 7fa5828a7740 1 --2- 172.21.15.16:0/3813390413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241e31b0 0x557f24594c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.257+0000 7fa5828a7740 1 --2- 172.21.15.16:0/3813390413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241c24d0 0x557f24594090 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.258+0000 7fa5828a7740 1 --2- 172.21.15.16:0/3813390413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241ba2f0 0x557f245939d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.258+0000 7fa55aa00640 0 client.25670 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.258+0000 7fa5828a7740 1 -- 172.21.15.16:0/3813390413 >> 172.21.15.16:0/3813390413 conn(0x557f241d7300 msgr2=0x557f24584360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.258+0000 7fa5828a7740 1 -- 172.21.15.16:0/3813390413 shutdown_connections 2024-09-21T03:47:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.258+0000 7fa5828a7740 1 -- 172.21.15.16:0/3813390413 wait complete. 2024-09-21T03:47:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.258+0000 7fa5828a7740 20 client.25670 trim_cache size 0 max 16384 2024-09-21T03:47:21.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.273+0000 7fa5828a7740 1 Processor -- start 2024-09-21T03:47:21.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.273+0000 7fa5828a7740 1 -- start start 2024-09-21T03:47:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.274+0000 7fa5828a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241ba2f0 0x557f2458ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.274+0000 7fa5828a7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 0x557f24592ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.274+0000 7fa5828a7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241e31b0 0x557f24584a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.274+0000 7fa5828a7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741ae810 con 0x557f241c24d0 2024-09-21T03:47:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.274+0000 7fa5828a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741b0350 con 0x557f241ba2f0 2024-09-21T03:47:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.274+0000 7fa5828a7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741aeee0 con 0x557f241e31b0 2024-09-21T03:47:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.274+0000 7fa579800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241ba2f0 0x557f2458ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.274+0000 7fa573e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 0x557f24592ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.274+0000 7fa579800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241ba2f0 0x557f2458ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:47082/0 (socket says 172.21.15.16:47082) 2024-09-21T03:47:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.274+0000 7fa579800640 1 -- 172.21.15.16:0/818360003 learned_addr learned my addr 172.21.15.16:0/818360003 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.274+0000 7fa573e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 0x557f24592ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59634/0 (socket says 172.21.15.16:59634) 2024-09-21T03:47:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.274+0000 7fa579800640 1 --2- 172.21.15.16:0/818360003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241e31b0 0x557f24584a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.275+0000 7fa579800640 1 -- 172.21.15.16:0/818360003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241e31b0 msgr2=0x557f24584a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.275+0000 7fa579800640 1 --2- 172.21.15.16:0/818360003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241e31b0 0x557f24584a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.275+0000 7fa579800640 1 -- 172.21.15.16:0/818360003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 msgr2=0x557f24592ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.275+0000 7fa579800640 1 --2- 172.21.15.16:0/818360003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 0x557f24592ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.275+0000 7fa579800640 1 -- 172.21.15.16:0/818360003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f245813c0 con 0x557f241ba2f0 2024-09-21T03:47:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.275+0000 7fa579800640 1 --2- 172.21.15.16:0/818360003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241ba2f0 0x557f2458ed80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x557f245a18e0 tx=0x7fa56c005a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12a2a7f41ef6bd4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.275+0000 7fa565600640 1 -- 172.21.15.16:0/818360003 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c011a30 con 0x557f241ba2f0 2024-09-21T03:47:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.275+0000 7fa565600640 1 -- 172.21.15.16:0/818360003 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa56c015040 con 0x557f241ba2f0 2024-09-21T03:47:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.275+0000 7fa565600640 1 -- 172.21.15.16:0/818360003 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c005640 con 0x557f241ba2f0 2024-09-21T03:47:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.276+0000 7fa5828a7740 1 -- 172.21.15.16:0/818360003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241ba2f0 msgr2=0x557f2458ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.276+0000 7fa5828a7740 1 --2- 172.21.15.16:0/818360003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241ba2f0 0x557f2458ed80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x557f245a18e0 tx=0x7fa56c005a50 comp rx=0 tx=0).stop 2024-09-21T03:47:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.276+0000 7fa5828a7740 1 -- 172.21.15.16:0/818360003 shutdown_connections 2024-09-21T03:47:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.276+0000 7fa5828a7740 1 --2- 172.21.15.16:0/818360003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241e31b0 0x557f24584a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.276+0000 7fa5828a7740 1 --2- 172.21.15.16:0/818360003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 0x557f24592ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.276+0000 7fa5828a7740 1 --2- 172.21.15.16:0/818360003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241ba2f0 0x557f2458ed80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.276+0000 7fa5828a7740 1 -- 172.21.15.16:0/818360003 >> 172.21.15.16:0/818360003 conn(0x557f241d7300 msgr2=0x557f24584360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.277+0000 7fa5828a7740 1 -- 172.21.15.16:0/818360003 shutdown_connections 2024-09-21T03:47:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.277+0000 7fa5828a7740 1 -- 172.21.15.16:0/818360003 wait complete. 2024-09-21T03:47:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.277+0000 7fa5828a7740 1 -- start start 2024-09-21T03:47:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.277+0000 7fa5828a7740 1 Processor -- start 2024-09-21T03:47:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.278+0000 7fa5828a7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241ba2f0 0x557f2459b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.278+0000 7fa5828a7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 0x557f2459a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.278+0000 7fa5828a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241e31b0 0x557f2459a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.278+0000 7fa5828a7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741afc80 con 0x557f241c24d0 2024-09-21T03:47:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.278+0000 7fa5828a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741af5b0 con 0x557f241e31b0 2024-09-21T03:47:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.278+0000 7fa5828a7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741ac7f0 con 0x557f241ba2f0 2024-09-21T03:47:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.278+0000 7fa5828a7740 10 client.0 _ll_register_callbacks cb 0x7fa560009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.278+0000 7fa579800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241e31b0 0x557f2459a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.278+0000 7fa579800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241e31b0 0x557f2459a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:47088/0 (socket says 172.21.15.16:47088) 2024-09-21T03:47:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.278+0000 7fa579800640 1 -- 172.21.15.16:0/596042368 learned_addr learned my addr 172.21.15.16:0/596042368 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.278+0000 7fa573e00640 1 --2- 172.21.15.16:0/596042368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 0x557f2459a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.278+0000 7fa579800640 1 --2- 172.21.15.16:0/596042368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241ba2f0 0x557f2459b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.279+0000 7fa573e00640 1 -- 172.21.15.16:0/596042368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241ba2f0 msgr2=0x557f2459b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.279+0000 7fa573e00640 1 --2- 172.21.15.16:0/596042368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241ba2f0 0x557f2459b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.279+0000 7fa573e00640 1 -- 172.21.15.16:0/596042368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241e31b0 msgr2=0x557f2459a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.279+0000 7fa573e00640 1 --2- 172.21.15.16:0/596042368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241e31b0 0x557f2459a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.279+0000 7fa573e00640 1 -- 172.21.15.16:0/596042368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f24598dc0 con 0x557f241c24d0 2024-09-21T03:47:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.279+0000 7fa573e00640 1 --2- 172.21.15.16:0/596042368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 0x557f2459a310 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa560000c00 tx=0x7fa560009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b90c1248aafed2a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.279+0000 7fa55aa00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:47:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.279+0000 7fa55aa00640 1 -- 172.21.15.16:0/596042368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560005350 con 0x557f241c24d0 2024-09-21T03:47:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.279+0000 7fa55aa00640 1 -- 172.21.15.16:0/596042368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fa560018070 con 0x557f241c24d0 2024-09-21T03:47:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.279+0000 7fa5828a7740 1 -- 172.21.15.16:0/596042368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa56c015040 con 0x557f241c24d0 2024-09-21T03:47:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.280+0000 7fa55aa00640 1 -- 172.21.15.16:0/596042368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa560037050 con 0x557f241c24d0 2024-09-21T03:47:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.280+0000 7fa5828a7740 10 client.15663 fetch_fsmap learned FSMap version 30 2024-09-21T03:47:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.280+0000 7fa5828a7740 1 -- 172.21.15.16:0/596042368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f2460bd80 con 0x557f241c24d0 2024-09-21T03:47:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.280+0000 7fa55aa00640 1 -- 172.21.15.16:0/596042368 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa56c015040 con 0x557f241c24d0 2024-09-21T03:47:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.280+0000 7fa55aa00640 1 -- 172.21.15.16:0/596042368 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa560010830 con 0x557f241c24d0 2024-09-21T03:47:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.280+0000 7fa5828a7740 10 client.15663 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:47:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.280+0000 7fa5828a7740 10 client.15663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.280+0000 7fa5828a7740 20 client.15663 populate_metadata read hostname 'smithi016' 2024-09-21T03:47:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.280+0000 7fa5828a7740 1 --2- 172.21.15.16:0/596042368 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557f241771c0 0x557f2460e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.280+0000 7fa5828a7740 4 client.15663 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:47:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.280+0000 7fa5828a7740 1 -- 172.21.15.16:0/596042368 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f244fb0b0 con 0x557f241771c0 2024-09-21T03:47:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.281+0000 7fa579800640 1 --2- 172.21.15.16:0/596042368 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557f241771c0 0x557f2460e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.282+0000 7fa579800640 1 --2- 172.21.15.16:0/596042368 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557f241771c0 0x557f2460e7f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a2608fd859abb70 server_cookie=52ab99d53554139d in_seq=0 out_seq=0 2024-09-21T03:47:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.282+0000 7fa55aa00640 10 client.15663 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.289+0000 7fa55aa00640 1 -- 172.21.15.16:0/596042368 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557f244fb0b0 con 0x557f241771c0 2024-09-21T03:47:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.289+0000 7fa55aa00640 10 client.15663 handle_command_reply: tid=0 2024-09-21T03:47:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.289+0000 7fa5828a7740 1 client.15663 shutdown 2024-09-21T03:47:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.290+0000 7fa5828a7740 1 -- 172.21.15.16:0/596042368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 msgr2=0x557f2459a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.290+0000 7fa5828a7740 1 --2- 172.21.15.16:0/596042368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 0x557f2459a310 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa560000c00 tx=0x7fa560009f20 comp rx=0 tx=0).stop 2024-09-21T03:47:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.290+0000 7fa5828a7740 1 -- 172.21.15.16:0/596042368 shutdown_connections 2024-09-21T03:47:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.290+0000 7fa5828a7740 1 --2- 172.21.15.16:0/596042368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f241e31b0 0x557f2459a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.290+0000 7fa5828a7740 1 --2- 172.21.15.16:0/596042368 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557f241771c0 0x557f2460e7f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.290+0000 7fa5828a7740 1 --2- 172.21.15.16:0/596042368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f241c24d0 0x557f2459a310 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.290+0000 7fa5828a7740 1 --2- 172.21.15.16:0/596042368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f241ba2f0 0x557f2459b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.290+0000 7fa5828a7740 1 -- 172.21.15.16:0/596042368 >> 172.21.15.16:0/596042368 conn(0x557f241d7300 msgr2=0x557f2458eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.290+0000 7fa55aa00640 0 client.15663 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.290+0000 7fa5828a7740 1 -- 172.21.15.16:0/596042368 shutdown_connections 2024-09-21T03:47:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.290+0000 7fa5828a7740 1 -- 172.21.15.16:0/596042368 wait complete. 2024-09-21T03:47:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.290+0000 7fa5828a7740 20 client.15663 trim_cache size 0 max 16384 2024-09-21T03:47:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:47:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:47:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:47:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:47:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.291+0000 7fa57a200640 1 -- 172.21.15.16:0/3734937931 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa55c0b6510 msgr2=0x7fa55c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.291+0000 7fa57a200640 1 --2- 172.21.15.16:0/3734937931 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa55c0b6510 0x7fa55c0b8a20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa5680cd9a0 tx=0x7fa56810b8b0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.291+0000 7fa57a200640 1 -- 172.21.15.16:0/3734937931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa57406b710 msgr2=0x7fa5741037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.291+0000 7fa57a200640 1 --2- 172.21.15.16:0/3734937931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa57406b710 0x7fa5741037c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa56802aa00 tx=0x7fa568003c10 comp rx=0 tx=0).stop 2024-09-21T03:47:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.291+0000 7fa57a200640 1 -- 172.21.15.16:0/3734937931 shutdown_connections 2024-09-21T03:47:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.291+0000 7fa57a200640 1 --2- 172.21.15.16:0/3734937931 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa55c0b6510 0x7fa55c0b8a20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.291+0000 7fa57a200640 1 --2- 172.21.15.16:0/3734937931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa574109280 0x7fa5741025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.291+0000 7fa57a200640 1 --2- 172.21.15.16:0/3734937931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5741066c0 0x7fa574102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.291+0000 7fa57a200640 1 --2- 172.21.15.16:0/3734937931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa57406b710 0x7fa5741037c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.291+0000 7fa57a200640 1 -- 172.21.15.16:0/3734937931 >> 172.21.15.16:0/3734937931 conn(0x7fa5740fdd80 msgr2=0x7fa574104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.291+0000 7fa57a200640 1 -- 172.21.15.16:0/3734937931 shutdown_connections 2024-09-21T03:47:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:21.291+0000 7fa57a200640 1 -- 172.21.15.16:0/3734937931 wait complete. 2024-09-21T03:47:23.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:22 smithi134 ceph-mon[35031]: pgmap v450: 129 pgs: 129 active+clean; 673 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T03:47:23.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:22 smithi123 ceph-mon[34673]: pgmap v450: 129 pgs: 129 active+clean; 673 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T03:47:23.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:22 smithi016 ceph-mon[30375]: pgmap v450: 129 pgs: 129 active+clean; 673 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T03:47:25.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:24 smithi134 ceph-mon[35031]: pgmap v451: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T03:47:25.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:24 smithi123 ceph-mon[34673]: pgmap v451: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T03:47:25.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:24 smithi016 ceph-mon[30375]: pgmap v451: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T03:47:26.733 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:26 smithi016 ceph-mon[30375]: pgmap v452: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:27.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:26 smithi134 ceph-mon[35031]: pgmap v452: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:27.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:26 smithi123 ceph-mon[34673]: pgmap v452: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:29.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:28 smithi134 ceph-mon[35031]: pgmap v453: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:29.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:28 smithi123 ceph-mon[34673]: pgmap v453: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:29.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:28 smithi016 ceph-mon[30375]: pgmap v453: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:30 smithi134 ceph-mon[35031]: pgmap v454: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:47:31.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:30 smithi123 ceph-mon[34673]: pgmap v454: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:47:31.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:30 smithi016 ceph-mon[30375]: pgmap v454: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T03:47:33.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:32 smithi134 ceph-mon[35031]: pgmap v455: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:33.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:32 smithi123 ceph-mon[34673]: pgmap v455: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:32 smithi016 ceph-mon[30375]: pgmap v455: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T03:47:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.486+0000 7f2eeb600640 1 -- 172.21.15.16:0/503267823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4104740 msgr2=0x7f2ee4104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.486+0000 7f2eeb600640 1 --2- 172.21.15.16:0/503267823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4104740 0x7f2ee4104b50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009900 tx=0x7f2ed002a410 comp rx=0 tx=0).stop 2024-09-21T03:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.486+0000 7f2eeb600640 1 -- 172.21.15.16:0/503267823 shutdown_connections 2024-09-21T03:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.486+0000 7f2eeb600640 1 --2- 172.21.15.16:0/503267823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ee41050e0 0x7f2ee410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.486+0000 7f2eeb600640 1 --2- 172.21.15.16:0/503267823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4104740 0x7f2ee4104b50 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.486+0000 7f2eeb600640 1 --2- 172.21.15.16:0/503267823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee41036c0 0x7f2ee4103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.486+0000 7f2eeb600640 1 -- 172.21.15.16:0/503267823 >> 172.21.15.16:0/503267823 conn(0x7f2ee40ff020 msgr2=0x7f2ee41014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.486+0000 7f2eeb600640 1 -- 172.21.15.16:0/503267823 shutdown_connections 2024-09-21T03:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.486+0000 7f2eeb600640 1 -- 172.21.15.16:0/503267823 wait complete. 2024-09-21T03:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.487+0000 7f2eeb600640 1 Processor -- start 2024-09-21T03:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.488+0000 7f2eeb600640 1 -- start start 2024-09-21T03:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.488+0000 7f2eeb600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee41036c0 0x7f2ee4199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.488+0000 7f2eeb600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4104740 0x7f2ee419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.488+0000 7f2eeb600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ee41050e0 0x7f2ee419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.488+0000 7f2eeb600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee410f310 con 0x7f2ee4104740 2024-09-21T03:47:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.488+0000 7f2eeb600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee410f190 con 0x7f2ee41036c0 2024-09-21T03:47:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.488+0000 7f2eeb600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee410f490 con 0x7f2ee41050e0 2024-09-21T03:47:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.488+0000 7f2ee3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4104740 0x7f2ee419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.488+0000 7f2ee9600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ee41050e0 0x7f2ee419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.488+0000 7f2ee8c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee41036c0 0x7f2ee4199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.488+0000 7f2ee3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4104740 0x7f2ee419a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44716/0 (socket says 172.21.15.16:44716) 2024-09-21T03:47:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.488+0000 7f2ee3e00640 1 -- 172.21.15.16:0/4278334472 learned_addr learned my addr 172.21.15.16:0/4278334472 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.489+0000 7f2ee9600640 1 -- 172.21.15.16:0/4278334472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee41036c0 msgr2=0x7f2ee4199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.489+0000 7f2ee9600640 1 --2- 172.21.15.16:0/4278334472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee41036c0 0x7f2ee4199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.489+0000 7f2ee9600640 1 -- 172.21.15.16:0/4278334472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4104740 msgr2=0x7f2ee419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.489+0000 7f2ee9600640 1 --2- 172.21.15.16:0/4278334472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4104740 0x7f2ee419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.489+0000 7f2ee9600640 1 -- 172.21.15.16:0/4278334472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee419b9a0 con 0x7f2ee41050e0 2024-09-21T03:47:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.489+0000 7f2ee9600640 1 --2- 172.21.15.16:0/4278334472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ee41050e0 0x7f2ee419b180 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009fd0 tx=0x7f2edc00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.489+0000 7f2ee1600640 1 -- 172.21.15.16:0/4278334472 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00ce20 con 0x7f2ee41050e0 2024-09-21T03:47:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.490+0000 7f2ee1600640 1 -- 172.21.15.16:0/4278334472 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2edc005020 con 0x7f2ee41050e0 2024-09-21T03:47:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.490+0000 7f2ee1600640 1 -- 172.21.15.16:0/4278334472 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc010690 con 0x7f2ee41050e0 2024-09-21T03:47:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.490+0000 7f2eeb600640 1 -- 172.21.15.16:0/4278334472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee419bc90 con 0x7f2ee41050e0 2024-09-21T03:47:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.490+0000 7f2eeb600640 1 -- 172.21.15.16:0/4278334472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee40757d0 con 0x7f2ee41050e0 2024-09-21T03:47:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.493+0000 7f2ef284f740 1 Processor -- start 2024-09-21T03:47:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.494+0000 7f2ee1600640 1 -- 172.21.15.16:0/4278334472 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2edc0054d0 con 0x7f2ee41050e0 2024-09-21T03:47:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.494+0000 7f2ef284f740 1 -- start start 2024-09-21T03:47:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.495+0000 7f2ef284f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7923131b0 0x55c7926ba0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.495+0000 7f2ef284f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922ea2f0 0x55c7926ba740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.495+0000 7f2ef284f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7922f24d0 0x55c7926bb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.495+0000 7f2ef284f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c79262d6b0 con 0x55c7922ea2f0 2024-09-21T03:47:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.495+0000 7f2ef284f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c79261cae0 con 0x55c7923131b0 2024-09-21T03:47:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.495+0000 7f2ef284f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c7926208f0 con 0x55c7922f24d0 2024-09-21T03:47:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.495+0000 7f2ee3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7923131b0 0x55c7926ba0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.495+0000 7f2ee3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7923131b0 0x55c7926ba0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:54154/0 (socket says 172.21.15.16:54154) 2024-09-21T03:47:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.495+0000 7f2ee3e00640 1 -- 172.21.15.16:0/2959562659 learned_addr learned my addr 172.21.15.16:0/2959562659 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.495+0000 7f2ee8c00640 1 --2- 172.21.15.16:0/2959562659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922ea2f0 0x55c7926ba740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.495+0000 7f2ee3e00640 1 --2- 172.21.15.16:0/2959562659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7922f24d0 0x55c7926bb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.496+0000 7f2ee8c00640 1 -- 172.21.15.16:0/2959562659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7922f24d0 msgr2=0x55c7926bb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.496+0000 7f2ee8c00640 1 --2- 172.21.15.16:0/2959562659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7922f24d0 0x55c7926bb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.496+0000 7f2ee8c00640 1 -- 172.21.15.16:0/2959562659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7923131b0 msgr2=0x55c7926ba0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.496+0000 7f2ee8c00640 1 --2- 172.21.15.16:0/2959562659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7923131b0 0x55c7926ba0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.496+0000 7f2ee8c00640 1 -- 172.21.15.16:0/2959562659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7925e9780 con 0x55c7922ea2f0 2024-09-21T03:47:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.496+0000 7f2ee3e00640 1 --2- 172.21.15.16:0/2959562659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7923131b0 0x55c7926ba0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.496+0000 7f2ee8c00640 1 --2- 172.21.15.16:0/2959562659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922ea2f0 0x55c7926ba740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800bbf0 tx=0x7f2ed80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910a02720c28b39 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.496+0000 7f2ee1600640 1 --2- 172.21.15.16:0/4278334472 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2ecc0ba9a0 0x7f2ecc0bceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.496+0000 7f2ed5c00640 1 -- 172.21.15.16:0/2959562659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8004030 con 0x55c7922ea2f0 2024-09-21T03:47:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.496+0000 7f2ee1600640 1 -- 172.21.15.16:0/4278334472 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f2edc0e7d10 con 0x7f2ee41050e0 2024-09-21T03:47:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.496+0000 7f2ed5c00640 1 -- 172.21.15.16:0/2959562659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ed80035f0 con 0x55c7922ea2f0 2024-09-21T03:47:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.496+0000 7f2ed5c00640 1 -- 172.21.15.16:0/2959562659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8011680 con 0x55c7922ea2f0 2024-09-21T03:47:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.496+0000 7f2ee3e00640 1 --2- 172.21.15.16:0/4278334472 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2ecc0ba9a0 0x7f2ecc0bceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.497+0000 7f2ef284f740 1 -- 172.21.15.16:0/2959562659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922ea2f0 msgr2=0x55c7926ba740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.497+0000 7f2ee3e00640 1 --2- 172.21.15.16:0/4278334472 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2ecc0ba9a0 0x7f2ecc0bceb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0031a50 tx=0x7f2ed0005d50 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.497+0000 7f2ef284f740 1 --2- 172.21.15.16:0/2959562659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922ea2f0 0x55c7926ba740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800bbf0 tx=0x7f2ed80107a0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.497+0000 7f2ef284f740 1 -- 172.21.15.16:0/2959562659 shutdown_connections 2024-09-21T03:47:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.497+0000 7f2ef284f740 1 --2- 172.21.15.16:0/2959562659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7922f24d0 0x55c7926bb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.497+0000 7f2ef284f740 1 --2- 172.21.15.16:0/2959562659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922ea2f0 0x55c7926ba740 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.497+0000 7f2ef284f740 1 --2- 172.21.15.16:0/2959562659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7923131b0 0x55c7926ba0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.497+0000 7f2ef284f740 1 -- 172.21.15.16:0/2959562659 >> 172.21.15.16:0/2959562659 conn(0x55c792307300 msgr2=0x55c7926b5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.498+0000 7f2ef284f740 1 -- 172.21.15.16:0/2959562659 shutdown_connections 2024-09-21T03:47:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.498+0000 7f2ef284f740 1 -- 172.21.15.16:0/2959562659 wait complete. 2024-09-21T03:47:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.498+0000 7f2ef284f740 1 -- start start 2024-09-21T03:47:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.498+0000 7f2ef284f740 1 Processor -- start 2024-09-21T03:47:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.499+0000 7f2ef284f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7922ea2f0 0x55c7926c3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.499+0000 7f2ef284f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 0x55c7926c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.499+0000 7f2ef284f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7923131b0 0x55c7926c4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.499+0000 7f2ef284f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c792554ab0 con 0x55c7922f24d0 2024-09-21T03:47:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.499+0000 7f2ef284f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c7925c3570 con 0x55c7923131b0 2024-09-21T03:47:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.499+0000 7f2ef284f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c7926bbdb0 con 0x55c7922ea2f0 2024-09-21T03:47:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.499+0000 7f2ef284f740 10 client.0 _ll_register_callbacks cb 0x55c7925d2100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.499+0000 7f2ee3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 0x55c7926c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.499+0000 7f2ee8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7922ea2f0 0x55c7926c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.499+0000 7f2ee3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 0x55c7926c3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44724/0 (socket says 172.21.15.16:44724) 2024-09-21T03:47:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.499+0000 7f2ee3e00640 1 -- 172.21.15.16:0/377966627 learned_addr learned my addr 172.21.15.16:0/377966627 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.499+0000 7f2ee9600640 1 --2- 172.21.15.16:0/377966627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7923131b0 0x55c7926c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.500+0000 7f2ee8c00640 1 -- 172.21.15.16:0/377966627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7923131b0 msgr2=0x55c7926c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.500+0000 7f2ee8c00640 1 --2- 172.21.15.16:0/377966627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7923131b0 0x55c7926c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.500+0000 7f2ee8c00640 1 -- 172.21.15.16:0/377966627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 msgr2=0x55c7926c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.500+0000 7f2ee8c00640 1 --2- 172.21.15.16:0/377966627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 0x55c7926c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.500+0000 7f2ee8c00640 1 -- 172.21.15.16:0/377966627 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c79237e790 con 0x55c7922ea2f0 2024-09-21T03:47:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.500+0000 7f2ee8c00640 1 --2- 172.21.15.16:0/377966627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7922ea2f0 0x55c7926c3760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8011010 tx=0x7f2ed800bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d247e12410ffe897 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.500+0000 7f2eca000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:47:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.500+0000 7f2eca000640 1 -- 172.21.15.16:0/377966627 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed802a060 con 0x55c7922ea2f0 2024-09-21T03:47:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.500+0000 7f2ef284f740 1 -- 172.21.15.16:0/377966627 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed80035f0 con 0x55c7922ea2f0 2024-09-21T03:47:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.500+0000 7f2eca000640 1 -- 172.21.15.16:0/377966627 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f2ed8003a30 con 0x55c7922ea2f0 2024-09-21T03:47:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.501+0000 7f2eca000640 1 -- 172.21.15.16:0/377966627 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2ed8031050 con 0x55c7922ea2f0 2024-09-21T03:47:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.501+0000 7f2ef284f740 10 client.25675 fetch_fsmap learned FSMap version 30 2024-09-21T03:47:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.501+0000 7f2ef284f740 1 -- 172.21.15.16:0/377966627 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c7923f7620 con 0x55c7922ea2f0 2024-09-21T03:47:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.501+0000 7f2eca000640 1 -- 172.21.15.16:0/377966627 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ed80035f0 con 0x55c7922ea2f0 2024-09-21T03:47:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.501+0000 7f2eca000640 1 -- 172.21.15.16:0/377966627 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2ed80046c0 con 0x55c7922ea2f0 2024-09-21T03:47:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.501+0000 7f2ef284f740 10 client.25675 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:47:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.501+0000 7f2ef284f740 10 client.25675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.501+0000 7f2ef284f740 20 client.25675 populate_metadata read hostname 'smithi016' 2024-09-21T03:47:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.501+0000 7f2ef284f740 1 --2- 172.21.15.16:0/377966627 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c7922d1d80 0x55c7926d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.501+0000 7f2ef284f740 4 client.25675 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.501+0000 7f2ef284f740 1 -- 172.21.15.16:0/377966627 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c79262f600 con 0x55c7922d1d80 2024-09-21T03:47:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.502+0000 7f2ee3e00640 1 --2- 172.21.15.16:0/377966627 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c7922d1d80 0x55c7926d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.502+0000 7f2ee3e00640 1 --2- 172.21.15.16:0/377966627 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c7922d1d80 0x55c7926d0b80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c52cc68a688d5ca1 server_cookie=6656060319f2f715 in_seq=0 out_seq=0 2024-09-21T03:47:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.502+0000 7f2eca000640 10 client.25675 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.503+0000 7f2eca000640 1 -- 172.21.15.16:0/377966627 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c79262f600 con 0x55c7922d1d80 2024-09-21T03:47:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.503+0000 7f2eca000640 10 client.25675 handle_command_reply: tid=0 2024-09-21T03:47:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.503+0000 7f2ef284f740 1 client.25675 shutdown 2024-09-21T03:47:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.504+0000 7f2ef284f740 1 -- 172.21.15.16:0/377966627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7922ea2f0 msgr2=0x55c7926c3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.504+0000 7f2ef284f740 1 --2- 172.21.15.16:0/377966627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7922ea2f0 0x55c7926c3760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8011010 tx=0x7f2ed800bfd0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.504+0000 7f2ef284f740 1 -- 172.21.15.16:0/377966627 shutdown_connections 2024-09-21T03:47:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.504+0000 7f2ef284f740 1 --2- 172.21.15.16:0/377966627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7923131b0 0x55c7926c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.504+0000 7f2ef284f740 1 --2- 172.21.15.16:0/377966627 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c7922d1d80 0x55c7926d0b80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.504+0000 7f2ef284f740 1 --2- 172.21.15.16:0/377966627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 0x55c7926c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.504+0000 7f2ef284f740 1 --2- 172.21.15.16:0/377966627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7922ea2f0 0x55c7926c3760 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.504+0000 7f2eca000640 0 client.25675 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.504+0000 7f2ef284f740 1 -- 172.21.15.16:0/377966627 >> 172.21.15.16:0/377966627 conn(0x55c792307300 msgr2=0x55c7926b40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.504+0000 7f2ef284f740 1 -- 172.21.15.16:0/377966627 shutdown_connections 2024-09-21T03:47:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.504+0000 7f2ef284f740 1 -- 172.21.15.16:0/377966627 wait complete. 2024-09-21T03:47:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.504+0000 7f2ef284f740 20 client.25675 trim_cache size 0 max 16384 2024-09-21T03:47:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.519+0000 7f2ef284f740 1 Processor -- start 2024-09-21T03:47:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.519+0000 7f2ef284f740 1 -- start start 2024-09-21T03:47:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.520+0000 7f2ef284f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7922ea2f0 0x55c7926c2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.520+0000 7f2ef284f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 0x55c7926cb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.520+0000 7f2ef284f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7923131b0 0x55c7926c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.520+0000 7f2ef284f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee41aa640 con 0x55c7922f24d0 2024-09-21T03:47:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.520+0000 7f2ef284f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee41ac180 con 0x55c7922ea2f0 2024-09-21T03:47:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.520+0000 7f2ef284f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee41aad10 con 0x55c7923131b0 2024-09-21T03:47:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.520+0000 7f2ee3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 0x55c7926cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.520+0000 7f2ee3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 0x55c7926cb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44740/0 (socket says 172.21.15.16:44740) 2024-09-21T03:47:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.520+0000 7f2ee3e00640 1 -- 172.21.15.16:0/896041255 learned_addr learned my addr 172.21.15.16:0/896041255 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.520+0000 7f2ee8c00640 1 --2- 172.21.15.16:0/896041255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7922ea2f0 0x55c7926c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.520+0000 7f2ee9600640 1 --2- 172.21.15.16:0/896041255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7923131b0 0x55c7926c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.521+0000 7f2ee3e00640 1 -- 172.21.15.16:0/896041255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7923131b0 msgr2=0x55c7926c1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.521+0000 7f2ee3e00640 1 --2- 172.21.15.16:0/896041255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7923131b0 0x55c7926c1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.521+0000 7f2ee3e00640 1 -- 172.21.15.16:0/896041255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7922ea2f0 msgr2=0x55c7926c2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.521+0000 7f2ee3e00640 1 --2- 172.21.15.16:0/896041255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7922ea2f0 0x55c7926c2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.521+0000 7f2ee3e00640 1 -- 172.21.15.16:0/896041255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7926c4d00 con 0x55c7922f24d0 2024-09-21T03:47:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.521+0000 7f2ee8c00640 1 --2- 172.21.15.16:0/896041255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7922ea2f0 0x55c7926c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.521+0000 7f2ee3e00640 1 --2- 172.21.15.16:0/896041255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 0x55c7926cb870 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0005dc0 tx=0x7f2ed0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22974044659c066f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.521+0000 7f2ed5200640 1 -- 172.21.15.16:0/896041255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0059070 con 0x55c7922f24d0 2024-09-21T03:47:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.521+0000 7f2ed5200640 1 -- 172.21.15.16:0/896041255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ed0004310 con 0x55c7922f24d0 2024-09-21T03:47:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.521+0000 7f2ed5200640 1 -- 172.21.15.16:0/896041255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0035040 con 0x55c7922f24d0 2024-09-21T03:47:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.522+0000 7f2ef284f740 1 -- 172.21.15.16:0/896041255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 msgr2=0x55c7926cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.522+0000 7f2ef284f740 1 --2- 172.21.15.16:0/896041255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 0x55c7926cb870 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0005dc0 tx=0x7f2ed0009260 comp rx=0 tx=0).stop 2024-09-21T03:47:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.522+0000 7f2ef284f740 1 -- 172.21.15.16:0/896041255 shutdown_connections 2024-09-21T03:47:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.522+0000 7f2ef284f740 1 --2- 172.21.15.16:0/896041255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7923131b0 0x55c7926c1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.522+0000 7f2ef284f740 1 --2- 172.21.15.16:0/896041255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 0x55c7926cb870 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.522+0000 7f2ef284f740 1 --2- 172.21.15.16:0/896041255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7922ea2f0 0x55c7926c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.522+0000 7f2ef284f740 1 -- 172.21.15.16:0/896041255 >> 172.21.15.16:0/896041255 conn(0x55c792307300 msgr2=0x55c7926baf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.522+0000 7f2ef284f740 1 -- 172.21.15.16:0/896041255 shutdown_connections 2024-09-21T03:47:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.522+0000 7f2ef284f740 1 -- 172.21.15.16:0/896041255 wait complete. 2024-09-21T03:47:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.523+0000 7f2ef284f740 1 -- start start 2024-09-21T03:47:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.523+0000 7f2ef284f740 1 Processor -- start 2024-09-21T03:47:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.523+0000 7f2ef284f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7922ea2f0 0x55c7926cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.523+0000 7f2ef284f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 0x55c7926ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.523+0000 7f2ef284f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7923131b0 0x55c7926ca710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.523+0000 7f2ef284f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee41abab0 con 0x55c7922f24d0 2024-09-21T03:47:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.523+0000 7f2ef284f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee41ab3e0 con 0x55c7922ea2f0 2024-09-21T03:47:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.523+0000 7f2ef284f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee41a8620 con 0x55c7923131b0 2024-09-21T03:47:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.523+0000 7f2ef284f740 10 client.0 _ll_register_callbacks cb 0x7f2ed0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.524+0000 7f2ee3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 0x55c7926ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.524+0000 7f2ee3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 0x55c7926ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44752/0 (socket says 172.21.15.16:44752) 2024-09-21T03:47:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.524+0000 7f2ee3e00640 1 -- 172.21.15.16:0/934974328 learned_addr learned my addr 172.21.15.16:0/934974328 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.524+0000 7f2ee9600640 1 --2- 172.21.15.16:0/934974328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7923131b0 0x55c7926ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.524+0000 7f2ee8c00640 1 --2- 172.21.15.16:0/934974328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7922ea2f0 0x55c7926cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.524+0000 7f2ee3e00640 1 -- 172.21.15.16:0/934974328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7923131b0 msgr2=0x55c7926ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.524+0000 7f2ee3e00640 1 --2- 172.21.15.16:0/934974328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7923131b0 0x55c7926ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.524+0000 7f2ee3e00640 1 -- 172.21.15.16:0/934974328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7922ea2f0 msgr2=0x55c7926cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.524+0000 7f2ee3e00640 1 --2- 172.21.15.16:0/934974328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7922ea2f0 0x55c7926cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.524+0000 7f2ee3e00640 1 -- 172.21.15.16:0/934974328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7926cae50 con 0x55c7922f24d0 2024-09-21T03:47:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.524+0000 7f2ee9600640 1 --2- 172.21.15.16:0/934974328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7923131b0 0x55c7926ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.524+0000 7f2ee8c00640 1 --2- 172.21.15.16:0/934974328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7922ea2f0 0x55c7926cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.524+0000 7f2ee3e00640 1 --2- 172.21.15.16:0/934974328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 0x55c7926ca1d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0040040 tx=0x7f2ed00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a815e828cc248ecf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.524+0000 7f2eca000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:47:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.525+0000 7f2eca000640 1 -- 172.21.15.16:0/934974328 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0059070 con 0x55c7922f24d0 2024-09-21T03:47:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.525+0000 7f2ef284f740 1 -- 172.21.15.16:0/934974328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed0004310 con 0x55c7922f24d0 2024-09-21T03:47:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.525+0000 7f2eca000640 1 -- 172.21.15.16:0/934974328 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f2ed0005310 con 0x55c7922f24d0 2024-09-21T03:47:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.525+0000 7f2eca000640 1 -- 172.21.15.16:0/934974328 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2ed0035040 con 0x55c7922f24d0 2024-09-21T03:47:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.525+0000 7f2ef284f740 10 client.15681 fetch_fsmap learned FSMap version 30 2024-09-21T03:47:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.525+0000 7f2ef284f740 1 -- 172.21.15.16:0/934974328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c79273b4c0 con 0x55c7922f24d0 2024-09-21T03:47:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.525+0000 7f2eca000640 1 -- 172.21.15.16:0/934974328 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ed0004310 con 0x55c7922f24d0 2024-09-21T03:47:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.526+0000 7f2eca000640 1 -- 172.21.15.16:0/934974328 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2ed0004a60 con 0x55c7922f24d0 2024-09-21T03:47:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.526+0000 7f2ef284f740 10 client.15681 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:47:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.526+0000 7f2ef284f740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.526+0000 7f2ef284f740 20 client.15681 populate_metadata read hostname 'smithi016' 2024-09-21T03:47:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.526+0000 7f2ef284f740 1 --2- 172.21.15.16:0/934974328 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c7922a71c0 0x55c79273de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.526+0000 7f2ef284f740 4 client.15681 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:47:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.526+0000 7f2ef284f740 1 -- 172.21.15.16:0/934974328 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c79262b0b0 con 0x55c7922a71c0 2024-09-21T03:47:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.526+0000 7f2ee8c00640 1 --2- 172.21.15.16:0/934974328 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c7922a71c0 0x55c79273de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.527+0000 7f2ee8c00640 1 --2- 172.21.15.16:0/934974328 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c7922a71c0 0x55c79273de20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e942173a423f3f server_cookie=3638aa5a2b8e4894 in_seq=0 out_seq=0 2024-09-21T03:47:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.527+0000 7f2eca000640 10 client.15681 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.532+0000 7f2eca000640 1 -- 172.21.15.16:0/934974328 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c79262b0b0 con 0x55c7922a71c0 2024-09-21T03:47:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.532+0000 7f2eca000640 10 client.15681 handle_command_reply: tid=0 2024-09-21T03:47:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.532+0000 7f2ef284f740 1 client.15681 shutdown 2024-09-21T03:47:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.532+0000 7f2ef284f740 1 -- 172.21.15.16:0/934974328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 msgr2=0x55c7926ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.532+0000 7f2ef284f740 1 --2- 172.21.15.16:0/934974328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 0x55c7926ca1d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0040040 tx=0x7f2ed00044c0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.533+0000 7f2ef284f740 1 -- 172.21.15.16:0/934974328 shutdown_connections 2024-09-21T03:47:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.533+0000 7f2ef284f740 1 --2- 172.21.15.16:0/934974328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7923131b0 0x55c7926ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.533+0000 7f2ef284f740 1 --2- 172.21.15.16:0/934974328 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c7922a71c0 0x55c79273de20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.533+0000 7f2ef284f740 1 --2- 172.21.15.16:0/934974328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7922f24d0 0x55c7926ca1d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.533+0000 7f2ef284f740 1 --2- 172.21.15.16:0/934974328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7922ea2f0 0x55c7926cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.533+0000 7f2ef284f740 1 -- 172.21.15.16:0/934974328 >> 172.21.15.16:0/934974328 conn(0x55c792307300 msgr2=0x55c7926c1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.533+0000 7f2eca000640 0 client.15681 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.533+0000 7f2ef284f740 1 -- 172.21.15.16:0/934974328 shutdown_connections 2024-09-21T03:47:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.533+0000 7f2ef284f740 1 -- 172.21.15.16:0/934974328 wait complete. 2024-09-21T03:47:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.533+0000 7f2ef284f740 20 client.15681 trim_cache size 0 max 16384 2024-09-21T03:47:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:47:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:47:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:47:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:47:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.533+0000 7f2eeb600640 1 -- 172.21.15.16:0/4278334472 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2ecc0ba9a0 msgr2=0x7f2ecc0bceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.533+0000 7f2eeb600640 1 --2- 172.21.15.16:0/4278334472 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2ecc0ba9a0 0x7f2ecc0bceb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0031a50 tx=0x7f2ed0005d50 comp rx=0 tx=0).stop 2024-09-21T03:47:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.533+0000 7f2eeb600640 1 -- 172.21.15.16:0/4278334472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ee41050e0 msgr2=0x7f2ee419b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.533+0000 7f2eeb600640 1 --2- 172.21.15.16:0/4278334472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ee41050e0 0x7f2ee419b180 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009fd0 tx=0x7f2edc00ef10 comp rx=0 tx=0).stop 2024-09-21T03:47:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.534+0000 7f2eeb600640 1 -- 172.21.15.16:0/4278334472 shutdown_connections 2024-09-21T03:47:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.534+0000 7f2eeb600640 1 --2- 172.21.15.16:0/4278334472 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2ecc0ba9a0 0x7f2ecc0bceb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.534+0000 7f2eeb600640 1 --2- 172.21.15.16:0/4278334472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ee41050e0 0x7f2ee419b180 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.534+0000 7f2eeb600640 1 --2- 172.21.15.16:0/4278334472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4104740 0x7f2ee419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.534+0000 7f2eeb600640 1 --2- 172.21.15.16:0/4278334472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee41036c0 0x7f2ee4199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.534+0000 7f2eeb600640 1 -- 172.21.15.16:0/4278334472 >> 172.21.15.16:0/4278334472 conn(0x7f2ee40ff020 msgr2=0x7f2ee4107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.534+0000 7f2eeb600640 1 -- 172.21.15.16:0/4278334472 shutdown_connections 2024-09-21T03:47:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:34.534+0000 7f2eeb600640 1 -- 172.21.15.16:0/4278334472 wait complete. 2024-09-21T03:47:35.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:34 smithi134 ceph-mon[35031]: pgmap v456: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T03:47:35.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:34 smithi123 ceph-mon[34673]: pgmap v456: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T03:47:35.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:34 smithi016 ceph-mon[30375]: pgmap v456: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T03:47:37.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:36 smithi134 ceph-mon[35031]: pgmap v457: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:47:37.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:36 smithi123 ceph-mon[34673]: pgmap v457: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:47:37.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:36 smithi016 ceph-mon[30375]: pgmap v457: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:47:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:38 smithi134 ceph-mon[35031]: pgmap v458: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:47:39.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:38 smithi123 ceph-mon[34673]: pgmap v458: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:47:39.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:38 smithi016 ceph-mon[30375]: pgmap v458: 129 pgs: 129 active+clean; 674 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T03:47:39.617 INFO:tasks.workunit.client.0.smithi016.stderr:Updating files: 59% (7525/12697) Updating files: 60% (7619/12697) Updating files: 61% (7746/12697) Updating files: 62% (7873/12697) Updating files: 63% (8000/12697) Updating files: 64% (8127/12697) Updating files: 65% (8254/12697) Updating files: 66% (8381/12697) Updating files: 67% (8507/12697) Updating files: 68% (8634/12697) Updating files: 69% (8761/12697) Updating files: 70% (8888/12697) Updating files: 71% (9015/12697) Updating files: 72% (9142/12697) Updating files: 73% (9269/12697) Updating files: 74% (9396/12697) Updating files: 75% (9523/12697) Updating files: 76% (9650/12697) Updating files: 77% (9777/12697) Updating files: 78% (9904/12697) Updating files: 79% (10031/12697) Updating files: 80% (10158/12697) Updating files: 81% (10285/12697) Updating files: 82% (10412/12697) Updating files: 83% (10539/12697) Updating files: 84% (10666/12697) Updating files: 85% (10793/12697) Updating files: 86% (10920/12697) Updating files: 87% (11047/12697) Updating files: 88% (11174/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 96% (12303/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-21T03:47:40.570 INFO:tasks.workunit.client.0.smithi016.stderr:Note: switching to '1843221895adccfb3b692060251dd6a0183bc410'. 2024-09-21T03:47:40.570 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:47:40.570 INFO:tasks.workunit.client.0.smithi016.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T03:47:40.570 INFO:tasks.workunit.client.0.smithi016.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T03:47:40.570 INFO:tasks.workunit.client.0.smithi016.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T03:47:40.570 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:47:40.570 INFO:tasks.workunit.client.0.smithi016.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T03:47:40.570 INFO:tasks.workunit.client.0.smithi016.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T03:47:40.570 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:47:40.570 INFO:tasks.workunit.client.0.smithi016.stderr: git switch -c 2024-09-21T03:47:40.570 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:47:40.570 INFO:tasks.workunit.client.0.smithi016.stderr:Or undo this operation with: 2024-09-21T03:47:40.571 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:47:40.571 INFO:tasks.workunit.client.0.smithi016.stderr: git switch - 2024-09-21T03:47:40.571 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:47:40.571 INFO:tasks.workunit.client.0.smithi016.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T03:47:40.571 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:47:40.571 INFO:tasks.workunit.client.0.smithi016.stderr:HEAD is now at 1843221895a Merge PR #56776 into wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:47:40.584 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T03:47:40.670 INFO:tasks.workunit.client.0.smithi016.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T03:47:40.675 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T03:47:40.675 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T03:47:41.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:40 smithi134 ceph-mon[35031]: pgmap v459: 129 pgs: 129 active+clean; 675 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T03:47:41.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:40 smithi123 ceph-mon[34673]: pgmap v459: 129 pgs: 129 active+clean; 675 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T03:47:41.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:40 smithi016 ceph-mon[30375]: pgmap v459: 129 pgs: 129 active+clean; 675 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T03:47:41.436 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T03:47:41.539 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T03:47:41.606 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T03:47:41.609 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T03:47:41.609 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T03:47:41.676 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T03:47:41.683 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:47:41.683 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T03:47:41.718 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-09-21T03:47:41.720 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-09-21T03:47:41.720 DEBUG:teuthology.orchestra.run.smithi016:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-09-21T03:47:41.808 INFO:tasks.workunit.client.0.smithi016.stdout:getting iogen 2024-09-21T03:47:41.808 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'getting iogen' 2024-09-21T03:47:41.809 INFO:tasks.workunit.client.0.smithi016.stderr:+ wget http://download.ceph.com/qa/iogen_3.1p0.tar 2024-09-21T03:47:41.856 INFO:tasks.workunit.client.0.smithi016.stderr:--2024-09-21 03:47:41-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-09-21T03:47:41.857 INFO:tasks.workunit.client.0.smithi016.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-21T03:47:41.878 INFO:tasks.workunit.client.0.smithi016.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-21T03:47:41.899 INFO:tasks.workunit.client.0.smithi016.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-21T03:47:41.899 INFO:tasks.workunit.client.0.smithi016.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-09-21T03:47:41.901 INFO:tasks.workunit.client.0.smithi016.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-09-21T03:47:41.905 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:47:41.905 INFO:tasks.workunit.client.0.smithi016.stderr: 0K ....... 100% 499M=0s 2024-09-21T03:47:41.905 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:47:41.905 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-21 03:47:41 (499 MB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-09-21T03:47:41.905 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:47:41.905 INFO:tasks.workunit.client.0.smithi016.stderr:+ tar -xvzf iogen_3.1p0.tar 2024-09-21T03:47:41.927 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0 2024-09-21T03:47:41.929 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/src 2024-09-21T03:47:41.930 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/src/iogen.8 2024-09-21T03:47:41.932 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/src/iogen.c 2024-09-21T03:47:41.933 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/README 2024-09-21T03:47:41.934 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/Makefile.bsd 2024-09-21T03:47:41.936 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/Makefile 2024-09-21T03:47:41.937 INFO:tasks.workunit.client.0.smithi016.stderr:+ cd iogen_3.1p0 2024-09-21T03:47:41.938 INFO:tasks.workunit.client.0.smithi016.stdout:making iogen 2024-09-21T03:47:41.938 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'making iogen' 2024-09-21T03:47:41.938 INFO:tasks.workunit.client.0.smithi016.stderr:+ make 2024-09-21T03:47:41.950 INFO:tasks.workunit.client.0.smithi016.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-09-21T03:47:42.135 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.0 2024-09-21T03:47:42.135 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:47:42.239 INFO:tasks.workunit.client.0.smithi016.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-09-21T03:47:42.417 INFO:tasks.workunit.client.0.smithi016.stdout:running iogen 2024-09-21T03:47:42.417 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'running iogen' 2024-09-21T03:47:42.417 INFO:tasks.workunit.client.0.smithi016.stderr:+ ./iogen -n 5 -s 2g 2024-09-21T03:47:42.418 INFO:tasks.workunit.client.0.smithi016.stderr:child spawned 88143 2024-09-21T03:47:42.418 INFO:tasks.workunit.client.0.smithi016.stderr:child spawned 88145 2024-09-21T03:47:42.419 INFO:tasks.workunit.client.0.smithi016.stderr:child spawned 88146 2024-09-21T03:47:42.419 INFO:tasks.workunit.client.0.smithi016.stderr:child spawned 88147 2024-09-21T03:47:42.419 INFO:tasks.workunit.client.0.smithi016.stderr:child spawned 88148 2024-09-21T03:47:42.420 INFO:tasks.workunit.client.0.smithi016.stdout:sleep for 10 min 2024-09-21T03:47:42.420 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'sleep for 10 min' 2024-09-21T03:47:42.420 INFO:tasks.workunit.client.0.smithi016.stderr:+ sleep 600 2024-09-21T03:47:42.603 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:43.037 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:42 smithi016 ceph-mon[30375]: pgmap v460: 129 pgs: 129 active+clean; 675 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:47:43.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:42 smithi134 ceph-mon[35031]: pgmap v460: 129 pgs: 129 active+clean; 675 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:47:43.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:42 smithi123 ceph-mon[34673]: pgmap v460: 129 pgs: 129 active+clean; 675 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:47:43.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.495+0000 7f094dc00640 1 -- 172.21.15.16:0/3514547379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09481092f0 msgr2=0x7f0948109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.495+0000 7f094dc00640 1 --2- 172.21.15.16:0/3514547379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09481092f0 0x7f0948109700 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0934009900 tx=0x7f093402a410 comp rx=0 tx=0).stop 2024-09-21T03:47:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.495+0000 7f094dc00640 1 -- 172.21.15.16:0/3514547379 shutdown_connections 2024-09-21T03:47:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.495+0000 7f094dc00640 1 --2- 172.21.15.16:0/3514547379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0948109c40 0x7f0948110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.495+0000 7f094dc00640 1 --2- 172.21.15.16:0/3514547379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09481092f0 0x7f0948109700 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.495+0000 7f094dc00640 1 --2- 172.21.15.16:0/3514547379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09481039b0 0x7f0948103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.495+0000 7f094dc00640 1 -- 172.21.15.16:0/3514547379 >> 172.21.15.16:0/3514547379 conn(0x7f09480ff440 msgr2=0x7f0948101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.495+0000 7f094dc00640 1 -- 172.21.15.16:0/3514547379 shutdown_connections 2024-09-21T03:47:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.495+0000 7f094dc00640 1 -- 172.21.15.16:0/3514547379 wait complete. 2024-09-21T03:47:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.497+0000 7f094dc00640 1 Processor -- start 2024-09-21T03:47:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.497+0000 7f094dc00640 1 -- start start 2024-09-21T03:47:43.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.498+0000 7f094dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09481039b0 0x7f094819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:43.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.498+0000 7f094dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09481092f0 0x7f094819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:43.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.498+0000 7f094dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0948109c40 0x7f09481a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:43.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.498+0000 7f094dc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0948113950 con 0x7f09481039b0 2024-09-21T03:47:43.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.498+0000 7f094dc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f09481137d0 con 0x7f09481092f0 2024-09-21T03:47:43.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.498+0000 7f094dc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0948113ad0 con 0x7f0948109c40 2024-09-21T03:47:43.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.499+0000 7f0947e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09481039b0 0x7f094819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:43.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.499+0000 7f0947e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09481039b0 0x7f094819c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60740/0 (socket says 172.21.15.16:60740) 2024-09-21T03:47:43.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.499+0000 7f0947e00640 1 -- 172.21.15.16:0/3461186785 learned_addr learned my addr 172.21.15.16:0/3461186785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:43.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.499+0000 7f0947400640 1 --2- 172.21.15.16:0/3461186785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09481092f0 0x7f094819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:43.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.499+0000 7f094d200640 1 --2- 172.21.15.16:0/3461186785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0948109c40 0x7f09481a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:43.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.500+0000 7f0947400640 1 -- 172.21.15.16:0/3461186785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0948109c40 msgr2=0x7f09481a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:43.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.500+0000 7f0947400640 1 --2- 172.21.15.16:0/3461186785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0948109c40 0x7f09481a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:43.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.500+0000 7f0947400640 1 -- 172.21.15.16:0/3461186785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09481039b0 msgr2=0x7f094819c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:43.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.500+0000 7f0947400640 1 --2- 172.21.15.16:0/3461186785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09481039b0 0x7f094819c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:43.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.500+0000 7f0947400640 1 -- 172.21.15.16:0/3461186785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09481a24b0 con 0x7f09481092f0 2024-09-21T03:47:43.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.500+0000 7f0947e00640 1 --2- 172.21.15.16:0/3461186785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09481039b0 0x7f094819c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:43.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.501+0000 7f0947400640 1 --2- 172.21.15.16:0/3461186785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09481092f0 0x7f094819cae0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0934009230 tx=0x7f0934003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:43.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.501+0000 7f0944c00640 1 -- 172.21.15.16:0/3461186785 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093403f070 con 0x7f09481092f0 2024-09-21T03:47:43.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.501+0000 7f0944c00640 1 -- 172.21.15.16:0/3461186785 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0934003e40 con 0x7f09481092f0 2024-09-21T03:47:43.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.501+0000 7f0944c00640 1 -- 172.21.15.16:0/3461186785 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934035030 con 0x7f09481092f0 2024-09-21T03:47:43.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.502+0000 7f094dc00640 1 -- 172.21.15.16:0/3461186785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09481a2740 con 0x7f09481092f0 2024-09-21T03:47:43.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.502+0000 7f094dc00640 1 -- 172.21.15.16:0/3461186785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09481a2ba0 con 0x7f09481092f0 2024-09-21T03:47:43.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.504+0000 7f094dc00640 1 -- 172.21.15.16:0/3461186785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0948103dc0 con 0x7f09481092f0 2024-09-21T03:47:43.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.508+0000 7f0944c00640 1 -- 172.21.15.16:0/3461186785 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0934042070 con 0x7f09481092f0 2024-09-21T03:47:43.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.511+0000 7f0944c00640 1 --2- 172.21.15.16:0/3461186785 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f092c0b65d0 0x7f092c0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:43.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.512+0000 7f0944c00640 1 -- 172.21.15.16:0/3461186785 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0934107d80 con 0x7f09481092f0 2024-09-21T03:47:43.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.512+0000 7f0947e00640 1 --2- 172.21.15.16:0/3461186785 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f092c0b65d0 0x7f092c0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:43.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.513+0000 7f0947e00640 1 --2- 172.21.15.16:0/3461186785 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f092c0b65d0 0x7f092c0b8ae0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0938010530 tx=0x7f093800b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:43.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:43.515+0000 7f0944c00640 1 -- 172.21.15.16:0/3461186785 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f09340d2640 con 0x7f09481092f0 2024-09-21T03:47:44.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:44.113+0000 7f094dc00640 1 -- 172.21.15.16:0/3461186785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09481a9070 con 0x7f09481092f0 2024-09-21T03:47:44.118 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:47:44.119 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:47:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:44.114+0000 7f0944c00640 1 -- 172.21.15.16:0/3461186785 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f09340d7580 con 0x7f09481092f0 2024-09-21T03:47:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:47:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:44.118+0000 7f094dc00640 1 -- 172.21.15.16:0/3461186785 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f092c0b65d0 msgr2=0x7f092c0b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:44.118+0000 7f094dc00640 1 --2- 172.21.15.16:0/3461186785 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f092c0b65d0 0x7f092c0b8ae0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0938010530 tx=0x7f093800b410 comp rx=0 tx=0).stop 2024-09-21T03:47:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:44.118+0000 7f094dc00640 1 -- 172.21.15.16:0/3461186785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09481092f0 msgr2=0x7f094819cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:44.118+0000 7f094dc00640 1 --2- 172.21.15.16:0/3461186785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09481092f0 0x7f094819cae0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0934009230 tx=0x7f0934003cd0 comp rx=0 tx=0).stop 2024-09-21T03:47:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:44.119+0000 7f094dc00640 1 -- 172.21.15.16:0/3461186785 shutdown_connections 2024-09-21T03:47:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:44.119+0000 7f094dc00640 1 --2- 172.21.15.16:0/3461186785 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f092c0b65d0 0x7f092c0b8ae0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:44.119+0000 7f094dc00640 1 --2- 172.21.15.16:0/3461186785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0948109c40 0x7f09481a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:44.119+0000 7f094dc00640 1 --2- 172.21.15.16:0/3461186785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09481092f0 0x7f094819cae0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:44.119+0000 7f094dc00640 1 --2- 172.21.15.16:0/3461186785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09481039b0 0x7f094819c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:44.119+0000 7f094dc00640 1 -- 172.21.15.16:0/3461186785 >> 172.21.15.16:0/3461186785 conn(0x7f09480ff440 msgr2=0x7f0948106930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:44.120+0000 7f094dc00640 1 -- 172.21.15.16:0/3461186785 shutdown_connections 2024-09-21T03:47:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:44.120+0000 7f094dc00640 1 -- 172.21.15.16:0/3461186785 wait complete. 2024-09-21T03:47:45.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:44 smithi134 ceph-mon[35031]: pgmap v461: 129 pgs: 129 active+clean; 833 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-09-21T03:47:45.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:44 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3461186785' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:45.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:44 smithi123 ceph-mon[34673]: pgmap v461: 129 pgs: 129 active+clean; 833 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-09-21T03:47:45.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:44 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3461186785' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:45.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:44 smithi016 ceph-mon[30375]: pgmap v461: 129 pgs: 129 active+clean; 833 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-09-21T03:47:45.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:44 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3461186785' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:47:46.724 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0' 2024-09-21T03:47:46.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0 2024-09-21T03:47:46.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:46 smithi016 ceph-mon[30375]: pgmap v462: 129 pgs: 129 active+clean; 73 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.0 MiB/s wr, 15 op/s 2024-09-21T03:47:47.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:46 smithi134 ceph-mon[35031]: pgmap v462: 129 pgs: 129 active+clean; 73 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.0 MiB/s wr, 15 op/s 2024-09-21T03:47:47.166 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:46 smithi123 ceph-mon[34673]: pgmap v462: 129 pgs: 129 active+clean; 73 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.0 MiB/s wr, 15 op/s 2024-09-21T03:47:47.471 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:47:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.973+0000 7efc5f200640 1 -- 172.21.15.16:0/1135880779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc5810bc00 msgr2=0x7efc58112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.974+0000 7efc5f200640 1 --2- 172.21.15.16:0/1135880779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc5810bc00 0x7efc58112620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efc50005fd0 tx=0x7efc5000aec0 comp rx=0 tx=0).stop 2024-09-21T03:47:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.974+0000 7efc5f200640 1 -- 172.21.15.16:0/1135880779 shutdown_connections 2024-09-21T03:47:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.975+0000 7efc5f200640 1 --2- 172.21.15.16:0/1135880779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc5810bc00 0x7efc58112620 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.975+0000 7efc5f200640 1 --2- 172.21.15.16:0/1135880779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc5810aff0 0x7efc5810b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.975+0000 7efc5f200640 1 --2- 172.21.15.16:0/1135880779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5810a5a0 0x7efc5810a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.975+0000 7efc5f200640 1 -- 172.21.15.16:0/1135880779 >> 172.21.15.16:0/1135880779 conn(0x7efc58070810 msgr2=0x7efc58072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.980+0000 7efc5f200640 1 -- 172.21.15.16:0/1135880779 shutdown_connections 2024-09-21T03:47:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.980+0000 7efc5f200640 1 -- 172.21.15.16:0/1135880779 wait complete. 2024-09-21T03:47:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.986+0000 7efc5f200640 1 Processor -- start 2024-09-21T03:47:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.986+0000 7efc5f200640 1 -- start start 2024-09-21T03:47:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.987+0000 7efc5f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc5810a5a0 0x7efc581a8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.988+0000 7efc5f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5810aff0 0x7efc581a9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.988+0000 7efc5f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc5810bc00 0x7efc581a9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.988+0000 7efc5f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc58114f80 con 0x7efc5810aff0 2024-09-21T03:47:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.988+0000 7efc57e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc5810a5a0 0x7efc581a8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.988+0000 7efc5d200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc5810bc00 0x7efc581a9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.988+0000 7efc57e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc5810a5a0 0x7efc581a8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:57258/0 (socket says 172.21.15.16:57258) 2024-09-21T03:47:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.988+0000 7efc57e00640 1 -- 172.21.15.16:0/1459820276 learned_addr learned my addr 172.21.15.16:0/1459820276 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.989+0000 7efc57400640 1 --2- 172.21.15.16:0/1459820276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5810aff0 0x7efc581a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.989+0000 7efc5f200640 1 -- 172.21.15.16:0/1459820276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc58114e00 con 0x7efc5810a5a0 2024-09-21T03:47:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.989+0000 7efc5f200640 1 -- 172.21.15.16:0/1459820276 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc58115100 con 0x7efc5810bc00 2024-09-21T03:47:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.990+0000 7efc57e00640 1 -- 172.21.15.16:0/1459820276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc5810bc00 msgr2=0x7efc581a9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.990+0000 7efc57e00640 1 --2- 172.21.15.16:0/1459820276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc5810bc00 0x7efc581a9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.990+0000 7efc57e00640 1 -- 172.21.15.16:0/1459820276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5810aff0 msgr2=0x7efc581a9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.990+0000 7efc57e00640 1 --2- 172.21.15.16:0/1459820276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5810aff0 0x7efc581a9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.990+0000 7efc57e00640 1 -- 172.21.15.16:0/1459820276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc581aa750 con 0x7efc5810a5a0 2024-09-21T03:47:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.991+0000 7efc5d200640 1 --2- 172.21.15.16:0/1459820276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc5810bc00 0x7efc581a9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.991+0000 7efc57e00640 1 --2- 172.21.15.16:0/1459820276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc5810a5a0 0x7efc581a8d10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efc4400b450 tx=0x7efc4400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.992+0000 7efc54c00640 1 -- 172.21.15.16:0/1459820276 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44015020 con 0x7efc5810a5a0 2024-09-21T03:47:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.992+0000 7efc5f200640 1 -- 172.21.15.16:0/1459820276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc581aaa40 con 0x7efc5810a5a0 2024-09-21T03:47:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.992+0000 7efc54c00640 1 -- 172.21.15.16:0/1459820276 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc44011d40 con 0x7efc5810a5a0 2024-09-21T03:47:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.992+0000 7efc54c00640 1 -- 172.21.15.16:0/1459820276 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4400f340 con 0x7efc5810a5a0 2024-09-21T03:47:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:47.993+0000 7efc5f200640 1 -- 172.21.15.16:0/1459820276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc581b1130 con 0x7efc5810a5a0 2024-09-21T03:47:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.015+0000 7efc662cb740 1 Processor -- start 2024-09-21T03:47:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.015+0000 7efc662cb740 1 -- start start 2024-09-21T03:47:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.016+0000 7efc662cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43eef1b0 0x556c442960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.016+0000 7efc662cb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43ec62f0 0x556c44296740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.016+0000 7efc662cb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c44297290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.016+0000 7efc662cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556c442096b0 con 0x556c43eef1b0 2024-09-21T03:47:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.016+0000 7efc662cb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556c441f8ae0 con 0x556c43ece4d0 2024-09-21T03:47:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.016+0000 7efc662cb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c441fc8f0 con 0x556c43ec62f0 2024-09-21T03:47:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.016+0000 7efc5d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43eef1b0 0x556c442960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.016+0000 7efc57400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c44297290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.016+0000 7efc57400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c44297290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:57274/0 (socket says 172.21.15.16:57274) 2024-09-21T03:47:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.016+0000 7efc57400640 1 -- 172.21.15.16:0/3156146428 learned_addr learned my addr 172.21.15.16:0/3156146428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.017+0000 7efc57400640 1 -- 172.21.15.16:0/3156146428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43ec62f0 msgr2=0x556c44296740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:47:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.017+0000 7efc57400640 1 --2- 172.21.15.16:0/3156146428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43ec62f0 0x556c44296740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.017+0000 7efc57400640 1 -- 172.21.15.16:0/3156146428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43eef1b0 msgr2=0x556c442960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.017+0000 7efc57400640 1 --2- 172.21.15.16:0/3156146428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43eef1b0 0x556c442960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.017+0000 7efc57400640 1 -- 172.21.15.16:0/3156146428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c441c5780 con 0x556c43ece4d0 2024-09-21T03:47:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.017+0000 7efc5d200640 1 --2- 172.21.15.16:0/3156146428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43eef1b0 0x556c442960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.018+0000 7efc57400640 1 --2- 172.21.15.16:0/3156146428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c44297290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efc48004570 tx=0x7efc48005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78217d42dcc941d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.018+0000 7efc4d600640 1 -- 172.21.15.16:0/3156146428 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4800be00 con 0x556c43ece4d0 2024-09-21T03:47:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.018+0000 7efc4d600640 1 -- 172.21.15.16:0/3156146428 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc480054d0 con 0x556c43ece4d0 2024-09-21T03:47:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.018+0000 7efc4d600640 1 -- 172.21.15.16:0/3156146428 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48004c60 con 0x556c43ece4d0 2024-09-21T03:47:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.019+0000 7efc54c00640 1 -- 172.21.15.16:0/1459820276 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7efc44018020 con 0x7efc5810a5a0 2024-09-21T03:47:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.019+0000 7efc662cb740 1 -- 172.21.15.16:0/3156146428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 msgr2=0x556c44297290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.019+0000 7efc662cb740 1 --2- 172.21.15.16:0/3156146428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c44297290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efc48004570 tx=0x7efc48005f90 comp rx=0 tx=0).stop 2024-09-21T03:47:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.020+0000 7efc662cb740 1 -- 172.21.15.16:0/3156146428 shutdown_connections 2024-09-21T03:47:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.020+0000 7efc662cb740 1 --2- 172.21.15.16:0/3156146428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c44297290 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.020+0000 7efc662cb740 1 --2- 172.21.15.16:0/3156146428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43ec62f0 0x556c44296740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.021+0000 7efc662cb740 1 --2- 172.21.15.16:0/3156146428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43eef1b0 0x556c442960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.021+0000 7efc662cb740 1 -- 172.21.15.16:0/3156146428 >> 172.21.15.16:0/3156146428 conn(0x556c43ee3300 msgr2=0x556c44291b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.021+0000 7efc662cb740 1 -- 172.21.15.16:0/3156146428 shutdown_connections 2024-09-21T03:47:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.021+0000 7efc662cb740 1 -- 172.21.15.16:0/3156146428 wait complete. 2024-09-21T03:47:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.022+0000 7efc662cb740 1 -- start start 2024-09-21T03:47:48.025 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.022+0000 7efc662cb740 1 Processor -- start 2024-09-21T03:47:48.026 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.023+0000 7efc54c00640 1 --2- 172.21.15.16:0/1459820276 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efc3c0b6520 0x7efc3c0b8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.026 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.024+0000 7efc54c00640 1 -- 172.21.15.16:0/1459820276 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7efc440e6670 con 0x7efc5810a5a0 2024-09-21T03:47:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.024+0000 7efc662cb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43ec62f0 0x556c4429f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.024+0000 7efc662cb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c4429feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.024+0000 7efc662cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43eef1b0 0x556c442a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.024+0000 7efc662cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556c44130ab0 con 0x556c43eef1b0 2024-09-21T03:47:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.024+0000 7efc662cb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556c4419f570 con 0x556c43ece4d0 2024-09-21T03:47:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.024+0000 7efc662cb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c44297db0 con 0x556c43ec62f0 2024-09-21T03:47:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.024+0000 7efc662cb740 10 client.0 _ll_register_callbacks cb 0x556c441ae100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.027+0000 7efc5d200640 1 --2- 172.21.15.16:0/1459820276 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efc3c0b6520 0x7efc3c0b8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.028+0000 7efc57400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c4429feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.028+0000 7efc57400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c4429feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:57286/0 (socket says 172.21.15.16:57286) 2024-09-21T03:47:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.028+0000 7efc57400640 1 -- 172.21.15.16:0/2913764394 learned_addr learned my addr 172.21.15.16:0/2913764394 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.028+0000 7efc5d200640 1 --2- 172.21.15.16:0/2913764394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43eef1b0 0x556c442a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.029+0000 7efc57400640 1 -- 172.21.15.16:0/2913764394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43ec62f0 msgr2=0x556c4429f7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:47:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.029+0000 7efc57400640 1 --2- 172.21.15.16:0/2913764394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43ec62f0 0x556c4429f7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.029+0000 7efc57400640 1 -- 172.21.15.16:0/2913764394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43eef1b0 msgr2=0x556c442a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.029+0000 7efc57400640 1 --2- 172.21.15.16:0/2913764394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43eef1b0 0x556c442a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.029+0000 7efc57400640 1 -- 172.21.15.16:0/2913764394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c43f5a790 con 0x556c43ece4d0 2024-09-21T03:47:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.030+0000 7efc5d200640 1 --2- 172.21.15.16:0/1459820276 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efc3c0b6520 0x7efc3c0b8a30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b010 tx=0x7efc50032460 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.031+0000 7efc57400640 1 --2- 172.21.15.16:0/2913764394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c4429feb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efc48005300 tx=0x7efc48005330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82c5cf958669d366 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.031+0000 7efc42000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:47:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.031+0000 7efc42000640 1 -- 172.21.15.16:0/2913764394 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48004210 con 0x556c43ece4d0 2024-09-21T03:47:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.031+0000 7efc42000640 1 -- 172.21.15.16:0/2913764394 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7efc4800be00 con 0x556c43ece4d0 2024-09-21T03:47:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.031+0000 7efc662cb740 1 -- 172.21.15.16:0/2913764394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc480054d0 con 0x556c43ece4d0 2024-09-21T03:47:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.032+0000 7efc42000640 1 -- 172.21.15.16:0/2913764394 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efc48031050 con 0x556c43ece4d0 2024-09-21T03:47:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.032+0000 7efc662cb740 10 client.25712 fetch_fsmap learned FSMap version 30 2024-09-21T03:47:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.032+0000 7efc662cb740 1 -- 172.21.15.16:0/2913764394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c43fd3620 con 0x556c43ece4d0 2024-09-21T03:47:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.032+0000 7efc42000640 1 -- 172.21.15.16:0/2913764394 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc480054d0 con 0x556c43ece4d0 2024-09-21T03:47:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.032+0000 7efc42000640 1 -- 172.21.15.16:0/2913764394 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efc480204b0 con 0x556c43ece4d0 2024-09-21T03:47:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.032+0000 7efc662cb740 10 client.25712 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:47:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.033+0000 7efc662cb740 10 client.25712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.033+0000 7efc662cb740 20 client.25712 populate_metadata read hostname 'smithi016' 2024-09-21T03:47:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.033+0000 7efc662cb740 1 --2- 172.21.15.16:0/2913764394 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556c43eadd80 0x556c442acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.033+0000 7efc662cb740 4 client.25712 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.033+0000 7efc662cb740 1 -- 172.21.15.16:0/2913764394 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c4420b600 con 0x556c43eadd80 2024-09-21T03:47:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.040+0000 7efc57e00640 1 --2- 172.21.15.16:0/2913764394 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556c43eadd80 0x556c442acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.043+0000 7efc42000640 10 client.25712 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.043+0000 7efc57e00640 1 --2- 172.21.15.16:0/2913764394 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556c43eadd80 0x556c442acd00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e2eda70d353aa56 server_cookie=8deaf8f843578bb in_seq=0 out_seq=0 2024-09-21T03:47:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.044+0000 7efc42000640 1 -- 172.21.15.16:0/2913764394 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556c4420b600 con 0x556c43eadd80 2024-09-21T03:47:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.044+0000 7efc42000640 10 client.25712 handle_command_reply: tid=0 2024-09-21T03:47:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.044+0000 7efc662cb740 1 client.25712 shutdown 2024-09-21T03:47:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.045+0000 7efc662cb740 1 -- 172.21.15.16:0/2913764394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 msgr2=0x556c4429feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.045+0000 7efc662cb740 1 --2- 172.21.15.16:0/2913764394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c4429feb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efc48005300 tx=0x7efc48005330 comp rx=0 tx=0).stop 2024-09-21T03:47:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.046+0000 7efc662cb740 1 -- 172.21.15.16:0/2913764394 shutdown_connections 2024-09-21T03:47:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.046+0000 7efc662cb740 1 --2- 172.21.15.16:0/2913764394 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556c43eadd80 0x556c442acd00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.046+0000 7efc662cb740 1 --2- 172.21.15.16:0/2913764394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43eef1b0 0x556c442a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.046+0000 7efc662cb740 1 --2- 172.21.15.16:0/2913764394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c4429feb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.046+0000 7efc662cb740 1 --2- 172.21.15.16:0/2913764394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43ec62f0 0x556c4429f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.046+0000 7efc662cb740 1 -- 172.21.15.16:0/2913764394 >> 172.21.15.16:0/2913764394 conn(0x556c43ee3300 msgr2=0x556c442900e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.046+0000 7efc42000640 0 client.25712 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.046+0000 7efc662cb740 1 -- 172.21.15.16:0/2913764394 shutdown_connections 2024-09-21T03:47:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.046+0000 7efc662cb740 1 -- 172.21.15.16:0/2913764394 wait complete. 2024-09-21T03:47:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.047+0000 7efc662cb740 20 client.25712 trim_cache size 0 max 16384 2024-09-21T03:47:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.071+0000 7efc662cb740 1 Processor -- start 2024-09-21T03:47:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.071+0000 7efc662cb740 1 -- start start 2024-09-21T03:47:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.072+0000 7efc662cb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43ec62f0 0x556c442914a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.072+0000 7efc662cb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c44290850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.072+0000 7efc662cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43eef1b0 0x556c442a7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.072+0000 7efc662cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc581bb460 con 0x556c43eef1b0 2024-09-21T03:47:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.072+0000 7efc662cb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc581bcfa0 con 0x556c43ece4d0 2024-09-21T03:47:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.072+0000 7efc662cb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc581bbb30 con 0x556c43ec62f0 2024-09-21T03:47:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.073+0000 7efc5d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43eef1b0 0x556c442a7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.073+0000 7efc5d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43eef1b0 0x556c442a7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60772/0 (socket says 172.21.15.16:60772) 2024-09-21T03:47:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.073+0000 7efc5d200640 1 -- 172.21.15.16:0/322790922 learned_addr learned my addr 172.21.15.16:0/322790922 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.073+0000 7efc57e00640 1 --2- 172.21.15.16:0/322790922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43ec62f0 0x556c442914a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.073+0000 7efc57400640 1 --2- 172.21.15.16:0/322790922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c44290850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.073+0000 7efc57e00640 1 -- 172.21.15.16:0/322790922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 msgr2=0x556c44290850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.073+0000 7efc57e00640 1 --2- 172.21.15.16:0/322790922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c44290850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.074+0000 7efc57e00640 1 -- 172.21.15.16:0/322790922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43eef1b0 msgr2=0x556c442a7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.074+0000 7efc57e00640 1 --2- 172.21.15.16:0/322790922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43eef1b0 0x556c442a7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.074+0000 7efc57e00640 1 -- 172.21.15.16:0/322790922 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c442a0d90 con 0x556c43ec62f0 2024-09-21T03:47:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.074+0000 7efc57e00640 1 --2- 172.21.15.16:0/322790922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43ec62f0 0x556c442914a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efc440a9710 tx=0x7efc440108a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b6875f66091a7b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.074+0000 7efc4cc00640 1 -- 172.21.15.16:0/322790922 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc440b5e10 con 0x556c43ec62f0 2024-09-21T03:47:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.075+0000 7efc4cc00640 1 -- 172.21.15.16:0/322790922 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc440e6300 con 0x556c43ec62f0 2024-09-21T03:47:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.075+0000 7efc4cc00640 1 -- 172.21.15.16:0/322790922 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc440ab700 con 0x556c43ec62f0 2024-09-21T03:47:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.077+0000 7efc662cb740 1 -- 172.21.15.16:0/322790922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43ec62f0 msgr2=0x556c442914a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.077+0000 7efc662cb740 1 --2- 172.21.15.16:0/322790922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43ec62f0 0x556c442914a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efc440a9710 tx=0x7efc440108a0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.077+0000 7efc662cb740 1 -- 172.21.15.16:0/322790922 shutdown_connections 2024-09-21T03:47:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.077+0000 7efc662cb740 1 --2- 172.21.15.16:0/322790922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43eef1b0 0x556c442a7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.077+0000 7efc662cb740 1 --2- 172.21.15.16:0/322790922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c44290850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.077+0000 7efc662cb740 1 --2- 172.21.15.16:0/322790922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43ec62f0 0x556c442914a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.077+0000 7efc662cb740 1 -- 172.21.15.16:0/322790922 >> 172.21.15.16:0/322790922 conn(0x556c43ee3300 msgr2=0x556c44295cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.077+0000 7efc662cb740 1 -- 172.21.15.16:0/322790922 shutdown_connections 2024-09-21T03:47:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.077+0000 7efc662cb740 1 -- 172.21.15.16:0/322790922 wait complete. 2024-09-21T03:47:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.078+0000 7efc662cb740 1 -- start start 2024-09-21T03:47:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.078+0000 7efc662cb740 1 Processor -- start 2024-09-21T03:47:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.079+0000 7efc662cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43ec62f0 0x556c442a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.079+0000 7efc662cb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c442a3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.079+0000 7efc662cb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43eef1b0 0x556c442a7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.079+0000 7efc662cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc581bc8d0 con 0x556c43ec62f0 2024-09-21T03:47:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.079+0000 7efc662cb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc581bc200 con 0x556c43ece4d0 2024-09-21T03:47:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.079+0000 7efc662cb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc581b9440 con 0x556c43eef1b0 2024-09-21T03:47:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.079+0000 7efc662cb740 10 client.0 _ll_register_callbacks cb 0x7efc48004470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.079+0000 7efc57400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c442a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.079+0000 7efc57400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c442a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:57314/0 (socket says 172.21.15.16:57314) 2024-09-21T03:47:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.079+0000 7efc57400640 1 -- 172.21.15.16:0/3972185281 learned_addr learned my addr 172.21.15.16:0/3972185281 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.079+0000 7efc57e00640 1 --2- 172.21.15.16:0/3972185281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43ec62f0 0x556c442a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.079+0000 7efc5d200640 1 --2- 172.21.15.16:0/3972185281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43eef1b0 0x556c442a7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.080+0000 7efc57400640 1 -- 172.21.15.16:0/3972185281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43eef1b0 msgr2=0x556c442a7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.080+0000 7efc57400640 1 --2- 172.21.15.16:0/3972185281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43eef1b0 0x556c442a7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.080+0000 7efc57400640 1 -- 172.21.15.16:0/3972185281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43ec62f0 msgr2=0x556c442a2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.080+0000 7efc57400640 1 --2- 172.21.15.16:0/3972185281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43ec62f0 0x556c442a2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.080+0000 7efc57400640 1 -- 172.21.15.16:0/3972185281 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c442a7790 con 0x556c43ece4d0 2024-09-21T03:47:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.080+0000 7efc57e00640 1 --2- 172.21.15.16:0/3972185281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43ec62f0 0x556c442a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.080+0000 7efc5d200640 1 --2- 172.21.15.16:0/3972185281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43eef1b0 0x556c442a7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:47:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.080+0000 7efc57400640 1 --2- 172.21.15.16:0/3972185281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c442a3440 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efc48020a20 tx=0x7efc48021fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43fd4ddac6109cff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.080+0000 7efc42000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:47:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.081+0000 7efc42000640 1 -- 172.21.15.16:0/3972185281 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48004210 con 0x556c43ece4d0 2024-09-21T03:47:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.081+0000 7efc42000640 1 -- 172.21.15.16:0/3972185281 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7efc4802e040 con 0x556c43ece4d0 2024-09-21T03:47:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.081+0000 7efc662cb740 1 -- 172.21.15.16:0/3972185281 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc440e6300 con 0x556c43ece4d0 2024-09-21T03:47:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.081+0000 7efc42000640 1 -- 172.21.15.16:0/3972185281 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efc48003de0 con 0x556c43ece4d0 2024-09-21T03:47:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.082+0000 7efc662cb740 10 client.25718 fetch_fsmap learned FSMap version 30 2024-09-21T03:47:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.082+0000 7efc662cb740 1 -- 172.21.15.16:0/3972185281 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c442a4a80 con 0x556c43ece4d0 2024-09-21T03:47:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.082+0000 7efc42000640 1 -- 172.21.15.16:0/3972185281 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc440e6300 con 0x556c43ece4d0 2024-09-21T03:47:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.082+0000 7efc42000640 1 -- 172.21.15.16:0/3972185281 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efc48016500 con 0x556c43ece4d0 2024-09-21T03:47:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.083+0000 7efc662cb740 10 client.25718 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:47:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.083+0000 7efc662cb740 10 client.25718 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:47:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.083+0000 7efc662cb740 20 client.25718 populate_metadata read hostname 'smithi016' 2024-09-21T03:47:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.083+0000 7efc662cb740 1 --2- 172.21.15.16:0/3972185281 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556c43e831c0 0x556c44319b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.083+0000 7efc662cb740 4 client.25718 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:47:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.083+0000 7efc662cb740 1 -- 172.21.15.16:0/3972185281 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c442070b0 con 0x556c43e831c0 2024-09-21T03:47:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.084+0000 7efc57e00640 1 --2- 172.21.15.16:0/3972185281 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556c43e831c0 0x556c44319b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.084+0000 7efc57e00640 1 --2- 172.21.15.16:0/3972185281 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556c43e831c0 0x556c44319b90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbf63b9e15b2c96f server_cookie=695b4f122ddcc584 in_seq=0 out_seq=0 2024-09-21T03:47:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.084+0000 7efc42000640 10 client.25718 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.127+0000 7efc42000640 1 -- 172.21.15.16:0/3972185281 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c442070b0 con 0x556c43e831c0 2024-09-21T03:47:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.127+0000 7efc42000640 10 client.25718 handle_command_reply: tid=0 2024-09-21T03:47:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.127+0000 7efc662cb740 1 client.25718 shutdown 2024-09-21T03:47:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.134+0000 7efc662cb740 1 -- 172.21.15.16:0/3972185281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 msgr2=0x556c442a3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.134+0000 7efc662cb740 1 --2- 172.21.15.16:0/3972185281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c442a3440 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efc48020a20 tx=0x7efc48021fc0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.134+0000 7efc662cb740 1 -- 172.21.15.16:0/3972185281 shutdown_connections 2024-09-21T03:47:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.134+0000 7efc662cb740 1 --2- 172.21.15.16:0/3972185281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c43eef1b0 0x556c442a7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.134+0000 7efc662cb740 1 --2- 172.21.15.16:0/3972185281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c43ece4d0 0x556c442a3440 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.134+0000 7efc662cb740 1 --2- 172.21.15.16:0/3972185281 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556c43e831c0 0x556c44319b90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.134+0000 7efc662cb740 1 --2- 172.21.15.16:0/3972185281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c43ec62f0 0x556c442a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.134+0000 7efc662cb740 1 -- 172.21.15.16:0/3972185281 >> 172.21.15.16:0/3972185281 conn(0x556c43ee3300 msgr2=0x556c442907f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.135+0000 7efc42000640 0 client.25718 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.135+0000 7efc662cb740 1 -- 172.21.15.16:0/3972185281 shutdown_connections 2024-09-21T03:47:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.135+0000 7efc662cb740 1 -- 172.21.15.16:0/3972185281 wait complete. 2024-09-21T03:47:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.136+0000 7efc662cb740 20 client.25718 trim_cache size 0 max 16384 2024-09-21T03:47:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:47:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:47:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:47:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:47:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.137+0000 7efc5f200640 1 -- 172.21.15.16:0/1459820276 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efc3c0b6520 msgr2=0x7efc3c0b8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.137+0000 7efc5f200640 1 --2- 172.21.15.16:0/1459820276 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efc3c0b6520 0x7efc3c0b8a30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b010 tx=0x7efc50032460 comp rx=0 tx=0).stop 2024-09-21T03:47:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.137+0000 7efc5f200640 1 -- 172.21.15.16:0/1459820276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc5810a5a0 msgr2=0x7efc581a8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.137+0000 7efc5f200640 1 --2- 172.21.15.16:0/1459820276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc5810a5a0 0x7efc581a8d10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efc4400b450 tx=0x7efc4400b910 comp rx=0 tx=0).stop 2024-09-21T03:47:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.137+0000 7efc5f200640 1 -- 172.21.15.16:0/1459820276 shutdown_connections 2024-09-21T03:47:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.137+0000 7efc5f200640 1 --2- 172.21.15.16:0/1459820276 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efc3c0b6520 0x7efc3c0b8a30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.137+0000 7efc5f200640 1 --2- 172.21.15.16:0/1459820276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc5810bc00 0x7efc581a9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.137+0000 7efc5f200640 1 --2- 172.21.15.16:0/1459820276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5810aff0 0x7efc581a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.137+0000 7efc5f200640 1 --2- 172.21.15.16:0/1459820276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc5810a5a0 0x7efc581a8d10 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efc4400b450 tx=0x7efc4400b910 comp rx=0 tx=0).stop 2024-09-21T03:47:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.137+0000 7efc5f200640 1 -- 172.21.15.16:0/1459820276 >> 172.21.15.16:0/1459820276 conn(0x7efc58070810 msgr2=0x7efc58117960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.137+0000 7efc5f200640 1 -- 172.21.15.16:0/1459820276 shutdown_connections 2024-09-21T03:47:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:47:48.137+0000 7efc5f200640 1 -- 172.21.15.16:0/1459820276 wait complete. 2024-09-21T03:47:48.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:48 smithi016 ceph-mon[30375]: pgmap v463: 129 pgs: 129 active+clean; 329 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-21T03:47:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.952+0000 7fd817c00640 1 -- 172.21.15.16:0/554200007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd81010d1c0 msgr2=0x7fd810075660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.952+0000 7fd817c00640 1 --2- 172.21.15.16:0/554200007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd81010d1c0 0x7fd810075660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009900 tx=0x7fd7fc02a400 comp rx=0 tx=0).stop 2024-09-21T03:47:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.952+0000 7fd817c00640 1 -- 172.21.15.16:0/554200007 shutdown_connections 2024-09-21T03:47:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.952+0000 7fd817c00640 1 --2- 172.21.15.16:0/554200007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd81010d1c0 0x7fd810075660 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.952+0000 7fd817c00640 1 --2- 172.21.15.16:0/554200007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd81010c870 0x7fd81010cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.952+0000 7fd817c00640 1 --2- 172.21.15.16:0/554200007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd81010a3c0 0x7fd81010aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.952+0000 7fd817c00640 1 -- 172.21.15.16:0/554200007 >> 172.21.15.16:0/554200007 conn(0x7fd810070e80 msgr2=0x7fd810071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.953+0000 7fd817c00640 1 -- 172.21.15.16:0/554200007 shutdown_connections 2024-09-21T03:47:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.953+0000 7fd817c00640 1 -- 172.21.15.16:0/554200007 wait complete. 2024-09-21T03:47:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.954+0000 7fd817c00640 1 Processor -- start 2024-09-21T03:47:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.954+0000 7fd817c00640 1 -- start start 2024-09-21T03:47:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.955+0000 7fd817c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd81010a3c0 0x7fd8100805d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.955+0000 7fd817c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd81010c870 0x7fd810080b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.955+0000 7fd817c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd81010d1c0 0x7fd810081050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.955+0000 7fd817c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd81007eb30 con 0x7fd81010a3c0 2024-09-21T03:47:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.955+0000 7fd817c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd81007e9b0 con 0x7fd81010d1c0 2024-09-21T03:47:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.955+0000 7fd817c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd81007ecb0 con 0x7fd81010c870 2024-09-21T03:47:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.956+0000 7fd80fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd81010c870 0x7fd810080b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.956+0000 7fd80fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd81010c870 0x7fd810080b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34054/0 (socket says 172.21.15.16:34054) 2024-09-21T03:47:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.956+0000 7fd80fe00640 1 -- 172.21.15.16:0/1950537308 learned_addr learned my addr 172.21.15.16:0/1950537308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.956+0000 7fd815c00640 1 --2- 172.21.15.16:0/1950537308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd81010d1c0 0x7fd810081050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.956+0000 7fd815200640 1 --2- 172.21.15.16:0/1950537308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd81010a3c0 0x7fd8100805d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.957+0000 7fd80fe00640 1 -- 172.21.15.16:0/1950537308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd81010d1c0 msgr2=0x7fd810081050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.957+0000 7fd80fe00640 1 --2- 172.21.15.16:0/1950537308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd81010d1c0 0x7fd810081050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.957+0000 7fd80fe00640 1 -- 172.21.15.16:0/1950537308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd81010a3c0 msgr2=0x7fd8100805d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.957+0000 7fd80fe00640 1 --2- 172.21.15.16:0/1950537308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd81010a3c0 0x7fd8100805d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.957+0000 7fd80fe00640 1 -- 172.21.15.16:0/1950537308 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8101bdb10 con 0x7fd81010c870 2024-09-21T03:47:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.958+0000 7fd80fe00640 1 --2- 172.21.15.16:0/1950537308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd81010c870 0x7fd810080b10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd800014110 tx=0x7fd8000145d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.960+0000 7fd80d600640 1 -- 172.21.15.16:0/1950537308 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800014e00 con 0x7fd81010c870 2024-09-21T03:47:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.960+0000 7fd80d600640 1 -- 172.21.15.16:0/1950537308 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd80000fbe0 con 0x7fd81010c870 2024-09-21T03:47:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.960+0000 7fd80d600640 1 -- 172.21.15.16:0/1950537308 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8000035f0 con 0x7fd81010c870 2024-09-21T03:47:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.966+0000 7fd817c00640 1 -- 172.21.15.16:0/1950537308 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8101bdda0 con 0x7fd81010c870 2024-09-21T03:47:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.966+0000 7fd817c00640 1 -- 172.21.15.16:0/1950537308 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8101be200 con 0x7fd81010c870 2024-09-21T03:47:48.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.976+0000 7fd80d600640 1 -- 172.21.15.16:0/1950537308 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd800022070 con 0x7fd81010c870 2024-09-21T03:47:48.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.979+0000 7fd80d600640 1 --2- 172.21.15.16:0/1950537308 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd7f40b65d0 0x7fd7f40b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.979+0000 7fd80d600640 1 -- 172.21.15.16:0/1950537308 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fd8000eed30 con 0x7fd81010c870 2024-09-21T03:47:48.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.980+0000 7fd81e882740 1 Processor -- start 2024-09-21T03:47:48.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.981+0000 7fd81e882740 1 -- start start 2024-09-21T03:47:48.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.982+0000 7fd81e882740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709458b840 0x557094933350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.982+0000 7fd81e882740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557094562980 0x557094933890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.982+0000 7fd81e882740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55709456ab60 0x557094933dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.982+0000 7fd81e882740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570948a8ca0 con 0x55709456ab60 2024-09-21T03:47:48.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.982+0000 7fd81e882740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5570948980d0 con 0x557094562980 2024-09-21T03:47:48.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.982+0000 7fd81e882740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55709489bee0 con 0x55709458b840 2024-09-21T03:47:48.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.983+0000 7fd80fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557094562980 0x557094933890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.983+0000 7fd80fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557094562980 0x557094933890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:57344/0 (socket says 172.21.15.16:57344) 2024-09-21T03:47:48.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.983+0000 7fd80fe00640 1 -- 172.21.15.16:0/539591861 learned_addr learned my addr 172.21.15.16:0/539591861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:48.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.984+0000 7fd815c00640 1 --2- 172.21.15.16:0/539591861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55709456ab60 0x557094933dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.984+0000 7fd80fe00640 1 -- 172.21.15.16:0/539591861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709458b840 msgr2=0x557094933350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:47:48.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.984+0000 7fd80fe00640 1 --2- 172.21.15.16:0/539591861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709458b840 0x557094933350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.984+0000 7fd80fe00640 1 -- 172.21.15.16:0/539591861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55709456ab60 msgr2=0x557094933dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:48.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.985+0000 7fd80fe00640 1 --2- 172.21.15.16:0/539591861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55709456ab60 0x557094933dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.985+0000 7fd80fe00640 1 -- 172.21.15.16:0/539591861 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55709484a400 con 0x557094562980 2024-09-21T03:47:48.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.985+0000 7fd815c00640 1 --2- 172.21.15.16:0/539591861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55709456ab60 0x557094933dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:47:48.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.986+0000 7fd80fe00640 1 --2- 172.21.15.16:0/539591861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557094562980 0x557094933890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd8000b4aa0 tx=0x7fd8000b4a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78cd05e4554965b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:48.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.986+0000 7fd806000640 1 -- 172.21.15.16:0/539591861 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8000b0280 con 0x557094562980 2024-09-21T03:47:48.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.986+0000 7fd806000640 1 -- 172.21.15.16:0/539591861 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8000b4620 con 0x557094562980 2024-09-21T03:47:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.988+0000 7fd81e882740 1 -- 172.21.15.16:0/539591861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557094562980 msgr2=0x557094933890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.988+0000 7fd81e882740 1 --2- 172.21.15.16:0/539591861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557094562980 0x557094933890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd8000b4aa0 tx=0x7fd8000b4a10 comp rx=0 tx=0).stop 2024-09-21T03:47:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.989+0000 7fd815200640 1 --2- 172.21.15.16:0/1950537308 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd7f40b65d0 0x7fd7f40b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.990+0000 7fd81e882740 1 -- 172.21.15.16:0/539591861 shutdown_connections 2024-09-21T03:47:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.990+0000 7fd81e882740 1 --2- 172.21.15.16:0/539591861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55709456ab60 0x557094933dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.990+0000 7fd81e882740 1 --2- 172.21.15.16:0/539591861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557094562980 0x557094933890 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.990+0000 7fd81e882740 1 --2- 172.21.15.16:0/539591861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709458b840 0x557094933350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.990+0000 7fd81e882740 1 -- 172.21.15.16:0/539591861 >> 172.21.15.16:0/539591861 conn(0x55709457f990 msgr2=0x557094931240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.990+0000 7fd81e882740 1 -- 172.21.15.16:0/539591861 shutdown_connections 2024-09-21T03:47:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.990+0000 7fd81e882740 1 -- 172.21.15.16:0/539591861 wait complete. 2024-09-21T03:47:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.991+0000 7fd81e882740 1 -- start start 2024-09-21T03:47:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.991+0000 7fd81e882740 1 Processor -- start 2024-09-21T03:47:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.992+0000 7fd81e882740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557094562980 0x55709493e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.992+0000 7fd81e882740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709456ab60 0x55709493eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.992+0000 7fd81e882740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55709458b840 0x55709493f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.993+0000 7fd80fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709456ab60 0x55709493eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.993+0000 7fd80fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709456ab60 0x55709493eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34060/0 (socket says 172.21.15.16:34060) 2024-09-21T03:47:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.993+0000 7fd815200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557094562980 0x55709493e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.993+0000 7fd815200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557094562980 0x55709493e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:57352/0 (socket says 172.21.15.16:57352) 2024-09-21T03:47:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.993+0000 7fd81e882740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570946f97a0 con 0x55709458b840 2024-09-21T03:47:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.993+0000 7fd81e882740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55709483da10 con 0x557094562980 2024-09-21T03:47:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.993+0000 7fd81e882740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557094952b20 con 0x55709456ab60 2024-09-21T03:47:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.993+0000 7fd81e882740 10 client.0 _ll_register_callbacks cb 0x55709484dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.994+0000 7fd815c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55709458b840 0x55709493f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.994+0000 7fd80fe00640 1 -- 172.21.15.16:0/443876236 learned_addr learned my addr 172.21.15.16:0/443876236 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.995+0000 7fd80fe00640 1 -- 172.21.15.16:0/443876236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557094562980 msgr2=0x55709493e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.995+0000 7fd80fe00640 1 --2- 172.21.15.16:0/443876236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557094562980 0x55709493e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.995+0000 7fd80fe00640 1 -- 172.21.15.16:0/443876236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55709458b840 msgr2=0x55709493f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.995+0000 7fd80fe00640 1 --2- 172.21.15.16:0/443876236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55709458b840 0x55709493f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.995+0000 7fd80fe00640 1 -- 172.21.15.16:0/443876236 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557094605b90 con 0x55709456ab60 2024-09-21T03:47:49.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.996+0000 7fd80fe00640 1 --2- 172.21.15.16:0/443876236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709456ab60 0x55709493eed0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd8000ec7f0 tx=0x7fd8000d6f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b910a3d382a760a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:49.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.996+0000 7fd7faa00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:47:49.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.997+0000 7fd7faa00640 1 -- 172.21.15.16:0/443876236 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8000d81b0 con 0x55709456ab60 2024-09-21T03:47:49.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.997+0000 7fd7faa00640 1 -- 172.21.15.16:0/443876236 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fd8000b9090 con 0x55709456ab60 2024-09-21T03:47:49.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.997+0000 7fd81e882740 1 -- 172.21.15.16:0/443876236 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8000b4620 con 0x55709456ab60 2024-09-21T03:47:49.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.997+0000 7fd7faa00640 1 -- 172.21.15.16:0/443876236 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd8000b95f0 con 0x55709456ab60 2024-09-21T03:47:49.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.998+0000 7fd81e882740 10 client.25708 fetch_fsmap learned FSMap version 30 2024-09-21T03:47:49.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.998+0000 7fd81e882740 1 -- 172.21.15.16:0/443876236 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55709495e780 con 0x55709456ab60 2024-09-21T03:47:49.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.998+0000 7fd7faa00640 1 -- 172.21.15.16:0/443876236 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8000b4620 con 0x55709456ab60 2024-09-21T03:47:49.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.999+0000 7fd7faa00640 1 -- 172.21.15.16:0/443876236 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd8000b37e0 con 0x55709456ab60 2024-09-21T03:47:49.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.999+0000 7fd81e882740 10 client.25708 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:47:49.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:48.999+0000 7fd81e882740 10 client.25708 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:47:49.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.000+0000 7fd81e882740 20 client.25708 populate_metadata read hostname 'smithi016' 2024-09-21T03:47:49.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.000+0000 7fd81e882740 1 --2- 172.21.15.16:0/443876236 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55709454a1c0 0x557094961ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:49.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.000+0000 7fd81e882740 4 client.25708 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:47:49.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.000+0000 7fd81e882740 1 -- 172.21.15.16:0/443876236 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570948aabf0 con 0x55709454a1c0 2024-09-21T03:47:49.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.002+0000 7fd815200640 1 --2- 172.21.15.16:0/443876236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557094562980 0x55709493e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:47:49.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.003+0000 7fd815200640 1 --2- 172.21.15.16:0/443876236 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55709454a1c0 0x557094961ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:49.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.006+0000 7fd7faa00640 10 client.25708 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:49.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.006+0000 7fd815200640 1 --2- 172.21.15.16:0/443876236 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55709454a1c0 0x557094961ae0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b3a7f10adef8008 server_cookie=6ecaf87d63d5c6ab in_seq=0 out_seq=0 2024-09-21T03:47:49.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.009+0000 7fd815200640 1 --2- 172.21.15.16:0/1950537308 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd7f40b65d0 0x7fd7f40b8ae0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x5570949358b0 tx=0x7fd808005bd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:49.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.009+0000 7fd7faa00640 1 -- 172.21.15.16:0/443876236 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5570948aabf0 con 0x55709454a1c0 2024-09-21T03:47:49.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.009+0000 7fd7faa00640 10 client.25708 handle_command_reply: tid=0 2024-09-21T03:47:49.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.009+0000 7fd81e882740 1 client.25708 shutdown 2024-09-21T03:47:49.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.010+0000 7fd81e882740 1 -- 172.21.15.16:0/443876236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709456ab60 msgr2=0x55709493eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:49.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.010+0000 7fd81e882740 1 --2- 172.21.15.16:0/443876236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709456ab60 0x55709493eed0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd8000ec7f0 tx=0x7fd8000d6f00 comp rx=0 tx=0).stop 2024-09-21T03:47:49.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.012+0000 7fd81e882740 1 -- 172.21.15.16:0/443876236 shutdown_connections 2024-09-21T03:47:49.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.012+0000 7fd81e882740 1 --2- 172.21.15.16:0/443876236 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55709454a1c0 0x557094961ae0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.012+0000 7fd81e882740 1 --2- 172.21.15.16:0/443876236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55709458b840 0x55709493f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.012+0000 7fd81e882740 1 --2- 172.21.15.16:0/443876236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709456ab60 0x55709493eed0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.012+0000 7fd81e882740 1 --2- 172.21.15.16:0/443876236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557094562980 0x55709493e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.012+0000 7fd7faa00640 0 client.25708 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:49.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.012+0000 7fd81e882740 1 -- 172.21.15.16:0/443876236 >> 172.21.15.16:0/443876236 conn(0x55709457f990 msgr2=0x55709492f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:49.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.013+0000 7fd81e882740 1 -- 172.21.15.16:0/443876236 shutdown_connections 2024-09-21T03:47:49.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.013+0000 7fd81e882740 1 -- 172.21.15.16:0/443876236 wait complete. 2024-09-21T03:47:49.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.013+0000 7fd81e882740 20 client.25708 trim_cache size 0 max 16384 2024-09-21T03:47:49.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.047+0000 7fd81e882740 1 Processor -- start 2024-09-21T03:47:49.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.047+0000 7fd81e882740 1 -- start start 2024-09-21T03:47:49.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.048+0000 7fd81e882740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557094562980 0x557094934d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.048+0000 7fd81e882740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709456ab60 0x557094940550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.048+0000 7fd81e882740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55709458b840 0x55709493ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.048+0000 7fd81e882740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd8101c81e0 con 0x557094562980 2024-09-21T03:47:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.048+0000 7fd81e882740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd8101c9d20 con 0x55709458b840 2024-09-21T03:47:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.048+0000 7fd81e882740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8101c88b0 con 0x55709456ab60 2024-09-21T03:47:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.049+0000 7fd815c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55709458b840 0x55709493ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.049+0000 7fd815c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55709458b840 0x55709493ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:57356/0 (socket says 172.21.15.16:57356) 2024-09-21T03:47:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.049+0000 7fd815c00640 1 -- 172.21.15.16:0/3734525816 learned_addr learned my addr 172.21.15.16:0/3734525816 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.049+0000 7fd815200640 1 --2- 172.21.15.16:0/3734525816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557094562980 0x557094934d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.049+0000 7fd80fe00640 1 --2- 172.21.15.16:0/3734525816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709456ab60 0x557094940550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.050+0000 7fd80fe00640 1 -- 172.21.15.16:0/3734525816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55709458b840 msgr2=0x55709493ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.050+0000 7fd80fe00640 1 --2- 172.21.15.16:0/3734525816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55709458b840 0x55709493ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.050+0000 7fd80fe00640 1 -- 172.21.15.16:0/3734525816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557094562980 msgr2=0x557094934d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.050+0000 7fd80fe00640 1 --2- 172.21.15.16:0/3734525816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557094562980 0x557094934d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.050+0000 7fd80fe00640 1 -- 172.21.15.16:0/3734525816 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55709492cc50 con 0x55709456ab60 2024-09-21T03:47:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.050+0000 7fd80fe00640 1 --2- 172.21.15.16:0/3734525816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709456ab60 0x557094940550 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd8000b1d80 tx=0x7fd800064190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=921810480b4bee36 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.050+0000 7fd805600640 1 -- 172.21.15.16:0/3734525816 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8000b9070 con 0x55709456ab60 2024-09-21T03:47:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.050+0000 7fd805600640 1 -- 172.21.15.16:0/3734525816 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8000b5210 con 0x55709456ab60 2024-09-21T03:47:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.050+0000 7fd805600640 1 -- 172.21.15.16:0/3734525816 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8000b9210 con 0x55709456ab60 2024-09-21T03:47:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.052+0000 7fd81e882740 1 -- 172.21.15.16:0/3734525816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709456ab60 msgr2=0x557094940550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.052+0000 7fd81e882740 1 --2- 172.21.15.16:0/3734525816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709456ab60 0x557094940550 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd8000b1d80 tx=0x7fd800064190 comp rx=0 tx=0).stop 2024-09-21T03:47:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.055+0000 7fd81e882740 1 -- 172.21.15.16:0/3734525816 shutdown_connections 2024-09-21T03:47:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.055+0000 7fd81e882740 1 --2- 172.21.15.16:0/3734525816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55709458b840 0x55709493ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.055+0000 7fd81e882740 1 --2- 172.21.15.16:0/3734525816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709456ab60 0x557094940550 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.055+0000 7fd81e882740 1 --2- 172.21.15.16:0/3734525816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557094562980 0x557094934d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.055+0000 7fd81e882740 1 -- 172.21.15.16:0/3734525816 >> 172.21.15.16:0/3734525816 conn(0x55709457f990 msgr2=0x55709493fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.055+0000 7fd81e882740 1 -- 172.21.15.16:0/3734525816 shutdown_connections 2024-09-21T03:47:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.055+0000 7fd81e882740 1 -- 172.21.15.16:0/3734525816 wait complete. 2024-09-21T03:47:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.056+0000 7fd81e882740 1 -- start start 2024-09-21T03:47:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.056+0000 7fd81e882740 1 Processor -- start 2024-09-21T03:47:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.057+0000 7fd81e882740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557094562980 0x557094963370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.058+0000 7fd81e882740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709456ab60 0x557094963950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.058+0000 7fd81e882740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55709458b840 0x5570949c5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.058+0000 7fd81e882740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd8101c9650 con 0x557094562980 2024-09-21T03:47:49.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.058+0000 7fd81e882740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd8101c8f80 con 0x55709458b840 2024-09-21T03:47:49.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.058+0000 7fd81e882740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8101c61c0 con 0x55709456ab60 2024-09-21T03:47:49.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.058+0000 7fd815c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55709458b840 0x5570949c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:49.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.058+0000 7fd815c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55709458b840 0x5570949c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:57372/0 (socket says 172.21.15.16:57372) 2024-09-21T03:47:49.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.058+0000 7fd815c00640 1 -- 172.21.15.16:0/3806649032 learned_addr learned my addr 172.21.15.16:0/3806649032 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:47:49.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.059+0000 7fd815c00640 1 -- 172.21.15.16:0/3806649032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709456ab60 msgr2=0x557094963950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:47:49.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.059+0000 7fd815c00640 1 --2- 172.21.15.16:0/3806649032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709456ab60 0x557094963950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.059+0000 7fd815c00640 1 -- 172.21.15.16:0/3806649032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557094562980 msgr2=0x557094963370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:47:49.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.059+0000 7fd815c00640 1 --2- 172.21.15.16:0/3806649032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557094562980 0x557094963370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.059+0000 7fd815c00640 1 -- 172.21.15.16:0/3806649032 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570949c58a0 con 0x55709458b840 2024-09-21T03:47:49.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.059+0000 7fd81e882740 10 client.0 _ll_register_callbacks cb 0x7fd8000b48e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:47:49.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.059+0000 7fd81e882740 1 -- 172.21.15.16:0/3806649032 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8000b5210 con 0x55709458b840 2024-09-21T03:47:49.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.061+0000 7fd7faa00640 10 client.25736 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:47:49.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.061+0000 7fd815c00640 1 --2- 172.21.15.16:0/3806649032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55709458b840 0x5570949c5230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009230 tx=0x7fd7fc0044f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8185157a0a77f3aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:47:49.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.062+0000 7fd7faa00640 1 -- 172.21.15.16:0/3806649032 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc035040 con 0x55709458b840 2024-09-21T03:47:49.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.062+0000 7fd7faa00640 1 -- 172.21.15.16:0/3806649032 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fd7fc005220 con 0x55709458b840 2024-09-21T03:47:49.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.062+0000 7fd7faa00640 1 -- 172.21.15.16:0/3806649032 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd7fc005440 con 0x55709458b840 2024-09-21T03:47:49.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.062+0000 7fd81e882740 10 client.25736 fetch_fsmap learned FSMap version 30 2024-09-21T03:47:49.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.062+0000 7fd81e882740 1 -- 172.21.15.16:0/3806649032 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5570949ccad0 con 0x55709458b840 2024-09-21T03:47:49.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.069+0000 7fd7faa00640 1 -- 172.21.15.16:0/3806649032 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd7fc044070 con 0x55709458b840 2024-09-21T03:47:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.069+0000 7fd81e882740 10 client.25736 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:47:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.069+0000 7fd81e882740 10 client.25736 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:47:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.069+0000 7fd81e882740 20 client.25736 populate_metadata read hostname 'smithi016' 2024-09-21T03:47:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.069+0000 7fd81e882740 1 --2- 172.21.15.16:0/3806649032 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55709451f570 0x5570949cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:47:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.069+0000 7fd81e882740 4 client.25736 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}] 2024-09-21T03:47:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.069+0000 7fd81e882740 1 -- 172.21.15.16:0/3806649032 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}) -- 0x5570948a66a0 con 0x55709451f570 2024-09-21T03:47:49.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.070+0000 7fd815200640 1 --2- 172.21.15.16:0/3806649032 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55709451f570 0x5570949cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:47:49.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.070+0000 7fd7faa00640 10 client.25736 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:49.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.070+0000 7fd815200640 1 --2- 172.21.15.16:0/3806649032 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55709451f570 0x5570949cf200 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=882024be4c1a3018 server_cookie=4039daaa2b9f7b9e in_seq=0 out_seq=0 2024-09-21T03:47:49.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:48 smithi134 ceph-mon[35031]: pgmap v463: 129 pgs: 129 active+clean; 329 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-21T03:47:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.141+0000 7fd7faa00640 1 -- 172.21.15.16:0/3806649032 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5570948a66a0 con 0x55709451f570 2024-09-21T03:47:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.141+0000 7fd7faa00640 10 client.25736 handle_command_reply: tid=0 2024-09-21T03:47:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.141+0000 7fd81e882740 1 client.25736 shutdown 2024-09-21T03:47:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.142+0000 7fd81e882740 1 -- 172.21.15.16:0/3806649032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55709458b840 msgr2=0x5570949c5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.142+0000 7fd81e882740 1 --2- 172.21.15.16:0/3806649032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55709458b840 0x5570949c5230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009230 tx=0x7fd7fc0044f0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.142+0000 7fd81e882740 1 -- 172.21.15.16:0/3806649032 shutdown_connections 2024-09-21T03:47:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.142+0000 7fd81e882740 1 --2- 172.21.15.16:0/3806649032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55709458b840 0x5570949c5230 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.142+0000 7fd81e882740 1 --2- 172.21.15.16:0/3806649032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55709456ab60 0x557094963950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.142+0000 7fd81e882740 1 --2- 172.21.15.16:0/3806649032 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55709451f570 0x5570949cf200 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.142+0000 7fd81e882740 1 --2- 172.21.15.16:0/3806649032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557094562980 0x557094963370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.142+0000 7fd7faa00640 0 client.25736 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:47:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.142+0000 7fd81e882740 1 -- 172.21.15.16:0/3806649032 >> 172.21.15.16:0/3806649032 conn(0x55709457f990 msgr2=0x55709495adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.142+0000 7fd81e882740 1 -- 172.21.15.16:0/3806649032 shutdown_connections 2024-09-21T03:47:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.142+0000 7fd81e882740 1 -- 172.21.15.16:0/3806649032 wait complete. 2024-09-21T03:47:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.142+0000 7fd81e882740 20 client.25736 trim_cache size 0 max 16384 2024-09-21T03:47:49.147 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:47:49.147 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:47:49.147 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:47:49.147 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 6, 2024-09-21T03:47:49.147 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:47:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "39a29551": { 2024-09-21T03:47:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 6, 2024-09-21T03:47:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:47:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:47:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:47:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:47:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:47:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:47:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 137.0, 2024-09-21T03:47:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:47:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:47:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:47:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:47:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:47:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:47:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:47:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:47:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:47:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:47:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:47:49.149 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:47:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.144+0000 7fd817c00640 1 -- 172.21.15.16:0/1950537308 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd7f40b65d0 msgr2=0x7fd7f40b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.144+0000 7fd817c00640 1 --2- 172.21.15.16:0/1950537308 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd7f40b65d0 0x7fd7f40b8ae0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x5570949358b0 tx=0x7fd808005bd0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.144+0000 7fd817c00640 1 -- 172.21.15.16:0/1950537308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd81010c870 msgr2=0x7fd810080b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:47:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.144+0000 7fd817c00640 1 --2- 172.21.15.16:0/1950537308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd81010c870 0x7fd810080b10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd800014110 tx=0x7fd8000145d0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.144+0000 7fd817c00640 1 -- 172.21.15.16:0/1950537308 shutdown_connections 2024-09-21T03:47:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.144+0000 7fd817c00640 1 --2- 172.21.15.16:0/1950537308 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd7f40b65d0 0x7fd7f40b8ae0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.144+0000 7fd817c00640 1 --2- 172.21.15.16:0/1950537308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd81010d1c0 0x7fd810081050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.144+0000 7fd817c00640 1 --2- 172.21.15.16:0/1950537308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd81010c870 0x7fd810080b10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.144+0000 7fd817c00640 1 --2- 172.21.15.16:0/1950537308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd81010a3c0 0x7fd8100805d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:47:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.144+0000 7fd817c00640 1 -- 172.21.15.16:0/1950537308 >> 172.21.15.16:0/1950537308 conn(0x7fd810070e80 msgr2=0x7fd81010fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:47:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.146+0000 7fd817c00640 1 -- 172.21.15.16:0/1950537308 shutdown_connections 2024-09-21T03:47:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:47:49.146+0000 7fd817c00640 1 -- 172.21.15.16:0/1950537308 wait complete. 2024-09-21T03:47:49.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:48 smithi123 ceph-mon[34673]: pgmap v463: 129 pgs: 129 active+clean; 329 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-21T03:47:50.460 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '39a29551', quiesce took 8.3 seconds. Will release after: 8.7 2024-09-21T03:47:51.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:50 smithi134 ceph-mon[35031]: pgmap v464: 129 pgs: 129 active+clean; 661 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 55 MiB/s wr, 34 op/s 2024-09-21T03:47:51.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:50 smithi123 ceph-mon[34673]: pgmap v464: 129 pgs: 129 active+clean; 661 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 55 MiB/s wr, 34 op/s 2024-09-21T03:47:51.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:50 smithi016 ceph-mon[30375]: pgmap v464: 129 pgs: 129 active+clean; 661 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 55 MiB/s wr, 34 op/s 2024-09-21T03:47:53.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:52 smithi134 ceph-mon[35031]: pgmap v465: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 96 MiB/s wr, 45 op/s 2024-09-21T03:47:53.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:52 smithi123 ceph-mon[34673]: pgmap v465: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 96 MiB/s wr, 45 op/s 2024-09-21T03:47:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:52 smithi016 ceph-mon[30375]: pgmap v465: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 96 MiB/s wr, 45 op/s 2024-09-21T03:47:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:54 smithi134 ceph-mon[35031]: pgmap v466: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 138 MiB/s wr, 63 op/s 2024-09-21T03:47:55.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:54 smithi123 ceph-mon[34673]: pgmap v466: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 138 MiB/s wr, 63 op/s 2024-09-21T03:47:55.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:54 smithi016 ceph-mon[30375]: pgmap v466: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 138 MiB/s wr, 63 op/s 2024-09-21T03:47:57.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:56 smithi134 ceph-mon[35031]: pgmap v467: 129 pgs: 129 active+clean; 2.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 197 MiB/s wr, 72 op/s 2024-09-21T03:47:57.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:56 smithi123 ceph-mon[34673]: pgmap v467: 129 pgs: 129 active+clean; 2.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 197 MiB/s wr, 72 op/s 2024-09-21T03:47:57.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:56 smithi016 ceph-mon[30375]: pgmap v467: 129 pgs: 129 active+clean; 2.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 197 MiB/s wr, 72 op/s 2024-09-21T03:47:59.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:47:58 smithi123 ceph-mon[34673]: pgmap v468: 129 pgs: 129 active+clean; 2.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 240 MiB/s wr, 82 op/s 2024-09-21T03:47:59.161 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:47:59.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:47:58 smithi016 ceph-mon[30375]: pgmap v468: 129 pgs: 129 active+clean; 2.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 240 MiB/s wr, 82 op/s 2024-09-21T03:47:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:47:58 smithi134 ceph-mon[35031]: pgmap v468: 129 pgs: 129 active+clean; 2.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 240 MiB/s wr, 82 op/s 2024-09-21T03:47:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:00.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.532+0000 7fd966e00640 1 -- 172.21.15.16:0/3605158765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd960075730 msgr2=0x7fd960075b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:00.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.532+0000 7fd966e00640 1 --2- 172.21.15.16:0/3605158765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd960075730 0x7fd960075b40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd950009900 tx=0x7fd95002a400 comp rx=0 tx=0).stop 2024-09-21T03:48:00.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.532+0000 7fd966e00640 1 -- 172.21.15.16:0/3605158765 shutdown_connections 2024-09-21T03:48:00.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.532+0000 7fd966e00640 1 --2- 172.21.15.16:0/3605158765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd960110160 0x7fd960112580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:00.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.532+0000 7fd966e00640 1 --2- 172.21.15.16:0/3605158765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd960076080 0x7fd96010fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:00.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.532+0000 7fd966e00640 1 --2- 172.21.15.16:0/3605158765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd960075730 0x7fd960075b40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:00.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.532+0000 7fd966e00640 1 -- 172.21.15.16:0/3605158765 >> 172.21.15.16:0/3605158765 conn(0x7fd9600ff590 msgr2=0x7fd9601019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:00.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.535+0000 7fd966e00640 1 -- 172.21.15.16:0/3605158765 shutdown_connections 2024-09-21T03:48:00.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.535+0000 7fd966e00640 1 -- 172.21.15.16:0/3605158765 wait complete. 2024-09-21T03:48:00.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.536+0000 7fd966e00640 1 Processor -- start 2024-09-21T03:48:00.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.537+0000 7fd966e00640 1 -- start start 2024-09-21T03:48:00.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.537+0000 7fd966e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd960075730 0x7fd960106c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:00.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.537+0000 7fd966e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd960076080 0x7fd9601071c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:00.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.537+0000 7fd966e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd960110160 0x7fd960107700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:00.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.537+0000 7fd966e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd960115cf0 con 0x7fd960075730 2024-09-21T03:48:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.537+0000 7fd966e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd960115b70 con 0x7fd960110160 2024-09-21T03:48:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.537+0000 7fd966e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd960115e70 con 0x7fd960076080 2024-09-21T03:48:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.538+0000 7fd964e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd960110160 0x7fd960107700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.538+0000 7fd964e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd960110160 0x7fd960107700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:52028/0 (socket says 172.21.15.16:52028) 2024-09-21T03:48:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.538+0000 7fd964e00640 1 -- 172.21.15.16:0/2511808099 learned_addr learned my addr 172.21.15.16:0/2511808099 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.538+0000 7fd964e00640 1 -- 172.21.15.16:0/2511808099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd960076080 msgr2=0x7fd9601071c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:48:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.538+0000 7fd964e00640 1 --2- 172.21.15.16:0/2511808099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd960076080 0x7fd9601071c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.539+0000 7fd964e00640 1 -- 172.21.15.16:0/2511808099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd960075730 msgr2=0x7fd960106c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.539+0000 7fd964e00640 1 --2- 172.21.15.16:0/2511808099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd960075730 0x7fd960106c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.539+0000 7fd964e00640 1 -- 172.21.15.16:0/2511808099 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9601a2400 con 0x7fd960110160 2024-09-21T03:48:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.539+0000 7fd964e00640 1 --2- 172.21.15.16:0/2511808099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd960110160 0x7fd960107700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd95800efd0 tx=0x7fd95800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.539+0000 7fd95cc00640 1 -- 172.21.15.16:0/2511808099 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd958003b70 con 0x7fd960110160 2024-09-21T03:48:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.539+0000 7fd95cc00640 1 -- 172.21.15.16:0/2511808099 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd958003d10 con 0x7fd960110160 2024-09-21T03:48:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.539+0000 7fd95cc00640 1 -- 172.21.15.16:0/2511808099 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd958010960 con 0x7fd960110160 2024-09-21T03:48:00.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.539+0000 7fd966e00640 1 -- 172.21.15.16:0/2511808099 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9601a26f0 con 0x7fd960110160 2024-09-21T03:48:00.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.540+0000 7fd966e00640 1 -- 172.21.15.16:0/2511808099 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9601a2be0 con 0x7fd960110160 2024-09-21T03:48:00.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.543+0000 7fd966e00640 1 -- 172.21.15.16:0/2511808099 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9400052c0 con 0x7fd960110160 2024-09-21T03:48:00.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.546+0000 7fd95cc00640 1 -- 172.21.15.16:0/2511808099 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd958010b00 con 0x7fd960110160 2024-09-21T03:48:00.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.549+0000 7fd95cc00640 1 --2- 172.21.15.16:0/2511808099 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd9440b66a0 0x7fd9440b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:00.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.550+0000 7fd95fe00640 1 --2- 172.21.15.16:0/2511808099 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd9440b66a0 0x7fd9440b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:00.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.550+0000 7fd95cc00640 1 -- 172.21.15.16:0/2511808099 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fd9580e78f0 con 0x7fd960110160 2024-09-21T03:48:00.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.551+0000 7fd95fe00640 1 --2- 172.21.15.16:0/2511808099 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd9440b66a0 0x7fd9440b8bb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd9500096c0 tx=0x7fd950031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:00.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:00.553+0000 7fd95cc00640 1 -- 172.21.15.16:0/2511808099 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd95801d060 con 0x7fd960110160 2024-09-21T03:48:00.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:00 smithi016 ceph-mon[30375]: pgmap v469: 129 pgs: 129 active+clean; 3.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 258 MiB/s wr, 88 op/s 2024-09-21T03:48:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:01.071+0000 7fd966e00640 1 -- 172.21.15.16:0/2511808099 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9400055b0 con 0x7fd960110160 2024-09-21T03:48:01.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:01.073+0000 7fd95cc00640 1 -- 172.21.15.16:0/2511808099 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7fd9580add70 con 0x7fd960110160 2024-09-21T03:48:01.080 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:48:01.081 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:48:01.081 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:48:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:01.081+0000 7fd966e00640 1 -- 172.21.15.16:0/2511808099 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd9440b66a0 msgr2=0x7fd9440b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:01.081+0000 7fd966e00640 1 --2- 172.21.15.16:0/2511808099 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd9440b66a0 0x7fd9440b8bb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd9500096c0 tx=0x7fd950031400 comp rx=0 tx=0).stop 2024-09-21T03:48:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:01.081+0000 7fd966e00640 1 -- 172.21.15.16:0/2511808099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd960110160 msgr2=0x7fd960107700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:01.081+0000 7fd966e00640 1 --2- 172.21.15.16:0/2511808099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd960110160 0x7fd960107700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd95800efd0 tx=0x7fd95800c6a0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:01.082+0000 7fd966e00640 1 -- 172.21.15.16:0/2511808099 shutdown_connections 2024-09-21T03:48:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:01.082+0000 7fd966e00640 1 --2- 172.21.15.16:0/2511808099 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fd9440b66a0 0x7fd9440b8bb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:01.082+0000 7fd966e00640 1 --2- 172.21.15.16:0/2511808099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd960110160 0x7fd960107700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:01.082+0000 7fd966e00640 1 --2- 172.21.15.16:0/2511808099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd960076080 0x7fd9601071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:01.082+0000 7fd966e00640 1 --2- 172.21.15.16:0/2511808099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd960075730 0x7fd960106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:01.082+0000 7fd966e00640 1 -- 172.21.15.16:0/2511808099 >> 172.21.15.16:0/2511808099 conn(0x7fd9600ff590 msgr2=0x7fd960109d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:01.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:01.083+0000 7fd966e00640 1 -- 172.21.15.16:0/2511808099 shutdown_connections 2024-09-21T03:48:01.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:01.083+0000 7fd966e00640 1 -- 172.21.15.16:0/2511808099 wait complete. 2024-09-21T03:48:01.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:00 smithi123 ceph-mon[34673]: pgmap v469: 129 pgs: 129 active+clean; 3.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 258 MiB/s wr, 88 op/s 2024-09-21T03:48:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:00 smithi134 ceph-mon[35031]: pgmap v469: 129 pgs: 129 active+clean; 3.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 258 MiB/s wr, 88 op/s 2024-09-21T03:48:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.494+0000 7f0072200640 1 -- 172.21.15.16:0/745468659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f006c1034e0 msgr2=0x7f006c103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.494+0000 7f0072200640 1 --2- 172.21.15.16:0/745468659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f006c1034e0 0x7f006c103950 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0060009900 tx=0x7f006002a410 comp rx=0 tx=0).stop 2024-09-21T03:48:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.494+0000 7f0072200640 1 -- 172.21.15.16:0/745468659 shutdown_connections 2024-09-21T03:48:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.494+0000 7f0072200640 1 --2- 172.21.15.16:0/745468659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f006c104f00 0x7f006c10b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.494+0000 7f0072200640 1 --2- 172.21.15.16:0/745468659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f006c104560 0x7f006c104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.494+0000 7f0072200640 1 --2- 172.21.15.16:0/745468659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f006c1034e0 0x7f006c103950 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.494+0000 7f0072200640 1 -- 172.21.15.16:0/745468659 >> 172.21.15.16:0/745468659 conn(0x7f006c0fee20 msgr2=0x7f006c1012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.494+0000 7f0072200640 1 -- 172.21.15.16:0/745468659 shutdown_connections 2024-09-21T03:48:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.494+0000 7f0072200640 1 -- 172.21.15.16:0/745468659 wait complete. 2024-09-21T03:48:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.495+0000 7f0072200640 1 Processor -- start 2024-09-21T03:48:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.495+0000 7f0072200640 1 -- start start 2024-09-21T03:48:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.495+0000 7f0072200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f006c1034e0 0x7f006c19bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.496+0000 7f0072200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f006c104560 0x7f006c19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.496+0000 7f0072200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f006c104f00 0x7f006c19d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.496+0000 7f0072200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f006c10f180 con 0x7f006c1034e0 2024-09-21T03:48:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.496+0000 7f0072200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f006c10f000 con 0x7f006c104f00 2024-09-21T03:48:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.496+0000 7f0072200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f006c10f300 con 0x7f006c104560 2024-09-21T03:48:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.496+0000 7f0070e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f006c1034e0 0x7f006c19bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.496+0000 7f0070e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f006c1034e0 0x7f006c19bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45148/0 (socket says 172.21.15.16:45148) 2024-09-21T03:48:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.496+0000 7f0070e00640 1 -- 172.21.15.16:0/3728809916 learned_addr learned my addr 172.21.15.16:0/3728809916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.496+0000 7f006be00640 1 --2- 172.21.15.16:0/3728809916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f006c104560 0x7f006c19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.496+0000 7f0071800640 1 --2- 172.21.15.16:0/3728809916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f006c104f00 0x7f006c19d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.496+0000 7f0070e00640 1 -- 172.21.15.16:0/3728809916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f006c104560 msgr2=0x7f006c19c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.496+0000 7f0070e00640 1 --2- 172.21.15.16:0/3728809916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f006c104560 0x7f006c19c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.496+0000 7f0070e00640 1 -- 172.21.15.16:0/3728809916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f006c104f00 msgr2=0x7f006c19d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.496+0000 7f0070e00640 1 --2- 172.21.15.16:0/3728809916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f006c104f00 0x7f006c19d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.496+0000 7f0070e00640 1 -- 172.21.15.16:0/3728809916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006c19d9e0 con 0x7f006c1034e0 2024-09-21T03:48:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.496+0000 7f006be00640 1 --2- 172.21.15.16:0/3728809916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f006c104560 0x7f006c19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:48:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.497+0000 7f0070e00640 1 --2- 172.21.15.16:0/3728809916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f006c1034e0 0x7f006c19bfa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0060031440 tx=0x7f0060003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.497+0000 7f0069600640 1 -- 172.21.15.16:0/3728809916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006003f070 con 0x7f006c1034e0 2024-09-21T03:48:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.497+0000 7f0069600640 1 -- 172.21.15.16:0/3728809916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0060031c50 con 0x7f006c1034e0 2024-09-21T03:48:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.497+0000 7f0072200640 1 -- 172.21.15.16:0/3728809916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006c19dc70 con 0x7f006c1034e0 2024-09-21T03:48:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.497+0000 7f0069600640 1 -- 172.21.15.16:0/3728809916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060035030 con 0x7f006c1034e0 2024-09-21T03:48:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.497+0000 7f0072200640 1 -- 172.21.15.16:0/3728809916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006c19df80 con 0x7f006c1034e0 2024-09-21T03:48:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.500+0000 7f007aa2e740 1 Processor -- start 2024-09-21T03:48:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.500+0000 7f007aa2e740 1 -- start start 2024-09-21T03:48:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.500+0000 7f007aa2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a046a1b0 0x5620a08112b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.501+0000 7f007aa2e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04412f0 0x5620a0811920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.501+0000 7f007aa2e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a04494d0 0x5620a0812470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.501+0000 7f0069600640 1 -- 172.21.15.16:0/3728809916 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0060042070 con 0x7f006c1034e0 2024-09-21T03:48:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.501+0000 7f007aa2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5620a07846b0 con 0x5620a046a1b0 2024-09-21T03:48:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.501+0000 7f007aa2e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5620a0773ae0 con 0x5620a04412f0 2024-09-21T03:48:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.501+0000 7f007aa2e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5620a07778f0 con 0x5620a04494d0 2024-09-21T03:48:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.501+0000 7f0071800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a046a1b0 0x5620a08112b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.501+0000 7f0070e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04412f0 0x5620a0811920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.501+0000 7f0071800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a046a1b0 0x5620a08112b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45162/0 (socket says 172.21.15.16:45162) 2024-09-21T03:48:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.501+0000 7f0071800640 1 -- 172.21.15.16:0/1737502677 learned_addr learned my addr 172.21.15.16:0/1737502677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.501+0000 7f006be00640 1 --2- 172.21.15.16:0/1737502677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a04494d0 0x5620a0812470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.501+0000 7f0070e00640 1 -- 172.21.15.16:0/1737502677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a04494d0 msgr2=0x5620a0812470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.501+0000 7f0070e00640 1 --2- 172.21.15.16:0/1737502677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a04494d0 0x5620a0812470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.501+0000 7f0070e00640 1 -- 172.21.15.16:0/1737502677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a046a1b0 msgr2=0x5620a08112b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.501+0000 7f0070e00640 1 --2- 172.21.15.16:0/1737502677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a046a1b0 0x5620a08112b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.501+0000 7f0070e00640 1 -- 172.21.15.16:0/1737502677 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620a0740780 con 0x5620a04412f0 2024-09-21T03:48:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.502+0000 7f0070e00640 1 --2- 172.21.15.16:0/1737502677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04412f0 0x5620a0811920 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0060105580 tx=0x7f00601101a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99cfeec9dd4ecdb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.502+0000 7f005e000640 1 -- 172.21.15.16:0/1737502677 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060110980 con 0x5620a04412f0 2024-09-21T03:48:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.502+0000 7f005e000640 1 -- 172.21.15.16:0/1737502677 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0060115070 con 0x5620a04412f0 2024-09-21T03:48:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.502+0000 7f005e000640 1 -- 172.21.15.16:0/1737502677 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060118370 con 0x5620a04412f0 2024-09-21T03:48:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.502+0000 7f0069600640 1 --2- 172.21.15.16:0/3728809916 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f00540b6510 0x7f00540b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.503+0000 7f0069600640 1 -- 172.21.15.16:0/3728809916 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0060107ef0 con 0x7f006c1034e0 2024-09-21T03:48:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.503+0000 7f0071800640 1 --2- 172.21.15.16:0/3728809916 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f00540b6510 0x7f00540b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.503+0000 7f007aa2e740 1 -- 172.21.15.16:0/1737502677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04412f0 msgr2=0x5620a0811920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.503+0000 7f007aa2e740 1 --2- 172.21.15.16:0/1737502677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04412f0 0x5620a0811920 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0060105580 tx=0x7f00601101a0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.503+0000 7f007aa2e740 1 -- 172.21.15.16:0/1737502677 shutdown_connections 2024-09-21T03:48:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.503+0000 7f007aa2e740 1 --2- 172.21.15.16:0/1737502677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a04494d0 0x5620a0812470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.503+0000 7f007aa2e740 1 --2- 172.21.15.16:0/1737502677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04412f0 0x5620a0811920 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.503+0000 7f007aa2e740 1 --2- 172.21.15.16:0/1737502677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a046a1b0 0x5620a08112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.503+0000 7f007aa2e740 1 -- 172.21.15.16:0/1737502677 >> 172.21.15.16:0/1737502677 conn(0x5620a045e300 msgr2=0x5620a080cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.503+0000 7f007aa2e740 1 -- 172.21.15.16:0/1737502677 shutdown_connections 2024-09-21T03:48:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.503+0000 7f007aa2e740 1 -- 172.21.15.16:0/1737502677 wait complete. 2024-09-21T03:48:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.503+0000 7f0071800640 1 --2- 172.21.15.16:0/3728809916 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f00540b6510 0x7f00540b8a20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0064009fd0 tx=0x7f00640096a0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.504+0000 7f007aa2e740 1 -- start start 2024-09-21T03:48:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.504+0000 7f007aa2e740 1 Processor -- start 2024-09-21T03:48:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.505+0000 7f007aa2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a04412f0 0x5620a081a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.505+0000 7f007aa2e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 0x5620a081ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.505+0000 7f007aa2e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a046a1b0 0x5620a081b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.505+0000 7f007aa2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5620a06abab0 con 0x5620a04412f0 2024-09-21T03:48:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.505+0000 7f007aa2e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5620a071a570 con 0x5620a04494d0 2024-09-21T03:48:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.505+0000 7f007aa2e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5620a0812d50 con 0x5620a046a1b0 2024-09-21T03:48:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.505+0000 7f007aa2e740 10 client.0 _ll_register_callbacks cb 0x5620a0729100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.505+0000 7f0070e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a04412f0 0x5620a081a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.505+0000 7f006be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 0x5620a081ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.505+0000 7f0070e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a04412f0 0x5620a081a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45164/0 (socket says 172.21.15.16:45164) 2024-09-21T03:48:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.505+0000 7f0070e00640 1 -- 172.21.15.16:0/1723359467 learned_addr learned my addr 172.21.15.16:0/1723359467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.505+0000 7f0071800640 1 --2- 172.21.15.16:0/1723359467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a046a1b0 0x5620a081b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.506+0000 7f006be00640 1 -- 172.21.15.16:0/1723359467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a046a1b0 msgr2=0x5620a081b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.506+0000 7f006be00640 1 --2- 172.21.15.16:0/1723359467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a046a1b0 0x5620a081b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.506+0000 7f006be00640 1 -- 172.21.15.16:0/1723359467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a04412f0 msgr2=0x5620a081a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.506+0000 7f006be00640 1 --2- 172.21.15.16:0/1723359467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a04412f0 0x5620a081a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.506+0000 7f006be00640 1 -- 172.21.15.16:0/1723359467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620a04d5790 con 0x5620a04494d0 2024-09-21T03:48:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.506+0000 7f006be00640 1 --2- 172.21.15.16:0/1723359467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 0x5620a081ad40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0058012d20 tx=0x7f005800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ac1457dc26f0486 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.506+0000 7f0052a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:48:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.506+0000 7f0052a00640 1 -- 172.21.15.16:0/1723359467 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058012110 con 0x5620a04494d0 2024-09-21T03:48:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.506+0000 7f0052a00640 1 -- 172.21.15.16:0/1723359467 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0058004050 con 0x5620a04494d0 2024-09-21T03:48:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.506+0000 7f007aa2e740 1 -- 172.21.15.16:0/1723359467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0060115070 con 0x5620a04494d0 2024-09-21T03:48:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.506+0000 7f0052a00640 1 -- 172.21.15.16:0/1723359467 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f00580122b0 con 0x5620a04494d0 2024-09-21T03:48:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.506+0000 7f007aa2e740 10 client.25754 fetch_fsmap learned FSMap version 30 2024-09-21T03:48:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.506+0000 7f007aa2e740 1 -- 172.21.15.16:0/1723359467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5620a054e620 con 0x5620a04494d0 2024-09-21T03:48:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.507+0000 7f0052a00640 1 -- 172.21.15.16:0/1723359467 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0060115070 con 0x5620a04494d0 2024-09-21T03:48:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.507+0000 7f0052a00640 1 -- 172.21.15.16:0/1723359467 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0058010430 con 0x5620a04494d0 2024-09-21T03:48:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.507+0000 7f007aa2e740 10 client.25754 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:48:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.507+0000 7f007aa2e740 10 client.25754 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.507+0000 7f007aa2e740 20 client.25754 populate_metadata read hostname 'smithi016' 2024-09-21T03:48:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.507+0000 7f007aa2e740 1 --2- 172.21.15.16:0/1723359467 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5620a0428d80 0x5620a0827e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.507+0000 7f007aa2e740 4 client.25754 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.507+0000 7f007aa2e740 1 -- 172.21.15.16:0/1723359467 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620a0786600 con 0x5620a0428d80 2024-09-21T03:48:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.507+0000 7f0070e00640 1 --2- 172.21.15.16:0/1723359467 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5620a0428d80 0x5620a0827e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.508+0000 7f0052a00640 10 client.25754 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.508+0000 7f0070e00640 1 --2- 172.21.15.16:0/1723359467 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5620a0428d80 0x5620a0827e20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baf297d52dd4f12b server_cookie=9171793a1bd8197f in_seq=0 out_seq=0 2024-09-21T03:48:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.508+0000 7f0052a00640 1 -- 172.21.15.16:0/1723359467 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5620a0786600 con 0x5620a0428d80 2024-09-21T03:48:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.508+0000 7f0052a00640 10 client.25754 handle_command_reply: tid=0 2024-09-21T03:48:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.508+0000 7f007aa2e740 1 client.25754 shutdown 2024-09-21T03:48:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.509+0000 7f007aa2e740 1 -- 172.21.15.16:0/1723359467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 msgr2=0x5620a081ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.509+0000 7f007aa2e740 1 --2- 172.21.15.16:0/1723359467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 0x5620a081ad40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0058012d20 tx=0x7f005800f790 comp rx=0 tx=0).stop 2024-09-21T03:48:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.509+0000 7f007aa2e740 1 -- 172.21.15.16:0/1723359467 shutdown_connections 2024-09-21T03:48:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.509+0000 7f007aa2e740 1 --2- 172.21.15.16:0/1723359467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a046a1b0 0x5620a081b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.509+0000 7f007aa2e740 1 --2- 172.21.15.16:0/1723359467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 0x5620a081ad40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.509+0000 7f007aa2e740 1 --2- 172.21.15.16:0/1723359467 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5620a0428d80 0x5620a0827e20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.509+0000 7f007aa2e740 1 --2- 172.21.15.16:0/1723359467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a04412f0 0x5620a081a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.509+0000 7f007aa2e740 1 -- 172.21.15.16:0/1723359467 >> 172.21.15.16:0/1723359467 conn(0x5620a045e300 msgr2=0x5620a0812320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.509+0000 7f0052a00640 0 client.25754 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.509+0000 7f007aa2e740 1 -- 172.21.15.16:0/1723359467 shutdown_connections 2024-09-21T03:48:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.509+0000 7f007aa2e740 1 -- 172.21.15.16:0/1723359467 wait complete. 2024-09-21T03:48:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.509+0000 7f007aa2e740 20 client.25754 trim_cache size 0 max 16384 2024-09-21T03:48:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.525+0000 7f007aa2e740 1 Processor -- start 2024-09-21T03:48:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.525+0000 7f007aa2e740 1 -- start start 2024-09-21T03:48:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.525+0000 7f007aa2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a04412f0 0x5620a0810ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.525+0000 7f007aa2e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 0x5620a080c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.525+0000 7f007aa2e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a046a1b0 0x5620a080cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.525+0000 7f007aa2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1ae760 con 0x5620a04412f0 2024-09-21T03:48:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.525+0000 7f007aa2e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1b02a0 con 0x5620a04494d0 2024-09-21T03:48:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.525+0000 7f007aa2e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1aee30 con 0x5620a046a1b0 2024-09-21T03:48:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.525+0000 7f0070e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a04412f0 0x5620a0810ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.525+0000 7f006be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 0x5620a080c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.525+0000 7f006be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 0x5620a080c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:52080/0 (socket says 172.21.15.16:52080) 2024-09-21T03:48:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.525+0000 7f0071800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a046a1b0 0x5620a080cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.525+0000 7f006be00640 1 -- 172.21.15.16:0/1810236557 learned_addr learned my addr 172.21.15.16:0/1810236557 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.526+0000 7f006be00640 1 -- 172.21.15.16:0/1810236557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a046a1b0 msgr2=0x5620a080cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.526+0000 7f006be00640 1 --2- 172.21.15.16:0/1810236557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a046a1b0 0x5620a080cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.526+0000 7f006be00640 1 -- 172.21.15.16:0/1810236557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a04412f0 msgr2=0x5620a0810ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.526+0000 7f006be00640 1 --2- 172.21.15.16:0/1810236557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a04412f0 0x5620a0810ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.526+0000 7f006be00640 1 -- 172.21.15.16:0/1810236557 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620a0811d20 con 0x5620a04494d0 2024-09-21T03:48:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.526+0000 7f0071800640 1 --2- 172.21.15.16:0/1810236557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a046a1b0 0x5620a080cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:48:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.526+0000 7f006be00640 1 --2- 172.21.15.16:0/1810236557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 0x5620a080c680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0058000c00 tx=0x7f0058012990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b48d4ab5fe158100 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.526+0000 7f005d600640 1 -- 172.21.15.16:0/1810236557 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058000e60 con 0x5620a04494d0 2024-09-21T03:48:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.526+0000 7f005d600640 1 -- 172.21.15.16:0/1810236557 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0058005590 con 0x5620a04494d0 2024-09-21T03:48:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.526+0000 7f005d600640 1 -- 172.21.15.16:0/1810236557 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058003830 con 0x5620a04494d0 2024-09-21T03:48:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.527+0000 7f007aa2e740 1 -- 172.21.15.16:0/1810236557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 msgr2=0x5620a080c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.527+0000 7f007aa2e740 1 --2- 172.21.15.16:0/1810236557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 0x5620a080c680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0058000c00 tx=0x7f0058012990 comp rx=0 tx=0).stop 2024-09-21T03:48:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.527+0000 7f007aa2e740 1 -- 172.21.15.16:0/1810236557 shutdown_connections 2024-09-21T03:48:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.527+0000 7f007aa2e740 1 --2- 172.21.15.16:0/1810236557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a046a1b0 0x5620a080cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.527+0000 7f007aa2e740 1 --2- 172.21.15.16:0/1810236557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 0x5620a080c680 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.527+0000 7f007aa2e740 1 --2- 172.21.15.16:0/1810236557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a04412f0 0x5620a0810ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.527+0000 7f007aa2e740 1 -- 172.21.15.16:0/1810236557 >> 172.21.15.16:0/1810236557 conn(0x5620a045e300 msgr2=0x5620a0818c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.527+0000 7f007aa2e740 1 -- 172.21.15.16:0/1810236557 shutdown_connections 2024-09-21T03:48:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.527+0000 7f007aa2e740 1 -- 172.21.15.16:0/1810236557 wait complete. 2024-09-21T03:48:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.528+0000 7f007aa2e740 1 -- start start 2024-09-21T03:48:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.528+0000 7f007aa2e740 1 Processor -- start 2024-09-21T03:48:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.528+0000 7f007aa2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a04412f0 0x5620a08220a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.528+0000 7f007aa2e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 0x5620a0822760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.528+0000 7f007aa2e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a046a1b0 0x5620a081e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.528+0000 7f007aa2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1afbd0 con 0x5620a04412f0 2024-09-21T03:48:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.528+0000 7f007aa2e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1af500 con 0x5620a04494d0 2024-09-21T03:48:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.528+0000 7f007aa2e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1ac740 con 0x5620a046a1b0 2024-09-21T03:48:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.528+0000 7f007aa2e740 10 client.0 _ll_register_callbacks cb 0x7f0058009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.528+0000 7f0070e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a04412f0 0x5620a08220a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.528+0000 7f0071800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a046a1b0 0x5620a081e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.528+0000 7f006be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 0x5620a0822760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.528+0000 7f0071800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a046a1b0 0x5620a081e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:55642/0 (socket says 172.21.15.16:55642) 2024-09-21T03:48:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.528+0000 7f0071800640 1 -- 172.21.15.16:0/264539971 learned_addr learned my addr 172.21.15.16:0/264539971 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.529+0000 7f006be00640 1 -- 172.21.15.16:0/264539971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a046a1b0 msgr2=0x5620a081e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.529+0000 7f006be00640 1 --2- 172.21.15.16:0/264539971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a046a1b0 0x5620a081e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.529+0000 7f006be00640 1 -- 172.21.15.16:0/264539971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a04412f0 msgr2=0x5620a08220a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.529+0000 7f006be00640 1 --2- 172.21.15.16:0/264539971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a04412f0 0x5620a08220a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.529+0000 7f006be00640 1 -- 172.21.15.16:0/264539971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620a081e7a0 con 0x5620a04494d0 2024-09-21T03:48:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.529+0000 7f006be00640 1 --2- 172.21.15.16:0/264539971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 0x5620a0822760 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0058009710 tx=0x7f00580128a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fe958e221effb5c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.529+0000 7f0052a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:48:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.529+0000 7f0052a00640 1 -- 172.21.15.16:0/264539971 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00580053e0 con 0x5620a04494d0 2024-09-21T03:48:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.530+0000 7f007aa2e740 1 -- 172.21.15.16:0/264539971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0058005590 con 0x5620a04494d0 2024-09-21T03:48:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.530+0000 7f0052a00640 1 -- 172.21.15.16:0/264539971 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0058003800 con 0x5620a04494d0 2024-09-21T03:48:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.530+0000 7f0052a00640 1 -- 172.21.15.16:0/264539971 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0058000e40 con 0x5620a04494d0 2024-09-21T03:48:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.530+0000 7f007aa2e740 10 client.25760 fetch_fsmap learned FSMap version 30 2024-09-21T03:48:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.530+0000 7f007aa2e740 1 -- 172.21.15.16:0/264539971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5620a081fe60 con 0x5620a04494d0 2024-09-21T03:48:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.530+0000 7f0052a00640 1 -- 172.21.15.16:0/264539971 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0058005590 con 0x5620a04494d0 2024-09-21T03:48:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.530+0000 7f0052a00640 1 -- 172.21.15.16:0/264539971 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f005803a030 con 0x5620a04494d0 2024-09-21T03:48:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.530+0000 7f007aa2e740 10 client.25760 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:48:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.530+0000 7f007aa2e740 10 client.25760 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.530+0000 7f007aa2e740 20 client.25760 populate_metadata read hostname 'smithi016' 2024-09-21T03:48:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.530+0000 7f007aa2e740 1 --2- 172.21.15.16:0/264539971 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5620a03fe1c0 0x5620a0895030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.530+0000 7f007aa2e740 4 client.25760 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:48:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.530+0000 7f007aa2e740 1 -- 172.21.15.16:0/264539971 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620a07820b0 con 0x5620a03fe1c0 2024-09-21T03:48:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.531+0000 7f0070e00640 1 --2- 172.21.15.16:0/264539971 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5620a03fe1c0 0x5620a0895030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.531+0000 7f0070e00640 1 --2- 172.21.15.16:0/264539971 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5620a03fe1c0 0x5620a0895030 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ead157ff808ef15b server_cookie=af8166ad72f537de in_seq=0 out_seq=0 2024-09-21T03:48:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.531+0000 7f0052a00640 10 client.25760 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.541+0000 7f0052a00640 1 -- 172.21.15.16:0/264539971 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5620a07820b0 con 0x5620a03fe1c0 2024-09-21T03:48:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.541+0000 7f0052a00640 10 client.25760 handle_command_reply: tid=0 2024-09-21T03:48:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.541+0000 7f007aa2e740 1 client.25760 shutdown 2024-09-21T03:48:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.542+0000 7f007aa2e740 1 -- 172.21.15.16:0/264539971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 msgr2=0x5620a0822760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.542+0000 7f007aa2e740 1 --2- 172.21.15.16:0/264539971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 0x5620a0822760 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0058009710 tx=0x7f00580128a0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.542+0000 7f007aa2e740 1 -- 172.21.15.16:0/264539971 shutdown_connections 2024-09-21T03:48:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.542+0000 7f007aa2e740 1 --2- 172.21.15.16:0/264539971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a046a1b0 0x5620a081e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.542+0000 7f007aa2e740 1 --2- 172.21.15.16:0/264539971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620a04494d0 0x5620a0822760 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.542+0000 7f007aa2e740 1 --2- 172.21.15.16:0/264539971 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5620a03fe1c0 0x5620a0895030 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.542+0000 7f007aa2e740 1 --2- 172.21.15.16:0/264539971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620a04412f0 0x5620a08220a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.542+0000 7f0052a00640 0 client.25760 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.542+0000 7f007aa2e740 1 -- 172.21.15.16:0/264539971 >> 172.21.15.16:0/264539971 conn(0x5620a045e300 msgr2=0x5620a081a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.542+0000 7f007aa2e740 1 -- 172.21.15.16:0/264539971 shutdown_connections 2024-09-21T03:48:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.542+0000 7f007aa2e740 1 -- 172.21.15.16:0/264539971 wait complete. 2024-09-21T03:48:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.542+0000 7f007aa2e740 20 client.25760 trim_cache size 0 max 16384 2024-09-21T03:48:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:48:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:48:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:48:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:48:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.543+0000 7f0072200640 1 -- 172.21.15.16:0/3728809916 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f00540b6510 msgr2=0x7f00540b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.543+0000 7f0072200640 1 --2- 172.21.15.16:0/3728809916 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f00540b6510 0x7f00540b8a20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0064009fd0 tx=0x7f00640096a0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.543+0000 7f0072200640 1 -- 172.21.15.16:0/3728809916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f006c1034e0 msgr2=0x7f006c19bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.543+0000 7f0072200640 1 --2- 172.21.15.16:0/3728809916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f006c1034e0 0x7f006c19bfa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0060031440 tx=0x7f0060003c10 comp rx=0 tx=0).stop 2024-09-21T03:48:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.543+0000 7f0072200640 1 -- 172.21.15.16:0/3728809916 shutdown_connections 2024-09-21T03:48:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.543+0000 7f0072200640 1 --2- 172.21.15.16:0/3728809916 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f00540b6510 0x7f00540b8a20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.543+0000 7f0072200640 1 --2- 172.21.15.16:0/3728809916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f006c104f00 0x7f006c19d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.543+0000 7f0072200640 1 --2- 172.21.15.16:0/3728809916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f006c104560 0x7f006c19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.543+0000 7f0072200640 1 --2- 172.21.15.16:0/3728809916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f006c1034e0 0x7f006c19bfa0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.543+0000 7f0072200640 1 -- 172.21.15.16:0/3728809916 >> 172.21.15.16:0/3728809916 conn(0x7f006c0fee20 msgr2=0x7f006c0755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.543+0000 7f0072200640 1 -- 172.21.15.16:0/3728809916 shutdown_connections 2024-09-21T03:48:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:01.543+0000 7f0072200640 1 -- 172.21.15.16:0/3728809916 wait complete. 2024-09-21T03:48:02.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:01 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2511808099' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:02.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:01 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2511808099' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:02.331 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db --set-id 39a29551 --release --await' 2024-09-21T03:48:02.331 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db --set-id 39a29551 --release --await 2024-09-21T03:48:02.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:01 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2511808099' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:03.136 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:02 smithi016 ceph-mon[30375]: pgmap v470: 129 pgs: 129 active+clean; 3.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 280 MiB/s wr, 89 op/s 2024-09-21T03:48:03.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:02 smithi123 ceph-mon[34673]: pgmap v470: 129 pgs: 129 active+clean; 3.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 280 MiB/s wr, 89 op/s 2024-09-21T03:48:03.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:02 smithi134 ceph-mon[35031]: pgmap v470: 129 pgs: 129 active+clean; 3.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 280 MiB/s wr, 89 op/s 2024-09-21T03:48:03.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.894+0000 7f9fd3400640 1 -- 172.21.15.16:0/2381643326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc105d70 msgr2=0x7f9fcc106180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:03.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.894+0000 7f9fd3400640 1 --2- 172.21.15.16:0/2381643326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc105d70 0x7f9fcc106180 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4009ca0 tx=0x7f9fc402a8e0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.894+0000 7f9fd3400640 1 -- 172.21.15.16:0/2381643326 shutdown_connections 2024-09-21T03:48:03.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.894+0000 7f9fd3400640 1 --2- 172.21.15.16:0/2381643326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc10c000 0x7f9fcc112830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.895+0000 7f9fd3400640 1 --2- 172.21.15.16:0/2381643326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fcc10b6b0 0x7f9fcc10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.895+0000 7f9fd3400640 1 --2- 172.21.15.16:0/2381643326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc105d70 0x7f9fcc106180 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.895+0000 7f9fd3400640 1 -- 172.21.15.16:0/2381643326 >> 172.21.15.16:0/2381643326 conn(0x7f9fcc101780 msgr2=0x7f9fcc103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:03.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.895+0000 7f9fd3400640 1 -- 172.21.15.16:0/2381643326 shutdown_connections 2024-09-21T03:48:03.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.895+0000 7f9fd3400640 1 -- 172.21.15.16:0/2381643326 wait complete. 2024-09-21T03:48:03.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.897+0000 7f9fd3400640 1 Processor -- start 2024-09-21T03:48:03.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.897+0000 7f9fd3400640 1 -- start start 2024-09-21T03:48:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.898+0000 7f9fd3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc105d70 0x7f9fcc07b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.898+0000 7f9fd3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc10b6b0 0x7f9fcc077920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.898+0000 7f9fd3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fcc10c000 0x7f9fcc078040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.898+0000 7f9fd3400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc115d10 con 0x7f9fcc10b6b0 2024-09-21T03:48:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.898+0000 7f9fd3400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc115b90 con 0x7f9fcc105d70 2024-09-21T03:48:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.898+0000 7f9fd3400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc115e90 con 0x7f9fcc10c000 2024-09-21T03:48:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.899+0000 7f9fd0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc105d70 0x7f9fcc07b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.899+0000 7f9fcbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc10b6b0 0x7f9fcc077920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.899+0000 7f9fcbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc10b6b0 0x7f9fcc077920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52044/0 (socket says 172.21.15.16:52044) 2024-09-21T03:48:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.899+0000 7f9fcbe00640 1 -- 172.21.15.16:0/2972594148 learned_addr learned my addr 172.21.15.16:0/2972594148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.899+0000 7f9fd0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc105d70 0x7f9fcc07b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:47664/0 (socket says 172.21.15.16:47664) 2024-09-21T03:48:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.900+0000 7f9fd0a00640 1 -- 172.21.15.16:0/2972594148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fcc10c000 msgr2=0x7f9fcc078040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.900+0000 7f9fd0a00640 1 --2- 172.21.15.16:0/2972594148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fcc10c000 0x7f9fcc078040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.900+0000 7f9fd0a00640 1 -- 172.21.15.16:0/2972594148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc10b6b0 msgr2=0x7f9fcc077920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.900+0000 7f9fd0a00640 1 --2- 172.21.15.16:0/2972594148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc10b6b0 0x7f9fcc077920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.900+0000 7f9fd0a00640 1 -- 172.21.15.16:0/2972594148 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc19c410 con 0x7f9fcc105d70 2024-09-21T03:48:03.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.900+0000 7f9fcbe00640 1 --2- 172.21.15.16:0/2972594148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc10b6b0 0x7f9fcc077920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:48:03.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.900+0000 7f9fd0a00640 1 --2- 172.21.15.16:0/2972594148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc105d70 0x7f9fcc07b640 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9fc402aed0 tx=0x7f9fc4003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:03.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.901+0000 7f9fc9600640 1 -- 172.21.15.16:0/2972594148 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4004040 con 0x7f9fcc105d70 2024-09-21T03:48:03.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.901+0000 7f9fc9600640 1 -- 172.21.15.16:0/2972594148 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc4004690 con 0x7f9fcc105d70 2024-09-21T03:48:03.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.901+0000 7f9fc9600640 1 -- 172.21.15.16:0/2972594148 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4030d70 con 0x7f9fcc105d70 2024-09-21T03:48:03.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.901+0000 7f9fd3400640 1 -- 172.21.15.16:0/2972594148 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fcc19c6a0 con 0x7f9fcc105d70 2024-09-21T03:48:03.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.902+0000 7f9fd3400640 1 -- 172.21.15.16:0/2972594148 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fcc19cb90 con 0x7f9fcc105d70 2024-09-21T03:48:03.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.907+0000 7f9fc9600640 1 -- 172.21.15.16:0/2972594148 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9fc40041e0 con 0x7f9fcc105d70 2024-09-21T03:48:03.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.910+0000 7f9fc9600640 1 --2- 172.21.15.16:0/2972594148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9fb00b65d0 0x7f9fb00b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:03.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.910+0000 7f9fc9600640 1 -- 172.21.15.16:0/2972594148 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f9fc41088e0 con 0x7f9fcc105d70 2024-09-21T03:48:03.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.911+0000 7f9fda1f6740 1 Processor -- start 2024-09-21T03:48:03.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.911+0000 7f9fcbe00640 1 --2- 172.21.15.16:0/2972594148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9fb00b65d0 0x7f9fb00b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:03.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.911+0000 7f9fda1f6740 1 -- start start 2024-09-21T03:48:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.912+0000 7f9fcbe00640 1 --2- 172.21.15.16:0/2972594148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9fb00b65d0 0x7f9fb00b8ae0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc103f80 tx=0x7f9fb800b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.913+0000 7f9fda1f6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c74b520 0x55cb0caf21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.913+0000 7f9fda1f6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c722660 0x55cb0caf2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.913+0000 7f9fda1f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c72a840 0x55cb0caf2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.913+0000 7f9fda1f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cb0ca69290 con 0x55cb0c72a840 2024-09-21T03:48:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.913+0000 7f9fda1f6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cb0ca586c0 con 0x55cb0c722660 2024-09-21T03:48:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.913+0000 7f9fda1f6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cb0ca5c4d0 con 0x55cb0c74b520 2024-09-21T03:48:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.914+0000 7f9fd0a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c74b520 0x55cb0caf21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.914+0000 7f9fd1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c72a840 0x55cb0caf2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:03.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.914+0000 7f9fcbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c722660 0x55cb0caf2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:03.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.914+0000 7f9fd1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c72a840 0x55cb0caf2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52046/0 (socket says 172.21.15.16:52046) 2024-09-21T03:48:03.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.914+0000 7f9fcbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c722660 0x55cb0caf2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:47678/0 (socket says 172.21.15.16:47678) 2024-09-21T03:48:03.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.914+0000 7f9fd1400640 1 -- 172.21.15.16:0/474497806 learned_addr learned my addr 172.21.15.16:0/474497806 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:03.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.915+0000 7f9fd1400640 1 -- 172.21.15.16:0/474497806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c74b520 msgr2=0x55cb0caf21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:03.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.915+0000 7f9fd1400640 1 --2- 172.21.15.16:0/474497806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c74b520 0x55cb0caf21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.916+0000 7f9fd1400640 1 -- 172.21.15.16:0/474497806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c722660 msgr2=0x55cb0caf2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:03.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.916+0000 7f9fd1400640 1 --2- 172.21.15.16:0/474497806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c722660 0x55cb0caf2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.916+0000 7f9fd1400640 1 -- 172.21.15.16:0/474497806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb0c9e5460 con 0x55cb0c72a840 2024-09-21T03:48:03.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.916+0000 7f9fcbe00640 1 --2- 172.21.15.16:0/474497806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c722660 0x55cb0caf2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:48:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.919+0000 7f9fd1400640 1 --2- 172.21.15.16:0/474497806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c72a840 0x55cb0caf2c40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00a9f0 tx=0x7f9fbc00aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa1c7f20e15561c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.920+0000 7f9fc1c00640 1 -- 172.21.15.16:0/474497806 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc00cde0 con 0x55cb0c72a840 2024-09-21T03:48:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.921+0000 7f9fc1c00640 1 -- 172.21.15.16:0/474497806 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fbc00f900 con 0x55cb0c72a840 2024-09-21T03:48:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.922+0000 7f9fc1c00640 1 -- 172.21.15.16:0/474497806 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc00cde0 con 0x55cb0c72a840 2024-09-21T03:48:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.923+0000 7f9fda1f6740 1 -- 172.21.15.16:0/474497806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c72a840 msgr2=0x55cb0caf2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.923+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/474497806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c72a840 0x55cb0caf2c40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00a9f0 tx=0x7f9fbc00aeb0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.927+0000 7f9fda1f6740 1 -- 172.21.15.16:0/474497806 shutdown_connections 2024-09-21T03:48:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.927+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/474497806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c72a840 0x55cb0caf2c40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.927+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/474497806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c722660 0x55cb0caf2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.927+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/474497806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c74b520 0x55cb0caf21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.927+0000 7f9fda1f6740 1 -- 172.21.15.16:0/474497806 >> 172.21.15.16:0/474497806 conn(0x55cb0c73f910 msgr2=0x55cb0caf00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.927+0000 7f9fda1f6740 1 -- 172.21.15.16:0/474497806 shutdown_connections 2024-09-21T03:48:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.927+0000 7f9fda1f6740 1 -- 172.21.15.16:0/474497806 wait complete. 2024-09-21T03:48:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.928+0000 7f9fda1f6740 1 -- start start 2024-09-21T03:48:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.928+0000 7f9fda1f6740 1 Processor -- start 2024-09-21T03:48:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.929+0000 7f9fda1f6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c722660 0x55cb0cafdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.929+0000 7f9fda1f6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c72a840 0x55cb0cafe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.929+0000 7f9fda1f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c74b520 0x55cb0cb09cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.929+0000 7f9fda1f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cb0c8b9bb0 con 0x55cb0c74b520 2024-09-21T03:48:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.929+0000 7f9fda1f6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cb0c9f36c0 con 0x55cb0c72a840 2024-09-21T03:48:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.929+0000 7f9fda1f6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cb0cb009a0 con 0x55cb0c722660 2024-09-21T03:48:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.929+0000 7f9fda1f6740 10 client.0 _ll_register_callbacks cb 0x55cb0ca0e490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.930+0000 7f9fd1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c74b520 0x55cb0cb09cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.930+0000 7f9fd1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c74b520 0x55cb0cb09cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52056/0 (socket says 172.21.15.16:52056) 2024-09-21T03:48:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.930+0000 7f9fd1400640 1 -- 172.21.15.16:0/10719943 learned_addr learned my addr 172.21.15.16:0/10719943 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.930+0000 7f9fcbe00640 1 --2- 172.21.15.16:0/10719943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c72a840 0x55cb0cafe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.931+0000 7f9fd1400640 1 -- 172.21.15.16:0/10719943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c722660 msgr2=0x55cb0cafdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.931+0000 7f9fd1400640 1 --2- 172.21.15.16:0/10719943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c722660 0x55cb0cafdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.931+0000 7f9fd1400640 1 -- 172.21.15.16:0/10719943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c72a840 msgr2=0x55cb0cafe260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.931+0000 7f9fd1400640 1 --2- 172.21.15.16:0/10719943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c72a840 0x55cb0cafe260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.931+0000 7f9fd1400640 1 -- 172.21.15.16:0/10719943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb0c7b6b00 con 0x55cb0c74b520 2024-09-21T03:48:03.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.931+0000 7f9fcbe00640 1 --2- 172.21.15.16:0/10719943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c72a840 0x55cb0cafe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:48:03.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.932+0000 7f9fd1400640 1 --2- 172.21.15.16:0/10719943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c74b520 0x55cb0cb09cc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00a9c0 tx=0x7f9fbc00f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c91493efa7f972 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:03.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.932+0000 7f9fb6000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:48:03.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.933+0000 7f9fb6000640 1 -- 172.21.15.16:0/10719943 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc028060 con 0x55cb0c74b520 2024-09-21T03:48:03.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.933+0000 7f9fda1f6740 1 -- 172.21.15.16:0/10719943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fbc00f900 con 0x55cb0c74b520 2024-09-21T03:48:03.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.933+0000 7f9fb6000640 1 -- 172.21.15.16:0/10719943 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f9fbc00fab0 con 0x55cb0c74b520 2024-09-21T03:48:03.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.934+0000 7f9fb6000640 1 -- 172.21.15.16:0/10719943 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9fbc02f050 con 0x55cb0c74b520 2024-09-21T03:48:03.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.935+0000 7f9fda1f6740 10 client.15738 fetch_fsmap learned FSMap version 30 2024-09-21T03:48:03.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.935+0000 7f9fda1f6740 1 -- 172.21.15.16:0/10719943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cb0c82d960 con 0x55cb0c74b520 2024-09-21T03:48:03.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.936+0000 7f9fb6000640 1 -- 172.21.15.16:0/10719943 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fbc00f900 con 0x55cb0c74b520 2024-09-21T03:48:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.937+0000 7f9fb6000640 1 -- 172.21.15.16:0/10719943 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9fbc018490 con 0x55cb0c74b520 2024-09-21T03:48:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.937+0000 7f9fda1f6740 10 client.15738 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:48:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.937+0000 7f9fda1f6740 10 client.15738 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:48:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.937+0000 7f9fda1f6740 20 client.15738 populate_metadata read hostname 'smithi016' 2024-09-21T03:48:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.937+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/10719943 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cb0c709ea0 0x55cb0cb0fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.937+0000 7f9fda1f6740 4 client.15738 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.937+0000 7f9fda1f6740 1 -- 172.21.15.16:0/10719943 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb0ca6b1e0 con 0x55cb0c709ea0 2024-09-21T03:48:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.938+0000 7f9fd0a00640 1 --2- 172.21.15.16:0/10719943 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cb0c709ea0 0x55cb0cb0fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:03.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.940+0000 7f9fb6000640 10 client.15738 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:03.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.940+0000 7f9fd0a00640 1 --2- 172.21.15.16:0/10719943 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cb0c709ea0 0x55cb0cb0fb20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b3ad9687c44b5e2 server_cookie=732eb378b15848b9 in_seq=0 out_seq=0 2024-09-21T03:48:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.942+0000 7f9fb6000640 1 -- 172.21.15.16:0/10719943 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cb0ca6b1e0 con 0x55cb0c709ea0 2024-09-21T03:48:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.942+0000 7f9fb6000640 10 client.15738 handle_command_reply: tid=0 2024-09-21T03:48:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.942+0000 7f9fda1f6740 1 client.15738 shutdown 2024-09-21T03:48:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.943+0000 7f9fda1f6740 1 -- 172.21.15.16:0/10719943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c74b520 msgr2=0x55cb0cb09cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.943+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/10719943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c74b520 0x55cb0cb09cc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00a9c0 tx=0x7f9fbc00f450 comp rx=0 tx=0).stop 2024-09-21T03:48:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.943+0000 7f9fda1f6740 1 -- 172.21.15.16:0/10719943 shutdown_connections 2024-09-21T03:48:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.943+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/10719943 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cb0c709ea0 0x55cb0cb0fb20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.943+0000 7f9fb6000640 0 client.15738 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.943+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/10719943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c74b520 0x55cb0cb09cc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.943+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/10719943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c72a840 0x55cb0cafe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.943+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/10719943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c722660 0x55cb0cafdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.943+0000 7f9fda1f6740 1 -- 172.21.15.16:0/10719943 >> 172.21.15.16:0/10719943 conn(0x55cb0c73f910 msgr2=0x55cb0caf5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.943+0000 7f9fda1f6740 1 -- 172.21.15.16:0/10719943 shutdown_connections 2024-09-21T03:48:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.944+0000 7f9fda1f6740 1 -- 172.21.15.16:0/10719943 wait complete. 2024-09-21T03:48:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:03.944+0000 7f9fda1f6740 20 client.15738 trim_cache size 0 max 16384 2024-09-21T03:48:04.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.017+0000 7f9fda1f6740 1 Processor -- start 2024-09-21T03:48:04.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.017+0000 7f9fda1f6740 1 -- start start 2024-09-21T03:48:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.019+0000 7f9fda1f6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c709ea0 0x55cb0cafca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.019+0000 7f9fda1f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c722660 0x55cb0caf79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.019+0000 7f9fda1f6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c72a840 0x55cb0caef660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.019+0000 7f9fda1f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1bf920 con 0x55cb0c722660 2024-09-21T03:48:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.019+0000 7f9fda1f6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1c1460 con 0x55cb0c72a840 2024-09-21T03:48:04.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.019+0000 7f9fda1f6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1bfff0 con 0x55cb0c709ea0 2024-09-21T03:48:04.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.020+0000 7f9fcbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c722660 0x55cb0caf79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.020+0000 7f9fcbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c722660 0x55cb0caf79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52068/0 (socket says 172.21.15.16:52068) 2024-09-21T03:48:04.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.020+0000 7f9fcbe00640 1 -- 172.21.15.16:0/473997958 learned_addr learned my addr 172.21.15.16:0/473997958 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:04.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.020+0000 7f9fd1400640 1 --2- 172.21.15.16:0/473997958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c72a840 0x55cb0caef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.021+0000 7f9fd0a00640 1 --2- 172.21.15.16:0/473997958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c709ea0 0x55cb0cafca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.021+0000 7f9fcbe00640 1 -- 172.21.15.16:0/473997958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c709ea0 msgr2=0x55cb0cafca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:04.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.021+0000 7f9fcbe00640 1 --2- 172.21.15.16:0/473997958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c709ea0 0x55cb0cafca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.021+0000 7f9fcbe00640 1 -- 172.21.15.16:0/473997958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c72a840 msgr2=0x55cb0caef660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:04.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.021+0000 7f9fcbe00640 1 --2- 172.21.15.16:0/473997958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c72a840 0x55cb0caef660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.021+0000 7f9fcbe00640 1 -- 172.21.15.16:0/473997958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb0caeb070 con 0x55cb0c722660 2024-09-21T03:48:04.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.021+0000 7f9fcbe00640 1 --2- 172.21.15.16:0/473997958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c722660 0x55cb0caf79a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8011b30 tx=0x7f9fb8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18d550f25650d3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.022+0000 7f9fc1200640 1 -- 172.21.15.16:0/473997958 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb802d070 con 0x55cb0c722660 2024-09-21T03:48:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.022+0000 7f9fc1200640 1 -- 172.21.15.16:0/473997958 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb8028410 con 0x55cb0c722660 2024-09-21T03:48:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.022+0000 7f9fc1200640 1 -- 172.21.15.16:0/473997958 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb801d610 con 0x55cb0c722660 2024-09-21T03:48:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.024+0000 7f9fda1f6740 1 -- 172.21.15.16:0/473997958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c722660 msgr2=0x55cb0caf79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.024+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/473997958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c722660 0x55cb0caf79a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8011b30 tx=0x7f9fb8013000 comp rx=0 tx=0).stop 2024-09-21T03:48:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.025+0000 7f9fda1f6740 1 -- 172.21.15.16:0/473997958 shutdown_connections 2024-09-21T03:48:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.025+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/473997958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c72a840 0x55cb0caef660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.025+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/473997958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c722660 0x55cb0caf79a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.025+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/473997958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c709ea0 0x55cb0cafca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.025+0000 7f9fda1f6740 1 -- 172.21.15.16:0/473997958 >> 172.21.15.16:0/473997958 conn(0x55cb0c73f910 msgr2=0x55cb0caf7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:04.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.027+0000 7f9fda1f6740 1 -- 172.21.15.16:0/473997958 shutdown_connections 2024-09-21T03:48:04.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.028+0000 7f9fda1f6740 1 -- 172.21.15.16:0/473997958 wait complete. 2024-09-21T03:48:04.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.029+0000 7f9fda1f6740 1 -- start start 2024-09-21T03:48:04.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.029+0000 7f9fda1f6740 1 Processor -- start 2024-09-21T03:48:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.031+0000 7f9fda1f6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c709ea0 0x55cb0cb11220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.031+0000 7f9fda1f6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c722660 0x55cb0cb11760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.031+0000 7f9fda1f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c72a840 0x55cb0cb792a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.031+0000 7f9fda1f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1c0d90 con 0x55cb0c72a840 2024-09-21T03:48:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.031+0000 7f9fda1f6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1c06c0 con 0x55cb0c709ea0 2024-09-21T03:48:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.031+0000 7f9fda1f6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc106180 con 0x55cb0c722660 2024-09-21T03:48:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.031+0000 7f9fda1f6740 10 client.0 _ll_register_callbacks cb 0x7f9fb800bcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.032+0000 7f9fd0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c709ea0 0x55cb0cb11220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.032+0000 7f9fd0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c709ea0 0x55cb0cb11220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:47708/0 (socket says 172.21.15.16:47708) 2024-09-21T03:48:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.032+0000 7f9fd0a00640 1 -- 172.21.15.16:0/4287131607 learned_addr learned my addr 172.21.15.16:0/4287131607 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.032+0000 7f9fcbe00640 1 --2- 172.21.15.16:0/4287131607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c722660 0x55cb0cb11760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.032+0000 7f9fd1400640 1 --2- 172.21.15.16:0/4287131607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c72a840 0x55cb0cb792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.033+0000 7f9fcbe00640 1 -- 172.21.15.16:0/4287131607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c709ea0 msgr2=0x55cb0cb11220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.033+0000 7f9fcbe00640 1 --2- 172.21.15.16:0/4287131607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c709ea0 0x55cb0cb11220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.033+0000 7f9fcbe00640 1 -- 172.21.15.16:0/4287131607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c72a840 msgr2=0x55cb0cb792a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.033+0000 7f9fcbe00640 1 --2- 172.21.15.16:0/4287131607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c72a840 0x55cb0cb792a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.033+0000 7f9fcbe00640 1 -- 172.21.15.16:0/4287131607 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb0cb797e0 con 0x55cb0c722660 2024-09-21T03:48:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.033+0000 7f9fd0a00640 1 --2- 172.21.15.16:0/4287131607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c709ea0 0x55cb0cb11220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:48:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.033+0000 7f9fd1400640 1 --2- 172.21.15.16:0/4287131607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c72a840 0x55cb0cb792a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:48:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.035+0000 7f9fcbe00640 1 --2- 172.21.15.16:0/4287131607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c722660 0x55cb0cb11760 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8011b00 tx=0x7f9fb8020cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c23cc718bf766e4f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.035+0000 7f9fb6000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:48:04.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.038+0000 7f9fb6000640 1 -- 172.21.15.16:0/4287131607 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8035020 con 0x55cb0c722660 2024-09-21T03:48:04.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.038+0000 7f9fb6000640 1 -- 172.21.15.16:0/4287131607 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f9fb8020e60 con 0x55cb0c722660 2024-09-21T03:48:04.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.039+0000 7f9fda1f6740 1 -- 172.21.15.16:0/4287131607 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb8028410 con 0x55cb0c722660 2024-09-21T03:48:04.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.039+0000 7f9fb6000640 1 -- 172.21.15.16:0/4287131607 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9fb802d040 con 0x55cb0c722660 2024-09-21T03:48:04.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.041+0000 7f9fda1f6740 10 client.25744 fetch_fsmap learned FSMap version 30 2024-09-21T03:48:04.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.041+0000 7f9fda1f6740 1 -- 172.21.15.16:0/4287131607 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cb0cb7a9e0 con 0x55cb0c722660 2024-09-21T03:48:04.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.042+0000 7f9fb6000640 1 -- 172.21.15.16:0/4287131607 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb8028410 con 0x55cb0c722660 2024-09-21T03:48:04.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.043+0000 7f9fb6000640 1 -- 172.21.15.16:0/4287131607 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9fb8030030 con 0x55cb0c722660 2024-09-21T03:48:04.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.043+0000 7f9fda1f6740 10 client.25744 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:48:04.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.043+0000 7f9fda1f6740 10 client.25744 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:48:04.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.043+0000 7f9fda1f6740 20 client.25744 populate_metadata read hostname 'smithi016' 2024-09-21T03:48:04.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.044+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/4287131607 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cb0c74b520 0x55cb0cb7d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:04.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.044+0000 7f9fda1f6740 4 client.25744 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "39a29551", "release": true, "await": true}] 2024-09-21T03:48:04.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.044+0000 7f9fda1f6740 1 -- 172.21.15.16:0/4287131607 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "set_id": "39a29551", "release": true, "await": true}) -- 0x55cb0ca66c90 con 0x55cb0c74b520 2024-09-21T03:48:04.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.045+0000 7f9fd0a00640 1 --2- 172.21.15.16:0/4287131607 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cb0c74b520 0x55cb0cb7d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:04.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.046+0000 7f9fd0a00640 1 --2- 172.21.15.16:0/4287131607 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cb0c74b520 0x55cb0cb7d070 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=973bbea249d4a925 server_cookie=722162e01343c44 in_seq=0 out_seq=0 2024-09-21T03:48:04.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.046+0000 7f9fb6000640 10 client.25744 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:04.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.051+0000 7f9fb6000640 1 -- 172.21.15.16:0/4287131607 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55cb0ca66c90 con 0x55cb0c74b520 2024-09-21T03:48:04.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.051+0000 7f9fb6000640 10 client.25744 handle_command_reply: tid=0 2024-09-21T03:48:04.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.051+0000 7f9fda1f6740 1 client.25744 shutdown 2024-09-21T03:48:04.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.060+0000 7f9fda1f6740 1 -- 172.21.15.16:0/4287131607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c722660 msgr2=0x55cb0cb11760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:04.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.060+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/4287131607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c722660 0x55cb0cb11760 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8011b00 tx=0x7f9fb8020cf0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.065+0000 7f9fda1f6740 1 -- 172.21.15.16:0/4287131607 shutdown_connections 2024-09-21T03:48:04.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.065+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/4287131607 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cb0c74b520 0x55cb0cb7d070 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.065+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/4287131607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb0c72a840 0x55cb0cb792a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.065+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/4287131607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb0c722660 0x55cb0cb11760 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.065+0000 7f9fda1f6740 1 --2- 172.21.15.16:0/4287131607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb0c709ea0 0x55cb0cb11220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.065+0000 7f9fda1f6740 1 -- 172.21.15.16:0/4287131607 >> 172.21.15.16:0/4287131607 conn(0x55cb0c73f910 msgr2=0x55cb0caf31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:04.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.065+0000 7f9fb6000640 0 client.25744 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:04.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.065+0000 7f9fda1f6740 1 -- 172.21.15.16:0/4287131607 shutdown_connections 2024-09-21T03:48:04.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.065+0000 7f9fda1f6740 1 -- 172.21.15.16:0/4287131607 wait complete. 2024-09-21T03:48:04.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.069+0000 7f9fda1f6740 20 client.25744 trim_cache size 0 max 16384 2024-09-21T03:48:04.088 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:48:04.088 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:48:04.088 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:48:04.088 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 8, 2024-09-21T03:48:04.088 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:48:04.088 INFO:teuthology.orchestra.run.smithi016.stdout: "39a29551": { 2024-09-21T03:48:04.088 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 8, 2024-09-21T03:48:04.088 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:48:04.088 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:48:04.088 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:48:04.088 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:48:04.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:48:04.089 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:48:04.089 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 137.0, 2024-09-21T03:48:04.089 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:48:04.089 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:48:04.089 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:48:04.089 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:48:04.089 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:48:04.089 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:48:04.089 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:48:04.089 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:48:04.090 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:48:04.090 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:48:04.090 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:48:04.090 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:48:04.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.087+0000 7f9fd3400640 1 -- 172.21.15.16:0/2972594148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9fb00b65d0 msgr2=0x7f9fb00b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:04.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.087+0000 7f9fd3400640 1 --2- 172.21.15.16:0/2972594148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9fb00b65d0 0x7f9fb00b8ae0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc103f80 tx=0x7f9fb800b410 comp rx=0 tx=0).stop 2024-09-21T03:48:04.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.087+0000 7f9fd3400640 1 -- 172.21.15.16:0/2972594148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc105d70 msgr2=0x7f9fcc07b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:04.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.087+0000 7f9fd3400640 1 --2- 172.21.15.16:0/2972594148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc105d70 0x7f9fcc07b640 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9fc402aed0 tx=0x7f9fc4003ba0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.093+0000 7f9fd3400640 1 -- 172.21.15.16:0/2972594148 shutdown_connections 2024-09-21T03:48:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.093+0000 7f9fd3400640 1 --2- 172.21.15.16:0/2972594148 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9fb00b65d0 0x7f9fb00b8ae0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.093+0000 7f9fd3400640 1 --2- 172.21.15.16:0/2972594148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fcc10c000 0x7f9fcc078040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.093+0000 7f9fd3400640 1 --2- 172.21.15.16:0/2972594148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc10b6b0 0x7f9fcc077920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.093+0000 7f9fd3400640 1 --2- 172.21.15.16:0/2972594148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc105d70 0x7f9fcc07b640 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.093+0000 7f9fd3400640 1 -- 172.21.15.16:0/2972594148 >> 172.21.15.16:0/2972594148 conn(0x7f9fcc101780 msgr2=0x7f9fcc109e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.094+0000 7f9fd3400640 1 -- 172.21.15.16:0/2972594148 shutdown_connections 2024-09-21T03:48:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:04.094+0000 7f9fd3400640 1 -- 172.21.15.16:0/2972594148 wait complete. 2024-09-21T03:48:05.126 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '39a29551', total seconds elapsed: 23.0 2024-09-21T03:48:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:04 smithi134 ceph-mon[35031]: pgmap v471: 129 pgs: 129 active+clean; 4.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 284 MiB/s wr, 91 op/s 2024-09-21T03:48:05.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:04 smithi123 ceph-mon[34673]: pgmap v471: 129 pgs: 129 active+clean; 4.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 284 MiB/s wr, 91 op/s 2024-09-21T03:48:05.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:04 smithi016 ceph-mon[30375]: pgmap v471: 129 pgs: 129 active+clean; 4.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 284 MiB/s wr, 91 op/s 2024-09-21T03:48:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:06 smithi016 ceph-mon[30375]: pgmap v472: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 288 MiB/s wr, 84 op/s 2024-09-21T03:48:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:06 smithi134 ceph-mon[35031]: pgmap v472: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 288 MiB/s wr, 84 op/s 2024-09-21T03:48:07.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:06 smithi123 ceph-mon[34673]: pgmap v472: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 288 MiB/s wr, 84 op/s 2024-09-21T03:48:09.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:08 smithi134 ceph-mon[35031]: pgmap v473: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 271 MiB/s wr, 80 op/s 2024-09-21T03:48:09.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:08 smithi123 ceph-mon[34673]: pgmap v473: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 271 MiB/s wr, 80 op/s 2024-09-21T03:48:09.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:08 smithi016 ceph-mon[30375]: pgmap v473: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 271 MiB/s wr, 80 op/s 2024-09-21T03:48:11.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:10 smithi134 ceph-mon[35031]: pgmap v474: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 244 MiB/s wr, 76 op/s 2024-09-21T03:48:11.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:10 smithi123 ceph-mon[34673]: pgmap v474: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 244 MiB/s wr, 76 op/s 2024-09-21T03:48:11.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:10 smithi016 ceph-mon[30375]: pgmap v474: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 244 MiB/s wr, 76 op/s 2024-09-21T03:48:13.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:12 smithi134 ceph-mon[35031]: pgmap v475: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 243 MiB/s wr, 80 op/s 2024-09-21T03:48:13.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:12 smithi123 ceph-mon[34673]: pgmap v475: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 243 MiB/s wr, 80 op/s 2024-09-21T03:48:13.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:12 smithi016 ceph-mon[30375]: pgmap v475: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 243 MiB/s wr, 80 op/s 2024-09-21T03:48:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.912+0000 7f7a8b400640 1 -- 172.21.15.16:0/1590358465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8406b710 msgr2=0x7f7a841040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.912+0000 7f7a8b400640 1 --2- 172.21.15.16:0/1590358465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8406b710 0x7f7a841040d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009900 tx=0x7f7a7802a420 comp rx=0 tx=0).stop 2024-09-21T03:48:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.916+0000 7f7a8b400640 1 -- 172.21.15.16:0/1590358465 shutdown_connections 2024-09-21T03:48:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.916+0000 7f7a8b400640 1 --2- 172.21.15.16:0/1590358465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a841071d0 0x7f7a841076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.916+0000 7f7a8b400640 1 --2- 172.21.15.16:0/1590358465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a84104610 0x7f7a84106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.916+0000 7f7a8b400640 1 --2- 172.21.15.16:0/1590358465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8406b710 0x7f7a841040d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.916+0000 7f7a8b400640 1 -- 172.21.15.16:0/1590358465 >> 172.21.15.16:0/1590358465 conn(0x7f7a840fdee0 msgr2=0x7f7a84100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.918+0000 7f7a8b400640 1 -- 172.21.15.16:0/1590358465 shutdown_connections 2024-09-21T03:48:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.918+0000 7f7a8b400640 1 -- 172.21.15.16:0/1590358465 wait complete. 2024-09-21T03:48:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.921+0000 7f7a8b400640 1 Processor -- start 2024-09-21T03:48:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.921+0000 7f7a8b400640 1 -- start start 2024-09-21T03:48:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.922+0000 7f7a8b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a8406b710 0x7f7a841a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.923+0000 7f7a8b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a84104610 0x7f7a841a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.923+0000 7f7a8b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a841071d0 0x7f7a841a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.923+0000 7f7a8b400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a841159c0 con 0x7f7a841071d0 2024-09-21T03:48:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.923+0000 7f7a8b400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a84115840 con 0x7f7a84104610 2024-09-21T03:48:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.923+0000 7f7a8b400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7a84115b40 con 0x7f7a8406b710 2024-09-21T03:48:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.923+0000 7f7a88a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a8406b710 0x7f7a841a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.923+0000 7f7a88a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a8406b710 0x7f7a841a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:35226/0 (socket says 172.21.15.16:35226) 2024-09-21T03:48:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.923+0000 7f7a88a00640 1 -- 172.21.15.16:0/1184284783 learned_addr learned my addr 172.21.15.16:0/1184284783 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.924+0000 7f7a83e00640 1 --2- 172.21.15.16:0/1184284783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a84104610 0x7f7a841a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.924+0000 7f7a89400640 1 --2- 172.21.15.16:0/1184284783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a841071d0 0x7f7a841a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.925+0000 7f7a89400640 1 -- 172.21.15.16:0/1184284783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a8406b710 msgr2=0x7f7a841a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.925+0000 7f7a89400640 1 --2- 172.21.15.16:0/1184284783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a8406b710 0x7f7a841a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.925+0000 7f7a89400640 1 -- 172.21.15.16:0/1184284783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a84104610 msgr2=0x7f7a841a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.925+0000 7f7a89400640 1 --2- 172.21.15.16:0/1184284783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a84104610 0x7f7a841a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.925+0000 7f7a89400640 1 -- 172.21.15.16:0/1184284783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a841a2050 con 0x7f7a841071d0 2024-09-21T03:48:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.925+0000 7f7a89400640 1 --2- 172.21.15.16:0/1184284783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a841071d0 0x7f7a841a1830 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00ca80 tx=0x7f7a7c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.926+0000 7f7a81600640 1 -- 172.21.15.16:0/1184284783 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c007d90 con 0x7f7a841071d0 2024-09-21T03:48:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.926+0000 7f7a81600640 1 -- 172.21.15.16:0/1184284783 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a7c004b70 con 0x7f7a841071d0 2024-09-21T03:48:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.926+0000 7f7a81600640 1 -- 172.21.15.16:0/1184284783 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c007d90 con 0x7f7a841071d0 2024-09-21T03:48:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.926+0000 7f7a8b400640 1 -- 172.21.15.16:0/1184284783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a841a2340 con 0x7f7a841071d0 2024-09-21T03:48:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.927+0000 7f7a8b400640 1 -- 172.21.15.16:0/1184284783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a841a8a30 con 0x7f7a841071d0 2024-09-21T03:48:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.934+0000 7f7a923c7740 1 Processor -- start 2024-09-21T03:48:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.937+0000 7f7a81600640 1 -- 172.21.15.16:0/1184284783 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f7a7c00f450 con 0x7f7a841071d0 2024-09-21T03:48:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.937+0000 7f7a923c7740 1 -- start start 2024-09-21T03:48:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.938+0000 7f7a923c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 0x55a73dfe40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.938+0000 7f7a923c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc142f0 0x55a73dfe4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.938+0000 7f7a923c7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc1c4d0 0x55a73dfe5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.938+0000 7f7a83e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 0x55a73dfe40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.938+0000 7f7a923c7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a73df576b0 con 0x55a73dc1c4d0 2024-09-21T03:48:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.938+0000 7f7a923c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a73df46ae0 con 0x55a73dc3d1b0 2024-09-21T03:48:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.938+0000 7f7a923c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a73df4a8f0 con 0x55a73dc142f0 2024-09-21T03:48:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.938+0000 7f7a83e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 0x55a73dfe40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:48882/0 (socket says 172.21.15.16:48882) 2024-09-21T03:48:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.938+0000 7f7a83e00640 1 -- 172.21.15.16:0/3656803944 learned_addr learned my addr 172.21.15.16:0/3656803944 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.938+0000 7f7a83e00640 1 --2- 172.21.15.16:0/3656803944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc1c4d0 0x55a73dfe5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.939+0000 7f7a83e00640 1 -- 172.21.15.16:0/3656803944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc142f0 msgr2=0x55a73dfe4740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:48:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.939+0000 7f7a83e00640 1 --2- 172.21.15.16:0/3656803944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc142f0 0x55a73dfe4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.939+0000 7f7a83e00640 1 -- 172.21.15.16:0/3656803944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc1c4d0 msgr2=0x55a73dfe5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.939+0000 7f7a83e00640 1 --2- 172.21.15.16:0/3656803944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc1c4d0 0x55a73dfe5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.939+0000 7f7a83e00640 1 -- 172.21.15.16:0/3656803944 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73df13780 con 0x55a73dc3d1b0 2024-09-21T03:48:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.940+0000 7f7a83e00640 1 --2- 172.21.15.16:0/3656803944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 0x55a73dfe40d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009fc0 tx=0x7f7a70009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f88972562d53985 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.940+0000 7f7a75c00640 1 -- 172.21.15.16:0/3656803944 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a70017470 con 0x55a73dc3d1b0 2024-09-21T03:48:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.940+0000 7f7a75c00640 1 -- 172.21.15.16:0/3656803944 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a7001c070 con 0x55a73dc3d1b0 2024-09-21T03:48:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.940+0000 7f7a75c00640 1 -- 172.21.15.16:0/3656803944 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a70015840 con 0x55a73dc3d1b0 2024-09-21T03:48:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.941+0000 7f7a923c7740 1 -- 172.21.15.16:0/3656803944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 msgr2=0x55a73dfe40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.941+0000 7f7a923c7740 1 --2- 172.21.15.16:0/3656803944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 0x55a73dfe40d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009fc0 tx=0x7f7a70009500 comp rx=0 tx=0).stop 2024-09-21T03:48:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.941+0000 7f7a923c7740 1 -- 172.21.15.16:0/3656803944 shutdown_connections 2024-09-21T03:48:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.941+0000 7f7a923c7740 1 --2- 172.21.15.16:0/3656803944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc1c4d0 0x55a73dfe5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.941+0000 7f7a923c7740 1 --2- 172.21.15.16:0/3656803944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc142f0 0x55a73dfe4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.941+0000 7f7a923c7740 1 --2- 172.21.15.16:0/3656803944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 0x55a73dfe40d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.941+0000 7f7a923c7740 1 -- 172.21.15.16:0/3656803944 >> 172.21.15.16:0/3656803944 conn(0x55a73dc31300 msgr2=0x55a73dfdfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.942+0000 7f7a923c7740 1 -- 172.21.15.16:0/3656803944 shutdown_connections 2024-09-21T03:48:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.942+0000 7f7a923c7740 1 -- 172.21.15.16:0/3656803944 wait complete. 2024-09-21T03:48:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.942+0000 7f7a923c7740 1 -- start start 2024-09-21T03:48:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.944+0000 7f7a81600640 1 --2- 172.21.15.16:0/1184284783 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7a6c0b6760 0x7f7a6c0b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.944+0000 7f7a81600640 1 -- 172.21.15.16:0/1184284783 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f7a7c0e73d0 con 0x7f7a841071d0 2024-09-21T03:48:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.945+0000 7f7a923c7740 1 Processor -- start 2024-09-21T03:48:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.945+0000 7f7a923c7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc142f0 0x55a73dffe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.945+0000 7f7a923c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc1c4d0 0x55a73dffee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.945+0000 7f7a923c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 0x55a73dfff9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.945+0000 7f7a923c7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a73de7eab0 con 0x55a73dc142f0 2024-09-21T03:48:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.945+0000 7f7a923c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a73deed570 con 0x55a73dc3d1b0 2024-09-21T03:48:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.945+0000 7f7a923c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a73dfe5db0 con 0x55a73dc1c4d0 2024-09-21T03:48:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.946+0000 7f7a923c7740 10 client.0 _ll_register_callbacks cb 0x55a73defc100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.946+0000 7f7a89400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 0x55a73dfff9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.946+0000 7f7a89400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 0x55a73dfff9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:48892/0 (socket says 172.21.15.16:48892) 2024-09-21T03:48:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.946+0000 7f7a89400640 1 -- 172.21.15.16:0/2240624907 learned_addr learned my addr 172.21.15.16:0/2240624907 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.946+0000 7f7a83e00640 1 --2- 172.21.15.16:0/1184284783 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7a6c0b6760 0x7f7a6c0b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.946+0000 7f7a89400640 1 -- 172.21.15.16:0/2240624907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc1c4d0 msgr2=0x55a73dffee70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:48:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.946+0000 7f7a89400640 1 --2- 172.21.15.16:0/2240624907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc1c4d0 0x55a73dffee70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.946+0000 7f7a89400640 1 -- 172.21.15.16:0/2240624907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc142f0 msgr2=0x55a73dffe7b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:48:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.947+0000 7f7a88a00640 1 --2- 172.21.15.16:0/2240624907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc142f0 0x55a73dffe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.947+0000 7f7a89400640 1 --2- 172.21.15.16:0/2240624907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc142f0 0x55a73dffe7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.947+0000 7f7a89400640 1 -- 172.21.15.16:0/2240624907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73dca8790 con 0x55a73dc3d1b0 2024-09-21T03:48:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.947+0000 7f7a89400640 1 --2- 172.21.15.16:0/2240624907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 0x55a73dfff9e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0ad190 tx=0x7f7a7c0acbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2af0bdeb1ac6d646 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.947+0000 7f7a83e00640 1 --2- 172.21.15.16:0/1184284783 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7a6c0b6760 0x7f7a6c0b8c70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7a7001b000 tx=0x7f7a70009b40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.947+0000 7f7a6a000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:48:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.948+0000 7f7a6a000640 1 -- 172.21.15.16:0/2240624907 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c01d030 con 0x55a73dc3d1b0 2024-09-21T03:48:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.949+0000 7f7a6a000640 1 -- 172.21.15.16:0/2240624907 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f7a7c0a8b50 con 0x55a73dc3d1b0 2024-09-21T03:48:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.949+0000 7f7a923c7740 1 -- 172.21.15.16:0/2240624907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a7001c070 con 0x55a73dc3d1b0 2024-09-21T03:48:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.949+0000 7f7a6a000640 1 -- 172.21.15.16:0/2240624907 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7a7c0e71b0 con 0x55a73dc3d1b0 2024-09-21T03:48:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.949+0000 7f7a923c7740 10 client.25787 fetch_fsmap learned FSMap version 30 2024-09-21T03:48:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.949+0000 7f7a923c7740 1 -- 172.21.15.16:0/2240624907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a73dd21620 con 0x55a73dc3d1b0 2024-09-21T03:48:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.950+0000 7f7a6a000640 1 -- 172.21.15.16:0/2240624907 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a7001c070 con 0x55a73dc3d1b0 2024-09-21T03:48:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.950+0000 7f7a6a000640 1 -- 172.21.15.16:0/2240624907 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7a7c0b2ec0 con 0x55a73dc3d1b0 2024-09-21T03:48:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.950+0000 7f7a923c7740 10 client.25787 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:48:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.950+0000 7f7a923c7740 10 client.25787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.950+0000 7f7a923c7740 20 client.25787 populate_metadata read hostname 'smithi016' 2024-09-21T03:48:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.950+0000 7f7a923c7740 1 --2- 172.21.15.16:0/2240624907 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a73dbfbd80 0x55a73e01cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.950+0000 7f7a923c7740 4 client.25787 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.950+0000 7f7a923c7740 1 -- 172.21.15.16:0/2240624907 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a73df59600 con 0x55a73dbfbd80 2024-09-21T03:48:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.951+0000 7f7a88a00640 1 --2- 172.21.15.16:0/2240624907 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a73dbfbd80 0x55a73e01cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.952+0000 7f7a88a00640 1 --2- 172.21.15.16:0/2240624907 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a73dbfbd80 0x55a73e01cee0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7bda13da657262e server_cookie=836d5866f1da537b in_seq=0 out_seq=0 2024-09-21T03:48:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.952+0000 7f7a6a000640 10 client.25787 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.953+0000 7f7a6a000640 1 -- 172.21.15.16:0/2240624907 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a73df59600 con 0x55a73dbfbd80 2024-09-21T03:48:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.953+0000 7f7a6a000640 10 client.25787 handle_command_reply: tid=0 2024-09-21T03:48:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.953+0000 7f7a923c7740 1 client.25787 shutdown 2024-09-21T03:48:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.954+0000 7f7a923c7740 1 -- 172.21.15.16:0/2240624907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 msgr2=0x55a73dfff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.954+0000 7f7a923c7740 1 --2- 172.21.15.16:0/2240624907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 0x55a73dfff9e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0ad190 tx=0x7f7a7c0acbc0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.955+0000 7f7a923c7740 1 -- 172.21.15.16:0/2240624907 shutdown_connections 2024-09-21T03:48:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.955+0000 7f7a923c7740 1 --2- 172.21.15.16:0/2240624907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 0x55a73dfff9e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.955+0000 7f7a923c7740 1 --2- 172.21.15.16:0/2240624907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc1c4d0 0x55a73dffee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.955+0000 7f7a923c7740 1 --2- 172.21.15.16:0/2240624907 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a73dbfbd80 0x55a73e01cee0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.955+0000 7f7a923c7740 1 --2- 172.21.15.16:0/2240624907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc142f0 0x55a73dffe7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.955+0000 7f7a923c7740 1 -- 172.21.15.16:0/2240624907 >> 172.21.15.16:0/2240624907 conn(0x55a73dc31300 msgr2=0x55a73dfde0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.955+0000 7f7a6a000640 0 client.25787 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.955+0000 7f7a923c7740 1 -- 172.21.15.16:0/2240624907 shutdown_connections 2024-09-21T03:48:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.955+0000 7f7a923c7740 1 -- 172.21.15.16:0/2240624907 wait complete. 2024-09-21T03:48:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.956+0000 7f7a923c7740 20 client.25787 trim_cache size 0 max 16384 2024-09-21T03:48:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.985+0000 7f7a923c7740 1 Processor -- start 2024-09-21T03:48:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.986+0000 7f7a923c7740 1 -- start start 2024-09-21T03:48:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.986+0000 7f7a923c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc142f0 0x55a73dfe8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.987+0000 7f7a923c7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc1c4d0 0x55a73dfe0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.987+0000 7f7a83e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc1c4d0 0x55a73dfe0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.988+0000 7f7a83e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc1c4d0 0x55a73dfe0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33212/0 (socket says 172.21.15.16:33212) 2024-09-21T03:48:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.988+0000 7f7a88a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc142f0 0x55a73dfe8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.988+0000 7f7a83e00640 1 -- 172.21.15.16:0/107197920 learned_addr learned my addr 172.21.15.16:0/107197920 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.988+0000 7f7a923c7740 1 --2- 172.21.15.16:0/107197920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 0x55a73dffd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.988+0000 7f7a923c7740 1 -- 172.21.15.16:0/107197920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8406e730 con 0x55a73dc1c4d0 2024-09-21T03:48:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.989+0000 7f7a923c7740 1 -- 172.21.15.16:0/107197920 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a84070270 con 0x55a73dc3d1b0 2024-09-21T03:48:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.989+0000 7f7a89400640 1 --2- 172.21.15.16:0/107197920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 0x55a73dffd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:14.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.989+0000 7f7a923c7740 1 -- 172.21.15.16:0/107197920 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8406ee00 con 0x55a73dc142f0 2024-09-21T03:48:14.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.990+0000 7f7a88a00640 1 -- 172.21.15.16:0/107197920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 msgr2=0x55a73dffd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:14.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.990+0000 7f7a88a00640 1 --2- 172.21.15.16:0/107197920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 0x55a73dffd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.990+0000 7f7a88a00640 1 -- 172.21.15.16:0/107197920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc1c4d0 msgr2=0x55a73dfe0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.990+0000 7f7a88a00640 1 --2- 172.21.15.16:0/107197920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc1c4d0 0x55a73dfe0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.990+0000 7f7a88a00640 1 -- 172.21.15.16:0/107197920 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73dfff1e0 con 0x55a73dc142f0 2024-09-21T03:48:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.991+0000 7f7a89400640 1 --2- 172.21.15.16:0/107197920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 0x55a73dffd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:48:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.992+0000 7f7a88a00640 1 --2- 172.21.15.16:0/107197920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc142f0 0x55a73dfe8b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7a78000c00 tx=0x7f7a780095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e74d3751dfa47173 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.992+0000 7f7a75200640 1 -- 172.21.15.16:0/107197920 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78035030 con 0x55a73dc142f0 2024-09-21T03:48:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.992+0000 7f7a75200640 1 -- 172.21.15.16:0/107197920 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a78049020 con 0x55a73dc142f0 2024-09-21T03:48:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.992+0000 7f7a75200640 1 -- 172.21.15.16:0/107197920 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78031860 con 0x55a73dc142f0 2024-09-21T03:48:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.994+0000 7f7a923c7740 1 -- 172.21.15.16:0/107197920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc142f0 msgr2=0x55a73dfe8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.994+0000 7f7a923c7740 1 --2- 172.21.15.16:0/107197920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc142f0 0x55a73dfe8b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7a78000c00 tx=0x7f7a780095f0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.994+0000 7f7a923c7740 1 -- 172.21.15.16:0/107197920 shutdown_connections 2024-09-21T03:48:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.994+0000 7f7a923c7740 1 --2- 172.21.15.16:0/107197920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc3d1b0 0x55a73dffd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.994+0000 7f7a923c7740 1 --2- 172.21.15.16:0/107197920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc1c4d0 0x55a73dfe0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.994+0000 7f7a923c7740 1 --2- 172.21.15.16:0/107197920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc142f0 0x55a73dfe8b80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.994+0000 7f7a923c7740 1 -- 172.21.15.16:0/107197920 >> 172.21.15.16:0/107197920 conn(0x55a73dc31300 msgr2=0x55a73dfdf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.995+0000 7f7a923c7740 1 -- 172.21.15.16:0/107197920 shutdown_connections 2024-09-21T03:48:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.995+0000 7f7a923c7740 1 -- 172.21.15.16:0/107197920 wait complete. 2024-09-21T03:48:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.995+0000 7f7a923c7740 1 -- start start 2024-09-21T03:48:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.996+0000 7f7a923c7740 1 Processor -- start 2024-09-21T03:48:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.997+0000 7f7a923c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc142f0 0x55a73e005700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.997+0000 7f7a923c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc1c4d0 0x55a73e005dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.997+0000 7f7a923c7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc3d1b0 0x55a73e004260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.998+0000 7f7a923c7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8406fba0 con 0x55a73dc3d1b0 2024-09-21T03:48:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.998+0000 7f7a923c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8406f4d0 con 0x55a73dc142f0 2024-09-21T03:48:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.998+0000 7f7a923c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7a84110820 con 0x55a73dc1c4d0 2024-09-21T03:48:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.998+0000 7f7a923c7740 10 client.0 _ll_register_callbacks cb 0x7f7a7c00fba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.998+0000 7f7a89400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc3d1b0 0x55a73e004260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.998+0000 7f7a89400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc3d1b0 0x55a73e004260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33222/0 (socket says 172.21.15.16:33222) 2024-09-21T03:48:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.998+0000 7f7a89400640 1 -- 172.21.15.16:0/744329566 learned_addr learned my addr 172.21.15.16:0/744329566 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.999+0000 7f7a89400640 1 -- 172.21.15.16:0/744329566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc1c4d0 msgr2=0x55a73e005dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:48:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.999+0000 7f7a89400640 1 --2- 172.21.15.16:0/744329566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc1c4d0 0x55a73e005dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.999+0000 7f7a89400640 1 -- 172.21.15.16:0/744329566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc142f0 msgr2=0x55a73e005700 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:48:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.999+0000 7f7a89400640 1 --2- 172.21.15.16:0/744329566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc142f0 0x55a73e005700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:14.999+0000 7f7a89400640 1 -- 172.21.15.16:0/744329566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73e004820 con 0x55a73dc3d1b0 2024-09-21T03:48:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.000+0000 7f7a89400640 1 --2- 172.21.15.16:0/744329566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc3d1b0 0x55a73e004260 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0aa470 tx=0x7f7a7c00f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0f5d3d7929d7e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.000+0000 7f7a6a000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:48:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.001+0000 7f7a6a000640 1 -- 172.21.15.16:0/744329566 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c01d070 con 0x55a73dc3d1b0 2024-09-21T03:48:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.001+0000 7f7a923c7740 1 -- 172.21.15.16:0/744329566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a78049020 con 0x55a73dc3d1b0 2024-09-21T03:48:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.002+0000 7f7a6a000640 1 -- 172.21.15.16:0/744329566 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f7a7c0abed0 con 0x55a73dc3d1b0 2024-09-21T03:48:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.002+0000 7f7a6a000640 1 -- 172.21.15.16:0/744329566 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7a7c0b64d0 con 0x55a73dc3d1b0 2024-09-21T03:48:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.003+0000 7f7a923c7740 10 client.15759 fetch_fsmap learned FSMap version 30 2024-09-21T03:48:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.003+0000 7f7a923c7740 1 -- 172.21.15.16:0/744329566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a73e0881a0 con 0x55a73dc3d1b0 2024-09-21T03:48:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.004+0000 7f7a6a000640 1 -- 172.21.15.16:0/744329566 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a78049020 con 0x55a73dc3d1b0 2024-09-21T03:48:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.005+0000 7f7a6a000640 1 -- 172.21.15.16:0/744329566 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7a7c0ac0d0 con 0x55a73dc3d1b0 2024-09-21T03:48:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.005+0000 7f7a923c7740 10 client.15759 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:48:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.005+0000 7f7a923c7740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.005+0000 7f7a923c7740 20 client.15759 populate_metadata read hostname 'smithi016' 2024-09-21T03:48:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.005+0000 7f7a923c7740 1 --2- 172.21.15.16:0/744329566 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a73dbd11c0 0x55a73e08abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.005+0000 7f7a923c7740 4 client.15759 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:48:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.005+0000 7f7a923c7740 1 -- 172.21.15.16:0/744329566 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a73df550b0 con 0x55a73dbd11c0 2024-09-21T03:48:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.007+0000 7f7a88a00640 1 --2- 172.21.15.16:0/744329566 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a73dbd11c0 0x55a73e08abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.007+0000 7f7a88a00640 1 --2- 172.21.15.16:0/744329566 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a73dbd11c0 0x55a73e08abe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f992efbe553e117 server_cookie=b62bc3f30deec54b in_seq=0 out_seq=0 2024-09-21T03:48:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.008+0000 7f7a6a000640 10 client.15759 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.021+0000 7f7a6a000640 1 -- 172.21.15.16:0/744329566 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a73df550b0 con 0x55a73dbd11c0 2024-09-21T03:48:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.021+0000 7f7a6a000640 10 client.15759 handle_command_reply: tid=0 2024-09-21T03:48:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.022+0000 7f7a923c7740 1 client.15759 shutdown 2024-09-21T03:48:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.022+0000 7f7a923c7740 1 -- 172.21.15.16:0/744329566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc3d1b0 msgr2=0x55a73e004260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.022+0000 7f7a923c7740 1 --2- 172.21.15.16:0/744329566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc3d1b0 0x55a73e004260 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0aa470 tx=0x7f7a7c00f950 comp rx=0 tx=0).stop 2024-09-21T03:48:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.022+0000 7f7a923c7740 1 -- 172.21.15.16:0/744329566 shutdown_connections 2024-09-21T03:48:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.022+0000 7f7a923c7740 1 --2- 172.21.15.16:0/744329566 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a73dbd11c0 0x55a73e08abe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.022+0000 7f7a923c7740 1 --2- 172.21.15.16:0/744329566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73dc3d1b0 0x55a73e004260 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.022+0000 7f7a923c7740 1 --2- 172.21.15.16:0/744329566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a73dc1c4d0 0x55a73e005dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.022+0000 7f7a923c7740 1 --2- 172.21.15.16:0/744329566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a73dc142f0 0x55a73e005700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.022+0000 7f7a923c7740 1 -- 172.21.15.16:0/744329566 >> 172.21.15.16:0/744329566 conn(0x55a73dc31300 msgr2=0x55a73dfdf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.022+0000 7f7a6a000640 0 client.15759 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.023+0000 7f7a923c7740 1 -- 172.21.15.16:0/744329566 shutdown_connections 2024-09-21T03:48:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.023+0000 7f7a923c7740 1 -- 172.21.15.16:0/744329566 wait complete. 2024-09-21T03:48:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.023+0000 7f7a923c7740 20 client.15759 trim_cache size 0 max 16384 2024-09-21T03:48:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:48:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:48:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:48:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:48:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.024+0000 7f7a8b400640 1 -- 172.21.15.16:0/1184284783 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7a6c0b6760 msgr2=0x7f7a6c0b8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.024+0000 7f7a8b400640 1 --2- 172.21.15.16:0/1184284783 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7a6c0b6760 0x7f7a6c0b8c70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7a7001b000 tx=0x7f7a70009b40 comp rx=0 tx=0).stop 2024-09-21T03:48:15.028 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.024+0000 7f7a8b400640 1 -- 172.21.15.16:0/1184284783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a841071d0 msgr2=0x7f7a841a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:15.028 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.024+0000 7f7a8b400640 1 --2- 172.21.15.16:0/1184284783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a841071d0 0x7f7a841a1830 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00ca80 tx=0x7f7a7c00cf40 comp rx=0 tx=0).stop 2024-09-21T03:48:15.028 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.025+0000 7f7a8b400640 1 -- 172.21.15.16:0/1184284783 shutdown_connections 2024-09-21T03:48:15.028 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.025+0000 7f7a8b400640 1 --2- 172.21.15.16:0/1184284783 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7a6c0b6760 0x7f7a6c0b8c70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.025+0000 7f7a8b400640 1 --2- 172.21.15.16:0/1184284783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a841071d0 0x7f7a841a1830 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.025+0000 7f7a8b400640 1 --2- 172.21.15.16:0/1184284783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a84104610 0x7f7a841a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.025+0000 7f7a8b400640 1 --2- 172.21.15.16:0/1184284783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a8406b710 0x7f7a841a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.025+0000 7f7a8b400640 1 -- 172.21.15.16:0/1184284783 >> 172.21.15.16:0/1184284783 conn(0x7f7a840fdee0 msgr2=0x7f7a841055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.028+0000 7f7a8b400640 1 -- 172.21.15.16:0/1184284783 shutdown_connections 2024-09-21T03:48:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:15.028+0000 7f7a8b400640 1 -- 172.21.15.16:0/1184284783 wait complete. 2024-09-21T03:48:15.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:14 smithi134 ceph-mon[35031]: pgmap v476: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 230 MiB/s wr, 85 op/s 2024-09-21T03:48:15.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:14 smithi123 ceph-mon[34673]: pgmap v476: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 230 MiB/s wr, 85 op/s 2024-09-21T03:48:15.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:14 smithi016 ceph-mon[30375]: pgmap v476: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 230 MiB/s wr, 85 op/s 2024-09-21T03:48:17.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:16 smithi134 ceph-mon[35031]: pgmap v477: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 239 MiB/s wr, 85 op/s 2024-09-21T03:48:17.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:16 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:48:17.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:16 smithi123 ceph-mon[34673]: pgmap v477: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 239 MiB/s wr, 85 op/s 2024-09-21T03:48:17.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:16 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:48:17.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:16 smithi016 ceph-mon[30375]: pgmap v477: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 239 MiB/s wr, 85 op/s 2024-09-21T03:48:17.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:16 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:48:18.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:18 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:48:18.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:18 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:48:18.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:18 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:48:18.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:18 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:48:18.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:18 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:48:18.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:18 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:48:18.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:18 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:48:18.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:18 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:48:18.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:18 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:48:19.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:19 smithi134 ceph-mon[35031]: pgmap v478: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 242 MiB/s wr, 87 op/s 2024-09-21T03:48:19.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:19 smithi123 ceph-mon[34673]: pgmap v478: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 242 MiB/s wr, 87 op/s 2024-09-21T03:48:19.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:19 smithi016 ceph-mon[30375]: pgmap v478: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 242 MiB/s wr, 87 op/s 2024-09-21T03:48:21.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:21 smithi134 ceph-mon[35031]: pgmap v479: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 235 MiB/s wr, 87 op/s 2024-09-21T03:48:21.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:21 smithi123 ceph-mon[34673]: pgmap v479: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 235 MiB/s wr, 87 op/s 2024-09-21T03:48:21.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:21 smithi016 ceph-mon[30375]: pgmap v479: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 235 MiB/s wr, 87 op/s 2024-09-21T03:48:23.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:23 smithi134 ceph-mon[35031]: pgmap v480: 129 pgs: 129 active+clean; 8.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 252 MiB/s wr, 89 op/s 2024-09-21T03:48:23.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:23 smithi123 ceph-mon[34673]: pgmap v480: 129 pgs: 129 active+clean; 8.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 252 MiB/s wr, 89 op/s 2024-09-21T03:48:23.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:23 smithi016 ceph-mon[30375]: pgmap v480: 129 pgs: 129 active+clean; 8.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 252 MiB/s wr, 89 op/s 2024-09-21T03:48:25.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:25 smithi134 ceph-mon[35031]: pgmap v481: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 264 MiB/s wr, 96 op/s 2024-09-21T03:48:25.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:25 smithi123 ceph-mon[34673]: pgmap v481: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 264 MiB/s wr, 96 op/s 2024-09-21T03:48:25.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:25 smithi016 ceph-mon[30375]: pgmap v481: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 264 MiB/s wr, 96 op/s 2024-09-21T03:48:27.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:27 smithi134 ceph-mon[35031]: pgmap v482: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 273 MiB/s wr, 162 op/s 2024-09-21T03:48:27.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:27 smithi123 ceph-mon[34673]: pgmap v482: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 273 MiB/s wr, 162 op/s 2024-09-21T03:48:27.417 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:27 smithi016 ceph-mon[30375]: pgmap v482: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 273 MiB/s wr, 162 op/s 2024-09-21T03:48:28.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.370+0000 7f6a6b200640 1 -- 172.21.15.16:0/1682585585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c10d040 msgr2=0x7f6a6c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.372+0000 7f6a6b200640 1 --2- 172.21.15.16:0/1682585585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c10d040 0x7f6a6c10f520 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6a58008410 tx=0x7f6a58005f90 comp rx=0 tx=0).stop 2024-09-21T03:48:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.379+0000 7f6a6b200640 1 -- 172.21.15.16:0/1682585585 shutdown_connections 2024-09-21T03:48:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.379+0000 7f6a6b200640 1 --2- 172.21.15.16:0/1682585585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a6c10fc60 0x7f6a6c112170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.379+0000 7f6a6b200640 1 --2- 172.21.15.16:0/1682585585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c10d040 0x7f6a6c10f520 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.379+0000 7f6a6b200640 1 --2- 172.21.15.16:0/1682585585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a6c10a5a0 0x7f6a6c10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.379+0000 7f6a6b200640 1 -- 172.21.15.16:0/1682585585 >> 172.21.15.16:0/1682585585 conn(0x7f6a6c070640 msgr2=0x7f6a6c072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.380+0000 7f6a6b200640 1 -- 172.21.15.16:0/1682585585 shutdown_connections 2024-09-21T03:48:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.380+0000 7f6a6b200640 1 -- 172.21.15.16:0/1682585585 wait complete. 2024-09-21T03:48:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.383+0000 7f6a6b200640 1 Processor -- start 2024-09-21T03:48:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.384+0000 7f6a6b200640 1 -- start start 2024-09-21T03:48:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.384+0000 7f6a6b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a6c10a5a0 0x7f6a6c1a0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.384+0000 7f6a6b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a6c10d040 0x7f6a6c1a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.385+0000 7f6a6b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c10fc60 0x7f6a6c1a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.385+0000 7f6a69e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a6c10a5a0 0x7f6a6c1a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.385+0000 7f6a69e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a6c10a5a0 0x7f6a6c1a0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37500/0 (socket says 172.21.15.16:37500) 2024-09-21T03:48:28.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.386+0000 7f6a69400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a6c10d040 0x7f6a6c1a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.386+0000 7f6a69e00640 1 -- 172.21.15.16:0/4229574334 learned_addr learned my addr 172.21.15.16:0/4229574334 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.386+0000 7f6a6a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c10fc60 0x7f6a6c1a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.386+0000 7f6a6b200640 1 -- 172.21.15.16:0/4229574334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c115740 con 0x7f6a6c10d040 2024-09-21T03:48:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.386+0000 7f6a6b200640 1 -- 172.21.15.16:0/4229574334 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c1155c0 con 0x7f6a6c10a5a0 2024-09-21T03:48:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.386+0000 7f6a6b200640 1 -- 172.21.15.16:0/4229574334 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c1158c0 con 0x7f6a6c10fc60 2024-09-21T03:48:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.387+0000 7f6a69e00640 1 -- 172.21.15.16:0/4229574334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c10fc60 msgr2=0x7f6a6c1a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.387+0000 7f6a69e00640 1 --2- 172.21.15.16:0/4229574334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c10fc60 0x7f6a6c1a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.387+0000 7f6a69e00640 1 -- 172.21.15.16:0/4229574334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a6c10d040 msgr2=0x7f6a6c1a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.388+0000 7f6a69e00640 1 --2- 172.21.15.16:0/4229574334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a6c10d040 0x7f6a6c1a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.388+0000 7f6a69e00640 1 -- 172.21.15.16:0/4229574334 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6c1a1d00 con 0x7f6a6c10a5a0 2024-09-21T03:48:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.388+0000 7f6a69e00640 1 --2- 172.21.15.16:0/4229574334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a6c10a5a0 0x7f6a6c1a0350 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00adf0 tx=0x7f6a5c007590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.388+0000 7f6a69400640 1 --2- 172.21.15.16:0/4229574334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a6c10d040 0x7f6a6c1a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:48:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.390+0000 7f6a62a00640 1 -- 172.21.15.16:0/4229574334 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c004220 con 0x7f6a6c10a5a0 2024-09-21T03:48:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.390+0000 7f6a62a00640 1 -- 172.21.15.16:0/4229574334 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a5c0043c0 con 0x7f6a6c10a5a0 2024-09-21T03:48:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.390+0000 7f6a6b200640 1 -- 172.21.15.16:0/4229574334 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a6c1a1ff0 con 0x7f6a6c10a5a0 2024-09-21T03:48:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.390+0000 7f6a62a00640 1 -- 172.21.15.16:0/4229574334 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c0035f0 con 0x7f6a6c10a5a0 2024-09-21T03:48:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.390+0000 7f6a6b200640 1 -- 172.21.15.16:0/4229574334 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a6c1a8770 con 0x7f6a6c10a5a0 2024-09-21T03:48:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.402+0000 7f6a77bf9740 1 Processor -- start 2024-09-21T03:48:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.404+0000 7f6a77bf9740 1 -- start start 2024-09-21T03:48:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.405+0000 7f6a77bf9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a984a1b0 0x55e6a9bf9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.405+0000 7f6a77bf9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a98212f0 0x55e6a9bfa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.405+0000 7f6a77bf9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98294d0 0x55e6a9bfac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.405+0000 7f6a77bf9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e6a9b646b0 con 0x55e6a984a1b0 2024-09-21T03:48:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.405+0000 7f6a77bf9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e6a9b53ae0 con 0x55e6a98294d0 2024-09-21T03:48:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.405+0000 7f6a77bf9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e6a9b578f0 con 0x55e6a98212f0 2024-09-21T03:48:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.408+0000 7f6a6a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a984a1b0 0x55e6a9bf9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.410+0000 7f6a69400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98294d0 0x55e6a9bfac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.412+0000 7f6a69400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98294d0 0x55e6a9bfac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37512/0 (socket says 172.21.15.16:37512) 2024-09-21T03:48:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.412+0000 7f6a69400640 1 -- 172.21.15.16:0/4001000971 learned_addr learned my addr 172.21.15.16:0/4001000971 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.412+0000 7f6a69e00640 1 --2- 172.21.15.16:0/4001000971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a98212f0 0x55e6a9bfa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.413+0000 7f6a69400640 1 -- 172.21.15.16:0/4001000971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a98212f0 msgr2=0x55e6a9bfa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.413+0000 7f6a69400640 1 --2- 172.21.15.16:0/4001000971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a98212f0 0x55e6a9bfa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.413+0000 7f6a69400640 1 -- 172.21.15.16:0/4001000971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a984a1b0 msgr2=0x55e6a9bf9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.413+0000 7f6a69400640 1 --2- 172.21.15.16:0/4001000971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a984a1b0 0x55e6a9bf9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.413+0000 7f6a69400640 1 -- 172.21.15.16:0/4001000971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6a9b20780 con 0x55e6a98294d0 2024-09-21T03:48:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.414+0000 7f6a69e00640 1 --2- 172.21.15.16:0/4001000971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a98212f0 0x55e6a9bfa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:48:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.414+0000 7f6a69400640 1 --2- 172.21.15.16:0/4001000971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98294d0 0x55e6a9bfac80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6a58033000 tx=0x7f6a5800af20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df575ae942c93994 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.415+0000 7f6a57000640 1 -- 172.21.15.16:0/4001000971 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a580323f0 con 0x55e6a98294d0 2024-09-21T03:48:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.415+0000 7f6a57000640 1 -- 172.21.15.16:0/4001000971 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a58040070 con 0x55e6a98294d0 2024-09-21T03:48:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.415+0000 7f6a57000640 1 -- 172.21.15.16:0/4001000971 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58032950 con 0x55e6a98294d0 2024-09-21T03:48:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.418+0000 7f6a77bf9740 1 -- 172.21.15.16:0/4001000971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98294d0 msgr2=0x55e6a9bfac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.418+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/4001000971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98294d0 0x55e6a9bfac80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6a58033000 tx=0x7f6a5800af20 comp rx=0 tx=0).stop 2024-09-21T03:48:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.418+0000 7f6a62a00640 1 -- 172.21.15.16:0/4229574334 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6a5c010050 con 0x7f6a6c10a5a0 2024-09-21T03:48:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.419+0000 7f6a77bf9740 1 -- 172.21.15.16:0/4001000971 shutdown_connections 2024-09-21T03:48:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.419+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/4001000971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98294d0 0x55e6a9bfac80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.419+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/4001000971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a98212f0 0x55e6a9bfa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.419+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/4001000971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a984a1b0 0x55e6a9bf9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.419+0000 7f6a77bf9740 1 -- 172.21.15.16:0/4001000971 >> 172.21.15.16:0/4001000971 conn(0x55e6a983e300 msgr2=0x55e6a9becd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.420+0000 7f6a77bf9740 1 -- 172.21.15.16:0/4001000971 shutdown_connections 2024-09-21T03:48:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.420+0000 7f6a77bf9740 1 -- 172.21.15.16:0/4001000971 wait complete. 2024-09-21T03:48:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.422+0000 7f6a77bf9740 1 -- start start 2024-09-21T03:48:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.423+0000 7f6a77bf9740 1 Processor -- start 2024-09-21T03:48:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.427+0000 7f6a77bf9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98212f0 0x55e6a9bf2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.428+0000 7f6a77bf9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a98294d0 0x55e6a9bf2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.428+0000 7f6a6a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98212f0 0x55e6a9bf2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.428+0000 7f6a6a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98212f0 0x55e6a9bf2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37524/0 (socket says 172.21.15.16:37524) 2024-09-21T03:48:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.428+0000 7f6a6a800640 1 -- 172.21.15.16:0/990024238 learned_addr learned my addr 172.21.15.16:0/990024238 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.429+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/990024238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a984a1b0 0x55e6a9bf36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.429+0000 7f6a77bf9740 1 -- 172.21.15.16:0/990024238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e6a9a8bab0 con 0x55e6a984a1b0 2024-09-21T03:48:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.429+0000 7f6a77bf9740 1 -- 172.21.15.16:0/990024238 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e6a9afa570 con 0x55e6a98212f0 2024-09-21T03:48:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.430+0000 7f6a77bf9740 1 -- 172.21.15.16:0/990024238 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e6a9bfb740 con 0x55e6a98294d0 2024-09-21T03:48:28.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.430+0000 7f6a69e00640 1 --2- 172.21.15.16:0/990024238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a98294d0 0x55e6a9bf2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.430+0000 7f6a6a800640 1 -- 172.21.15.16:0/990024238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a98294d0 msgr2=0x55e6a9bf2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:28.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.430+0000 7f6a6a800640 1 --2- 172.21.15.16:0/990024238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a98294d0 0x55e6a9bf2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.430+0000 7f6a6a800640 1 -- 172.21.15.16:0/990024238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a984a1b0 msgr2=0x55e6a9bf36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:28.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.430+0000 7f6a77bf9740 10 client.0 _ll_register_callbacks cb 0x55e6a9b09100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:28.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.431+0000 7f6a6a800640 1 --2- 172.21.15.16:0/990024238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a984a1b0 0x55e6a9bf36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.431+0000 7f6a6a800640 1 -- 172.21.15.16:0/990024238 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6a98b5790 con 0x55e6a98212f0 2024-09-21T03:48:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.432+0000 7f6a77bf9740 1 -- 172.21.15.16:0/990024238 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a58040070 con 0x55e6a98212f0 2024-09-21T03:48:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.432+0000 7f6a6a800640 1 --2- 172.21.15.16:0/990024238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98212f0 0x55e6a9bf2480 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009fd0 tx=0x7f6a6400ce80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c9e048ba2025a12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.432+0000 7f6a4b400640 10 client.25802 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:48:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.433+0000 7f6a4b400640 1 -- 172.21.15.16:0/990024238 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6400bc70 con 0x55e6a98212f0 2024-09-21T03:48:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.433+0000 7f6a4b400640 1 -- 172.21.15.16:0/990024238 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f6a6400be10 con 0x55e6a98212f0 2024-09-21T03:48:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.433+0000 7f6a4b400640 1 -- 172.21.15.16:0/990024238 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6a64026050 con 0x55e6a98212f0 2024-09-21T03:48:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.433+0000 7f6a77bf9740 10 client.25802 fetch_fsmap learned FSMap version 30 2024-09-21T03:48:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.433+0000 7f6a77bf9740 1 -- 172.21.15.16:0/990024238 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55e6a992e620 con 0x55e6a98212f0 2024-09-21T03:48:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.435+0000 7f6a4b400640 1 -- 172.21.15.16:0/990024238 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6a6400b040 con 0x55e6a98212f0 2024-09-21T03:48:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.435+0000 7f6a77bf9740 10 client.25802 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:48:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.435+0000 7f6a77bf9740 10 client.25802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.435+0000 7f6a77bf9740 20 client.25802 populate_metadata read hostname 'smithi016' 2024-09-21T03:48:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.435+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/990024238 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e6a9808d80 0x55e6a9c18f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.435+0000 7f6a77bf9740 4 client.25802 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.435+0000 7f6a77bf9740 1 -- 172.21.15.16:0/990024238 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6a9b66600 con 0x55e6a9808d80 2024-09-21T03:48:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.436+0000 7f6a69e00640 1 --2- 172.21.15.16:0/990024238 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e6a9808d80 0x55e6a9c18f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.438+0000 7f6a69e00640 1 --2- 172.21.15.16:0/990024238 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e6a9808d80 0x55e6a9c18f60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d89e8a99bec9f18 server_cookie=11fa4bea47334561 in_seq=0 out_seq=0 2024-09-21T03:48:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.438+0000 7f6a4b400640 10 client.25802 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.439+0000 7f6a4b400640 1 -- 172.21.15.16:0/990024238 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e6a9b66600 con 0x55e6a9808d80 2024-09-21T03:48:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.439+0000 7f6a4b400640 10 client.25802 handle_command_reply: tid=0 2024-09-21T03:48:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.440+0000 7f6a77bf9740 1 client.25802 shutdown 2024-09-21T03:48:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.440+0000 7f6a77bf9740 1 -- 172.21.15.16:0/990024238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98212f0 msgr2=0x55e6a9bf2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.440+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/990024238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98212f0 0x55e6a9bf2480 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009fd0 tx=0x7f6a6400ce80 comp rx=0 tx=0).stop 2024-09-21T03:48:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.440+0000 7f6a77bf9740 1 -- 172.21.15.16:0/990024238 shutdown_connections 2024-09-21T03:48:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.441+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/990024238 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e6a9808d80 0x55e6a9c18f60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.441+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/990024238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a984a1b0 0x55e6a9bf36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.441+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/990024238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a98294d0 0x55e6a9bf2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.441+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/990024238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98212f0 0x55e6a9bf2480 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.441+0000 7f6a77bf9740 1 -- 172.21.15.16:0/990024238 >> 172.21.15.16:0/990024238 conn(0x55e6a983e300 msgr2=0x55e6a9beb360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.441+0000 7f6a4b400640 0 client.25802 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.441+0000 7f6a77bf9740 1 -- 172.21.15.16:0/990024238 shutdown_connections 2024-09-21T03:48:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.441+0000 7f6a77bf9740 1 -- 172.21.15.16:0/990024238 wait complete. 2024-09-21T03:48:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.442+0000 7f6a77bf9740 20 client.25802 trim_cache size 0 max 16384 2024-09-21T03:48:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.458+0000 7f6a62a00640 1 --2- 172.21.15.16:0/4229574334 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6a500b65f0 0x7f6a500b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.459+0000 7f6a62a00640 1 -- 172.21.15.16:0/4229574334 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f6a5c0e6d40 con 0x7f6a6c10a5a0 2024-09-21T03:48:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.460+0000 7f6a69400640 1 --2- 172.21.15.16:0/4229574334 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6a500b65f0 0x7f6a500b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.464+0000 7f6a69400640 1 --2- 172.21.15.16:0/4229574334 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6a500b65f0 0x7f6a500b8b00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800af90 tx=0x7f6a58005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.475+0000 7f6a77bf9740 1 Processor -- start 2024-09-21T03:48:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.476+0000 7f6a77bf9740 1 -- start start 2024-09-21T03:48:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.476+0000 7f6a77bf9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a98212f0 0x55e6a9bec680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.476+0000 7f6a77bf9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98294d0 0x55e6a9bfe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.476+0000 7f6a77bf9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a984a1b0 0x55e6a9bed180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.477+0000 7f6a69e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a98212f0 0x55e6a9bec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.477+0000 7f6a69400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98294d0 0x55e6a9bfe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.477+0000 7f6a69400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98294d0 0x55e6a9bfe290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37528/0 (socket says 172.21.15.16:37528) 2024-09-21T03:48:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.478+0000 7f6a6a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a984a1b0 0x55e6a9bed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.478+0000 7f6a6a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a984a1b0 0x55e6a9bed180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:48902/0 (socket says 172.21.15.16:48902) 2024-09-21T03:48:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.478+0000 7f6a69e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a98212f0 0x55e6a9bec680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51250/0 (socket says 172.21.15.16:51250) 2024-09-21T03:48:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.478+0000 7f6a77bf9740 1 -- 172.21.15.16:0/3916304979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c00fd00 con 0x55e6a98212f0 2024-09-21T03:48:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.478+0000 7f6a77bf9740 1 -- 172.21.15.16:0/3916304979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c011840 con 0x55e6a98294d0 2024-09-21T03:48:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.478+0000 7f6a77bf9740 1 -- 172.21.15.16:0/3916304979 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c0103d0 con 0x55e6a984a1b0 2024-09-21T03:48:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.478+0000 7f6a69400640 1 -- 172.21.15.16:0/3916304979 learned_addr learned my addr 172.21.15.16:0/3916304979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.479+0000 7f6a69400640 1 -- 172.21.15.16:0/3916304979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a984a1b0 msgr2=0x55e6a9bed180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.479+0000 7f6a69400640 1 --2- 172.21.15.16:0/3916304979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a984a1b0 0x55e6a9bed180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.479+0000 7f6a69400640 1 -- 172.21.15.16:0/3916304979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a98212f0 msgr2=0x55e6a9bec680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.479+0000 7f6a69400640 1 --2- 172.21.15.16:0/3916304979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a98212f0 0x55e6a9bec680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.480+0000 7f6a69400640 1 -- 172.21.15.16:0/3916304979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6a9c01360 con 0x55e6a98294d0 2024-09-21T03:48:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.481+0000 7f6a69400640 1 --2- 172.21.15.16:0/3916304979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98294d0 0x55e6a9bfe290 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6a58004650 tx=0x7f6a58043000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=482c350bfbedca19 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.483+0000 7f6a56600640 1 -- 172.21.15.16:0/3916304979 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5804c040 con 0x55e6a98294d0 2024-09-21T03:48:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.483+0000 7f6a56600640 1 -- 172.21.15.16:0/3916304979 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a58055070 con 0x55e6a98294d0 2024-09-21T03:48:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.483+0000 7f6a56600640 1 -- 172.21.15.16:0/3916304979 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5802ad50 con 0x55e6a98294d0 2024-09-21T03:48:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.485+0000 7f6a77bf9740 1 -- 172.21.15.16:0/3916304979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98294d0 msgr2=0x55e6a9bfe290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.485+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/3916304979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98294d0 0x55e6a9bfe290 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6a58004650 tx=0x7f6a58043000 comp rx=0 tx=0).stop 2024-09-21T03:48:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.487+0000 7f6a77bf9740 1 -- 172.21.15.16:0/3916304979 shutdown_connections 2024-09-21T03:48:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.487+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/3916304979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a984a1b0 0x55e6a9bed180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.487+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/3916304979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98294d0 0x55e6a9bfe290 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.487+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/3916304979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a98212f0 0x55e6a9bec680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.487+0000 7f6a77bf9740 1 -- 172.21.15.16:0/3916304979 >> 172.21.15.16:0/3916304979 conn(0x55e6a983e300 msgr2=0x55e6a9bfdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.487+0000 7f6a77bf9740 1 -- 172.21.15.16:0/3916304979 shutdown_connections 2024-09-21T03:48:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.487+0000 7f6a77bf9740 1 -- 172.21.15.16:0/3916304979 wait complete. 2024-09-21T03:48:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.488+0000 7f6a77bf9740 1 -- start start 2024-09-21T03:48:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.488+0000 7f6a77bf9740 1 Processor -- start 2024-09-21T03:48:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.492+0000 7f6a77bf9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98212f0 0x55e6a9c2a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.493+0000 7f6a69e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98212f0 0x55e6a9c2a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.493+0000 7f6a69e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98212f0 0x55e6a9c2a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37532/0 (socket says 172.21.15.16:37532) 2024-09-21T03:48:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.493+0000 7f6a69e00640 1 -- 172.21.15.16:0/342844259 learned_addr learned my addr 172.21.15.16:0/342844259 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.493+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/342844259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a98294d0 0x55e6a9c2a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.494+0000 7f6a69400640 1 --2- 172.21.15.16:0/342844259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a98294d0 0x55e6a9c2a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.494+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/342844259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a984a1b0 0x55e6a9c28d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.494+0000 7f6a6a800640 1 --2- 172.21.15.16:0/342844259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a984a1b0 0x55e6a9c28d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.495+0000 7f6a77bf9740 1 -- 172.21.15.16:0/342844259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c011170 con 0x55e6a984a1b0 2024-09-21T03:48:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.495+0000 7f6a77bf9740 1 -- 172.21.15.16:0/342844259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c010aa0 con 0x55e6a98212f0 2024-09-21T03:48:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.495+0000 7f6a77bf9740 1 -- 172.21.15.16:0/342844259 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c00dce0 con 0x55e6a98294d0 2024-09-21T03:48:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.495+0000 7f6a77bf9740 10 client.0 _ll_register_callbacks cb 0x7f6a640097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.495+0000 7f6a69400640 1 -- 172.21.15.16:0/342844259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98212f0 msgr2=0x55e6a9c2a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.495+0000 7f6a69400640 1 --2- 172.21.15.16:0/342844259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98212f0 0x55e6a9c2a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.495+0000 7f6a69400640 1 -- 172.21.15.16:0/342844259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a984a1b0 msgr2=0x55e6a9c28d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.495+0000 7f6a69400640 1 --2- 172.21.15.16:0/342844259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a984a1b0 0x55e6a9c28d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.495+0000 7f6a69400640 1 -- 172.21.15.16:0/342844259 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6a9c2ae90 con 0x55e6a98294d0 2024-09-21T03:48:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.496+0000 7f6a69400640 1 --2- 172.21.15.16:0/342844259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a98294d0 0x55e6a9c2a5c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6a58005940 tx=0x7f6a5800a770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=288dc199430bee1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.496+0000 7f6a4b400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:48:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.496+0000 7f6a4b400640 1 -- 172.21.15.16:0/342844259 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5804c040 con 0x55e6a98294d0 2024-09-21T03:48:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.496+0000 7f6a4b400640 1 -- 172.21.15.16:0/342844259 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f6a5802acd0 con 0x55e6a98294d0 2024-09-21T03:48:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.497+0000 7f6a77bf9740 1 -- 172.21.15.16:0/342844259 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a58055070 con 0x55e6a98294d0 2024-09-21T03:48:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.498+0000 7f6a4b400640 1 -- 172.21.15.16:0/342844259 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6a5805f050 con 0x55e6a98294d0 2024-09-21T03:48:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.498+0000 7f6a77bf9740 10 client.25765 fetch_fsmap learned FSMap version 30 2024-09-21T03:48:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.498+0000 7f6a77bf9740 1 -- 172.21.15.16:0/342844259 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e6a9c83e70 con 0x55e6a98294d0 2024-09-21T03:48:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.499+0000 7f6a4b400640 1 -- 172.21.15.16:0/342844259 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a58055070 con 0x55e6a98294d0 2024-09-21T03:48:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.500+0000 7f6a4b400640 1 -- 172.21.15.16:0/342844259 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6a580054e0 con 0x55e6a98294d0 2024-09-21T03:48:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.500+0000 7f6a77bf9740 10 client.25765 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:48:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.500+0000 7f6a77bf9740 10 client.25765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.500+0000 7f6a77bf9740 20 client.25765 populate_metadata read hostname 'smithi016' 2024-09-21T03:48:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.500+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/342844259 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e6a97de1c0 0x55e6a9c86300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.500+0000 7f6a77bf9740 4 client.25765 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:48:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.500+0000 7f6a77bf9740 1 -- 172.21.15.16:0/342844259 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6a9b620b0 con 0x55e6a97de1c0 2024-09-21T03:48:28.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.501+0000 7f6a69e00640 1 --2- 172.21.15.16:0/342844259 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e6a97de1c0 0x55e6a9c86300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:28.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.502+0000 7f6a69e00640 1 --2- 172.21.15.16:0/342844259 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e6a97de1c0 0x55e6a9c86300 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b72322186a9d2da0 server_cookie=1abec341d90f942a in_seq=0 out_seq=0 2024-09-21T03:48:28.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.502+0000 7f6a4b400640 10 client.25765 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.703+0000 7f6a4b400640 1 -- 172.21.15.16:0/342844259 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e6a9b620b0 con 0x55e6a97de1c0 2024-09-21T03:48:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.703+0000 7f6a4b400640 10 client.25765 handle_command_reply: tid=0 2024-09-21T03:48:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.703+0000 7f6a77bf9740 1 client.25765 shutdown 2024-09-21T03:48:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.704+0000 7f6a77bf9740 1 -- 172.21.15.16:0/342844259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a98294d0 msgr2=0x55e6a9c2a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.704+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/342844259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a98294d0 0x55e6a9c2a5c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6a58005940 tx=0x7f6a5800a770 comp rx=0 tx=0).stop 2024-09-21T03:48:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.704+0000 7f6a77bf9740 1 -- 172.21.15.16:0/342844259 shutdown_connections 2024-09-21T03:48:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.704+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/342844259 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e6a97de1c0 0x55e6a9c86300 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.705+0000 7f6a4b400640 0 client.25765 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.705+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/342844259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a984a1b0 0x55e6a9c28d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.705+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/342844259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6a98294d0 0x55e6a9c2a5c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.705+0000 7f6a77bf9740 1 --2- 172.21.15.16:0/342844259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6a98212f0 0x55e6a9c2a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.705+0000 7f6a77bf9740 1 -- 172.21.15.16:0/342844259 >> 172.21.15.16:0/342844259 conn(0x55e6a983e300 msgr2=0x55e6a9bf0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.705+0000 7f6a77bf9740 1 -- 172.21.15.16:0/342844259 shutdown_connections 2024-09-21T03:48:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.706+0000 7f6a77bf9740 1 -- 172.21.15.16:0/342844259 wait complete. 2024-09-21T03:48:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.706+0000 7f6a77bf9740 20 client.25765 trim_cache size 0 max 16384 2024-09-21T03:48:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:48:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:48:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:48:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:48:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.707+0000 7f6a6b200640 1 -- 172.21.15.16:0/4229574334 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6a500b65f0 msgr2=0x7f6a500b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.707+0000 7f6a6b200640 1 --2- 172.21.15.16:0/4229574334 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6a500b65f0 0x7f6a500b8b00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800af90 tx=0x7f6a58005f90 comp rx=0 tx=0).stop 2024-09-21T03:48:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.707+0000 7f6a6b200640 1 -- 172.21.15.16:0/4229574334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a6c10a5a0 msgr2=0x7f6a6c1a0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.707+0000 7f6a6b200640 1 --2- 172.21.15.16:0/4229574334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a6c10a5a0 0x7f6a6c1a0350 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00adf0 tx=0x7f6a5c007590 comp rx=0 tx=0).stop 2024-09-21T03:48:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.708+0000 7f6a6b200640 1 -- 172.21.15.16:0/4229574334 shutdown_connections 2024-09-21T03:48:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.708+0000 7f6a6b200640 1 --2- 172.21.15.16:0/4229574334 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6a500b65f0 0x7f6a500b8b00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.708+0000 7f6a6b200640 1 --2- 172.21.15.16:0/4229574334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a6c10fc60 0x7f6a6c1a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.708+0000 7f6a6b200640 1 --2- 172.21.15.16:0/4229574334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a6c10d040 0x7f6a6c1a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.708+0000 7f6a6b200640 1 --2- 172.21.15.16:0/4229574334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a6c10a5a0 0x7f6a6c1a0350 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.708+0000 7f6a6b200640 1 -- 172.21.15.16:0/4229574334 >> 172.21.15.16:0/4229574334 conn(0x7f6a6c070640 msgr2=0x7f6a6c10b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.708+0000 7f6a6b200640 1 -- 172.21.15.16:0/4229574334 shutdown_connections 2024-09-21T03:48:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:28.709+0000 7f6a6b200640 1 -- 172.21.15.16:0/4229574334 wait complete. 2024-09-21T03:48:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:29 smithi016 ceph-mon[30375]: pgmap v483: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 268 MiB/s wr, 279 op/s 2024-09-21T03:48:29.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:29 smithi134 ceph-mon[35031]: pgmap v483: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 268 MiB/s wr, 279 op/s 2024-09-21T03:48:29.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:29 smithi123 ceph-mon[34673]: pgmap v483: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 268 MiB/s wr, 279 op/s 2024-09-21T03:48:31.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:31 smithi134 ceph-mon[35031]: pgmap v484: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 249 MiB/s wr, 411 op/s 2024-09-21T03:48:31.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:31 smithi123 ceph-mon[34673]: pgmap v484: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 249 MiB/s wr, 411 op/s 2024-09-21T03:48:31.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:31 smithi016 ceph-mon[30375]: pgmap v484: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 249 MiB/s wr, 411 op/s 2024-09-21T03:48:33.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:33 smithi123 ceph-mon[34673]: pgmap v485: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 257 MiB/s wr, 612 op/s 2024-09-21T03:48:33.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:33 smithi016 ceph-mon[30375]: pgmap v485: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 257 MiB/s wr, 612 op/s 2024-09-21T03:48:33.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:33 smithi134 ceph-mon[35031]: pgmap v485: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 257 MiB/s wr, 612 op/s 2024-09-21T03:48:35.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:35 smithi123 ceph-mon[34673]: pgmap v486: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 255 MiB/s wr, 774 op/s 2024-09-21T03:48:35.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:35 smithi016 ceph-mon[30375]: pgmap v486: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 255 MiB/s wr, 774 op/s 2024-09-21T03:48:35.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:35 smithi134 ceph-mon[35031]: pgmap v486: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 255 MiB/s wr, 774 op/s 2024-09-21T03:48:36.699 INFO:tasks.quiescer.fs.[cephfs]:Total duration (55.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T03:48:36.700 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.8 2024-09-21T03:48:36.700 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:48:37.137 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:37 smithi016 ceph-mon[30375]: pgmap v487: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 947 op/s 2024-09-21T03:48:37.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:37 smithi123 ceph-mon[34673]: pgmap v487: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 947 op/s 2024-09-21T03:48:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:37.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:37 smithi134 ceph-mon[35031]: pgmap v487: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 947 op/s 2024-09-21T03:48:38.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:38 smithi016 ceph-mon[30375]: pgmap v488: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 256 MiB/s wr, 1.06k op/s 2024-09-21T03:48:38.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:38 smithi134 ceph-mon[35031]: pgmap v488: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 256 MiB/s wr, 1.06k op/s 2024-09-21T03:48:38.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:38 smithi123 ceph-mon[34673]: pgmap v488: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 256 MiB/s wr, 1.06k op/s 2024-09-21T03:48:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.343+0000 7f427ce00640 1 -- 172.21.15.16:0/1896609969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4278077940 msgr2=0x7f4278077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.344+0000 7f427ce00640 1 --2- 172.21.15.16:0/1896609969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4278077940 0x7f4278077d50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4270009ca0 tx=0x7f427002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:48:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.346+0000 7f427ce00640 1 -- 172.21.15.16:0/1896609969 shutdown_connections 2024-09-21T03:48:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.346+0000 7f427ce00640 1 --2- 172.21.15.16:0/1896609969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4278114580 0x7f42781169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.346+0000 7f427ce00640 1 --2- 172.21.15.16:0/1896609969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4278078290 0x7f4278114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.346+0000 7f427ce00640 1 --2- 172.21.15.16:0/1896609969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4278077940 0x7f4278077d50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.346+0000 7f427ce00640 1 -- 172.21.15.16:0/1896609969 >> 172.21.15.16:0/1896609969 conn(0x7f4278101630 msgr2=0x7f4278103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.347+0000 7f427ce00640 1 -- 172.21.15.16:0/1896609969 shutdown_connections 2024-09-21T03:48:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.347+0000 7f427ce00640 1 -- 172.21.15.16:0/1896609969 wait complete. 2024-09-21T03:48:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.348+0000 7f427ce00640 1 Processor -- start 2024-09-21T03:48:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.349+0000 7f427ce00640 1 -- start start 2024-09-21T03:48:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.350+0000 7f427ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4278077940 0x7f4278113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.351+0000 7f427ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4278078290 0x7f427810d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.351+0000 7f4277400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4278077940 0x7f4278113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.351+0000 7f427ce00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4278114580 0x7f427810d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.352+0000 7f4276a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4278078290 0x7f427810d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.352+0000 7f4277400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4278077940 0x7f4278113df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58680/0 (socket says 172.21.15.16:58680) 2024-09-21T03:48:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.352+0000 7f4276a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4278078290 0x7f427810d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:33822/0 (socket says 172.21.15.16:33822) 2024-09-21T03:48:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.353+0000 7f427ce00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4278119f30 con 0x7f4278077940 2024-09-21T03:48:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.353+0000 7f427ce00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4278119db0 con 0x7f4278078290 2024-09-21T03:48:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.353+0000 7f427ce00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f427811a0b0 con 0x7f4278114580 2024-09-21T03:48:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.353+0000 7f4277e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4278114580 0x7f427810d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.353+0000 7f4277e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4278114580 0x7f427810d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:37672/0 (socket says 172.21.15.16:37672) 2024-09-21T03:48:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.353+0000 7f4277e00640 1 -- 172.21.15.16:0/2943077272 learned_addr learned my addr 172.21.15.16:0/2943077272 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.354+0000 7f4277e00640 1 -- 172.21.15.16:0/2943077272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4278078290 msgr2=0x7f427810d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.354+0000 7f4277e00640 1 --2- 172.21.15.16:0/2943077272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4278078290 0x7f427810d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.354+0000 7f4277e00640 1 -- 172.21.15.16:0/2943077272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4278077940 msgr2=0x7f4278113df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.354+0000 7f4277e00640 1 --2- 172.21.15.16:0/2943077272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4278077940 0x7f4278113df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.354+0000 7f4277e00640 1 -- 172.21.15.16:0/2943077272 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427810de60 con 0x7f4278114580 2024-09-21T03:48:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.354+0000 7f4277e00640 1 --2- 172.21.15.16:0/2943077272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4278114580 0x7f427810d640 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f426400e9b0 tx=0x7f426400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.354+0000 7f426fe00640 1 -- 172.21.15.16:0/2943077272 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426400cd80 con 0x7f4278114580 2024-09-21T03:48:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.354+0000 7f427ce00640 1 -- 172.21.15.16:0/2943077272 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4278112b20 con 0x7f4278114580 2024-09-21T03:48:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.354+0000 7f426fe00640 1 -- 172.21.15.16:0/2943077272 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42640035f0 con 0x7f4278114580 2024-09-21T03:48:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.355+0000 7f426fe00640 1 -- 172.21.15.16:0/2943077272 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4264010670 con 0x7f4278114580 2024-09-21T03:48:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.355+0000 7f427ce00640 1 -- 172.21.15.16:0/2943077272 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4278113060 con 0x7f4278114580 2024-09-21T03:48:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.356+0000 7f427ce00640 1 -- 172.21.15.16:0/2943077272 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4278077d50 con 0x7f4278114580 2024-09-21T03:48:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.358+0000 7f426fe00640 1 -- 172.21.15.16:0/2943077272 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4264004000 con 0x7f4278114580 2024-09-21T03:48:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.361+0000 7f426fe00640 1 --2- 172.21.15.16:0/2943077272 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f425c0b65d0 0x7f425c0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.361+0000 7f426fe00640 1 -- 172.21.15.16:0/2943077272 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f42640e7a00 con 0x7f4278114580 2024-09-21T03:48:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.363+0000 7f4277400640 1 --2- 172.21.15.16:0/2943077272 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f425c0b65d0 0x7f425c0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.365+0000 7f4277400640 1 --2- 172.21.15.16:0/2943077272 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f425c0b65d0 0x7f425c0b8ae0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4270006010 tx=0x7f427002f390 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.366+0000 7f426fe00640 1 -- 172.21.15.16:0/2943077272 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f42640adf10 con 0x7f4278114580 2024-09-21T03:48:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.884+0000 7f427ce00640 1 -- 172.21.15.16:0/2943077272 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f427810e3c0 con 0x7f4278114580 2024-09-21T03:48:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.885+0000 7f426fe00640 1 -- 172.21.15.16:0/2943077272 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f427810e3c0 con 0x7f4278114580 2024-09-21T03:48:39.895 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:48:39.896 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:48:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:48:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.891+0000 7f427ce00640 1 -- 172.21.15.16:0/2943077272 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f425c0b65d0 msgr2=0x7f425c0b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.891+0000 7f427ce00640 1 --2- 172.21.15.16:0/2943077272 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f425c0b65d0 0x7f425c0b8ae0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4270006010 tx=0x7f427002f390 comp rx=0 tx=0).stop 2024-09-21T03:48:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.891+0000 7f427ce00640 1 -- 172.21.15.16:0/2943077272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4278114580 msgr2=0x7f427810d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.891+0000 7f427ce00640 1 --2- 172.21.15.16:0/2943077272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4278114580 0x7f427810d640 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f426400e9b0 tx=0x7f426400ee70 comp rx=0 tx=0).stop 2024-09-21T03:48:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.892+0000 7f427ce00640 1 -- 172.21.15.16:0/2943077272 shutdown_connections 2024-09-21T03:48:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.892+0000 7f427ce00640 1 --2- 172.21.15.16:0/2943077272 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f425c0b65d0 0x7f425c0b8ae0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.892+0000 7f427ce00640 1 --2- 172.21.15.16:0/2943077272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4278114580 0x7f427810d640 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.892+0000 7f427ce00640 1 --2- 172.21.15.16:0/2943077272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4278078290 0x7f427810d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.892+0000 7f427ce00640 1 --2- 172.21.15.16:0/2943077272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4278077940 0x7f4278113df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:39.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.892+0000 7f427ce00640 1 -- 172.21.15.16:0/2943077272 >> 172.21.15.16:0/2943077272 conn(0x7f4278101630 msgr2=0x7f4278116630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:39.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.892+0000 7f427ce00640 1 -- 172.21.15.16:0/2943077272 shutdown_connections 2024-09-21T03:48:39.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:39.892+0000 7f427ce00640 1 -- 172.21.15.16:0/2943077272 wait complete. 2024-09-21T03:48:40.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:40 smithi134 ceph-mon[35031]: pgmap v489: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 234 MiB/s wr, 1.02k op/s 2024-09-21T03:48:40.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:40 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2943077272' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:40.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:40 smithi123 ceph-mon[34673]: pgmap v489: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 234 MiB/s wr, 1.02k op/s 2024-09-21T03:48:40.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:40 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2943077272' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:40.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:40 smithi016 ceph-mon[30375]: pgmap v489: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 234 MiB/s wr, 1.02k op/s 2024-09-21T03:48:40.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:40 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2943077272' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:48:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.120+0000 7f6928a00640 1 -- 172.21.15.16:0/3513906043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f692410b700 msgr2=0x7f69241120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:42.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.120+0000 7f6928a00640 1 --2- 172.21.15.16:0/3513906043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f692410b700 0x7f69241120a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f691c009ca0 tx=0x7f691c02a8b0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.123+0000 7f6928a00640 1 -- 172.21.15.16:0/3513906043 shutdown_connections 2024-09-21T03:48:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.123+0000 7f6928a00640 1 --2- 172.21.15.16:0/3513906043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f692410b700 0x7f69241120a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.123+0000 7f6928a00640 1 --2- 172.21.15.16:0/3513906043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f692410ad60 0x7f692410b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.123+0000 7f6928a00640 1 --2- 172.21.15.16:0/3513906043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924109ce0 0x7f692410a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.123+0000 7f6928a00640 1 -- 172.21.15.16:0/3513906043 >> 172.21.15.16:0/3513906043 conn(0x7f6924079060 msgr2=0x7f692407b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.129+0000 7f6928a00640 1 -- 172.21.15.16:0/3513906043 shutdown_connections 2024-09-21T03:48:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.129+0000 7f6928a00640 1 -- 172.21.15.16:0/3513906043 wait complete. 2024-09-21T03:48:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.131+0000 7f6928a00640 1 Processor -- start 2024-09-21T03:48:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.131+0000 7f6928a00640 1 -- start start 2024-09-21T03:48:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.132+0000 7f6928a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924109ce0 0x7f69240721f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.132+0000 7f6928a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f692410ad60 0x7f6924072860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.133+0000 7f6928a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f692410b700 0x7f6924073410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.133+0000 7f6928a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6924115960 con 0x7f6924109ce0 2024-09-21T03:48:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.133+0000 7f6922800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f692410b700 0x7f6924073410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.133+0000 7f6922800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f692410b700 0x7f6924073410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:60754/0 (socket says 172.21.15.16:60754) 2024-09-21T03:48:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.133+0000 7f6922800640 1 -- 172.21.15.16:0/4031656325 learned_addr learned my addr 172.21.15.16:0/4031656325 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.133+0000 7f6928a00640 1 -- 172.21.15.16:0/4031656325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69241157e0 con 0x7f692410ad60 2024-09-21T03:48:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.133+0000 7f6921400640 1 --2- 172.21.15.16:0/4031656325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f692410ad60 0x7f6924072860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.133+0000 7f6928a00640 1 -- 172.21.15.16:0/4031656325 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6924115ae0 con 0x7f692410b700 2024-09-21T03:48:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.134+0000 7f6921e00640 1 --2- 172.21.15.16:0/4031656325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924109ce0 0x7f69240721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.136+0000 7f6921e00640 1 -- 172.21.15.16:0/4031656325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f692410b700 msgr2=0x7f6924073410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.136+0000 7f6921e00640 1 --2- 172.21.15.16:0/4031656325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f692410b700 0x7f6924073410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.136+0000 7f6921e00640 1 -- 172.21.15.16:0/4031656325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f692410ad60 msgr2=0x7f6924072860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.136+0000 7f6921e00640 1 --2- 172.21.15.16:0/4031656325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f692410ad60 0x7f6924072860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.136+0000 7f6921e00640 1 -- 172.21.15.16:0/4031656325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6924073ba0 con 0x7f6924109ce0 2024-09-21T03:48:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.137+0000 7f6921400640 1 --2- 172.21.15.16:0/4031656325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f692410ad60 0x7f6924072860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:48:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.137+0000 7f6922800640 1 --2- 172.21.15.16:0/4031656325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f692410b700 0x7f6924073410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:48:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.139+0000 7f6921e00640 1 --2- 172.21.15.16:0/4031656325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924109ce0 0x7f69240721f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f691000c970 tx=0x7f691000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.139+0000 7f691aa00640 1 -- 172.21.15.16:0/4031656325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910007bb0 con 0x7f6924109ce0 2024-09-21T03:48:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.139+0000 7f691aa00640 1 -- 172.21.15.16:0/4031656325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6910007d50 con 0x7f6924109ce0 2024-09-21T03:48:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.139+0000 7f691aa00640 1 -- 172.21.15.16:0/4031656325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910018660 con 0x7f6924109ce0 2024-09-21T03:48:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.140+0000 7f6928a00640 1 -- 172.21.15.16:0/4031656325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6924073e90 con 0x7f6924109ce0 2024-09-21T03:48:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.140+0000 7f6928a00640 1 -- 172.21.15.16:0/4031656325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6924074400 con 0x7f6924109ce0 2024-09-21T03:48:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.146+0000 7f692f722740 1 Processor -- start 2024-09-21T03:48:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.147+0000 7f692f722740 1 -- start start 2024-09-21T03:48:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.148+0000 7f692f722740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 0x557e4ba430d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.148+0000 7f692f722740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b6732f0 0x557e4ba43740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.148+0000 7f692f722740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b67b4d0 0x557e4ba44290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.148+0000 7f692f722740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557e4b9b66b0 con 0x557e4b67b4d0 2024-09-21T03:48:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.148+0000 7f692f722740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557e4b9a5ae0 con 0x557e4b69c1b0 2024-09-21T03:48:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.148+0000 7f692f722740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557e4b9a98f0 con 0x557e4b6732f0 2024-09-21T03:48:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.148+0000 7f6922800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 0x557e4ba430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.148+0000 7f6922800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 0x557e4ba430d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:58530/0 (socket says 172.21.15.16:58530) 2024-09-21T03:48:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.148+0000 7f6922800640 1 -- 172.21.15.16:0/2153266078 learned_addr learned my addr 172.21.15.16:0/2153266078 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.148+0000 7f6921400640 1 --2- 172.21.15.16:0/2153266078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b67b4d0 0x557e4ba44290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.149+0000 7f6922800640 1 -- 172.21.15.16:0/2153266078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b6732f0 msgr2=0x557e4ba43740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:48:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.149+0000 7f6922800640 1 --2- 172.21.15.16:0/2153266078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b6732f0 0x557e4ba43740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.149+0000 7f6922800640 1 -- 172.21.15.16:0/2153266078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b67b4d0 msgr2=0x557e4ba44290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.149+0000 7f6922800640 1 --2- 172.21.15.16:0/2153266078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b67b4d0 0x557e4ba44290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.149+0000 7f6922800640 1 -- 172.21.15.16:0/2153266078 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e4b972780 con 0x557e4b69c1b0 2024-09-21T03:48:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.149+0000 7f6922800640 1 --2- 172.21.15.16:0/2153266078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 0x557e4ba430d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f691c030e00 tx=0x7f691c0303a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b5ba7a40f49282f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.150+0000 7f690f000640 1 -- 172.21.15.16:0/2153266078 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c030550 con 0x557e4b69c1b0 2024-09-21T03:48:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.150+0000 7f690f000640 1 -- 172.21.15.16:0/2153266078 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691c036030 con 0x557e4b69c1b0 2024-09-21T03:48:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.150+0000 7f690f000640 1 -- 172.21.15.16:0/2153266078 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c0316a0 con 0x557e4b69c1b0 2024-09-21T03:48:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.150+0000 7f691aa00640 1 -- 172.21.15.16:0/4031656325 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f69100188c0 con 0x7f6924109ce0 2024-09-21T03:48:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.151+0000 7f692f722740 1 -- 172.21.15.16:0/2153266078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 msgr2=0x557e4ba430d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.151+0000 7f692f722740 1 --2- 172.21.15.16:0/2153266078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 0x557e4ba430d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f691c030e00 tx=0x7f691c0303a0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.152+0000 7f692f722740 1 -- 172.21.15.16:0/2153266078 shutdown_connections 2024-09-21T03:48:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.152+0000 7f692f722740 1 --2- 172.21.15.16:0/2153266078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b67b4d0 0x557e4ba44290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.152+0000 7f692f722740 1 --2- 172.21.15.16:0/2153266078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b6732f0 0x557e4ba43740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.152+0000 7f692f722740 1 --2- 172.21.15.16:0/2153266078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 0x557e4ba430d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.152+0000 7f692f722740 1 -- 172.21.15.16:0/2153266078 >> 172.21.15.16:0/2153266078 conn(0x557e4b690300 msgr2=0x557e4ba3eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.152+0000 7f692f722740 1 -- 172.21.15.16:0/2153266078 shutdown_connections 2024-09-21T03:48:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.153+0000 7f692f722740 1 -- 172.21.15.16:0/2153266078 wait complete. 2024-09-21T03:48:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.154+0000 7f692f722740 1 -- start start 2024-09-21T03:48:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.154+0000 7f692f722740 1 Processor -- start 2024-09-21T03:48:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.154+0000 7f691aa00640 1 --2- 172.21.15.16:0/4031656325 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f69080b64a0 0x7f69080b89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.155+0000 7f691aa00640 1 -- 172.21.15.16:0/4031656325 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f69100e76d0 con 0x7f6924109ce0 2024-09-21T03:48:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.156+0000 7f6922800640 1 --2- 172.21.15.16:0/4031656325 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f69080b64a0 0x7f69080b89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.157+0000 7f6922800640 1 --2- 172.21.15.16:0/4031656325 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f69080b64a0 0x7f69080b89b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f691c030dd0 tx=0x7f691c030820 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.157+0000 7f692f722740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b6732f0 0x557e4ba4c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.157+0000 7f692f722740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b67b4d0 0x557e4ba4ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.157+0000 7f692f722740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b69c1b0 0x557e4ba4d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.157+0000 7f692f722740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557e4b8ddab0 con 0x557e4b69c1b0 2024-09-21T03:48:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.157+0000 7f692f722740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557e4b94c570 con 0x557e4b6732f0 2024-09-21T03:48:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.157+0000 7f692f722740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557e4ba44d50 con 0x557e4b67b4d0 2024-09-21T03:48:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.157+0000 7f692f722740 10 client.0 _ll_register_callbacks cb 0x557e4b95b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.157+0000 7f6921e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b6732f0 0x557e4ba4c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.157+0000 7f6921e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b6732f0 0x557e4ba4c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:58544/0 (socket says 172.21.15.16:58544) 2024-09-21T03:48:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.157+0000 7f6921e00640 1 -- 172.21.15.16:0/2325235870 learned_addr learned my addr 172.21.15.16:0/2325235870 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.157+0000 7f6922800640 1 --2- 172.21.15.16:0/2325235870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b69c1b0 0x557e4ba4d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.158+0000 7f6921e00640 1 -- 172.21.15.16:0/2325235870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b67b4d0 msgr2=0x557e4ba4ce50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:48:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.158+0000 7f6921e00640 1 --2- 172.21.15.16:0/2325235870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b67b4d0 0x557e4ba4ce50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.158+0000 7f6921e00640 1 -- 172.21.15.16:0/2325235870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b69c1b0 msgr2=0x557e4ba4d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.158+0000 7f6921e00640 1 --2- 172.21.15.16:0/2325235870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b69c1b0 0x557e4ba4d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.158+0000 7f6921e00640 1 -- 172.21.15.16:0/2325235870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e4b707790 con 0x557e4b6732f0 2024-09-21T03:48:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.158+0000 7f692f722740 1 -- 172.21.15.16:0/2325235870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f691c036030 con 0x557e4b6732f0 2024-09-21T03:48:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.158+0000 7f6921e00640 1 --2- 172.21.15.16:0/2325235870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b6732f0 0x557e4ba4c790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f69100aa6c0 tx=0x7f69100b6d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e91a0593b263d860 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.158+0000 7f6903400640 10 client.25826 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:48:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.158+0000 7f6903400640 1 -- 172.21.15.16:0/2325235870 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910094de0 con 0x557e4b6732f0 2024-09-21T03:48:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.159+0000 7f6903400640 1 -- 172.21.15.16:0/2325235870 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f69100a7b50 con 0x557e4b6732f0 2024-09-21T03:48:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.159+0000 7f6903400640 1 -- 172.21.15.16:0/2325235870 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69100a7d90 con 0x557e4b6732f0 2024-09-21T03:48:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.159+0000 7f692f722740 10 client.25826 fetch_fsmap learned FSMap version 30 2024-09-21T03:48:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.159+0000 7f692f722740 1 -- 172.21.15.16:0/2325235870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=30}) -- 0x557e4b780620 con 0x557e4b6732f0 2024-09-21T03:48:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.160+0000 7f6903400640 1 -- 172.21.15.16:0/2325235870 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f69100ac6b0 con 0x557e4b6732f0 2024-09-21T03:48:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.160+0000 7f692f722740 10 client.25826 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:48:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.160+0000 7f692f722740 10 client.25826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.160+0000 7f692f722740 20 client.25826 populate_metadata read hostname 'smithi016' 2024-09-21T03:48:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.160+0000 7f692f722740 1 --2- 172.21.15.16:0/2325235870 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557e4b65ad80 0x557e4ba5a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.160+0000 7f692f722740 4 client.25826 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.160+0000 7f692f722740 1 -- 172.21.15.16:0/2325235870 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e4b9b8600 con 0x557e4b65ad80 2024-09-21T03:48:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.160+0000 7f6921400640 1 --2- 172.21.15.16:0/2325235870 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557e4b65ad80 0x557e4ba5a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.161+0000 7f6921400640 1 --2- 172.21.15.16:0/2325235870 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557e4b65ad80 0x557e4ba5a210 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d6578eb6c36dcf9 server_cookie=2fa324a840eb44b9 in_seq=0 out_seq=0 2024-09-21T03:48:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.161+0000 7f6903400640 10 client.25826 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.163+0000 7f6903400640 1 -- 172.21.15.16:0/2325235870 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557e4b9b8600 con 0x557e4b65ad80 2024-09-21T03:48:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.163+0000 7f6903400640 10 client.25826 handle_command_reply: tid=0 2024-09-21T03:48:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.163+0000 7f692f722740 1 client.25826 shutdown 2024-09-21T03:48:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.164+0000 7f692f722740 1 -- 172.21.15.16:0/2325235870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b6732f0 msgr2=0x557e4ba4c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.164+0000 7f692f722740 1 --2- 172.21.15.16:0/2325235870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b6732f0 0x557e4ba4c790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f69100aa6c0 tx=0x7f69100b6d50 comp rx=0 tx=0).stop 2024-09-21T03:48:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.164+0000 7f692f722740 1 -- 172.21.15.16:0/2325235870 shutdown_connections 2024-09-21T03:48:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.164+0000 7f692f722740 1 --2- 172.21.15.16:0/2325235870 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557e4b65ad80 0x557e4ba5a210 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.165+0000 7f692f722740 1 --2- 172.21.15.16:0/2325235870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b69c1b0 0x557e4ba4d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.165+0000 7f692f722740 1 --2- 172.21.15.16:0/2325235870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b67b4d0 0x557e4ba4ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.165+0000 7f692f722740 1 --2- 172.21.15.16:0/2325235870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b6732f0 0x557e4ba4c790 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.165+0000 7f692f722740 1 -- 172.21.15.16:0/2325235870 >> 172.21.15.16:0/2325235870 conn(0x557e4b690300 msgr2=0x557e4ba3d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.165+0000 7f6903400640 0 client.25826 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.167+0000 7f692f722740 1 -- 172.21.15.16:0/2325235870 shutdown_connections 2024-09-21T03:48:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.168+0000 7f692f722740 1 -- 172.21.15.16:0/2325235870 wait complete. 2024-09-21T03:48:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.168+0000 7f692f722740 20 client.25826 trim_cache size 0 max 16384 2024-09-21T03:48:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.191+0000 7f692f722740 1 Processor -- start 2024-09-21T03:48:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.191+0000 7f692f722740 1 -- start start 2024-09-21T03:48:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.192+0000 7f692f722740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b6732f0 0x557e4ba3ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.192+0000 7f692f722740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b67b4d0 0x557e4ba4ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.192+0000 7f692f722740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 0x557e4ba4b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.192+0000 7f692f722740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69241b2bb0 con 0x557e4b6732f0 2024-09-21T03:48:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.192+0000 7f692f722740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69241b46f0 con 0x557e4b69c1b0 2024-09-21T03:48:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.192+0000 7f692f722740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f69241b3280 con 0x557e4b67b4d0 2024-09-21T03:48:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.192+0000 7f6921400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b67b4d0 0x557e4ba4ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.192+0000 7f6922800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 0x557e4ba4b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.192+0000 7f6921400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b67b4d0 0x557e4ba4ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:60786/0 (socket says 172.21.15.16:60786) 2024-09-21T03:48:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.192+0000 7f6921400640 1 -- 172.21.15.16:0/423411756 learned_addr learned my addr 172.21.15.16:0/423411756 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.192+0000 7f6921e00640 1 --2- 172.21.15.16:0/423411756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b6732f0 0x557e4ba3ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.192+0000 7f6921400640 1 -- 172.21.15.16:0/423411756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 msgr2=0x557e4ba4b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.192+0000 7f6921400640 1 --2- 172.21.15.16:0/423411756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 0x557e4ba4b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.192+0000 7f6921400640 1 -- 172.21.15.16:0/423411756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b6732f0 msgr2=0x557e4ba3ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.192+0000 7f6921400640 1 --2- 172.21.15.16:0/423411756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b6732f0 0x557e4ba3ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.192+0000 7f6921400640 1 -- 172.21.15.16:0/423411756 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e4ba4dd30 con 0x557e4b67b4d0 2024-09-21T03:48:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.192+0000 7f6922800640 1 --2- 172.21.15.16:0/423411756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 0x557e4ba4b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:48:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.193+0000 7f6921e00640 1 --2- 172.21.15.16:0/423411756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b6732f0 0x557e4ba3ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:48:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.193+0000 7f6921400640 1 --2- 172.21.15.16:0/423411756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b67b4d0 0x557e4ba4ace0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x557e4ba5a980 tx=0x7f691400a2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8444eb6000607cf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.193+0000 7f690e600640 1 -- 172.21.15.16:0/423411756 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914010810 con 0x557e4b67b4d0 2024-09-21T03:48:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.193+0000 7f690e600640 1 -- 172.21.15.16:0/423411756 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691401e070 con 0x557e4b67b4d0 2024-09-21T03:48:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.193+0000 7f690e600640 1 -- 172.21.15.16:0/423411756 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914023d50 con 0x557e4b67b4d0 2024-09-21T03:48:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.194+0000 7f692f722740 1 -- 172.21.15.16:0/423411756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b67b4d0 msgr2=0x557e4ba4ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.194+0000 7f692f722740 1 --2- 172.21.15.16:0/423411756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b67b4d0 0x557e4ba4ace0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x557e4ba5a980 tx=0x7f691400a2d0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.194+0000 7f692f722740 1 -- 172.21.15.16:0/423411756 shutdown_connections 2024-09-21T03:48:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.194+0000 7f692f722740 1 --2- 172.21.15.16:0/423411756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 0x557e4ba4b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.194+0000 7f692f722740 1 --2- 172.21.15.16:0/423411756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b67b4d0 0x557e4ba4ace0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.194+0000 7f692f722740 1 --2- 172.21.15.16:0/423411756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b6732f0 0x557e4ba3ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.194+0000 7f692f722740 1 -- 172.21.15.16:0/423411756 >> 172.21.15.16:0/423411756 conn(0x557e4b690300 msgr2=0x557e4ba43f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.194+0000 7f692f722740 1 -- 172.21.15.16:0/423411756 shutdown_connections 2024-09-21T03:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.194+0000 7f692f722740 1 -- 172.21.15.16:0/423411756 wait complete. 2024-09-21T03:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.195+0000 7f692f722740 1 -- start start 2024-09-21T03:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.195+0000 7f692f722740 1 Processor -- start 2024-09-21T03:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.195+0000 7f692f722740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b6732f0 0x557e4ba53b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.195+0000 7f692f722740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b67b4d0 0x557e4ba54240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.195+0000 7f692f722740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 0x557e4ba529a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.195+0000 7f692f722740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69241b4020 con 0x557e4b67b4d0 2024-09-21T03:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.195+0000 7f692f722740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69241b3950 con 0x557e4b69c1b0 2024-09-21T03:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.195+0000 7f692f722740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6924082f20 con 0x557e4b6732f0 2024-09-21T03:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.196+0000 7f692f722740 10 client.0 _ll_register_callbacks cb 0x7f691000af00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.196+0000 7f6922800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 0x557e4ba529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.196+0000 7f6922800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 0x557e4ba529a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:58554/0 (socket says 172.21.15.16:58554) 2024-09-21T03:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.196+0000 7f6922800640 1 -- 172.21.15.16:0/3149115104 learned_addr learned my addr 172.21.15.16:0/3149115104 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.196+0000 7f6921400640 1 --2- 172.21.15.16:0/3149115104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b67b4d0 0x557e4ba54240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.196+0000 7f6921e00640 1 --2- 172.21.15.16:0/3149115104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b6732f0 0x557e4ba53b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.196+0000 7f6922800640 1 -- 172.21.15.16:0/3149115104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b6732f0 msgr2=0x557e4ba53b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.196+0000 7f6922800640 1 --2- 172.21.15.16:0/3149115104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b6732f0 0x557e4ba53b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.196+0000 7f6922800640 1 -- 172.21.15.16:0/3149115104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b67b4d0 msgr2=0x557e4ba54240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.196+0000 7f6922800640 1 --2- 172.21.15.16:0/3149115104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b67b4d0 0x557e4ba54240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.196+0000 7f6922800640 1 -- 172.21.15.16:0/3149115104 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e4ba53110 con 0x557e4b69c1b0 2024-09-21T03:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.196+0000 7f6921400640 1 --2- 172.21.15.16:0/3149115104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b67b4d0 0x557e4ba54240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.196+0000 7f6921e00640 1 --2- 172.21.15.16:0/3149115104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b6732f0 0x557e4ba53b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.197+0000 7f6922800640 1 --2- 172.21.15.16:0/3149115104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 0x557e4ba529a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f691c045000 tx=0x7f691c02a8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7d736f12556f49c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.197+0000 7f6903400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.197+0000 7f6903400640 1 -- 172.21.15.16:0/3149115104 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c05c650 con 0x557e4b69c1b0 2024-09-21T03:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.197+0000 7f6903400640 1 -- 172.21.15.16:0/3149115104 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f691c05c7f0 con 0x557e4b69c1b0 2024-09-21T03:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.197+0000 7f692f722740 1 -- 172.21.15.16:0/3149115104 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f691401e070 con 0x557e4b69c1b0 2024-09-21T03:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.197+0000 7f6903400640 1 -- 172.21.15.16:0/3149115104 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f691c05c9f0 con 0x557e4b69c1b0 2024-09-21T03:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.197+0000 7f692f722740 10 client.25832 fetch_fsmap learned FSMap version 30 2024-09-21T03:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.197+0000 7f692f722740 1 -- 172.21.15.16:0/3149115104 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557e4ba51fa0 con 0x557e4b69c1b0 2024-09-21T03:48:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.198+0000 7f6903400640 1 -- 172.21.15.16:0/3149115104 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691401e070 con 0x557e4b69c1b0 2024-09-21T03:48:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.198+0000 7f6903400640 1 -- 172.21.15.16:0/3149115104 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f691c043070 con 0x557e4b69c1b0 2024-09-21T03:48:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.198+0000 7f692f722740 10 client.25832 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:48:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.198+0000 7f692f722740 10 client.25832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.198+0000 7f692f722740 20 client.25832 populate_metadata read hostname 'smithi016' 2024-09-21T03:48:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.198+0000 7f692f722740 1 --2- 172.21.15.16:0/3149115104 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557e4b6301c0 0x557e4bac6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.199+0000 7f692f722740 4 client.25832 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:48:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.199+0000 7f692f722740 1 -- 172.21.15.16:0/3149115104 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e4b9b40b0 con 0x557e4b6301c0 2024-09-21T03:48:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.199+0000 7f6921e00640 1 --2- 172.21.15.16:0/3149115104 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557e4b6301c0 0x557e4bac6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.199+0000 7f6921e00640 1 --2- 172.21.15.16:0/3149115104 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557e4b6301c0 0x557e4bac6e10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bb093d0fcaf5d87 server_cookie=36b95650a8bf21d9 in_seq=0 out_seq=0 2024-09-21T03:48:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.199+0000 7f6903400640 10 client.25832 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.214+0000 7f6903400640 1 -- 172.21.15.16:0/3149115104 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e4b9b40b0 con 0x557e4b6301c0 2024-09-21T03:48:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.214+0000 7f6903400640 10 client.25832 handle_command_reply: tid=0 2024-09-21T03:48:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.214+0000 7f692f722740 1 client.25832 shutdown 2024-09-21T03:48:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.214+0000 7f692f722740 1 -- 172.21.15.16:0/3149115104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 msgr2=0x557e4ba529a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.214+0000 7f692f722740 1 --2- 172.21.15.16:0/3149115104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 0x557e4ba529a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f691c045000 tx=0x7f691c02a8b0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.215+0000 7f692f722740 1 -- 172.21.15.16:0/3149115104 shutdown_connections 2024-09-21T03:48:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.215+0000 7f692f722740 1 --2- 172.21.15.16:0/3149115104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e4b69c1b0 0x557e4ba529a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.215+0000 7f692f722740 1 --2- 172.21.15.16:0/3149115104 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557e4b6301c0 0x557e4bac6e10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.215+0000 7f692f722740 1 --2- 172.21.15.16:0/3149115104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e4b67b4d0 0x557e4ba54240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.215+0000 7f692f722740 1 --2- 172.21.15.16:0/3149115104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e4b6732f0 0x557e4ba53b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.215+0000 7f692f722740 1 -- 172.21.15.16:0/3149115104 >> 172.21.15.16:0/3149115104 conn(0x557e4b690300 msgr2=0x557e4ba4ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.215+0000 7f6903400640 0 client.25832 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.215+0000 7f692f722740 1 -- 172.21.15.16:0/3149115104 shutdown_connections 2024-09-21T03:48:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.216+0000 7f692f722740 1 -- 172.21.15.16:0/3149115104 wait complete. 2024-09-21T03:48:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.216+0000 7f692f722740 20 client.25832 trim_cache size 0 max 16384 2024-09-21T03:48:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:48:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:48:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:48:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:48:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.218+0000 7f6928a00640 1 -- 172.21.15.16:0/4031656325 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f69080b64a0 msgr2=0x7f69080b89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.218+0000 7f6928a00640 1 --2- 172.21.15.16:0/4031656325 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f69080b64a0 0x7f69080b89b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f691c030dd0 tx=0x7f691c030820 comp rx=0 tx=0).stop 2024-09-21T03:48:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.218+0000 7f6928a00640 1 -- 172.21.15.16:0/4031656325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924109ce0 msgr2=0x7f69240721f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.218+0000 7f6928a00640 1 --2- 172.21.15.16:0/4031656325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924109ce0 0x7f69240721f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f691000c970 tx=0x7f691000cd70 comp rx=0 tx=0).stop 2024-09-21T03:48:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.220+0000 7f6928a00640 1 -- 172.21.15.16:0/4031656325 shutdown_connections 2024-09-21T03:48:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.224+0000 7f6928a00640 1 --2- 172.21.15.16:0/4031656325 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f69080b64a0 0x7f69080b89b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.224+0000 7f6928a00640 1 --2- 172.21.15.16:0/4031656325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f692410b700 0x7f6924073410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.224+0000 7f6928a00640 1 --2- 172.21.15.16:0/4031656325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f692410ad60 0x7f6924072860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.224+0000 7f6928a00640 1 --2- 172.21.15.16:0/4031656325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924109ce0 0x7f69240721f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.224+0000 7f6928a00640 1 -- 172.21.15.16:0/4031656325 >> 172.21.15.16:0/4031656325 conn(0x7f6924079060 msgr2=0x7f692410e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.224+0000 7f6928a00640 1 -- 172.21.15.16:0/4031656325 shutdown_connections 2024-09-21T03:48:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:42.224+0000 7f6928a00640 1 -- 172.21.15.16:0/4031656325 wait complete. 2024-09-21T03:48:42.422 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0' 2024-09-21T03:48:42.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0 2024-09-21T03:48:42.596 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:42 smithi016 ceph-mon[30375]: pgmap v490: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 239 MiB/s wr, 1k op/s 2024-09-21T03:48:42.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:42 smithi134 ceph-mon[35031]: pgmap v490: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 239 MiB/s wr, 1k op/s 2024-09-21T03:48:42.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:42 smithi123 ceph-mon[34673]: pgmap v490: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 239 MiB/s wr, 1k op/s 2024-09-21T03:48:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:48:44.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:44 smithi134 ceph-mon[35031]: pgmap v491: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 236 MiB/s wr, 918 op/s 2024-09-21T03:48:44.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:44 smithi123 ceph-mon[34673]: pgmap v491: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 236 MiB/s wr, 918 op/s 2024-09-21T03:48:44.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:44 smithi016 ceph-mon[30375]: pgmap v491: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 236 MiB/s wr, 918 op/s 2024-09-21T03:48:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.290+0000 7f43ad400640 1 -- 172.21.15.16:0/3297390730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43a81027c0 msgr2=0x7f43a8102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.292+0000 7f43ad400640 1 --2- 172.21.15.16:0/3297390730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43a81027c0 0x7f43a8102bd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4398009930 tx=0x7f439802a420 comp rx=0 tx=0).stop 2024-09-21T03:48:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.292+0000 7f43ad400640 1 -- 172.21.15.16:0/3297390730 shutdown_connections 2024-09-21T03:48:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.292+0000 7f43ad400640 1 --2- 172.21.15.16:0/3297390730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43a8107c00 0x7f43a81080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.292+0000 7f43ad400640 1 --2- 172.21.15.16:0/3297390730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a8103110 0x7f43a8107640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.292+0000 7f43ad400640 1 --2- 172.21.15.16:0/3297390730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43a81027c0 0x7f43a8102bd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.292+0000 7f43ad400640 1 -- 172.21.15.16:0/3297390730 >> 172.21.15.16:0/3297390730 conn(0x7f43a8079760 msgr2=0x7f43a8079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:45.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.295+0000 7f43ad400640 1 -- 172.21.15.16:0/3297390730 shutdown_connections 2024-09-21T03:48:45.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.296+0000 7f43ad400640 1 -- 172.21.15.16:0/3297390730 wait complete. 2024-09-21T03:48:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.299+0000 7f43ad400640 1 Processor -- start 2024-09-21T03:48:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.299+0000 7f43ad400640 1 -- start start 2024-09-21T03:48:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.304+0000 7f43ad400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a81027c0 0x7f43a81a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.304+0000 7f43ad400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43a8103110 0x7f43a81a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.306+0000 7f43a5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43a8103110 0x7f43a81a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.306+0000 7f43ad400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43a8107c00 0x7f43a81a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.306+0000 7f43a5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43a8103110 0x7f43a81a1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:58566/0 (socket says 172.21.15.16:58566) 2024-09-21T03:48:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.305+0000 7f43a6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a81027c0 0x7f43a81a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.306+0000 7f43ad400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43a811a100 con 0x7f43a81027c0 2024-09-21T03:48:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.306+0000 7f43ad400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43a8119f80 con 0x7f43a8103110 2024-09-21T03:48:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.306+0000 7f43ad400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f43a811a280 con 0x7f43a8107c00 2024-09-21T03:48:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.306+0000 7f43a5e00640 1 -- 172.21.15.16:0/1798558043 learned_addr learned my addr 172.21.15.16:0/1798558043 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.307+0000 7f43a7200640 1 --2- 172.21.15.16:0/1798558043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43a8107c00 0x7f43a81a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.307+0000 7f43a6800640 1 -- 172.21.15.16:0/1798558043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43a8107c00 msgr2=0x7f43a81a63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.307+0000 7f43a6800640 1 --2- 172.21.15.16:0/1798558043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43a8107c00 0x7f43a81a63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.307+0000 7f43a6800640 1 -- 172.21.15.16:0/1798558043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43a8103110 msgr2=0x7f43a81a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:45.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.307+0000 7f43a6800640 1 --2- 172.21.15.16:0/1798558043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43a8103110 0x7f43a81a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.308+0000 7f43a6800640 1 -- 172.21.15.16:0/1798558043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a81a6a20 con 0x7f43a81027c0 2024-09-21T03:48:45.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.308+0000 7f43a6800640 1 --2- 172.21.15.16:0/1798558043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a81027c0 0x7f43a81a0b70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4398009230 tx=0x7f4398003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:45.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.309+0000 7f439f400640 1 -- 172.21.15.16:0/1798558043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439803f070 con 0x7f43a81027c0 2024-09-21T03:48:45.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.309+0000 7f43ad400640 1 -- 172.21.15.16:0/1798558043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a81a6cb0 con 0x7f43a81027c0 2024-09-21T03:48:45.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.310+0000 7f43ad400640 1 -- 172.21.15.16:0/1798558043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a81a71b0 con 0x7f43a81027c0 2024-09-21T03:48:45.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.310+0000 7f439f400640 1 -- 172.21.15.16:0/1798558043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4398003e10 con 0x7f43a81027c0 2024-09-21T03:48:45.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.310+0000 7f439f400640 1 -- 172.21.15.16:0/1798558043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398035030 con 0x7f43a81027c0 2024-09-21T03:48:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.317+0000 7f43b4220740 1 Processor -- start 2024-09-21T03:48:45.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.317+0000 7f43b4220740 1 -- start start 2024-09-21T03:48:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.318+0000 7f43b4220740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12dc840 0x5627f1684350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.318+0000 7f43b4220740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627f12b3980 0x5627f1684890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.318+0000 7f43b4220740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12bbb60 0x5627f1684dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.318+0000 7f43b4220740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5627f15f9ca0 con 0x5627f12b3980 2024-09-21T03:48:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.318+0000 7f43b4220740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5627f15e90d0 con 0x5627f12dc840 2024-09-21T03:48:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.318+0000 7f43b4220740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5627f15ecee0 con 0x5627f12bbb60 2024-09-21T03:48:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.319+0000 7f439f400640 1 -- 172.21.15.16:0/1798558043 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4398042070 con 0x7f43a81027c0 2024-09-21T03:48:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.320+0000 7f43a7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12dc840 0x5627f1684350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.320+0000 7f43a7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12dc840 0x5627f1684350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:58580/0 (socket says 172.21.15.16:58580) 2024-09-21T03:48:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.320+0000 7f43a7200640 1 -- 172.21.15.16:0/1227801687 learned_addr learned my addr 172.21.15.16:0/1227801687 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.320+0000 7f43a5e00640 1 --2- 172.21.15.16:0/1227801687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12bbb60 0x5627f1684dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.321+0000 7f43a7200640 1 -- 172.21.15.16:0/1227801687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12bbb60 msgr2=0x5627f1684dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.321+0000 7f43a7200640 1 --2- 172.21.15.16:0/1227801687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12bbb60 0x5627f1684dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.321+0000 7f43a7200640 1 -- 172.21.15.16:0/1227801687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627f12b3980 msgr2=0x5627f1684890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.321+0000 7f43a7200640 1 --2- 172.21.15.16:0/1227801687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627f12b3980 0x5627f1684890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.321+0000 7f43a7200640 1 -- 172.21.15.16:0/1227801687 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f159b400 con 0x5627f12dc840 2024-09-21T03:48:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.321+0000 7f43a7200640 1 --2- 172.21.15.16:0/1227801687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12dc840 0x5627f1684350 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009f50 tx=0x7f43a00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88b7a7d9fffd6acc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.322+0000 7f4393400640 1 -- 172.21.15.16:0/1227801687 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0004d20 con 0x5627f12dc840 2024-09-21T03:48:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.322+0000 7f4393400640 1 -- 172.21.15.16:0/1227801687 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a0004050 con 0x5627f12dc840 2024-09-21T03:48:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.322+0000 7f4393400640 1 -- 172.21.15.16:0/1227801687 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0005280 con 0x5627f12dc840 2024-09-21T03:48:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.322+0000 7f439f400640 1 --2- 172.21.15.16:0/1798558043 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f438c0b6600 0x7f438c0b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.325+0000 7f43a6800640 1 --2- 172.21.15.16:0/1798558043 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f438c0b6600 0x7f438c0b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:45.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.326+0000 7f439f400640 1 -- 172.21.15.16:0/1798558043 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f4398107a60 con 0x7f43a81027c0 2024-09-21T03:48:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.329+0000 7f43b4220740 1 -- 172.21.15.16:0/1227801687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12dc840 msgr2=0x5627f1684350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.329+0000 7f43b4220740 1 --2- 172.21.15.16:0/1227801687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12dc840 0x5627f1684350 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009f50 tx=0x7f43a00118d0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.329+0000 7f43a6800640 1 --2- 172.21.15.16:0/1798558043 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f438c0b6600 0x7f438c0b8b10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4398105200 tx=0x7f439810b1e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.329+0000 7f43b4220740 1 -- 172.21.15.16:0/1227801687 shutdown_connections 2024-09-21T03:48:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.329+0000 7f43b4220740 1 --2- 172.21.15.16:0/1227801687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12bbb60 0x5627f1684dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.329+0000 7f43b4220740 1 --2- 172.21.15.16:0/1227801687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627f12b3980 0x5627f1684890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.329+0000 7f43b4220740 1 --2- 172.21.15.16:0/1227801687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12dc840 0x5627f1684350 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.329+0000 7f43b4220740 1 -- 172.21.15.16:0/1227801687 >> 172.21.15.16:0/1227801687 conn(0x5627f12d0990 msgr2=0x5627f1682240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.330+0000 7f43b4220740 1 -- 172.21.15.16:0/1227801687 shutdown_connections 2024-09-21T03:48:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.330+0000 7f43b4220740 1 -- 172.21.15.16:0/1227801687 wait complete. 2024-09-21T03:48:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.331+0000 7f43b4220740 1 -- start start 2024-09-21T03:48:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.331+0000 7f43b4220740 1 Processor -- start 2024-09-21T03:48:45.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.332+0000 7f43b4220740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12b3980 0x5627f168f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.332+0000 7f43b4220740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12bbb60 0x5627f168fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.332+0000 7f43b4220740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627f12dc840 0x5627f16903b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.332+0000 7f43b4220740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5627f144a7a0 con 0x5627f12dc840 2024-09-21T03:48:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.332+0000 7f43b4220740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5627f158ea10 con 0x5627f12b3980 2024-09-21T03:48:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.332+0000 7f43b4220740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5627f1692b30 con 0x5627f12bbb60 2024-09-21T03:48:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.333+0000 7f43b4220740 10 client.0 _ll_register_callbacks cb 0x5627f159ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.333+0000 7f43a7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12b3980 0x5627f168f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.333+0000 7f43a7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12b3980 0x5627f168f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:58586/0 (socket says 172.21.15.16:58586) 2024-09-21T03:48:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.333+0000 7f43a7200640 1 -- 172.21.15.16:0/4264004439 learned_addr learned my addr 172.21.15.16:0/4264004439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:45.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.334+0000 7f43a7200640 1 -- 172.21.15.16:0/4264004439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12bbb60 msgr2=0x5627f168fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:45.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.334+0000 7f43a7200640 1 --2- 172.21.15.16:0/4264004439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12bbb60 0x5627f168fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.334+0000 7f43a7200640 1 -- 172.21.15.16:0/4264004439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627f12dc840 msgr2=0x5627f16903b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:45.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.334+0000 7f43a7200640 1 --2- 172.21.15.16:0/4264004439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627f12dc840 0x5627f16903b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.334+0000 7f43a7200640 1 -- 172.21.15.16:0/4264004439 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627f1356b90 con 0x5627f12b3980 2024-09-21T03:48:45.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.335+0000 7f43a7200640 1 --2- 172.21.15.16:0/4264004439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12b3980 0x5627f168f930 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f43a0003ea0 tx=0x7f43a00048a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be0ce1920729ba56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:45.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.336+0000 7f4387e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:48:45.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.336+0000 7f4387e00640 1 -- 172.21.15.16:0/4264004439 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a00044d0 con 0x5627f12b3980 2024-09-21T03:48:45.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.336+0000 7f4387e00640 1 -- 172.21.15.16:0/4264004439 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f43a0004a00 con 0x5627f12b3980 2024-09-21T03:48:45.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.336+0000 7f43b4220740 1 -- 172.21.15.16:0/4264004439 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a0004050 con 0x5627f12b3980 2024-09-21T03:48:45.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.337+0000 7f4387e00640 1 -- 172.21.15.16:0/4264004439 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f43a0025480 con 0x5627f12b3980 2024-09-21T03:48:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.338+0000 7f43b4220740 10 client.25844 fetch_fsmap learned FSMap version 30 2024-09-21T03:48:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.338+0000 7f43b4220740 1 -- 172.21.15.16:0/4264004439 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627f169e790 con 0x5627f12b3980 2024-09-21T03:48:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.342+0000 7f4387e00640 1 -- 172.21.15.16:0/4264004439 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a0004050 con 0x5627f12b3980 2024-09-21T03:48:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.343+0000 7f4387e00640 1 -- 172.21.15.16:0/4264004439 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f43a0025620 con 0x5627f12b3980 2024-09-21T03:48:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.343+0000 7f43b4220740 10 client.25844 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:48:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.343+0000 7f43b4220740 10 client.25844 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:48:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.343+0000 7f43b4220740 20 client.25844 populate_metadata read hostname 'smithi016' 2024-09-21T03:48:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.343+0000 7f43b4220740 1 --2- 172.21.15.16:0/4264004439 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5627f129b1c0 0x5627f16a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.344+0000 7f43b4220740 4 client.25844 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.344+0000 7f43b4220740 1 -- 172.21.15.16:0/4264004439 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627f15fbbf0 con 0x5627f129b1c0 2024-09-21T03:48:45.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.344+0000 7f43a5e00640 1 --2- 172.21.15.16:0/4264004439 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5627f129b1c0 0x5627f16a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:45.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.346+0000 7f43a5e00640 1 --2- 172.21.15.16:0/4264004439 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5627f129b1c0 0x5627f16a1ba0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d691ab7d10dff5b7 server_cookie=4710cb1ca2cb9b3b in_seq=0 out_seq=0 2024-09-21T03:48:45.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.346+0000 7f4387e00640 10 client.25844 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:45.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.352+0000 7f4387e00640 1 -- 172.21.15.16:0/4264004439 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5627f15fbbf0 con 0x5627f129b1c0 2024-09-21T03:48:45.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.352+0000 7f4387e00640 10 client.25844 handle_command_reply: tid=0 2024-09-21T03:48:45.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.353+0000 7f43b4220740 1 client.25844 shutdown 2024-09-21T03:48:45.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.354+0000 7f43b4220740 1 -- 172.21.15.16:0/4264004439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12b3980 msgr2=0x5627f168f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:45.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.354+0000 7f43b4220740 1 --2- 172.21.15.16:0/4264004439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12b3980 0x5627f168f930 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f43a0003ea0 tx=0x7f43a00048a0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.356+0000 7f43b4220740 1 -- 172.21.15.16:0/4264004439 shutdown_connections 2024-09-21T03:48:45.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.356+0000 7f43b4220740 1 --2- 172.21.15.16:0/4264004439 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5627f129b1c0 0x5627f16a1ba0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.356+0000 7f43b4220740 1 --2- 172.21.15.16:0/4264004439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627f12dc840 0x5627f16903b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.356+0000 7f43b4220740 1 --2- 172.21.15.16:0/4264004439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12bbb60 0x5627f168fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.356+0000 7f43b4220740 1 --2- 172.21.15.16:0/4264004439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12b3980 0x5627f168f930 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.356+0000 7f43b4220740 1 -- 172.21.15.16:0/4264004439 >> 172.21.15.16:0/4264004439 conn(0x5627f12d0990 msgr2=0x5627f16809a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:45.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.358+0000 7f4387e00640 0 client.25844 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:45.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.359+0000 7f43b4220740 1 -- 172.21.15.16:0/4264004439 shutdown_connections 2024-09-21T03:48:45.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.360+0000 7f43b4220740 1 -- 172.21.15.16:0/4264004439 wait complete. 2024-09-21T03:48:45.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.361+0000 7f43b4220740 20 client.25844 trim_cache size 0 max 16384 2024-09-21T03:48:45.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.396+0000 7f43b4220740 1 Processor -- start 2024-09-21T03:48:45.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.397+0000 7f43b4220740 1 -- start start 2024-09-21T03:48:45.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.398+0000 7f43b4220740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12b3980 0x5627f16914c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.398+0000 7f43b4220740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12bbb60 0x5627f169c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.398+0000 7f43b4220740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627f12dc840 0x5627f1685d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.398+0000 7f43b4220740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4388020d60 con 0x5627f12dc840 2024-09-21T03:48:45.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.398+0000 7f43b4220740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43880228a0 con 0x5627f12b3980 2024-09-21T03:48:45.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.398+0000 7f43b4220740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4388021430 con 0x5627f12bbb60 2024-09-21T03:48:45.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.399+0000 7f43a5e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12bbb60 0x5627f169c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:45.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.399+0000 7f43a5e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12bbb60 0x5627f169c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:60848/0 (socket says 172.21.15.16:60848) 2024-09-21T03:48:45.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.399+0000 7f43a5e00640 1 -- 172.21.15.16:0/318230194 learned_addr learned my addr 172.21.15.16:0/318230194 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:45.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.399+0000 7f43a7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12b3980 0x5627f16914c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:45.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.400+0000 7f43a7200640 1 --2- 172.21.15.16:0/318230194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627f12dc840 0x5627f1685d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:45.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.401+0000 7f43a5e00640 1 -- 172.21.15.16:0/318230194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12b3980 msgr2=0x5627f16914c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:45.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.401+0000 7f43a5e00640 1 --2- 172.21.15.16:0/318230194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12b3980 0x5627f16914c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.401+0000 7f43a5e00640 1 -- 172.21.15.16:0/318230194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627f12dc840 msgr2=0x5627f1685d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:45.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.401+0000 7f43a5e00640 1 --2- 172.21.15.16:0/318230194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627f12dc840 0x5627f1685d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.401+0000 7f43a5e00640 1 -- 172.21.15.16:0/318230194 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f167dc50 con 0x5627f12bbb60 2024-09-21T03:48:45.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.402+0000 7f43a5e00640 1 --2- 172.21.15.16:0/318230194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12bbb60 0x5627f169c000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f439400adf0 tx=0x7f4394012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99e20ca327d1bb27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:45.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.404+0000 7f4392a00640 1 -- 172.21.15.16:0/318230194 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439401f040 con 0x5627f12bbb60 2024-09-21T03:48:45.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.404+0000 7f4392a00640 1 -- 172.21.15.16:0/318230194 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4394012e50 con 0x5627f12bbb60 2024-09-21T03:48:45.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.404+0000 7f4392a00640 1 -- 172.21.15.16:0/318230194 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394010960 con 0x5627f12bbb60 2024-09-21T03:48:45.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.405+0000 7f43b4220740 1 -- 172.21.15.16:0/318230194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12bbb60 msgr2=0x5627f169c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:45.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.405+0000 7f43b4220740 1 --2- 172.21.15.16:0/318230194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12bbb60 0x5627f169c000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f439400adf0 tx=0x7f4394012820 comp rx=0 tx=0).stop 2024-09-21T03:48:45.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.406+0000 7f43b4220740 1 -- 172.21.15.16:0/318230194 shutdown_connections 2024-09-21T03:48:45.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.406+0000 7f43b4220740 1 --2- 172.21.15.16:0/318230194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627f12dc840 0x5627f1685d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.406+0000 7f43b4220740 1 --2- 172.21.15.16:0/318230194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12bbb60 0x5627f169c000 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.406+0000 7f43b4220740 1 --2- 172.21.15.16:0/318230194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12b3980 0x5627f16914c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.406+0000 7f43b4220740 1 -- 172.21.15.16:0/318230194 >> 172.21.15.16:0/318230194 conn(0x5627f12d0990 msgr2=0x5627f169ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:45.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.407+0000 7f43b4220740 1 -- 172.21.15.16:0/318230194 shutdown_connections 2024-09-21T03:48:45.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.407+0000 7f43b4220740 1 -- 172.21.15.16:0/318230194 wait complete. 2024-09-21T03:48:45.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.408+0000 7f43b4220740 1 -- start start 2024-09-21T03:48:45.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.408+0000 7f43b4220740 1 Processor -- start 2024-09-21T03:48:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.410+0000 7f43b4220740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12b3980 0x5627f1693e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.411+0000 7f43b4220740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12bbb60 0x5627f16943b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.412+0000 7f43b4220740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627f12dc840 0x5627f16948f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.412+0000 7f43b4220740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43880221d0 con 0x5627f12dc840 2024-09-21T03:48:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.412+0000 7f43b4220740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4388021b00 con 0x5627f12bbb60 2024-09-21T03:48:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.412+0000 7f43b4220740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f438801ed00 con 0x5627f12b3980 2024-09-21T03:48:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.412+0000 7f43b4220740 10 client.0 _ll_register_callbacks cb 0x7f439400a680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.412+0000 7f43a7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12b3980 0x5627f1693e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.412+0000 7f43a7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12b3980 0x5627f1693e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:60864/0 (socket says 172.21.15.16:60864) 2024-09-21T03:48:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.412+0000 7f43a7200640 1 -- 172.21.15.16:0/3203039124 learned_addr learned my addr 172.21.15.16:0/3203039124 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.413+0000 7f43a7200640 1 -- 172.21.15.16:0/3203039124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12bbb60 msgr2=0x5627f16943b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:48:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.413+0000 7f43a7200640 1 --2- 172.21.15.16:0/3203039124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12bbb60 0x5627f16943b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.413+0000 7f43a7200640 1 -- 172.21.15.16:0/3203039124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627f12dc840 msgr2=0x5627f16948f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.413+0000 7f43a7200640 1 --2- 172.21.15.16:0/3203039124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627f12dc840 0x5627f16948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.413+0000 7f43a7200640 1 -- 172.21.15.16:0/3203039124 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627f1683000 con 0x5627f12b3980 2024-09-21T03:48:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.415+0000 7f43a7200640 1 --2- 172.21.15.16:0/3203039124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12b3980 0x5627f1693e70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f43a0017cc0 tx=0x7f43a000f760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f5d7c30e7946195 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.415+0000 7f4387e00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:48:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.416+0000 7f4387e00640 1 -- 172.21.15.16:0/3203039124 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0025480 con 0x5627f12b3980 2024-09-21T03:48:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.417+0000 7f4387e00640 1 -- 172.21.15.16:0/3203039124 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f43a000ede0 con 0x5627f12b3980 2024-09-21T03:48:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.417+0000 7f43b4220740 1 -- 172.21.15.16:0/3203039124 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4394012e50 con 0x5627f12b3980 2024-09-21T03:48:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.417+0000 7f4387e00640 1 -- 172.21.15.16:0/3203039124 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f43a000e3b0 con 0x5627f12b3980 2024-09-21T03:48:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.418+0000 7f43b4220740 10 client.25792 fetch_fsmap learned FSMap version 30 2024-09-21T03:48:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.418+0000 7f43b4220740 1 -- 172.21.15.16:0/3203039124 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627f170c620 con 0x5627f12b3980 2024-09-21T03:48:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.418+0000 7f4387e00640 1 -- 172.21.15.16:0/3203039124 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4394012e50 con 0x5627f12b3980 2024-09-21T03:48:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.418+0000 7f4387e00640 1 -- 172.21.15.16:0/3203039124 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f43a0025620 con 0x5627f12b3980 2024-09-21T03:48:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.420+0000 7f43b4220740 10 client.25792 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:48:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.420+0000 7f43b4220740 10 client.25792 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:48:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.420+0000 7f43b4220740 20 client.25792 populate_metadata read hostname 'smithi016' 2024-09-21T03:48:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.420+0000 7f43b4220740 1 --2- 172.21.15.16:0/3203039124 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5627f1270570 0x5627f170eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.421+0000 7f43b4220740 4 client.25792 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}] 2024-09-21T03:48:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.421+0000 7f43b4220740 1 -- 172.21.15.16:0/3203039124 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}) -- 0x5627f15f76a0 con 0x5627f1270570 2024-09-21T03:48:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.421+0000 7f43a5e00640 1 --2- 172.21.15.16:0/3203039124 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5627f1270570 0x5627f170eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.422+0000 7f43a5e00640 1 --2- 172.21.15.16:0/3203039124 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5627f1270570 0x5627f170eac0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c448008998dabf server_cookie=de34bd24380bce78 in_seq=0 out_seq=0 2024-09-21T03:48:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.422+0000 7f4387e00640 10 client.25792 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.491+0000 7f4387e00640 1 -- 172.21.15.16:0/3203039124 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5627f15f76a0 con 0x5627f1270570 2024-09-21T03:48:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.491+0000 7f4387e00640 10 client.25792 handle_command_reply: tid=0 2024-09-21T03:48:45.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.491+0000 7f43b4220740 1 client.25792 shutdown 2024-09-21T03:48:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.493+0000 7f43b4220740 1 -- 172.21.15.16:0/3203039124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12b3980 msgr2=0x5627f1693e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.493+0000 7f43b4220740 1 --2- 172.21.15.16:0/3203039124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12b3980 0x5627f1693e70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f43a0017cc0 tx=0x7f43a000f760 comp rx=0 tx=0).stop 2024-09-21T03:48:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.494+0000 7f43b4220740 1 -- 172.21.15.16:0/3203039124 shutdown_connections 2024-09-21T03:48:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.494+0000 7f43b4220740 1 --2- 172.21.15.16:0/3203039124 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5627f1270570 0x5627f170eac0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.494+0000 7f43b4220740 1 --2- 172.21.15.16:0/3203039124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627f12dc840 0x5627f16948f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.494+0000 7f43b4220740 1 --2- 172.21.15.16:0/3203039124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627f12bbb60 0x5627f16943b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.494+0000 7f43b4220740 1 --2- 172.21.15.16:0/3203039124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627f12b3980 0x5627f1693e70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.494+0000 7f43b4220740 1 -- 172.21.15.16:0/3203039124 >> 172.21.15.16:0/3203039124 conn(0x5627f12d0990 msgr2=0x5627f169b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.494+0000 7f4387e00640 0 client.25792 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.495+0000 7f43b4220740 1 -- 172.21.15.16:0/3203039124 shutdown_connections 2024-09-21T03:48:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.496+0000 7f43b4220740 1 -- 172.21.15.16:0/3203039124 wait complete. 2024-09-21T03:48:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.496+0000 7f43b4220740 20 client.25792 trim_cache size 0 max 16384 2024-09-21T03:48:45.499 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:48:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:48:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:48:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 10, 2024-09-21T03:48:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:48:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "b00a0f67": { 2024-09-21T03:48:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 10, 2024-09-21T03:48:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:48:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:48:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:48:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:48:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:48:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:48:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 147.8, 2024-09-21T03:48:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:48:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:48:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:48:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:48:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:48:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:48:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:48:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:48:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:48:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:48:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:48:45.501 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:48:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.497+0000 7f43ad400640 1 -- 172.21.15.16:0/1798558043 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f438c0b6600 msgr2=0x7f438c0b8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.497+0000 7f43ad400640 1 --2- 172.21.15.16:0/1798558043 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f438c0b6600 0x7f438c0b8b10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4398105200 tx=0x7f439810b1e0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.497+0000 7f43ad400640 1 -- 172.21.15.16:0/1798558043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a81027c0 msgr2=0x7f43a81a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.497+0000 7f43ad400640 1 --2- 172.21.15.16:0/1798558043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a81027c0 0x7f43a81a0b70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4398009230 tx=0x7f4398003be0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.498+0000 7f43ad400640 1 -- 172.21.15.16:0/1798558043 shutdown_connections 2024-09-21T03:48:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.498+0000 7f43ad400640 1 --2- 172.21.15.16:0/1798558043 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f438c0b6600 0x7f438c0b8b10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.498+0000 7f43ad400640 1 --2- 172.21.15.16:0/1798558043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43a8107c00 0x7f43a81a63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.498+0000 7f43ad400640 1 --2- 172.21.15.16:0/1798558043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43a8103110 0x7f43a81a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.498+0000 7f43ad400640 1 --2- 172.21.15.16:0/1798558043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a81027c0 0x7f43a81a0b70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.498+0000 7f43ad400640 1 -- 172.21.15.16:0/1798558043 >> 172.21.15.16:0/1798558043 conn(0x7f43a8079760 msgr2=0x7f43a8105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.498+0000 7f43ad400640 1 -- 172.21.15.16:0/1798558043 shutdown_connections 2024-09-21T03:48:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:48:45.499+0000 7f43ad400640 1 -- 172.21.15.16:0/1798558043 wait complete. 2024-09-21T03:48:46.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:46 smithi134 ceph-mon[35031]: pgmap v492: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 244 MiB/s wr, 903 op/s 2024-09-21T03:48:46.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:46 smithi123 ceph-mon[34673]: pgmap v492: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 244 MiB/s wr, 903 op/s 2024-09-21T03:48:46.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:46 smithi016 ceph-mon[30375]: pgmap v492: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 244 MiB/s wr, 903 op/s 2024-09-21T03:48:47.396 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b00a0f67', quiesce took 10.7 seconds. Will release after: 17.1 2024-09-21T03:48:48.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:48 smithi134 ceph-mon[35031]: pgmap v493: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 236 MiB/s wr, 855 op/s 2024-09-21T03:48:48.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:48 smithi123 ceph-mon[34673]: pgmap v493: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 236 MiB/s wr, 855 op/s 2024-09-21T03:48:48.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:48 smithi016 ceph-mon[30375]: pgmap v493: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 236 MiB/s wr, 855 op/s 2024-09-21T03:48:50.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:50 smithi134 ceph-mon[35031]: pgmap v494: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 213 MiB/s wr, 767 op/s 2024-09-21T03:48:50.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:50 smithi123 ceph-mon[34673]: pgmap v494: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 213 MiB/s wr, 767 op/s 2024-09-21T03:48:50.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:50 smithi016 ceph-mon[30375]: pgmap v494: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 213 MiB/s wr, 767 op/s 2024-09-21T03:48:52.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:52 smithi134 ceph-mon[35031]: pgmap v495: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 230 MiB/s wr, 856 op/s 2024-09-21T03:48:52.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:52 smithi123 ceph-mon[34673]: pgmap v495: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 230 MiB/s wr, 856 op/s 2024-09-21T03:48:52.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:52 smithi016 ceph-mon[30375]: pgmap v495: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 230 MiB/s wr, 856 op/s 2024-09-21T03:48:54.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:54 smithi134 ceph-mon[35031]: pgmap v496: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 239 MiB/s wr, 953 op/s 2024-09-21T03:48:54.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:54 smithi123 ceph-mon[34673]: pgmap v496: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 239 MiB/s wr, 953 op/s 2024-09-21T03:48:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:54 smithi016 ceph-mon[30375]: pgmap v496: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 239 MiB/s wr, 953 op/s 2024-09-21T03:48:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.580+0000 7f0c09c00640 1 -- 172.21.15.16:0/3582817108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c041050d0 msgr2=0x7f0c0410ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.580+0000 7f0c09c00640 1 --2- 172.21.15.16:0/3582817108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c041050d0 0x7f0c0410ba70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009ca0 tx=0x7f0bfc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.581+0000 7f0c09c00640 1 -- 172.21.15.16:0/3582817108 shutdown_connections 2024-09-21T03:48:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.581+0000 7f0c09c00640 1 --2- 172.21.15.16:0/3582817108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c041050d0 0x7f0c0410ba70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.581+0000 7f0c09c00640 1 --2- 172.21.15.16:0/3582817108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c04104730 0x7f0c04104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.581+0000 7f0c09c00640 1 --2- 172.21.15.16:0/3582817108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c041036b0 0x7f0c04103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.581+0000 7f0c09c00640 1 -- 172.21.15.16:0/3582817108 >> 172.21.15.16:0/3582817108 conn(0x7f0c040feff0 msgr2=0x7f0c041014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.582+0000 7f0c09c00640 1 -- 172.21.15.16:0/3582817108 shutdown_connections 2024-09-21T03:48:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.582+0000 7f0c09c00640 1 -- 172.21.15.16:0/3582817108 wait complete. 2024-09-21T03:48:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.583+0000 7f0c09c00640 1 Processor -- start 2024-09-21T03:48:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.585+0000 7f0c09c00640 1 -- start start 2024-09-21T03:48:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.586+0000 7f0c09c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c041036b0 0x7f0c04197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.586+0000 7f0c09c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04104730 0x7f0c04198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.586+0000 7f0c09c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c041050d0 0x7f0c04198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.586+0000 7f0c09c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0410f350 con 0x7f0c04104730 2024-09-21T03:48:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.586+0000 7f0c09c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0410f1d0 con 0x7f0c041036b0 2024-09-21T03:48:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.586+0000 7f0c09c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0410f4d0 con 0x7f0c041050d0 2024-09-21T03:48:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.586+0000 7f0c02a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c041036b0 0x7f0c04197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.586+0000 7f0c02a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c041036b0 0x7f0c04197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:59442/0 (socket says 172.21.15.16:59442) 2024-09-21T03:48:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.586+0000 7f0c02a00640 1 -- 172.21.15.16:0/1980274842 learned_addr learned my addr 172.21.15.16:0/1980274842 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.586+0000 7f0c03400640 1 --2- 172.21.15.16:0/1980274842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c041050d0 0x7f0c04198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.586+0000 7f0c02000640 1 --2- 172.21.15.16:0/1980274842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04104730 0x7f0c04198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.587+0000 7f0c02a00640 1 -- 172.21.15.16:0/1980274842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c041050d0 msgr2=0x7f0c04198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.587+0000 7f0c02a00640 1 --2- 172.21.15.16:0/1980274842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c041050d0 0x7f0c04198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.587+0000 7f0c02a00640 1 -- 172.21.15.16:0/1980274842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04104730 msgr2=0x7f0c04198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.587+0000 7f0c02a00640 1 --2- 172.21.15.16:0/1980274842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04104730 0x7f0c04198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.587+0000 7f0c02a00640 1 -- 172.21.15.16:0/1980274842 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c04199730 con 0x7f0c041036b0 2024-09-21T03:48:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.587+0000 7f0c02a00640 1 --2- 172.21.15.16:0/1980274842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c041036b0 0x7f0c04197cf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400c890 tx=0x7f0bf400cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.588+0000 7f0bfb400640 1 -- 172.21.15.16:0/1980274842 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4004220 con 0x7f0c041036b0 2024-09-21T03:48:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.588+0000 7f0c09c00640 1 -- 172.21.15.16:0/1980274842 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c04199a20 con 0x7f0c041036b0 2024-09-21T03:48:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.588+0000 7f0c09c00640 1 -- 172.21.15.16:0/1980274842 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c0419c0f0 con 0x7f0c041036b0 2024-09-21T03:48:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.588+0000 7f0bfb400640 1 -- 172.21.15.16:0/1980274842 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bf40043c0 con 0x7f0c041036b0 2024-09-21T03:48:55.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.588+0000 7f0bfb400640 1 -- 172.21.15.16:0/1980274842 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4004a00 con 0x7f0c041036b0 2024-09-21T03:48:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.592+0000 7f0c10c91740 1 Processor -- start 2024-09-21T03:48:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.593+0000 7f0c10c91740 1 -- start start 2024-09-21T03:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.593+0000 7f0c10c91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067dc1b0 0x55c606b830d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.593+0000 7f0c10c91740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067b32f0 0x55c606b83740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.593+0000 7f0c10c91740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067bb4d0 0x55c606b84290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.593+0000 7f0c10c91740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c606af66b0 con 0x55c6067b32f0 2024-09-21T03:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.593+0000 7f0c10c91740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c606ae5ae0 con 0x55c6067bb4d0 2024-09-21T03:48:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.593+0000 7f0c10c91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c606ae98f0 con 0x55c6067dc1b0 2024-09-21T03:48:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.594+0000 7f0c03400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067dc1b0 0x55c606b830d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.594+0000 7f0c03400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067dc1b0 0x55c606b830d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:56130/0 (socket says 172.21.15.16:56130) 2024-09-21T03:48:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.594+0000 7f0c03400640 1 -- 172.21.15.16:0/2892153334 learned_addr learned my addr 172.21.15.16:0/2892153334 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.595+0000 7f0c02000640 1 --2- 172.21.15.16:0/2892153334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067bb4d0 0x55c606b84290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.595+0000 7f0c03400640 1 -- 172.21.15.16:0/2892153334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067bb4d0 msgr2=0x55c606b84290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.595+0000 7f0c03400640 1 --2- 172.21.15.16:0/2892153334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067bb4d0 0x55c606b84290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.595+0000 7f0c03400640 1 -- 172.21.15.16:0/2892153334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067b32f0 msgr2=0x55c606b83740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:48:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.595+0000 7f0c03400640 1 --2- 172.21.15.16:0/2892153334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067b32f0 0x55c606b83740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.595+0000 7f0c03400640 1 -- 172.21.15.16:0/2892153334 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c606ab2780 con 0x55c6067dc1b0 2024-09-21T03:48:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.595+0000 7f0bfb400640 1 -- 172.21.15.16:0/1980274842 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0bf4004ba0 con 0x7f0c041036b0 2024-09-21T03:48:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.596+0000 7f0c03400640 1 --2- 172.21.15.16:0/2892153334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067dc1b0 0x55c606b830d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc02f530 tx=0x7f0bfc009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d98ce8abfdd3531 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.598+0000 7f0bfb400640 1 --2- 172.21.15.16:0/1980274842 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0be80b6510 0x7f0be80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.598+0000 7f0bfb400640 1 -- 172.21.15.16:0/1980274842 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0bf40e7350 con 0x7f0c041036b0 2024-09-21T03:48:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.599+0000 7f0c02a00640 1 --2- 172.21.15.16:0/1980274842 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0be80b6510 0x7f0be80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.599+0000 7f0befe00640 1 -- 172.21.15.16:0/2892153334 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc031650 con 0x55c6067dc1b0 2024-09-21T03:48:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.599+0000 7f0befe00640 1 -- 172.21.15.16:0/2892153334 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bfc036040 con 0x55c6067dc1b0 2024-09-21T03:48:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.599+0000 7f0befe00640 1 -- 172.21.15.16:0/2892153334 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc004760 con 0x55c6067dc1b0 2024-09-21T03:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.600+0000 7f0c10c91740 1 -- 172.21.15.16:0/2892153334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067dc1b0 msgr2=0x55c606b830d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.600+0000 7f0c10c91740 1 --2- 172.21.15.16:0/2892153334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067dc1b0 0x55c606b830d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc02f530 tx=0x7f0bfc009800 comp rx=0 tx=0).stop 2024-09-21T03:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.600+0000 7f0c02a00640 1 --2- 172.21.15.16:0/1980274842 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0be80b6510 0x7f0be80b8a20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40e4aa0 tx=0x7f0bf400a520 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.601+0000 7f0c10c91740 1 -- 172.21.15.16:0/2892153334 shutdown_connections 2024-09-21T03:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.601+0000 7f0c10c91740 1 --2- 172.21.15.16:0/2892153334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067bb4d0 0x55c606b84290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.601+0000 7f0c10c91740 1 --2- 172.21.15.16:0/2892153334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067b32f0 0x55c606b83740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.601+0000 7f0c10c91740 1 --2- 172.21.15.16:0/2892153334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067dc1b0 0x55c606b830d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.601+0000 7f0c10c91740 1 -- 172.21.15.16:0/2892153334 >> 172.21.15.16:0/2892153334 conn(0x55c6067d0300 msgr2=0x55c606b7eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.601+0000 7f0c10c91740 1 -- 172.21.15.16:0/2892153334 shutdown_connections 2024-09-21T03:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.601+0000 7f0c10c91740 1 -- 172.21.15.16:0/2892153334 wait complete. 2024-09-21T03:48:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.602+0000 7f0c10c91740 1 -- start start 2024-09-21T03:48:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.602+0000 7f0c10c91740 1 Processor -- start 2024-09-21T03:48:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.603+0000 7f0c10c91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067b32f0 0x55c606b8c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.603+0000 7f0c10c91740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067bb4d0 0x55c606b8ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.603+0000 7f0c10c91740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067dc1b0 0x55c606b8d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.603+0000 7f0c10c91740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c606a1dab0 con 0x55c6067dc1b0 2024-09-21T03:48:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.603+0000 7f0c10c91740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c606a8c570 con 0x55c6067bb4d0 2024-09-21T03:48:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.603+0000 7f0c10c91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c606b84d50 con 0x55c6067b32f0 2024-09-21T03:48:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.603+0000 7f0c10c91740 10 client.0 _ll_register_callbacks cb 0x55c606a9b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.603+0000 7f0c03400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067b32f0 0x55c606b8c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.603+0000 7f0c03400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067b32f0 0x55c606b8c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:56138/0 (socket says 172.21.15.16:56138) 2024-09-21T03:48:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.603+0000 7f0c03400640 1 -- 172.21.15.16:0/1042114324 learned_addr learned my addr 172.21.15.16:0/1042114324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.604+0000 7f0c03400640 1 --2- 172.21.15.16:0/1042114324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067dc1b0 0x55c606b8d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.604+0000 7f0c02000640 1 --2- 172.21.15.16:0/1042114324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067bb4d0 0x55c606b8ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.604+0000 7f0c03400640 1 -- 172.21.15.16:0/1042114324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067bb4d0 msgr2=0x55c606b8ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.604+0000 7f0c03400640 1 --2- 172.21.15.16:0/1042114324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067bb4d0 0x55c606b8ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.604+0000 7f0c03400640 1 -- 172.21.15.16:0/1042114324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067dc1b0 msgr2=0x55c606b8d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.604+0000 7f0c03400640 1 --2- 172.21.15.16:0/1042114324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067dc1b0 0x55c606b8d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.605+0000 7f0c03400640 1 -- 172.21.15.16:0/1042114324 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c606847790 con 0x55c6067b32f0 2024-09-21T03:48:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.605+0000 7f0c02000640 1 --2- 172.21.15.16:0/1042114324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067bb4d0 0x55c606b8ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:48:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.605+0000 7f0c03400640 1 --2- 172.21.15.16:0/1042114324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067b32f0 0x55c606b8c7c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc02ade0 tx=0x7f0bfc03e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc6f10c76db4525b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.605+0000 7f0becc00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:48:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.606+0000 7f0becc00640 1 -- 172.21.15.16:0/1042114324 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc003970 con 0x55c6067b32f0 2024-09-21T03:48:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.606+0000 7f0c10c91740 1 -- 172.21.15.16:0/1042114324 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bfc036040 con 0x55c6067b32f0 2024-09-21T03:48:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.606+0000 7f0becc00640 1 -- 172.21.15.16:0/1042114324 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0bfc003b10 con 0x55c6067b32f0 2024-09-21T03:48:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.606+0000 7f0becc00640 1 -- 172.21.15.16:0/1042114324 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0bfc003d10 con 0x55c6067b32f0 2024-09-21T03:48:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.607+0000 7f0c10c91740 10 client.25804 fetch_fsmap learned FSMap version 30 2024-09-21T03:48:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.607+0000 7f0c10c91740 1 -- 172.21.15.16:0/1042114324 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c6068c0620 con 0x55c6067b32f0 2024-09-21T03:48:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.607+0000 7f0becc00640 1 -- 172.21.15.16:0/1042114324 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bfc036040 con 0x55c6067b32f0 2024-09-21T03:48:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.607+0000 7f0becc00640 1 -- 172.21.15.16:0/1042114324 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0bfc02f940 con 0x55c6067b32f0 2024-09-21T03:48:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.607+0000 7f0c10c91740 10 client.25804 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:48:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.607+0000 7f0c10c91740 10 client.25804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.607+0000 7f0c10c91740 20 client.25804 populate_metadata read hostname 'smithi016' 2024-09-21T03:48:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.607+0000 7f0c10c91740 1 --2- 172.21.15.16:0/1042114324 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c60679ad80 0x55c606b99b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.607+0000 7f0c10c91740 4 client.25804 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:48:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.608+0000 7f0c10c91740 1 -- 172.21.15.16:0/1042114324 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c606af8600 con 0x55c60679ad80 2024-09-21T03:48:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.608+0000 7f0c02000640 1 --2- 172.21.15.16:0/1042114324 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c60679ad80 0x55c606b99b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.608+0000 7f0c02000640 1 --2- 172.21.15.16:0/1042114324 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c60679ad80 0x55c606b99b20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ac82cc0ccd7df5 server_cookie=f2b03ee357246356 in_seq=0 out_seq=0 2024-09-21T03:48:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.608+0000 7f0becc00640 10 client.25804 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.609+0000 7f0becc00640 1 -- 172.21.15.16:0/1042114324 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c606af8600 con 0x55c60679ad80 2024-09-21T03:48:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.609+0000 7f0becc00640 10 client.25804 handle_command_reply: tid=0 2024-09-21T03:48:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.610+0000 7f0c10c91740 1 client.25804 shutdown 2024-09-21T03:48:55.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.612+0000 7f0c10c91740 1 -- 172.21.15.16:0/1042114324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067b32f0 msgr2=0x55c606b8c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:55.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.612+0000 7f0c10c91740 1 --2- 172.21.15.16:0/1042114324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067b32f0 0x55c606b8c7c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc02ade0 tx=0x7f0bfc03e040 comp rx=0 tx=0).stop 2024-09-21T03:48:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.613+0000 7f0c10c91740 1 -- 172.21.15.16:0/1042114324 shutdown_connections 2024-09-21T03:48:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.613+0000 7f0c10c91740 1 --2- 172.21.15.16:0/1042114324 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c60679ad80 0x55c606b99b20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.613+0000 7f0c10c91740 1 --2- 172.21.15.16:0/1042114324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067dc1b0 0x55c606b8d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.613+0000 7f0c10c91740 1 --2- 172.21.15.16:0/1042114324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067bb4d0 0x55c606b8ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.613+0000 7f0c10c91740 1 --2- 172.21.15.16:0/1042114324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067b32f0 0x55c606b8c7c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.613+0000 7f0becc00640 0 client.25804 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.613+0000 7f0c10c91740 1 -- 172.21.15.16:0/1042114324 >> 172.21.15.16:0/1042114324 conn(0x55c6067d0300 msgr2=0x55c606b7d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.613+0000 7f0c10c91740 1 -- 172.21.15.16:0/1042114324 shutdown_connections 2024-09-21T03:48:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.613+0000 7f0c10c91740 1 -- 172.21.15.16:0/1042114324 wait complete. 2024-09-21T03:48:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.613+0000 7f0c10c91740 20 client.25804 trim_cache size 0 max 16384 2024-09-21T03:48:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.638+0000 7f0c10c91740 1 Processor -- start 2024-09-21T03:48:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.638+0000 7f0c10c91740 1 -- start start 2024-09-21T03:48:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.639+0000 7f0c10c91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067b32f0 0x55c606b82cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.639+0000 7f0c10c91740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067bb4d0 0x55c606b83f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.639+0000 7f0c10c91740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067dc1b0 0x55c606b7e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.639+0000 7f0c10c91740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0be400fc40 con 0x55c6067bb4d0 2024-09-21T03:48:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.639+0000 7f0c10c91740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0be4011780 con 0x55c6067dc1b0 2024-09-21T03:48:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.639+0000 7f0c10c91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0be4010310 con 0x55c6067b32f0 2024-09-21T03:48:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.639+0000 7f0c03400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067b32f0 0x55c606b82cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.639+0000 7f0c03400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067b32f0 0x55c606b82cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:56140/0 (socket says 172.21.15.16:56140) 2024-09-21T03:48:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.639+0000 7f0c03400640 1 -- 172.21.15.16:0/1017094810 learned_addr learned my addr 172.21.15.16:0/1017094810 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.639+0000 7f0c03400640 1 --2- 172.21.15.16:0/1017094810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067dc1b0 0x55c606b7e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.640+0000 7f0c02000640 1 --2- 172.21.15.16:0/1017094810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067bb4d0 0x55c606b83f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.640+0000 7f0c03400640 1 -- 172.21.15.16:0/1017094810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067b32f0 msgr2=0x55c606b82cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.640+0000 7f0c03400640 1 --2- 172.21.15.16:0/1017094810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067b32f0 0x55c606b82cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.640+0000 7f0c03400640 1 -- 172.21.15.16:0/1017094810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067bb4d0 msgr2=0x55c606b83f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.640+0000 7f0c03400640 1 --2- 172.21.15.16:0/1017094810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067bb4d0 0x55c606b83f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.640+0000 7f0c03400640 1 -- 172.21.15.16:0/1017094810 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c606b8c0d0 con 0x55c6067dc1b0 2024-09-21T03:48:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.640+0000 7f0c02000640 1 --2- 172.21.15.16:0/1017094810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067bb4d0 0x55c606b83f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:48:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.641+0000 7f0c03400640 1 --2- 172.21.15.16:0/1017094810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067dc1b0 0x55c606b7e0f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0041d0 tx=0x7f0bfc009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=152376a4c1f71208 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.641+0000 7f0bef400640 1 -- 172.21.15.16:0/1017094810 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc0305c0 con 0x55c6067dc1b0 2024-09-21T03:48:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.641+0000 7f0bef400640 1 -- 172.21.15.16:0/1017094810 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bfc036030 con 0x55c6067dc1b0 2024-09-21T03:48:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.641+0000 7f0bef400640 1 -- 172.21.15.16:0/1017094810 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc02f610 con 0x55c6067dc1b0 2024-09-21T03:48:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.642+0000 7f0c10c91740 1 -- 172.21.15.16:0/1017094810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067dc1b0 msgr2=0x55c606b7e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.642+0000 7f0c10c91740 1 --2- 172.21.15.16:0/1017094810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067dc1b0 0x55c606b7e0f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0041d0 tx=0x7f0bfc009a20 comp rx=0 tx=0).stop 2024-09-21T03:48:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.644+0000 7f0c10c91740 1 -- 172.21.15.16:0/1017094810 shutdown_connections 2024-09-21T03:48:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.644+0000 7f0c10c91740 1 --2- 172.21.15.16:0/1017094810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067dc1b0 0x55c606b7e0f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.644+0000 7f0c10c91740 1 --2- 172.21.15.16:0/1017094810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067bb4d0 0x55c606b83f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.644+0000 7f0c10c91740 1 --2- 172.21.15.16:0/1017094810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067b32f0 0x55c606b82cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.644+0000 7f0c10c91740 1 -- 172.21.15.16:0/1017094810 >> 172.21.15.16:0/1017094810 conn(0x55c6067d0300 msgr2=0x55c606b8b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.644+0000 7f0c10c91740 1 -- 172.21.15.16:0/1017094810 shutdown_connections 2024-09-21T03:48:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.644+0000 7f0c10c91740 1 -- 172.21.15.16:0/1017094810 wait complete. 2024-09-21T03:48:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.646+0000 7f0c10c91740 1 -- start start 2024-09-21T03:48:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.646+0000 7f0c10c91740 1 Processor -- start 2024-09-21T03:48:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.648+0000 7f0c10c91740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067b32f0 0x55c606b94340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.648+0000 7f0c10c91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067bb4d0 0x55c606b93000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.648+0000 7f0c10c91740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067dc1b0 0x55c606b93540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.648+0000 7f0c10c91740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0be40110b0 con 0x55c6067b32f0 2024-09-21T03:48:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.648+0000 7f0c10c91740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0be40109e0 con 0x55c6067dc1b0 2024-09-21T03:48:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.648+0000 7f0c10c91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0be400dc20 con 0x55c6067bb4d0 2024-09-21T03:48:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.648+0000 7f0c10c91740 10 client.0 _ll_register_callbacks cb 0x7f0bf0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:48:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.648+0000 7f0c02000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067bb4d0 0x55c606b93000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.648+0000 7f0c03400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067b32f0 0x55c606b94340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.648+0000 7f0c03400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067b32f0 0x55c606b94340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60664/0 (socket says 172.21.15.16:60664) 2024-09-21T03:48:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.648+0000 7f0c03400640 1 -- 172.21.15.16:0/695159467 learned_addr learned my addr 172.21.15.16:0/695159467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:48:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.648+0000 7f0c03400640 1 --2- 172.21.15.16:0/695159467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067dc1b0 0x55c606b93540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.648+0000 7f0c03400640 1 -- 172.21.15.16:0/695159467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067bb4d0 msgr2=0x55c606b93000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.648+0000 7f0c03400640 1 --2- 172.21.15.16:0/695159467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067bb4d0 0x55c606b93000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.648+0000 7f0c03400640 1 -- 172.21.15.16:0/695159467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067dc1b0 msgr2=0x55c606b93540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:48:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.648+0000 7f0c03400640 1 --2- 172.21.15.16:0/695159467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067dc1b0 0x55c606b93540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.649+0000 7f0c03400640 1 -- 172.21.15.16:0/695159467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c606b93a80 con 0x55c6067b32f0 2024-09-21T03:48:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.649+0000 7f0becc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:48:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.649+0000 7f0c03400640 1 --2- 172.21.15.16:0/695159467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067b32f0 0x55c606b94340 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc02a900 tx=0x7f0bfc02bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcd5437cf74b910 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:48:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.649+0000 7f0becc00640 1 -- 172.21.15.16:0/695159467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc004b40 con 0x55c6067b32f0 2024-09-21T03:48:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.649+0000 7f0c10c91740 1 -- 172.21.15.16:0/695159467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bfc036030 con 0x55c6067b32f0 2024-09-21T03:48:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.649+0000 7f0becc00640 1 -- 172.21.15.16:0/695159467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0bfc004ce0 con 0x55c6067b32f0 2024-09-21T03:48:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.650+0000 7f0becc00640 1 -- 172.21.15.16:0/695159467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0bfc02f610 con 0x55c6067b32f0 2024-09-21T03:48:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.650+0000 7f0c10c91740 10 client.15807 fetch_fsmap learned FSMap version 30 2024-09-21T03:48:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.650+0000 7f0c10c91740 1 -- 172.21.15.16:0/695159467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c606b92880 con 0x55c6067b32f0 2024-09-21T03:48:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.650+0000 7f0becc00640 1 -- 172.21.15.16:0/695159467 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bfc036030 con 0x55c6067b32f0 2024-09-21T03:48:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.651+0000 7f0becc00640 1 -- 172.21.15.16:0/695159467 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0bfc030cc0 con 0x55c6067b32f0 2024-09-21T03:48:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.651+0000 7f0c10c91740 10 client.15807 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:48:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.651+0000 7f0c10c91740 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:48:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.651+0000 7f0c10c91740 20 client.15807 populate_metadata read hostname 'smithi016' 2024-09-21T03:48:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.651+0000 7f0c10c91740 1 --2- 172.21.15.16:0/695159467 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c6067701c0 0x55c606c06a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:48:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.651+0000 7f0c10c91740 4 client.15807 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:48:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.651+0000 7f0c10c91740 1 -- 172.21.15.16:0/695159467 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c606af40b0 con 0x55c6067701c0 2024-09-21T03:48:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.651+0000 7f0c02000640 1 --2- 172.21.15.16:0/695159467 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c6067701c0 0x55c606c06a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:48:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.652+0000 7f0c02000640 1 --2- 172.21.15.16:0/695159467 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c6067701c0 0x55c606c06a80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6596eb96182f8bcf server_cookie=f1994e5bc96d848d in_seq=0 out_seq=0 2024-09-21T03:48:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.652+0000 7f0becc00640 10 client.15807 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.671+0000 7f0becc00640 1 -- 172.21.15.16:0/695159467 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c606af40b0 con 0x55c6067701c0 2024-09-21T03:48:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.671+0000 7f0becc00640 10 client.15807 handle_command_reply: tid=0 2024-09-21T03:48:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.671+0000 7f0c10c91740 1 client.15807 shutdown 2024-09-21T03:48:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.671+0000 7f0c10c91740 1 -- 172.21.15.16:0/695159467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067b32f0 msgr2=0x55c606b94340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.671+0000 7f0c10c91740 1 --2- 172.21.15.16:0/695159467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067b32f0 0x55c606b94340 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc02a900 tx=0x7f0bfc02bfd0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.671+0000 7f0c10c91740 1 -- 172.21.15.16:0/695159467 shutdown_connections 2024-09-21T03:48:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.671+0000 7f0c10c91740 1 --2- 172.21.15.16:0/695159467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6067dc1b0 0x55c606b93540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.671+0000 7f0c10c91740 1 --2- 172.21.15.16:0/695159467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6067bb4d0 0x55c606b93000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.671+0000 7f0c10c91740 1 --2- 172.21.15.16:0/695159467 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c6067701c0 0x55c606c06a80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.671+0000 7f0c10c91740 1 --2- 172.21.15.16:0/695159467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6067b32f0 0x55c606b94340 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.671+0000 7f0c10c91740 1 -- 172.21.15.16:0/695159467 >> 172.21.15.16:0/695159467 conn(0x55c6067d0300 msgr2=0x55c606b8b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.671+0000 7f0becc00640 0 client.15807 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:48:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.672+0000 7f0c10c91740 1 -- 172.21.15.16:0/695159467 shutdown_connections 2024-09-21T03:48:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.672+0000 7f0c10c91740 1 -- 172.21.15.16:0/695159467 wait complete. 2024-09-21T03:48:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.672+0000 7f0c10c91740 20 client.15807 trim_cache size 0 max 16384 2024-09-21T03:48:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:48:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:48:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:48:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:48:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.673+0000 7f0c09c00640 1 -- 172.21.15.16:0/1980274842 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0be80b6510 msgr2=0x7f0be80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.673+0000 7f0c09c00640 1 --2- 172.21.15.16:0/1980274842 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0be80b6510 0x7f0be80b8a20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40e4aa0 tx=0x7f0bf400a520 comp rx=0 tx=0).stop 2024-09-21T03:48:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.673+0000 7f0c09c00640 1 -- 172.21.15.16:0/1980274842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c041036b0 msgr2=0x7f0c04197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:48:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.673+0000 7f0c09c00640 1 --2- 172.21.15.16:0/1980274842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c041036b0 0x7f0c04197cf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400c890 tx=0x7f0bf400cd50 comp rx=0 tx=0).stop 2024-09-21T03:48:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.673+0000 7f0c09c00640 1 -- 172.21.15.16:0/1980274842 shutdown_connections 2024-09-21T03:48:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.673+0000 7f0c09c00640 1 --2- 172.21.15.16:0/1980274842 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0be80b6510 0x7f0be80b8a20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.673+0000 7f0c09c00640 1 --2- 172.21.15.16:0/1980274842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c041050d0 0x7f0c04198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.673+0000 7f0c09c00640 1 --2- 172.21.15.16:0/1980274842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04104730 0x7f0c04198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.673+0000 7f0c09c00640 1 --2- 172.21.15.16:0/1980274842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c041036b0 0x7f0c04197cf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:48:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.673+0000 7f0c09c00640 1 -- 172.21.15.16:0/1980274842 >> 172.21.15.16:0/1980274842 conn(0x7f0c040feff0 msgr2=0x7f0c04105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:48:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.674+0000 7f0c09c00640 1 -- 172.21.15.16:0/1980274842 shutdown_connections 2024-09-21T03:48:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:48:55.674+0000 7f0c09c00640 1 -- 172.21.15.16:0/1980274842 wait complete. 2024-09-21T03:48:56.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:56 smithi134 ceph-mon[35031]: pgmap v497: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 249 MiB/s wr, 1.10k op/s 2024-09-21T03:48:56.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:56 smithi123 ceph-mon[34673]: pgmap v497: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 249 MiB/s wr, 1.10k op/s 2024-09-21T03:48:56.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:56 smithi016 ceph-mon[30375]: pgmap v497: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 249 MiB/s wr, 1.10k op/s 2024-09-21T03:48:58.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:48:58 smithi134 ceph-mon[35031]: pgmap v498: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 245 MiB/s wr, 1.21k op/s 2024-09-21T03:48:58.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:48:58 smithi123 ceph-mon[34673]: pgmap v498: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 245 MiB/s wr, 1.21k op/s 2024-09-21T03:48:58.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:48:58 smithi016 ceph-mon[30375]: pgmap v498: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 245 MiB/s wr, 1.21k op/s 2024-09-21T03:49:00.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:00 smithi123 ceph-mon[34673]: pgmap v499: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 222 MiB/s wr, 1.18k op/s 2024-09-21T03:49:00.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:00 smithi016 ceph-mon[30375]: pgmap v499: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 222 MiB/s wr, 1.18k op/s 2024-09-21T03:49:01.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:00 smithi134 ceph-mon[35031]: pgmap v499: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 222 MiB/s wr, 1.18k op/s 2024-09-21T03:49:02.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:02 smithi123 ceph-mon[34673]: pgmap v500: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 205 MiB/s wr, 1.14k op/s 2024-09-21T03:49:02.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:02 smithi016 ceph-mon[30375]: pgmap v500: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 205 MiB/s wr, 1.14k op/s 2024-09-21T03:49:03.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:02 smithi134 ceph-mon[35031]: pgmap v500: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 205 MiB/s wr, 1.14k op/s 2024-09-21T03:49:04.497 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:49:04.893 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:04.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:04 smithi123 ceph-mon[34673]: pgmap v501: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 162 MiB/s wr, 960 op/s 2024-09-21T03:49:04.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:04 smithi016 ceph-mon[30375]: pgmap v501: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 162 MiB/s wr, 960 op/s 2024-09-21T03:49:05.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:04 smithi134 ceph-mon[35031]: pgmap v501: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 162 MiB/s wr, 960 op/s 2024-09-21T03:49:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.559+0000 7fb7d2a00640 1 -- 172.21.15.16:0/1686237138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc1049b0 msgr2=0x7fb7cc104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.559+0000 7fb7d2a00640 1 --2- 172.21.15.16:0/1686237138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc1049b0 0x7fb7cc104dc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009900 tx=0x7fb7bc02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:49:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.559+0000 7fb7d2a00640 1 -- 172.21.15.16:0/1686237138 shutdown_connections 2024-09-21T03:49:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.559+0000 7fb7d2a00640 1 --2- 172.21.15.16:0/1686237138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7cc10de10 0x7fb7cc110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.559+0000 7fb7d2a00640 1 --2- 172.21.15.16:0/1686237138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7cc105300 0x7fb7cc10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.559+0000 7fb7d2a00640 1 --2- 172.21.15.16:0/1686237138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc1049b0 0x7fb7cc104dc0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.559+0000 7fb7d2a00640 1 -- 172.21.15.16:0/1686237138 >> 172.21.15.16:0/1686237138 conn(0x7fb7cc079760 msgr2=0x7fb7cc079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.559+0000 7fb7d2a00640 1 -- 172.21.15.16:0/1686237138 shutdown_connections 2024-09-21T03:49:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.559+0000 7fb7d2a00640 1 -- 172.21.15.16:0/1686237138 wait complete. 2024-09-21T03:49:05.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.561+0000 7fb7d2a00640 1 Processor -- start 2024-09-21T03:49:05.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.561+0000 7fb7d2a00640 1 -- start start 2024-09-21T03:49:05.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.561+0000 7fb7d2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc1049b0 0x7fb7cc104100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:05.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.561+0000 7fb7d2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7cc105300 0x7fb7cc1026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:05.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.561+0000 7fb7d2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7cc10de10 0x7fb7cc102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:05.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.561+0000 7fb7d2a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc113a90 con 0x7fb7cc1049b0 2024-09-21T03:49:05.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.561+0000 7fb7d2a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc113910 con 0x7fb7cc10de10 2024-09-21T03:49:05.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.561+0000 7fb7d2a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc113c10 con 0x7fb7cc105300 2024-09-21T03:49:05.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.561+0000 7fb7cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc1049b0 0x7fb7cc104100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:05.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.562+0000 7fb7cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc1049b0 0x7fb7cc104100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41620/0 (socket says 172.21.15.16:41620) 2024-09-21T03:49:05.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.562+0000 7fb7cbe00640 1 -- 172.21.15.16:0/4231029343 learned_addr learned my addr 172.21.15.16:0/4231029343 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:05.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.562+0000 7fb7c3e00640 1 --2- 172.21.15.16:0/4231029343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7cc105300 0x7fb7cc1026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:05.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.562+0000 7fb7d0a00640 1 --2- 172.21.15.16:0/4231029343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7cc10de10 0x7fb7cc102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:05.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.562+0000 7fb7cbe00640 1 -- 172.21.15.16:0/4231029343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7cc105300 msgr2=0x7fb7cc1026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:05.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.562+0000 7fb7cbe00640 1 --2- 172.21.15.16:0/4231029343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7cc105300 0x7fb7cc1026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:05.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.562+0000 7fb7cbe00640 1 -- 172.21.15.16:0/4231029343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7cc10de10 msgr2=0x7fb7cc102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:05.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.562+0000 7fb7cbe00640 1 --2- 172.21.15.16:0/4231029343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7cc10de10 0x7fb7cc102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:05.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.562+0000 7fb7cbe00640 1 -- 172.21.15.16:0/4231029343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7cc103450 con 0x7fb7cc1049b0 2024-09-21T03:49:05.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.562+0000 7fb7cbe00640 1 --2- 172.21.15.16:0/4231029343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc1049b0 0x7fb7cc104100 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc005940 tx=0x7fb7bc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:05.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.563+0000 7fb7c9600640 1 -- 172.21.15.16:0/4231029343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc03f030 con 0x7fb7cc1049b0 2024-09-21T03:49:05.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.563+0000 7fb7c9600640 1 -- 172.21.15.16:0/4231029343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7bc0044c0 con 0x7fb7cc1049b0 2024-09-21T03:49:05.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.563+0000 7fb7d2a00640 1 -- 172.21.15.16:0/4231029343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7cc1ace00 con 0x7fb7cc1049b0 2024-09-21T03:49:05.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.563+0000 7fb7c9600640 1 -- 172.21.15.16:0/4231029343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc035070 con 0x7fb7cc1049b0 2024-09-21T03:49:05.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.563+0000 7fb7d2a00640 1 -- 172.21.15.16:0/4231029343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7cc1ad250 con 0x7fb7cc1049b0 2024-09-21T03:49:05.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.564+0000 7fb7d2a00640 1 -- 172.21.15.16:0/4231029343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7cc06c2c0 con 0x7fb7cc1049b0 2024-09-21T03:49:05.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.567+0000 7fb7c9600640 1 -- 172.21.15.16:0/4231029343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb7bc042030 con 0x7fb7cc1049b0 2024-09-21T03:49:05.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.568+0000 7fb7c9600640 1 --2- 172.21.15.16:0/4231029343 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb7b00b6590 0x7fb7b00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:05.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.568+0000 7fb7c9600640 1 -- 172.21.15.16:0/4231029343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb7bc107c00 con 0x7fb7cc1049b0 2024-09-21T03:49:05.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.568+0000 7fb7c3e00640 1 --2- 172.21.15.16:0/4231029343 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb7b00b6590 0x7fb7b00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:05.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.569+0000 7fb7c3e00640 1 --2- 172.21.15.16:0/4231029343 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb7b00b6590 0x7fb7b00b8aa0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc10fb90 tx=0x7fb7b4005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:05.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.571+0000 7fb7c9600640 1 -- 172.21.15.16:0/4231029343 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb7bc0d2460 con 0x7fb7cc1049b0 2024-09-21T03:49:05.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.814+0000 7fb7d2a00640 1 -- 172.21.15.16:0/4231029343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7cc104dc0 con 0x7fb7cc1049b0 2024-09-21T03:49:05.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.815+0000 7fb7c9600640 1 -- 172.21.15.16:0/4231029343 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7fb7cc104dc0 con 0x7fb7cc1049b0 2024-09-21T03:49:05.817 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:49:05.817 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:49:05.818 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:49:05.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.818+0000 7fb7d2a00640 1 -- 172.21.15.16:0/4231029343 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb7b00b6590 msgr2=0x7fb7b00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:05.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.818+0000 7fb7d2a00640 1 --2- 172.21.15.16:0/4231029343 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb7b00b6590 0x7fb7b00b8aa0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc10fb90 tx=0x7fb7b4005f90 comp rx=0 tx=0).stop 2024-09-21T03:49:05.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.818+0000 7fb7d2a00640 1 -- 172.21.15.16:0/4231029343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc1049b0 msgr2=0x7fb7cc104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:05.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.818+0000 7fb7d2a00640 1 --2- 172.21.15.16:0/4231029343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc1049b0 0x7fb7cc104100 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc005940 tx=0x7fb7bc0040c0 comp rx=0 tx=0).stop 2024-09-21T03:49:05.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.818+0000 7fb7d2a00640 1 -- 172.21.15.16:0/4231029343 shutdown_connections 2024-09-21T03:49:05.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.818+0000 7fb7d2a00640 1 --2- 172.21.15.16:0/4231029343 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb7b00b6590 0x7fb7b00b8aa0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:05.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.818+0000 7fb7d2a00640 1 --2- 172.21.15.16:0/4231029343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7cc10de10 0x7fb7cc102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:05.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.818+0000 7fb7d2a00640 1 --2- 172.21.15.16:0/4231029343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7cc105300 0x7fb7cc1026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:05.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.818+0000 7fb7d2a00640 1 --2- 172.21.15.16:0/4231029343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7cc1049b0 0x7fb7cc104100 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.818+0000 7fb7d2a00640 1 -- 172.21.15.16:0/4231029343 >> 172.21.15.16:0/4231029343 conn(0x7fb7cc079760 msgr2=0x7fb7cc107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.818+0000 7fb7d2a00640 1 -- 172.21.15.16:0/4231029343 shutdown_connections 2024-09-21T03:49:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:05.818+0000 7fb7d2a00640 1 -- 172.21.15.16:0/4231029343 wait complete. 2024-09-21T03:49:06.734 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db --set-id b00a0f67 --release --await' 2024-09-21T03:49:06.734 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db --set-id b00a0f67 --release --await 2024-09-21T03:49:06.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:06 smithi123 ceph-mon[34673]: pgmap v502: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 117 MiB/s wr, 738 op/s 2024-09-21T03:49:06.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:06 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/4231029343' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:06.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:06 smithi016 ceph-mon[30375]: pgmap v502: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 117 MiB/s wr, 738 op/s 2024-09-21T03:49:06.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:06 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/4231029343' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:07.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:06 smithi134 ceph-mon[35031]: pgmap v502: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 117 MiB/s wr, 738 op/s 2024-09-21T03:49:07.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:06 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/4231029343' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:07.116 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:07.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.794+0000 7fbc9d800640 1 -- 172.21.15.16:0/3177078642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc9810c010 msgr2=0x7fbc98112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.794+0000 7fbc9d800640 1 --2- 172.21.15.16:0/3177078642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc9810c010 0x7fbc98112840 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009ca0 tx=0x7fbc9002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.794+0000 7fbc9d800640 1 -- 172.21.15.16:0/3177078642 shutdown_connections 2024-09-21T03:49:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.794+0000 7fbc9d800640 1 --2- 172.21.15.16:0/3177078642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc9810c010 0x7fbc98112840 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.794+0000 7fbc9d800640 1 --2- 172.21.15.16:0/3177078642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9810b6c0 0x7fbc9810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.794+0000 7fbc9d800640 1 --2- 172.21.15.16:0/3177078642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc98105d80 0x7fbc98106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.794+0000 7fbc9d800640 1 -- 172.21.15.16:0/3177078642 >> 172.21.15.16:0/3177078642 conn(0x7fbc98101810 msgr2=0x7fbc98103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.794+0000 7fbc9d800640 1 -- 172.21.15.16:0/3177078642 shutdown_connections 2024-09-21T03:49:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.794+0000 7fbc9d800640 1 -- 172.21.15.16:0/3177078642 wait complete. 2024-09-21T03:49:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.795+0000 7fbc9d800640 1 Processor -- start 2024-09-21T03:49:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.795+0000 7fbc9d800640 1 -- start start 2024-09-21T03:49:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.796+0000 7fbc9d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc98105d80 0x7fbc9819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.796+0000 7fbc9d800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc9810b6c0 0x7fbc9819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.796+0000 7fbc9d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9810c010 0x7fbc981a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.796+0000 7fbc9d800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc98115d20 con 0x7fbc98105d80 2024-09-21T03:49:07.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.796+0000 7fbc9d800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbc98115ba0 con 0x7fbc9810c010 2024-09-21T03:49:07.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.796+0000 7fbc9d800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc98115ea0 con 0x7fbc9810b6c0 2024-09-21T03:49:07.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.796+0000 7fbc96a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc98105d80 0x7fbc9819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.796+0000 7fbc96000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc9810b6c0 0x7fbc9819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.796+0000 7fbc96a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc98105d80 0x7fbc9819c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41630/0 (socket says 172.21.15.16:41630) 2024-09-21T03:49:07.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.796+0000 7fbc96a00640 1 -- 172.21.15.16:0/845543121 learned_addr learned my addr 172.21.15.16:0/845543121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:07.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.796+0000 7fbc97400640 1 --2- 172.21.15.16:0/845543121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9810c010 0x7fbc981a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.797+0000 7fbc96000640 1 -- 172.21.15.16:0/845543121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9810c010 msgr2=0x7fbc981a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.797+0000 7fbc96000640 1 --2- 172.21.15.16:0/845543121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9810c010 0x7fbc981a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.797+0000 7fbc96000640 1 -- 172.21.15.16:0/845543121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc98105d80 msgr2=0x7fbc9819c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.797+0000 7fbc96000640 1 --2- 172.21.15.16:0/845543121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc98105d80 0x7fbc9819c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.797+0000 7fbc96000640 1 -- 172.21.15.16:0/845543121 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc981a25c0 con 0x7fbc9810b6c0 2024-09-21T03:49:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.797+0000 7fbc96a00640 1 --2- 172.21.15.16:0/845543121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc98105d80 0x7fbc9819c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.797+0000 7fbc96000640 1 --2- 172.21.15.16:0/845543121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc9810b6c0 0x7fbc9819cbf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc9806caa0 tx=0x7fbc8400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.797+0000 7fbc8f400640 1 -- 172.21.15.16:0/845543121 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8400cc40 con 0x7fbc9810b6c0 2024-09-21T03:49:07.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.797+0000 7fbc9d800640 1 -- 172.21.15.16:0/845543121 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc981a28b0 con 0x7fbc9810b6c0 2024-09-21T03:49:07.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.797+0000 7fbc8f400640 1 -- 172.21.15.16:0/845543121 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc8400cde0 con 0x7fbc9810b6c0 2024-09-21T03:49:07.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.797+0000 7fbc8f400640 1 -- 172.21.15.16:0/845543121 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84004a00 con 0x7fbc9810b6c0 2024-09-21T03:49:07.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.797+0000 7fbc9d800640 1 -- 172.21.15.16:0/845543121 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc981a9010 con 0x7fbc9810b6c0 2024-09-21T03:49:07.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.801+0000 7fbca45e8740 1 Processor -- start 2024-09-21T03:49:07.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.801+0000 7fbca45e8740 1 -- start start 2024-09-21T03:49:07.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.801+0000 7fbc8f400640 1 -- 172.21.15.16:0/845543121 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fbc84004ba0 con 0x7fbc9810b6c0 2024-09-21T03:49:07.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.801+0000 7fbca45e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55779926d520 0x5577996141c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.801+0000 7fbca45e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557799244660 0x557799614700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.801+0000 7fbca45e8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55779924c840 0x557799614c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.801+0000 7fbca45e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55779958b290 con 0x55779926d520 2024-09-21T03:49:07.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.801+0000 7fbca45e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55779957a6c0 con 0x557799244660 2024-09-21T03:49:07.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.801+0000 7fbca45e8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55779957e4d0 con 0x55779924c840 2024-09-21T03:49:07.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.802+0000 7fbc97400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55779926d520 0x5577996141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.802+0000 7fbc97400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55779926d520 0x5577996141c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41638/0 (socket says 172.21.15.16:41638) 2024-09-21T03:49:07.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.802+0000 7fbc97400640 1 -- 172.21.15.16:0/3463049828 learned_addr learned my addr 172.21.15.16:0/3463049828 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:07.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.802+0000 7fbc96a00640 1 --2- 172.21.15.16:0/3463049828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557799244660 0x557799614700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.802+0000 7fbc96000640 1 --2- 172.21.15.16:0/3463049828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55779924c840 0x557799614c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.802+0000 7fbc97400640 1 -- 172.21.15.16:0/3463049828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55779924c840 msgr2=0x557799614c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.802+0000 7fbc97400640 1 --2- 172.21.15.16:0/3463049828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55779924c840 0x557799614c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.802+0000 7fbc97400640 1 -- 172.21.15.16:0/3463049828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557799244660 msgr2=0x557799614700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.802+0000 7fbc97400640 1 --2- 172.21.15.16:0/3463049828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557799244660 0x557799614700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.802+0000 7fbc97400640 1 -- 172.21.15.16:0/3463049828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557799507460 con 0x55779926d520 2024-09-21T03:49:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.802+0000 7fbc96000640 1 --2- 172.21.15.16:0/3463049828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55779924c840 0x557799614c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.803+0000 7fbc97400640 1 --2- 172.21.15.16:0/3463049828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55779926d520 0x5577996141c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009ca0 tx=0x7fbc9002f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b82d106c4c8136e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.803+0000 7fbc83e00640 1 -- 172.21.15.16:0/3463049828 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90031db0 con 0x55779926d520 2024-09-21T03:49:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.803+0000 7fbc83e00640 1 -- 172.21.15.16:0/3463049828 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc90036030 con 0x55779926d520 2024-09-21T03:49:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.803+0000 7fbc83e00640 1 -- 172.21.15.16:0/3463049828 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc900042d0 con 0x55779926d520 2024-09-21T03:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.803+0000 7fbc8f400640 1 --2- 172.21.15.16:0/845543121 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbc7c0b6510 0x7fbc7c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.803+0000 7fbc8f400640 1 -- 172.21.15.16:0/845543121 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fbc840e75e0 con 0x7fbc9810b6c0 2024-09-21T03:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.804+0000 7fbc96a00640 1 --2- 172.21.15.16:0/845543121 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbc7c0b6510 0x7fbc7c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.804+0000 7fbca45e8740 1 -- 172.21.15.16:0/3463049828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55779926d520 msgr2=0x5577996141c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.804+0000 7fbca45e8740 1 --2- 172.21.15.16:0/3463049828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55779926d520 0x5577996141c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009ca0 tx=0x7fbc9002f310 comp rx=0 tx=0).stop 2024-09-21T03:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.804+0000 7fbca45e8740 1 -- 172.21.15.16:0/3463049828 shutdown_connections 2024-09-21T03:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.804+0000 7fbca45e8740 1 --2- 172.21.15.16:0/3463049828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55779924c840 0x557799614c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.804+0000 7fbca45e8740 1 --2- 172.21.15.16:0/3463049828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557799244660 0x557799614700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.804+0000 7fbca45e8740 1 --2- 172.21.15.16:0/3463049828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55779926d520 0x5577996141c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.804+0000 7fbca45e8740 1 -- 172.21.15.16:0/3463049828 >> 172.21.15.16:0/3463049828 conn(0x557799261910 msgr2=0x5577996120b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.804+0000 7fbca45e8740 1 -- 172.21.15.16:0/3463049828 shutdown_connections 2024-09-21T03:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.804+0000 7fbc96a00640 1 --2- 172.21.15.16:0/845543121 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbc7c0b6510 0x7fbc7c0b8a20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x557799612e40 tx=0x7fbc8800b4a0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.804+0000 7fbca45e8740 1 -- 172.21.15.16:0/3463049828 wait complete. 2024-09-21T03:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.805+0000 7fbca45e8740 1 -- start start 2024-09-21T03:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.805+0000 7fbca45e8740 1 Processor -- start 2024-09-21T03:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.806+0000 7fbca45e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557799244660 0x55779961fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.806+0000 7fbca45e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 0x5577996203b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.806+0000 7fbca45e8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55779926d520 0x55779962bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.806+0000 7fbca45e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5577993dbbb0 con 0x557799244660 2024-09-21T03:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.806+0000 7fbca45e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5577995156c0 con 0x55779924c840 2024-09-21T03:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.806+0000 7fbca45e8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5577996227c0 con 0x55779926d520 2024-09-21T03:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.806+0000 7fbca45e8740 10 client.0 _ll_register_callbacks cb 0x557799530490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.806+0000 7fbc96000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 0x5577996203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.806+0000 7fbc96a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557799244660 0x55779961fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.806+0000 7fbc96000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 0x5577996203b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34258/0 (socket says 172.21.15.16:34258) 2024-09-21T03:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.806+0000 7fbc96000640 1 -- 172.21.15.16:0/1614340459 learned_addr learned my addr 172.21.15.16:0/1614340459 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.806+0000 7fbc97400640 1 --2- 172.21.15.16:0/1614340459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55779926d520 0x55779962bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.807+0000 7fbc96000640 1 -- 172.21.15.16:0/1614340459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55779926d520 msgr2=0x55779962bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.807+0000 7fbc96000640 1 --2- 172.21.15.16:0/1614340459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55779926d520 0x55779962bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.807+0000 7fbc96000640 1 -- 172.21.15.16:0/1614340459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557799244660 msgr2=0x55779961fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.807+0000 7fbc96000640 1 --2- 172.21.15.16:0/1614340459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557799244660 0x55779961fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.807+0000 7fbc96000640 1 -- 172.21.15.16:0/1614340459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577992d8b00 con 0x55779924c840 2024-09-21T03:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.807+0000 7fbc96000640 1 --2- 172.21.15.16:0/1614340459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 0x5577996203b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbc840a9930 tx=0x7fbc84005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8118a72f6eafb08c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.807+0000 7fbc80c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:49:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.807+0000 7fbc80c00640 1 -- 172.21.15.16:0/1614340459 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8401c070 con 0x55779924c840 2024-09-21T03:49:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.807+0000 7fbc80c00640 1 -- 172.21.15.16:0/1614340459 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fbc840ac5c0 con 0x55779924c840 2024-09-21T03:49:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.807+0000 7fbca45e8740 1 -- 172.21.15.16:0/1614340459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc90036030 con 0x55779924c840 2024-09-21T03:49:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.808+0000 7fbc80c00640 1 -- 172.21.15.16:0/1614340459 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbc840acad0 con 0x55779924c840 2024-09-21T03:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.808+0000 7fbca45e8740 10 client.25883 fetch_fsmap learned FSMap version 30 2024-09-21T03:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.808+0000 7fbca45e8740 1 -- 172.21.15.16:0/1614340459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55779934f960 con 0x55779924c840 2024-09-21T03:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.808+0000 7fbc80c00640 1 -- 172.21.15.16:0/1614340459 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc90036030 con 0x55779924c840 2024-09-21T03:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.808+0000 7fbc80c00640 1 -- 172.21.15.16:0/1614340459 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fbc840d0ce0 con 0x55779924c840 2024-09-21T03:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.808+0000 7fbca45e8740 10 client.25883 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.808+0000 7fbca45e8740 10 client.25883 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.808+0000 7fbca45e8740 20 client.25883 populate_metadata read hostname 'smithi016' 2024-09-21T03:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.808+0000 7fbca45e8740 1 --2- 172.21.15.16:0/1614340459 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55779922bea0 0x557799631ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.808+0000 7fbca45e8740 4 client.25883 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.808+0000 7fbca45e8740 1 -- 172.21.15.16:0/1614340459 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55779958d1e0 con 0x55779922bea0 2024-09-21T03:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.809+0000 7fbc96a00640 1 --2- 172.21.15.16:0/1614340459 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55779922bea0 0x557799631ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.809+0000 7fbc96a00640 1 --2- 172.21.15.16:0/1614340459 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55779922bea0 0x557799631ae0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c15579e0a0292b9 server_cookie=d9dd1cd69fafcb1d in_seq=0 out_seq=0 2024-09-21T03:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.809+0000 7fbc80c00640 10 client.25883 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.810+0000 7fbc80c00640 1 -- 172.21.15.16:0/1614340459 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55779958d1e0 con 0x55779922bea0 2024-09-21T03:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.810+0000 7fbc80c00640 10 client.25883 handle_command_reply: tid=0 2024-09-21T03:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.810+0000 7fbca45e8740 1 client.25883 shutdown 2024-09-21T03:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.810+0000 7fbca45e8740 1 -- 172.21.15.16:0/1614340459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 msgr2=0x5577996203b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.810+0000 7fbca45e8740 1 --2- 172.21.15.16:0/1614340459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 0x5577996203b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbc840a9930 tx=0x7fbc84005250 comp rx=0 tx=0).stop 2024-09-21T03:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.810+0000 7fbca45e8740 1 -- 172.21.15.16:0/1614340459 shutdown_connections 2024-09-21T03:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.810+0000 7fbca45e8740 1 --2- 172.21.15.16:0/1614340459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55779926d520 0x55779962bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.810+0000 7fbca45e8740 1 --2- 172.21.15.16:0/1614340459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 0x5577996203b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.810+0000 7fbca45e8740 1 --2- 172.21.15.16:0/1614340459 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55779922bea0 0x557799631ae0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.810+0000 7fbca45e8740 1 --2- 172.21.15.16:0/1614340459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557799244660 0x55779961fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.810+0000 7fbc80c00640 0 client.25883 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.810+0000 7fbca45e8740 1 -- 172.21.15.16:0/1614340459 >> 172.21.15.16:0/1614340459 conn(0x557799261910 msgr2=0x5577996173a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:07.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.811+0000 7fbca45e8740 1 -- 172.21.15.16:0/1614340459 shutdown_connections 2024-09-21T03:49:07.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.811+0000 7fbca45e8740 1 -- 172.21.15.16:0/1614340459 wait complete. 2024-09-21T03:49:07.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.811+0000 7fbca45e8740 20 client.25883 trim_cache size 0 max 16384 2024-09-21T03:49:07.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.829+0000 7fbca45e8740 1 Processor -- start 2024-09-21T03:49:07.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.829+0000 7fbca45e8740 1 -- start start 2024-09-21T03:49:07.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.829+0000 7fbca45e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557799244660 0x557799619440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.829+0000 7fbca45e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 0x557799619980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.829+0000 7fbca45e8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55779926d520 0x557799611660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.829+0000 7fbca45e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981b2cd0 con 0x557799244660 2024-09-21T03:49:07.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.829+0000 7fbca45e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981b4810 con 0x55779924c840 2024-09-21T03:49:07.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.829+0000 7fbca45e8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981b33a0 con 0x55779926d520 2024-09-21T03:49:07.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.829+0000 7fbc96a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557799244660 0x557799619440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.830+0000 7fbc96000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 0x557799619980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.830+0000 7fbc96a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557799244660 0x557799619440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41664/0 (socket says 172.21.15.16:41664) 2024-09-21T03:49:07.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.830+0000 7fbc96a00640 1 -- 172.21.15.16:0/2456640096 learned_addr learned my addr 172.21.15.16:0/2456640096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:07.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.830+0000 7fbc97400640 1 --2- 172.21.15.16:0/2456640096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55779926d520 0x557799611660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.830+0000 7fbc96000640 1 -- 172.21.15.16:0/2456640096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55779926d520 msgr2=0x557799611660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:07.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.830+0000 7fbc96000640 1 --2- 172.21.15.16:0/2456640096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55779926d520 0x557799611660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.830+0000 7fbc96000640 1 -- 172.21.15.16:0/2456640096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557799244660 msgr2=0x557799619440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:07.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.830+0000 7fbc96000640 1 --2- 172.21.15.16:0/2456640096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557799244660 0x557799619440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.830+0000 7fbc96000640 1 -- 172.21.15.16:0/2456640096 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557799616a50 con 0x55779924c840 2024-09-21T03:49:07.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.830+0000 7fbc96a00640 1 --2- 172.21.15.16:0/2456640096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557799244660 0x557799619440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:07.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.830+0000 7fbc96000640 1 --2- 172.21.15.16:0/2456640096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 0x557799619980 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbc84004610 tx=0x7fbc840096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a716eed188d8f5a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:07.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.831+0000 7fbc83400640 1 -- 172.21.15.16:0/2456640096 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8401c070 con 0x55779924c840 2024-09-21T03:49:07.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.831+0000 7fbc83400640 1 -- 172.21.15.16:0/2456640096 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc84094070 con 0x55779924c840 2024-09-21T03:49:07.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.831+0000 7fbc83400640 1 -- 172.21.15.16:0/2456640096 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc840b1e40 con 0x55779924c840 2024-09-21T03:49:07.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.831+0000 7fbca45e8740 1 -- 172.21.15.16:0/2456640096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 msgr2=0x557799619980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:07.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.831+0000 7fbca45e8740 1 --2- 172.21.15.16:0/2456640096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 0x557799619980 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbc84004610 tx=0x7fbc840096e0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.832+0000 7fbca45e8740 1 -- 172.21.15.16:0/2456640096 shutdown_connections 2024-09-21T03:49:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.832+0000 7fbca45e8740 1 --2- 172.21.15.16:0/2456640096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55779926d520 0x557799611660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.832+0000 7fbca45e8740 1 --2- 172.21.15.16:0/2456640096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 0x557799619980 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.832+0000 7fbca45e8740 1 --2- 172.21.15.16:0/2456640096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557799244660 0x557799619440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.832+0000 7fbca45e8740 1 -- 172.21.15.16:0/2456640096 >> 172.21.15.16:0/2456640096 conn(0x557799261910 msgr2=0x557799621150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.832+0000 7fbca45e8740 1 -- 172.21.15.16:0/2456640096 shutdown_connections 2024-09-21T03:49:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.832+0000 7fbca45e8740 1 -- 172.21.15.16:0/2456640096 wait complete. 2024-09-21T03:49:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.832+0000 7fbca45e8740 1 -- start start 2024-09-21T03:49:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.832+0000 7fbca45e8740 1 Processor -- start 2024-09-21T03:49:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.833+0000 7fbca45e8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557799244660 0x557799622e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.833+0000 7fbca45e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 0x557799623340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.833+0000 7fbca45e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55779926d520 0x55779969c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.833+0000 7fbca45e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981b4140 con 0x55779926d520 2024-09-21T03:49:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.833+0000 7fbca45e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981b3a70 con 0x55779924c840 2024-09-21T03:49:07.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.833+0000 7fbca45e8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981b0cb0 con 0x557799244660 2024-09-21T03:49:07.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.833+0000 7fbca45e8740 10 client.0 _ll_register_callbacks cb 0x7fbc840052f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:07.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.833+0000 7fbc97400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55779926d520 0x55779969c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.833+0000 7fbc96000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 0x557799623340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.833+0000 7fbc96000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 0x557799623340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34266/0 (socket says 172.21.15.16:34266) 2024-09-21T03:49:07.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.833+0000 7fbc97400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55779926d520 0x55779969c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41678/0 (socket says 172.21.15.16:41678) 2024-09-21T03:49:07.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.833+0000 7fbc96000640 1 -- 172.21.15.16:0/1978342533 learned_addr learned my addr 172.21.15.16:0/1978342533 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:07.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.833+0000 7fbc96a00640 1 --2- 172.21.15.16:0/1978342533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557799244660 0x557799622e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.834+0000 7fbc97400640 1 -- 172.21.15.16:0/1978342533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557799244660 msgr2=0x557799622e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:07.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.834+0000 7fbc97400640 1 --2- 172.21.15.16:0/1978342533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557799244660 0x557799622e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.834+0000 7fbc97400640 1 -- 172.21.15.16:0/1978342533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 msgr2=0x557799623340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:07.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.834+0000 7fbc97400640 1 --2- 172.21.15.16:0/1978342533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 0x557799623340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.834+0000 7fbc97400640 1 -- 172.21.15.16:0/1978342533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557799623b70 con 0x55779926d520 2024-09-21T03:49:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.834+0000 7fbc97400640 1 --2- 172.21.15.16:0/1978342533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55779926d520 0x55779969c4a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbc9002ade0 tx=0x7fbc9002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6811d99b37bd972 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.834+0000 7fbc80c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:49:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.834+0000 7fbc80c00640 1 -- 172.21.15.16:0/1978342533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9004a040 con 0x55779926d520 2024-09-21T03:49:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.834+0000 7fbca45e8740 1 -- 172.21.15.16:0/1978342533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc84094070 con 0x55779926d520 2024-09-21T03:49:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.834+0000 7fbc80c00640 1 -- 172.21.15.16:0/1978342533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fbc90031450 con 0x55779926d520 2024-09-21T03:49:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.835+0000 7fbc80c00640 1 -- 172.21.15.16:0/1978342533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbc90031670 con 0x55779926d520 2024-09-21T03:49:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.835+0000 7fbca45e8740 10 client.15831 fetch_fsmap learned FSMap version 30 2024-09-21T03:49:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.835+0000 7fbca45e8740 1 -- 172.21.15.16:0/1978342533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55779969da00 con 0x55779926d520 2024-09-21T03:49:07.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.835+0000 7fbc80c00640 1 -- 172.21.15.16:0/1978342533 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc84094070 con 0x55779926d520 2024-09-21T03:49:07.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.835+0000 7fbc80c00640 1 -- 172.21.15.16:0/1978342533 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fbc90043070 con 0x55779926d520 2024-09-21T03:49:07.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.835+0000 7fbca45e8740 10 client.15831 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:49:07.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.835+0000 7fbca45e8740 10 client.15831 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:49:07.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.835+0000 7fbca45e8740 20 client.15831 populate_metadata read hostname 'smithi016' 2024-09-21T03:49:07.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.835+0000 7fbca45e8740 1 --2- 172.21.15.16:0/1978342533 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557799201250 0x55779969ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:07.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.835+0000 7fbca45e8740 4 client.15831 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b00a0f67", "release": true, "await": true}] 2024-09-21T03:49:07.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.835+0000 7fbca45e8740 1 -- 172.21.15.16:0/1978342533 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b00a0f67", "release": true, "await": true}) -- 0x557799620740 con 0x557799201250 2024-09-21T03:49:07.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.836+0000 7fbc96a00640 1 --2- 172.21.15.16:0/1978342533 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557799201250 0x55779969ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:07.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.836+0000 7fbc96a00640 1 --2- 172.21.15.16:0/1978342533 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557799201250 0x55779969ff00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3c373f85ada22db server_cookie=c1b3bc6df763d754 in_seq=0 out_seq=0 2024-09-21T03:49:07.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.836+0000 7fbc80c00640 10 client.15831 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:07.844 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:49:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:49:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:49:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 12, 2024-09-21T03:49:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:49:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "b00a0f67": { 2024-09-21T03:49:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 12, 2024-09-21T03:49:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:49:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:49:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:49:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:49:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:49:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:49:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 147.8, 2024-09-21T03:49:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:49:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:49:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:49:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:49:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:49:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:49:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:49:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:49:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:49:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:49:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:49:07.846 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:49:07.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.840+0000 7fbc80c00640 1 -- 172.21.15.16:0/1978342533 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557799620740 con 0x557799201250 2024-09-21T03:49:07.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.840+0000 7fbc80c00640 10 client.15831 handle_command_reply: tid=0 2024-09-21T03:49:07.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.840+0000 7fbca45e8740 1 client.15831 shutdown 2024-09-21T03:49:07.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.841+0000 7fbca45e8740 1 -- 172.21.15.16:0/1978342533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55779926d520 msgr2=0x55779969c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:07.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.841+0000 7fbca45e8740 1 --2- 172.21.15.16:0/1978342533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55779926d520 0x55779969c4a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbc9002ade0 tx=0x7fbc9002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.841+0000 7fbca45e8740 1 -- 172.21.15.16:0/1978342533 shutdown_connections 2024-09-21T03:49:07.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.841+0000 7fbca45e8740 1 --2- 172.21.15.16:0/1978342533 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557799201250 0x55779969ff00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.841+0000 7fbca45e8740 1 --2- 172.21.15.16:0/1978342533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55779926d520 0x55779969c4a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.841+0000 7fbca45e8740 1 --2- 172.21.15.16:0/1978342533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55779924c840 0x557799623340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.841+0000 7fbca45e8740 1 --2- 172.21.15.16:0/1978342533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557799244660 0x557799622e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.841+0000 7fbca45e8740 1 -- 172.21.15.16:0/1978342533 >> 172.21.15.16:0/1978342533 conn(0x557799261910 msgr2=0x5577996172c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:07.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.841+0000 7fbc80c00640 0 client.15831 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:07.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.841+0000 7fbca45e8740 1 -- 172.21.15.16:0/1978342533 shutdown_connections 2024-09-21T03:49:07.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.841+0000 7fbca45e8740 1 -- 172.21.15.16:0/1978342533 wait complete. 2024-09-21T03:49:07.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.842+0000 7fbca45e8740 20 client.15831 trim_cache size 0 max 16384 2024-09-21T03:49:07.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.844+0000 7fbc9d800640 1 -- 172.21.15.16:0/845543121 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbc7c0b6510 msgr2=0x7fbc7c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:07.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.844+0000 7fbc9d800640 1 --2- 172.21.15.16:0/845543121 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbc7c0b6510 0x7fbc7c0b8a20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x557799612e40 tx=0x7fbc8800b4a0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.844+0000 7fbc9d800640 1 -- 172.21.15.16:0/845543121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc9810b6c0 msgr2=0x7fbc9819cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:07.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.844+0000 7fbc9d800640 1 --2- 172.21.15.16:0/845543121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc9810b6c0 0x7fbc9819cbf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc9806caa0 tx=0x7fbc8400ee50 comp rx=0 tx=0).stop 2024-09-21T03:49:07.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.845+0000 7fbc9d800640 1 -- 172.21.15.16:0/845543121 shutdown_connections 2024-09-21T03:49:07.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.845+0000 7fbc9d800640 1 --2- 172.21.15.16:0/845543121 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbc7c0b6510 0x7fbc7c0b8a20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.845+0000 7fbc9d800640 1 --2- 172.21.15.16:0/845543121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9810c010 0x7fbc981a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.845+0000 7fbc9d800640 1 --2- 172.21.15.16:0/845543121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc9810b6c0 0x7fbc9819cbf0 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc9806caa0 tx=0x7fbc8400ee50 comp rx=0 tx=0).stop 2024-09-21T03:49:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.845+0000 7fbc9d800640 1 --2- 172.21.15.16:0/845543121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc98105d80 0x7fbc9819c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.845+0000 7fbc9d800640 1 -- 172.21.15.16:0/845543121 >> 172.21.15.16:0/845543121 conn(0x7fbc98101810 msgr2=0x7fbc9810acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.846+0000 7fbc9d800640 1 -- 172.21.15.16:0/845543121 shutdown_connections 2024-09-21T03:49:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:07.846+0000 7fbc9d800640 1 -- 172.21.15.16:0/845543121 wait complete. 2024-09-21T03:49:08.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:08 smithi123 ceph-mon[34673]: pgmap v503: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 69 MiB/s wr, 461 op/s 2024-09-21T03:49:08.921 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:08 smithi016 ceph-mon[30375]: pgmap v503: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 69 MiB/s wr, 461 op/s 2024-09-21T03:49:09.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:08 smithi134 ceph-mon[35031]: pgmap v503: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 69 MiB/s wr, 461 op/s 2024-09-21T03:49:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.250+0000 7f0be4e00640 1 -- 172.21.15.16:0/1227351472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be0113c30 msgr2=0x7f0be0116120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.250+0000 7f0be4e00640 1 --2- 172.21.15.16:0/1227351472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be0113c30 0x7f0be0116120 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8009ca0 tx=0x7f0bd802a8f0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.253+0000 7f0be4e00640 1 -- 172.21.15.16:0/1227351472 shutdown_connections 2024-09-21T03:49:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.253+0000 7f0be4e00640 1 --2- 172.21.15.16:0/1227351472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be0113c30 0x7f0be0116120 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.253+0000 7f0be4e00640 1 --2- 172.21.15.16:0/1227351472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0be010cd20 0x7f0be010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.253+0000 7f0be4e00640 1 --2- 172.21.15.16:0/1227351472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be010a3c0 0x7f0be010c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.253+0000 7f0be4e00640 1 -- 172.21.15.16:0/1227351472 >> 172.21.15.16:0/1227351472 conn(0x7f0be0070ed0 msgr2=0x7f0be0073380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.258+0000 7f0be4e00640 1 -- 172.21.15.16:0/1227351472 shutdown_connections 2024-09-21T03:49:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.258+0000 7f0be4e00640 1 -- 172.21.15.16:0/1227351472 wait complete. 2024-09-21T03:49:09.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.259+0000 7f0be4e00640 1 Processor -- start 2024-09-21T03:49:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.260+0000 7f0be4e00640 1 -- start start 2024-09-21T03:49:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.260+0000 7f0be4e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0be010a3c0 0x7f0be010daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.260+0000 7f0be4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be010cd20 0x7f0be010e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.261+0000 7f0bdf400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0be010a3c0 0x7f0be010daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.261+0000 7f0bdf400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0be010a3c0 0x7f0be010daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:35324/0 (socket says 172.21.15.16:35324) 2024-09-21T03:49:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.262+0000 7f0bdf400640 1 -- 172.21.15.16:0/3163023630 learned_addr learned my addr 172.21.15.16:0/3163023630 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.262+0000 7f0be4e00640 1 --2- 172.21.15.16:0/3163023630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be0113c30 0x7f0be010ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.262+0000 7f0be4e00640 1 -- 172.21.15.16:0/3163023630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0119b90 con 0x7f0be010cd20 2024-09-21T03:49:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.262+0000 7f0be4e00640 1 -- 172.21.15.16:0/3163023630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0119a10 con 0x7f0be0113c30 2024-09-21T03:49:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.262+0000 7f0be4e00640 1 -- 172.21.15.16:0/3163023630 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0119d10 con 0x7f0be010a3c0 2024-09-21T03:49:09.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.262+0000 7f0bdfe00640 1 --2- 172.21.15.16:0/3163023630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be0113c30 0x7f0be010ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:09.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.263+0000 7f0bdfe00640 1 -- 172.21.15.16:0/3163023630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0be010a3c0 msgr2=0x7f0be010daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:09.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.263+0000 7f0bdfe00640 1 --2- 172.21.15.16:0/3163023630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0be010a3c0 0x7f0be010daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.263+0000 7f0bdfe00640 1 -- 172.21.15.16:0/3163023630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be010cd20 msgr2=0x7f0be010e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:09.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.263+0000 7f0bdfe00640 1 --2- 172.21.15.16:0/3163023630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be010cd20 0x7f0be010e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.263+0000 7f0bdfe00640 1 -- 172.21.15.16:0/3163023630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be010f450 con 0x7f0be0113c30 2024-09-21T03:49:09.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.263+0000 7f0bdfe00640 1 --2- 172.21.15.16:0/3163023630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be0113c30 0x7f0be010ecc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8009bf0 tx=0x7f0bd802f8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:09.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.264+0000 7f0bd7e00640 1 -- 172.21.15.16:0/3163023630 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd802fde0 con 0x7f0be0113c30 2024-09-21T03:49:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.264+0000 7f0be4e00640 1 -- 172.21.15.16:0/3163023630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be010f6e0 con 0x7f0be0113c30 2024-09-21T03:49:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.264+0000 7f0bd7e00640 1 -- 172.21.15.16:0/3163023630 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd802d070 con 0x7f0be0113c30 2024-09-21T03:49:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.266+0000 7f0bd7e00640 1 -- 172.21.15.16:0/3163023630 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd802fde0 con 0x7f0be0113c30 2024-09-21T03:49:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.267+0000 7f0be4e00640 1 -- 172.21.15.16:0/3163023630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be010ff10 con 0x7f0be0113c30 2024-09-21T03:49:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.277+0000 7f0bed47b740 1 Processor -- start 2024-09-21T03:49:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.280+0000 7f0bd7e00640 1 -- 172.21.15.16:0/3163023630 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0bd8030c10 con 0x7f0be0113c30 2024-09-21T03:49:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.280+0000 7f0bed47b740 1 -- start start 2024-09-21T03:49:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.281+0000 7f0bed47b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4831b0 0x55814f82a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.283+0000 7f0bed47b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f45a2f0 0x55814f82a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.283+0000 7f0bed47b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f4624d0 0x55814f82b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.283+0000 7f0bed47b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55814f79d6b0 con 0x55814f4624d0 2024-09-21T03:49:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.283+0000 7f0bed47b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55814f78cae0 con 0x55814f45a2f0 2024-09-21T03:49:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.283+0000 7f0bed47b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55814f7908f0 con 0x55814f4831b0 2024-09-21T03:49:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.284+0000 7f0bdea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4831b0 0x55814f82a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.284+0000 7f0bdea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4831b0 0x55814f82a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:35340/0 (socket says 172.21.15.16:35340) 2024-09-21T03:49:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.284+0000 7f0bdea00640 1 -- 172.21.15.16:0/1843236361 learned_addr learned my addr 172.21.15.16:0/1843236361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.285+0000 7f0bdea00640 1 --2- 172.21.15.16:0/1843236361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f4624d0 0x55814f82b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.285+0000 7f0bdf400640 1 --2- 172.21.15.16:0/1843236361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f45a2f0 0x55814f82a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.289+0000 7f0bdea00640 1 -- 172.21.15.16:0/1843236361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f45a2f0 msgr2=0x55814f82a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.289+0000 7f0bdea00640 1 --2- 172.21.15.16:0/1843236361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f45a2f0 0x55814f82a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.289+0000 7f0bdea00640 1 -- 172.21.15.16:0/1843236361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f4624d0 msgr2=0x55814f82b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.289+0000 7f0bdea00640 1 --2- 172.21.15.16:0/1843236361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f4624d0 0x55814f82b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.289+0000 7f0bdea00640 1 -- 172.21.15.16:0/1843236361 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55814f759780 con 0x55814f4831b0 2024-09-21T03:49:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.290+0000 7f0bd7e00640 1 --2- 172.21.15.16:0/3163023630 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0bc40b65d0 0x7f0bc40b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.290+0000 7f0bdea00640 1 --2- 172.21.15.16:0/1843236361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4831b0 0x55814f82a2b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0001bb0 tx=0x7f0bd0001280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c5a085cc44e609e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.291+0000 7f0bd4c00640 1 -- 172.21.15.16:0/1843236361 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd000ea70 con 0x55814f4831b0 2024-09-21T03:49:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.291+0000 7f0bd7e00640 1 -- 172.21.15.16:0/3163023630 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0bd810b170 con 0x7f0be0113c30 2024-09-21T03:49:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.291+0000 7f0bdf400640 1 --2- 172.21.15.16:0/3163023630 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0bc40b65d0 0x7f0bc40b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.294+0000 7f0bd4c00640 1 -- 172.21.15.16:0/1843236361 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd000ec10 con 0x55814f4831b0 2024-09-21T03:49:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.294+0000 7f0bd4c00640 1 -- 172.21.15.16:0/1843236361 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0008b60 con 0x55814f4831b0 2024-09-21T03:49:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.296+0000 7f0bed47b740 1 -- 172.21.15.16:0/1843236361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4831b0 msgr2=0x55814f82a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.296+0000 7f0bed47b740 1 --2- 172.21.15.16:0/1843236361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4831b0 0x55814f82a2b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0001bb0 tx=0x7f0bd0001280 comp rx=0 tx=0).stop 2024-09-21T03:49:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.298+0000 7f0bdf400640 1 --2- 172.21.15.16:0/3163023630 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0bc40b65d0 0x7f0bc40b8ae0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00ade0 tx=0x7f0bcc00a380 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.298+0000 7f0bed47b740 1 -- 172.21.15.16:0/1843236361 shutdown_connections 2024-09-21T03:49:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.298+0000 7f0bed47b740 1 --2- 172.21.15.16:0/1843236361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f4624d0 0x55814f82b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.298+0000 7f0bed47b740 1 --2- 172.21.15.16:0/1843236361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f45a2f0 0x55814f82a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.299+0000 7f0bed47b740 1 --2- 172.21.15.16:0/1843236361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4831b0 0x55814f82a2b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.299+0000 7f0bed47b740 1 -- 172.21.15.16:0/1843236361 >> 172.21.15.16:0/1843236361 conn(0x55814f477300 msgr2=0x55814f825d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.301+0000 7f0bed47b740 1 -- 172.21.15.16:0/1843236361 shutdown_connections 2024-09-21T03:49:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.301+0000 7f0bed47b740 1 -- 172.21.15.16:0/1843236361 wait complete. 2024-09-21T03:49:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.302+0000 7f0bed47b740 1 -- start start 2024-09-21T03:49:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.302+0000 7f0bed47b740 1 Processor -- start 2024-09-21T03:49:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.304+0000 7f0bed47b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f45a2f0 0x55814f8339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.304+0000 7f0bed47b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4624d0 0x55814f8340b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.305+0000 7f0bdf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f45a2f0 0x55814f8339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.305+0000 7f0bdea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4624d0 0x55814f8340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.306+0000 7f0bdea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4624d0 0x55814f8340b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:35342/0 (socket says 172.21.15.16:35342) 2024-09-21T03:49:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.306+0000 7f0bdf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f45a2f0 0x55814f8339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41710/0 (socket says 172.21.15.16:41710) 2024-09-21T03:49:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.306+0000 7f0bed47b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f4831b0 0x55814f834c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.306+0000 7f0bdea00640 1 -- 172.21.15.16:0/4083168817 learned_addr learned my addr 172.21.15.16:0/4083168817 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.306+0000 7f0bdfe00640 1 --2- 172.21.15.16:0/4083168817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f4831b0 0x55814f834c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.306+0000 7f0bed47b740 1 -- 172.21.15.16:0/4083168817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55814f6c4ab0 con 0x55814f45a2f0 2024-09-21T03:49:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.306+0000 7f0bed47b740 1 -- 172.21.15.16:0/4083168817 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55814f733570 con 0x55814f4831b0 2024-09-21T03:49:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.306+0000 7f0bed47b740 1 -- 172.21.15.16:0/4083168817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55814f82bf90 con 0x55814f4624d0 2024-09-21T03:49:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.307+0000 7f0bdf400640 1 -- 172.21.15.16:0/4083168817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4624d0 msgr2=0x55814f8340b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.307+0000 7f0bdf400640 1 --2- 172.21.15.16:0/4083168817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4624d0 0x55814f8340b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.307+0000 7f0bdf400640 1 -- 172.21.15.16:0/4083168817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f4831b0 msgr2=0x55814f834c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.307+0000 7f0bdf400640 1 --2- 172.21.15.16:0/4083168817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f4831b0 0x55814f834c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.308+0000 7f0bdf400640 1 -- 172.21.15.16:0/4083168817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55814f4ee790 con 0x55814f45a2f0 2024-09-21T03:49:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.309+0000 7f0bed47b740 10 client.0 _ll_register_callbacks cb 0x55814f742100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.313+0000 7f0bed47b740 1 -- 172.21.15.16:0/4083168817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bd000ec10 con 0x55814f45a2f0 2024-09-21T03:49:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.319+0000 7f0bdf400640 1 --2- 172.21.15.16:0/4083168817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f45a2f0 0x55814f8339f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc010c90 tx=0x7f0bcc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d62a8f1fb1cb0f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.319+0000 7f0bc9600640 10 client.15837 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:49:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.321+0000 7f0bc9600640 1 -- 172.21.15.16:0/4083168817 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc02c060 con 0x55814f45a2f0 2024-09-21T03:49:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.321+0000 7f0bc9600640 1 -- 172.21.15.16:0/4083168817 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0bcc022030 con 0x55814f45a2f0 2024-09-21T03:49:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.321+0000 7f0bc9600640 1 -- 172.21.15.16:0/4083168817 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0bcc02c060 con 0x55814f45a2f0 2024-09-21T03:49:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.325+0000 7f0bed47b740 10 client.15837 fetch_fsmap learned FSMap version 30 2024-09-21T03:49:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.325+0000 7f0bed47b740 1 -- 172.21.15.16:0/4083168817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55814f567620 con 0x55814f45a2f0 2024-09-21T03:49:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.326+0000 7f0bc9600640 1 -- 172.21.15.16:0/4083168817 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0bcc00f8a0 con 0x55814f45a2f0 2024-09-21T03:49:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.326+0000 7f0bed47b740 10 client.15837 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:49:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.326+0000 7f0bed47b740 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.326+0000 7f0bed47b740 20 client.15837 populate_metadata read hostname 'smithi016' 2024-09-21T03:49:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.326+0000 7f0bed47b740 1 --2- 172.21.15.16:0/4083168817 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55814f441d80 0x55814f8414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.326+0000 7f0bed47b740 4 client.15837 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.326+0000 7f0bed47b740 1 -- 172.21.15.16:0/4083168817 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55814f79f600 con 0x55814f441d80 2024-09-21T03:49:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.327+0000 7f0bdea00640 1 --2- 172.21.15.16:0/4083168817 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55814f441d80 0x55814f8414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.328+0000 7f0bdea00640 1 --2- 172.21.15.16:0/4083168817 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55814f441d80 0x55814f8414a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6329e8171c1d432 server_cookie=ac2dece9c584b5cc in_seq=0 out_seq=0 2024-09-21T03:49:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.328+0000 7f0bc9600640 10 client.15837 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.329+0000 7f0bc9600640 1 -- 172.21.15.16:0/4083168817 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55814f79f600 con 0x55814f441d80 2024-09-21T03:49:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.329+0000 7f0bc9600640 10 client.15837 handle_command_reply: tid=0 2024-09-21T03:49:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.329+0000 7f0bed47b740 1 client.15837 shutdown 2024-09-21T03:49:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.330+0000 7f0bed47b740 1 -- 172.21.15.16:0/4083168817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f45a2f0 msgr2=0x55814f8339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.330+0000 7f0bed47b740 1 --2- 172.21.15.16:0/4083168817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f45a2f0 0x55814f8339f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc010c90 tx=0x7f0bcc004730 comp rx=0 tx=0).stop 2024-09-21T03:49:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.331+0000 7f0bed47b740 1 -- 172.21.15.16:0/4083168817 shutdown_connections 2024-09-21T03:49:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.331+0000 7f0bed47b740 1 --2- 172.21.15.16:0/4083168817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f4831b0 0x55814f834c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.331+0000 7f0bed47b740 1 --2- 172.21.15.16:0/4083168817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4624d0 0x55814f8340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.331+0000 7f0bed47b740 1 --2- 172.21.15.16:0/4083168817 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55814f441d80 0x55814f8414a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.331+0000 7f0bed47b740 1 --2- 172.21.15.16:0/4083168817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f45a2f0 0x55814f8339f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.331+0000 7f0bed47b740 1 -- 172.21.15.16:0/4083168817 >> 172.21.15.16:0/4083168817 conn(0x55814f477300 msgr2=0x55814f824360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.331+0000 7f0bc9600640 0 client.15837 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.331+0000 7f0bed47b740 1 -- 172.21.15.16:0/4083168817 shutdown_connections 2024-09-21T03:49:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.331+0000 7f0bed47b740 1 -- 172.21.15.16:0/4083168817 wait complete. 2024-09-21T03:49:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.331+0000 7f0bed47b740 20 client.15837 trim_cache size 0 max 16384 2024-09-21T03:49:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.394+0000 7f0bed47b740 1 Processor -- start 2024-09-21T03:49:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.394+0000 7f0bed47b740 1 -- start start 2024-09-21T03:49:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.394+0000 7f0bed47b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f45a2f0 0x55814f82a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.394+0000 7f0bed47b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f4624d0 0x55814f82ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.394+0000 7f0bed47b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4831b0 0x55814f826180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.395+0000 7f0bed47b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0be01bb300 con 0x55814f4624d0 2024-09-21T03:49:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.395+0000 7f0bed47b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0be01bce40 con 0x55814f45a2f0 2024-09-21T03:49:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.395+0000 7f0bed47b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0be01bb9d0 con 0x55814f4831b0 2024-09-21T03:49:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.395+0000 7f0bdf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f45a2f0 0x55814f82a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.395+0000 7f0bdf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f45a2f0 0x55814f82a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34308/0 (socket says 172.21.15.16:34308) 2024-09-21T03:49:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.395+0000 7f0bdf400640 1 -- 172.21.15.16:0/2702537822 learned_addr learned my addr 172.21.15.16:0/2702537822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.395+0000 7f0bdf400640 1 -- 172.21.15.16:0/2702537822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4831b0 msgr2=0x55814f826180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:49:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.395+0000 7f0bdf400640 1 --2- 172.21.15.16:0/2702537822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4831b0 0x55814f826180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.395+0000 7f0bdf400640 1 -- 172.21.15.16:0/2702537822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f4624d0 msgr2=0x55814f82ea40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:49:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.395+0000 7f0bdf400640 1 --2- 172.21.15.16:0/2702537822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f4624d0 0x55814f82ea40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.395+0000 7f0bdf400640 1 -- 172.21.15.16:0/2702537822 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55814f441fb0 con 0x55814f45a2f0 2024-09-21T03:49:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.396+0000 7f0bdf400640 1 --2- 172.21.15.16:0/2702537822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f45a2f0 0x55814f82a860 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc01c840 tx=0x7f0bcc021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b16ed0b47ed774de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.396+0000 7f0bcbe00640 1 -- 172.21.15.16:0/2702537822 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc040070 con 0x55814f45a2f0 2024-09-21T03:49:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.396+0000 7f0bcbe00640 1 -- 172.21.15.16:0/2702537822 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd000ec10 con 0x55814f45a2f0 2024-09-21T03:49:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.396+0000 7f0bcbe00640 1 -- 172.21.15.16:0/2702537822 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc004730 con 0x55814f45a2f0 2024-09-21T03:49:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.402+0000 7f0bed47b740 1 -- 172.21.15.16:0/2702537822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f45a2f0 msgr2=0x55814f82a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.402+0000 7f0bed47b740 1 --2- 172.21.15.16:0/2702537822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f45a2f0 0x55814f82a860 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc01c840 tx=0x7f0bcc021000 comp rx=0 tx=0).stop 2024-09-21T03:49:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.402+0000 7f0bed47b740 1 -- 172.21.15.16:0/2702537822 shutdown_connections 2024-09-21T03:49:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.402+0000 7f0bed47b740 1 --2- 172.21.15.16:0/2702537822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4831b0 0x55814f826180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.402+0000 7f0bed47b740 1 --2- 172.21.15.16:0/2702537822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f4624d0 0x55814f82ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.402+0000 7f0bed47b740 1 --2- 172.21.15.16:0/2702537822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f45a2f0 0x55814f82a860 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc01c840 tx=0x7f0bcc021000 comp rx=0 tx=0).stop 2024-09-21T03:49:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.403+0000 7f0bed47b740 1 -- 172.21.15.16:0/2702537822 >> 172.21.15.16:0/2702537822 conn(0x55814f477300 msgr2=0x55814f82e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.403+0000 7f0bed47b740 1 -- 172.21.15.16:0/2702537822 shutdown_connections 2024-09-21T03:49:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.403+0000 7f0bed47b740 1 -- 172.21.15.16:0/2702537822 wait complete. 2024-09-21T03:49:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.404+0000 7f0bed47b740 1 -- start start 2024-09-21T03:49:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.404+0000 7f0bed47b740 1 Processor -- start 2024-09-21T03:49:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.404+0000 7f0bed47b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4624d0 0x55814f838500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.404+0000 7f0bed47b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f4831b0 0x55814f838bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.404+0000 7f0bed47b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f4171c0 0x55814f839730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.404+0000 7f0bed47b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0be01bc770 con 0x55814f4831b0 2024-09-21T03:49:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.404+0000 7f0bed47b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0be01bc0a0 con 0x55814f4171c0 2024-09-21T03:49:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.404+0000 7f0bed47b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0be01b9280 con 0x55814f4624d0 2024-09-21T03:49:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.405+0000 7f0bed47b740 10 client.0 _ll_register_callbacks cb 0x7f0bcc00a790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.405+0000 7f0bdf400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4624d0 0x55814f838500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.405+0000 7f0bdf400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4624d0 0x55814f838500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:35360/0 (socket says 172.21.15.16:35360) 2024-09-21T03:49:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.405+0000 7f0bdf400640 1 -- 172.21.15.16:0/960646966 learned_addr learned my addr 172.21.15.16:0/960646966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.405+0000 7f0bdea00640 1 --2- 172.21.15.16:0/960646966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f4831b0 0x55814f838bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.405+0000 7f0bdfe00640 1 --2- 172.21.15.16:0/960646966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f4171c0 0x55814f839730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.405+0000 7f0bdfe00640 1 -- 172.21.15.16:0/960646966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4624d0 msgr2=0x55814f838500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.405+0000 7f0bdfe00640 1 --2- 172.21.15.16:0/960646966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4624d0 0x55814f838500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.405+0000 7f0bdfe00640 1 -- 172.21.15.16:0/960646966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f4831b0 msgr2=0x55814f838bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.405+0000 7f0bdfe00640 1 --2- 172.21.15.16:0/960646966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f4831b0 0x55814f838bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.405+0000 7f0bdfe00640 1 -- 172.21.15.16:0/960646966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55814f839e90 con 0x55814f4171c0 2024-09-21T03:49:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.406+0000 7f0bdea00640 1 --2- 172.21.15.16:0/960646966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f4831b0 0x55814f838bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.406+0000 7f0bdfe00640 1 --2- 172.21.15.16:0/960646966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f4171c0 0x55814f839730 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0be010f3a0 tx=0x7f0bd8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=655abee93bee1a8f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.406+0000 7f0bc9600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:49:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.406+0000 7f0bc9600640 1 -- 172.21.15.16:0/960646966 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd810c060 con 0x55814f4171c0 2024-09-21T03:49:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.406+0000 7f0bc9600640 1 -- 172.21.15.16:0/960646966 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0bd80d21b0 con 0x55814f4171c0 2024-09-21T03:49:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.406+0000 7f0bed47b740 1 -- 172.21.15.16:0/960646966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bd000ec10 con 0x55814f4171c0 2024-09-21T03:49:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.406+0000 7f0bc9600640 1 -- 172.21.15.16:0/960646966 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0bd80cd5b0 con 0x55814f4171c0 2024-09-21T03:49:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.406+0000 7f0bed47b740 10 client.25907 fetch_fsmap learned FSMap version 30 2024-09-21T03:49:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.406+0000 7f0bed47b740 1 -- 172.21.15.16:0/960646966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55814f83b1f0 con 0x55814f4171c0 2024-09-21T03:49:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.407+0000 7f0bc9600640 1 -- 172.21.15.16:0/960646966 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd000ec10 con 0x55814f4171c0 2024-09-21T03:49:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.407+0000 7f0bc9600640 1 -- 172.21.15.16:0/960646966 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0bd8107b20 con 0x55814f4171c0 2024-09-21T03:49:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.407+0000 7f0bed47b740 10 client.25907 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:49:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.407+0000 7f0bed47b740 10 client.25907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.407+0000 7f0bed47b740 20 client.25907 populate_metadata read hostname 'smithi016' 2024-09-21T03:49:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.407+0000 7f0bed47b740 1 --2- 172.21.15.16:0/960646966 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55814f8abea0 0x55814f8ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.407+0000 7f0bed47b740 4 client.25907 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:49:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.407+0000 7f0bed47b740 1 -- 172.21.15.16:0/960646966 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55814f79b0b0 con 0x55814f8abea0 2024-09-21T03:49:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.408+0000 7f0bdf400640 1 --2- 172.21.15.16:0/960646966 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55814f8abea0 0x55814f8ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.408+0000 7f0bc9600640 10 client.25907 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.408+0000 7f0bdf400640 1 --2- 172.21.15.16:0/960646966 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55814f8abea0 0x55814f8ae2c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e93de5d718853688 server_cookie=ee0dd665e4bc40b in_seq=0 out_seq=0 2024-09-21T03:49:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.965+0000 7f0bc9600640 1 -- 172.21.15.16:0/960646966 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55814f79b0b0 con 0x55814f8abea0 2024-09-21T03:49:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.965+0000 7f0bc9600640 10 client.25907 handle_command_reply: tid=0 2024-09-21T03:49:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.965+0000 7f0bed47b740 1 client.25907 shutdown 2024-09-21T03:49:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.966+0000 7f0bed47b740 1 -- 172.21.15.16:0/960646966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f4171c0 msgr2=0x55814f839730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.966+0000 7f0bed47b740 1 --2- 172.21.15.16:0/960646966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f4171c0 0x55814f839730 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0be010f3a0 tx=0x7f0bd8005fd0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.966+0000 7f0bed47b740 1 -- 172.21.15.16:0/960646966 shutdown_connections 2024-09-21T03:49:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.966+0000 7f0bed47b740 1 --2- 172.21.15.16:0/960646966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55814f4171c0 0x55814f839730 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.966+0000 7f0bed47b740 1 --2- 172.21.15.16:0/960646966 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55814f8abea0 0x55814f8ae2c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.966+0000 7f0bed47b740 1 --2- 172.21.15.16:0/960646966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55814f4831b0 0x55814f838bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.966+0000 7f0bed47b740 1 --2- 172.21.15.16:0/960646966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55814f4624d0 0x55814f838500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.966+0000 7f0bed47b740 1 -- 172.21.15.16:0/960646966 >> 172.21.15.16:0/960646966 conn(0x55814f477300 msgr2=0x55814f82ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.966+0000 7f0bc9600640 0 client.25907 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.966+0000 7f0bed47b740 1 -- 172.21.15.16:0/960646966 shutdown_connections 2024-09-21T03:49:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.966+0000 7f0bed47b740 1 -- 172.21.15.16:0/960646966 wait complete. 2024-09-21T03:49:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.969+0000 7f0bed47b740 20 client.25907 trim_cache size 0 max 16384 2024-09-21T03:49:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:49:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:49:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:49:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:49:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.971+0000 7f0be4e00640 1 -- 172.21.15.16:0/3163023630 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0bc40b65d0 msgr2=0x7f0bc40b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.971+0000 7f0be4e00640 1 --2- 172.21.15.16:0/3163023630 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0bc40b65d0 0x7f0bc40b8ae0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00ade0 tx=0x7f0bcc00a380 comp rx=0 tx=0).stop 2024-09-21T03:49:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.971+0000 7f0be4e00640 1 -- 172.21.15.16:0/3163023630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be0113c30 msgr2=0x7f0be010ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.971+0000 7f0be4e00640 1 --2- 172.21.15.16:0/3163023630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be0113c30 0x7f0be010ecc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8009bf0 tx=0x7f0bd802f8e0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.972+0000 7f0be4e00640 1 -- 172.21.15.16:0/3163023630 shutdown_connections 2024-09-21T03:49:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.972+0000 7f0be4e00640 1 --2- 172.21.15.16:0/3163023630 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0bc40b65d0 0x7f0bc40b8ae0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.972+0000 7f0be4e00640 1 --2- 172.21.15.16:0/3163023630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be0113c30 0x7f0be010ecc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.972+0000 7f0be4e00640 1 --2- 172.21.15.16:0/3163023630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be010cd20 0x7f0be010e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.972+0000 7f0be4e00640 1 --2- 172.21.15.16:0/3163023630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0be010a3c0 0x7f0be010daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:09.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.972+0000 7f0be4e00640 1 -- 172.21.15.16:0/3163023630 >> 172.21.15.16:0/3163023630 conn(0x7f0be0070ed0 msgr2=0x7f0be010b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:09.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.972+0000 7f0be4e00640 1 -- 172.21.15.16:0/3163023630 shutdown_connections 2024-09-21T03:49:09.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:09.972+0000 7f0be4e00640 1 -- 172.21.15.16:0/3163023630 wait complete. 2024-09-21T03:49:10.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:10 smithi016 ceph-mon[30375]: pgmap v504: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 29 MiB/s wr, 212 op/s 2024-09-21T03:49:10.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:10 smithi123 ceph-mon[34673]: pgmap v504: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 29 MiB/s wr, 212 op/s 2024-09-21T03:49:11.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:10 smithi134 ceph-mon[35031]: pgmap v504: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 29 MiB/s wr, 212 op/s 2024-09-21T03:49:12.053 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b00a0f67', total seconds elapsed: 35.4 2024-09-21T03:49:12.053 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.8 2024-09-21T03:49:12.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:49:12.925 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:13.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:12 smithi016 ceph-mon[30375]: pgmap v505: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 26 MiB/s wr, 170 op/s 2024-09-21T03:49:13.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:12 smithi134 ceph-mon[35031]: pgmap v505: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 26 MiB/s wr, 170 op/s 2024-09-21T03:49:13.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:12 smithi123 ceph-mon[34673]: pgmap v505: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 26 MiB/s wr, 170 op/s 2024-09-21T03:49:14.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.404+0000 7f6f9aa00640 1 -- 172.21.15.16:0/3020944078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f9410a590 msgr2=0x7f6f9410a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:14.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.404+0000 7f6f9aa00640 1 --2- 172.21.15.16:0/3020944078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f9410a590 0x7f6f9410a9a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009be0 tx=0x7f6f8c02eaf0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.405+0000 7f6f9aa00640 1 -- 172.21.15.16:0/3020944078 shutdown_connections 2024-09-21T03:49:14.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.405+0000 7f6f9aa00640 1 --2- 172.21.15.16:0/3020944078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f94116640 0x7f6f94118b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.405+0000 7f6f9aa00640 1 --2- 172.21.15.16:0/3020944078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f9410aee0 0x7f6f94115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.405+0000 7f6f9aa00640 1 --2- 172.21.15.16:0/3020944078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f9410a590 0x7f6f9410a9a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.405+0000 7f6f9aa00640 1 -- 172.21.15.16:0/3020944078 >> 172.21.15.16:0/3020944078 conn(0x7f6f94070ec0 msgr2=0x7f6f940712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:14.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.405+0000 7f6f9aa00640 1 -- 172.21.15.16:0/3020944078 shutdown_connections 2024-09-21T03:49:14.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.405+0000 7f6f9aa00640 1 -- 172.21.15.16:0/3020944078 wait complete. 2024-09-21T03:49:14.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.407+0000 7f6f9aa00640 1 Processor -- start 2024-09-21T03:49:14.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.407+0000 7f6f9aa00640 1 -- start start 2024-09-21T03:49:14.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.408+0000 7f6f9aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f9410a590 0x7f6f941a9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.408+0000 7f6f9aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f9410aee0 0x7f6f941a9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.408+0000 7f6f9aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f94116640 0x7f6f941ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.409+0000 7f6f93e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f9410a590 0x7f6f941a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.409+0000 7f6f93e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f9410a590 0x7f6f941a9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:33958/0 (socket says 172.21.15.16:33958) 2024-09-21T03:49:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.409+0000 7f6f93e00640 1 -- 172.21.15.16:0/105796644 learned_addr learned my addr 172.21.15.16:0/105796644 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.409+0000 7f6f9aa00640 1 -- 172.21.15.16:0/105796644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9411a100 con 0x7f6f94116640 2024-09-21T03:49:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.409+0000 7f6f93400640 1 --2- 172.21.15.16:0/105796644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f9410aee0 0x7f6f941a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.410+0000 7f6f9aa00640 1 -- 172.21.15.16:0/105796644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6f94119f80 con 0x7f6f9410a590 2024-09-21T03:49:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.410+0000 7f6f9aa00640 1 -- 172.21.15.16:0/105796644 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9411a280 con 0x7f6f9410aee0 2024-09-21T03:49:14.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.411+0000 7f6f98a00640 1 --2- 172.21.15.16:0/105796644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f94116640 0x7f6f941ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.411+0000 7f6f93e00640 1 -- 172.21.15.16:0/105796644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f9410aee0 msgr2=0x7f6f941a9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:14.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.411+0000 7f6f93e00640 1 --2- 172.21.15.16:0/105796644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f9410aee0 0x7f6f941a9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.411+0000 7f6f93e00640 1 -- 172.21.15.16:0/105796644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f94116640 msgr2=0x7f6f941ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:14.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.411+0000 7f6f93e00640 1 --2- 172.21.15.16:0/105796644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f94116640 0x7f6f941ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.411+0000 7f6f93e00640 1 -- 172.21.15.16:0/105796644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f941af0e0 con 0x7f6f9410a590 2024-09-21T03:49:14.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.412+0000 7f6f93e00640 1 --2- 172.21.15.16:0/105796644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f9410a590 0x7f6f941a9170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c031740 tx=0x7f6f8c031770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:14.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.412+0000 7f6f90c00640 1 -- 172.21.15.16:0/105796644 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c003d80 con 0x7f6f9410a590 2024-09-21T03:49:14.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.412+0000 7f6f90c00640 1 -- 172.21.15.16:0/105796644 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f8c0104f0 con 0x7f6f9410a590 2024-09-21T03:49:14.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.412+0000 7f6f90c00640 1 -- 172.21.15.16:0/105796644 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c004670 con 0x7f6f9410a590 2024-09-21T03:49:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.413+0000 7f6f9aa00640 1 -- 172.21.15.16:0/105796644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f941af370 con 0x7f6f9410a590 2024-09-21T03:49:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.413+0000 7f6f9aa00640 1 -- 172.21.15.16:0/105796644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f941b5b30 con 0x7f6f9410a590 2024-09-21T03:49:14.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.420+0000 7f6f90c00640 1 -- 172.21.15.16:0/105796644 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6f8c04b430 con 0x7f6f9410a590 2024-09-21T03:49:14.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.423+0000 7f6f9aa00640 1 -- 172.21.15.16:0/105796644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f941aa2a0 con 0x7f6f9410a590 2024-09-21T03:49:14.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.424+0000 7f6f90c00640 1 --2- 172.21.15.16:0/105796644 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6f780b6610 0x7f6f780b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:14.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.425+0000 7f6f90c00640 1 -- 172.21.15.16:0/105796644 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f6f8c108970 con 0x7f6f9410a590 2024-09-21T03:49:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.429+0000 7f6f93400640 1 --2- 172.21.15.16:0/105796644 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6f780b6610 0x7f6f780b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.433+0000 7f6f90c00640 1 -- 172.21.15.16:0/105796644 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6f8c03e070 con 0x7f6f9410a590 2024-09-21T03:49:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.440+0000 7f6f93400640 1 --2- 172.21.15.16:0/105796644 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6f780b6610 0x7f6f780b8b20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6f94071630 tx=0x7f6f8000b390 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.944+0000 7f6f9aa00640 1 -- 172.21.15.16:0/105796644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f941b5fe0 con 0x7f6f9410a590 2024-09-21T03:49:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.948+0000 7f6f90c00640 1 -- 172.21.15.16:0/105796644 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f6f8c036070 con 0x7f6f9410a590 2024-09-21T03:49:14.952 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:49:14.952 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:49:14.952 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:49:14.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.953+0000 7f6f9aa00640 1 -- 172.21.15.16:0/105796644 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6f780b6610 msgr2=0x7f6f780b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:14.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.953+0000 7f6f9aa00640 1 --2- 172.21.15.16:0/105796644 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6f780b6610 0x7f6f780b8b20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6f94071630 tx=0x7f6f8000b390 comp rx=0 tx=0).stop 2024-09-21T03:49:14.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.953+0000 7f6f9aa00640 1 -- 172.21.15.16:0/105796644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f9410a590 msgr2=0x7f6f941a9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:14.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.953+0000 7f6f9aa00640 1 --2- 172.21.15.16:0/105796644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f9410a590 0x7f6f941a9170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c031740 tx=0x7f6f8c031770 comp rx=0 tx=0).stop 2024-09-21T03:49:14.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.954+0000 7f6f9aa00640 1 -- 172.21.15.16:0/105796644 shutdown_connections 2024-09-21T03:49:14.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.954+0000 7f6f9aa00640 1 --2- 172.21.15.16:0/105796644 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6f780b6610 0x7f6f780b8b20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.954+0000 7f6f9aa00640 1 --2- 172.21.15.16:0/105796644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f94116640 0x7f6f941ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.954+0000 7f6f9aa00640 1 --2- 172.21.15.16:0/105796644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f9410aee0 0x7f6f941a9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.954+0000 7f6f9aa00640 1 --2- 172.21.15.16:0/105796644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f9410a590 0x7f6f941a9170 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:14.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.954+0000 7f6f9aa00640 1 -- 172.21.15.16:0/105796644 >> 172.21.15.16:0/105796644 conn(0x7f6f94070ec0 msgr2=0x7f6f9410df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:14.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.954+0000 7f6f9aa00640 1 -- 172.21.15.16:0/105796644 shutdown_connections 2024-09-21T03:49:14.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:14.954+0000 7f6f9aa00640 1 -- 172.21.15.16:0/105796644 wait complete. 2024-09-21T03:49:15.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:14 smithi016 ceph-mon[30375]: pgmap v506: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 43 MiB/s wr, 250 op/s 2024-09-21T03:49:15.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:14 smithi134 ceph-mon[35031]: pgmap v506: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 43 MiB/s wr, 250 op/s 2024-09-21T03:49:15.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:14 smithi123 ceph-mon[34673]: pgmap v506: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 43 MiB/s wr, 250 op/s 2024-09-21T03:49:16.057 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:15 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/105796644' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:16.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:15 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/105796644' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:16.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:15 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/105796644' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:17.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:16 smithi134 ceph-mon[35031]: pgmap v507: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 83 MiB/s wr, 455 op/s 2024-09-21T03:49:17.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:16 smithi123 ceph-mon[34673]: pgmap v507: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 83 MiB/s wr, 455 op/s 2024-09-21T03:49:17.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:16 smithi016 ceph-mon[30375]: pgmap v507: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 83 MiB/s wr, 455 op/s 2024-09-21T03:49:17.999 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0' 2024-09-21T03:49:17.999 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0 2024-09-21T03:49:18.101 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:17 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:49:18.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:17 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:49:18.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:17 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:49:19.072 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:19.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:18 smithi134 ceph-mon[35031]: pgmap v508: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 119 MiB/s wr, 641 op/s 2024-09-21T03:49:19.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:18 smithi123 ceph-mon[34673]: pgmap v508: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 119 MiB/s wr, 641 op/s 2024-09-21T03:49:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:18 smithi016 ceph-mon[30375]: pgmap v508: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 119 MiB/s wr, 641 op/s 2024-09-21T03:49:20.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:49:20.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:49:20.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:19 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:49:20.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:49:20.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:49:20.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:19 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:49:20.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:49:20.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:49:20.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:19 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:49:20.992 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:20 smithi016 ceph-mon[30375]: pgmap v509: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 145 MiB/s wr, 780 op/s 2024-09-21T03:49:21.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:20 smithi134 ceph-mon[35031]: pgmap v509: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 145 MiB/s wr, 780 op/s 2024-09-21T03:49:21.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.093+0000 7fa853200640 1 -- 172.21.15.16:0/2392057655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa84c077380 msgr2=0x7fa84c0757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.093+0000 7fa853200640 1 --2- 172.21.15.16:0/2392057655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa84c077380 0x7fa84c0757b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa840009900 tx=0x7fa84002a440 comp rx=0 tx=0).stop 2024-09-21T03:49:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.093+0000 7fa853200640 1 -- 172.21.15.16:0/2392057655 shutdown_connections 2024-09-21T03:49:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.093+0000 7fa853200640 1 --2- 172.21.15.16:0/2392057655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa84c114290 0x7fa84c116760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.093+0000 7fa853200640 1 --2- 172.21.15.16:0/2392057655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa84c075dd0 0x7fa84c0761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.093+0000 7fa853200640 1 --2- 172.21.15.16:0/2392057655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa84c077380 0x7fa84c0757b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.093+0000 7fa853200640 1 -- 172.21.15.16:0/2392057655 >> 172.21.15.16:0/2392057655 conn(0x7fa84c0ff430 msgr2=0x7fa84c101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.094+0000 7fa853200640 1 -- 172.21.15.16:0/2392057655 shutdown_connections 2024-09-21T03:49:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.095+0000 7fa853200640 1 -- 172.21.15.16:0/2392057655 wait complete. 2024-09-21T03:49:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.097+0000 7fa853200640 1 Processor -- start 2024-09-21T03:49:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.097+0000 7fa853200640 1 -- start start 2024-09-21T03:49:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.098+0000 7fa853200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa84c075dd0 0x7fa84c113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.098+0000 7fa853200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa84c077380 0x7fa84c10dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.098+0000 7fa853200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa84c114290 0x7fa84c10e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.098+0000 7fa853200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1080e0 con 0x7fa84c077380 2024-09-21T03:49:21.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.098+0000 7fa853200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c107f60 con 0x7fa84c075dd0 2024-09-21T03:49:21.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.098+0000 7fa853200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c108260 con 0x7fa84c114290 2024-09-21T03:49:21.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.098+0000 7fa851e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa84c075dd0 0x7fa84c113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:21.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.098+0000 7fa851e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa84c075dd0 0x7fa84c113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:33968/0 (socket says 172.21.15.16:33968) 2024-09-21T03:49:21.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.098+0000 7fa851e00640 1 -- 172.21.15.16:0/1822257286 learned_addr learned my addr 172.21.15.16:0/1822257286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:21.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.098+0000 7fa852800640 1 --2- 172.21.15.16:0/1822257286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa84c114290 0x7fa84c10e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:21.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.099+0000 7fa851e00640 1 -- 172.21.15.16:0/1822257286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa84c114290 msgr2=0x7fa84c10e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:21.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.099+0000 7fa851e00640 1 --2- 172.21.15.16:0/1822257286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa84c114290 0x7fa84c10e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.099+0000 7fa851e00640 1 -- 172.21.15.16:0/1822257286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa84c077380 msgr2=0x7fa84c10dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:21.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.099+0000 7fa851e00640 1 --2- 172.21.15.16:0/1822257286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa84c077380 0x7fa84c10dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.099+0000 7fa851e00640 1 -- 172.21.15.16:0/1822257286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84c10e840 con 0x7fa84c075dd0 2024-09-21T03:49:21.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.099+0000 7fa852800640 1 --2- 172.21.15.16:0/1822257286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa84c114290 0x7fa84c10e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:21.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.100+0000 7fa851e00640 1 --2- 172.21.15.16:0/1822257286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa84c075dd0 0x7fa84c113950 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa84002a9b0 tx=0x7fa840003d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:21.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.100+0000 7fa846a00640 1 -- 172.21.15.16:0/1822257286 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840061070 con 0x7fa84c075dd0 2024-09-21T03:49:21.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.100+0000 7fa846a00640 1 -- 172.21.15.16:0/1822257286 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa84005dd30 con 0x7fa84c075dd0 2024-09-21T03:49:21.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.100+0000 7fa846a00640 1 -- 172.21.15.16:0/1822257286 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840035030 con 0x7fa84c075dd0 2024-09-21T03:49:21.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.101+0000 7fa853200640 1 -- 172.21.15.16:0/1822257286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84c06edf0 con 0x7fa84c075dd0 2024-09-21T03:49:21.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.101+0000 7fa853200640 1 -- 172.21.15.16:0/1822257286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa84c06f280 con 0x7fa84c075dd0 2024-09-21T03:49:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.114+0000 7fa85b485740 1 Processor -- start 2024-09-21T03:49:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.114+0000 7fa85b485740 1 -- start start 2024-09-21T03:49:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.114+0000 7fa85b485740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 0x56442aeeb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.114+0000 7fa85b485740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab1a980 0x56442aeeba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.114+0000 7fa85b485740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab22b60 0x56442aeebf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.114+0000 7fa85b485740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56442ae60ca0 con 0x56442ab22b60 2024-09-21T03:49:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.114+0000 7fa85b485740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56442ae500d0 con 0x56442ab1a980 2024-09-21T03:49:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.114+0000 7fa85b485740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56442ae53ee0 con 0x56442ab43840 2024-09-21T03:49:21.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.116+0000 7fa852800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 0x56442aeeb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:21.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.116+0000 7fa852800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 0x56442aeeb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:42794/0 (socket says 172.21.15.16:42794) 2024-09-21T03:49:21.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.116+0000 7fa852800640 1 -- 172.21.15.16:0/1211666299 learned_addr learned my addr 172.21.15.16:0/1211666299 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:21.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.117+0000 7fa852800640 1 -- 172.21.15.16:0/1211666299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab1a980 msgr2=0x56442aeeba50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:49:21.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.117+0000 7fa852800640 1 --2- 172.21.15.16:0/1211666299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab1a980 0x56442aeeba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.117+0000 7fa852800640 1 -- 172.21.15.16:0/1211666299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab22b60 msgr2=0x56442aeebf90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:49:21.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.117+0000 7fa851400640 1 --2- 172.21.15.16:0/1211666299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab22b60 0x56442aeebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:21.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.117+0000 7fa852800640 1 --2- 172.21.15.16:0/1211666299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab22b60 0x56442aeebf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.117+0000 7fa852800640 1 -- 172.21.15.16:0/1211666299 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56442ae02400 con 0x56442ab43840 2024-09-21T03:49:21.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.118+0000 7fa852800640 1 --2- 172.21.15.16:0/1211666299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 0x56442aeeb510 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c450 tx=0x7fa84800c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f074dff5d06767e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:21.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.119+0000 7fa83b000640 1 -- 172.21.15.16:0/1211666299 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84801a070 con 0x56442ab43840 2024-09-21T03:49:21.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.119+0000 7fa83b000640 1 -- 172.21.15.16:0/1211666299 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8480078e0 con 0x56442ab43840 2024-09-21T03:49:21.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.119+0000 7fa83b000640 1 -- 172.21.15.16:0/1211666299 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8480158d0 con 0x56442ab43840 2024-09-21T03:49:21.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.120+0000 7fa85b485740 1 -- 172.21.15.16:0/1211666299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 msgr2=0x56442aeeb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:21.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.120+0000 7fa85b485740 1 --2- 172.21.15.16:0/1211666299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 0x56442aeeb510 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c450 tx=0x7fa84800c910 comp rx=0 tx=0).stop 2024-09-21T03:49:21.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.121+0000 7fa846a00640 1 -- 172.21.15.16:0/1822257286 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa840064070 con 0x7fa84c075dd0 2024-09-21T03:49:21.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.123+0000 7fa85b485740 1 -- 172.21.15.16:0/1211666299 shutdown_connections 2024-09-21T03:49:21.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.123+0000 7fa85b485740 1 --2- 172.21.15.16:0/1211666299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab22b60 0x56442aeebf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.123+0000 7fa85b485740 1 --2- 172.21.15.16:0/1211666299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab1a980 0x56442aeeba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.123+0000 7fa85b485740 1 --2- 172.21.15.16:0/1211666299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 0x56442aeeb510 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.123+0000 7fa85b485740 1 -- 172.21.15.16:0/1211666299 >> 172.21.15.16:0/1211666299 conn(0x56442ab37990 msgr2=0x56442aee9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:21.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.124+0000 7fa846a00640 1 --2- 172.21.15.16:0/1822257286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa8340b65d0 0x7fa8340b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.128+0000 7fa846a00640 1 -- 172.21.15.16:0/1822257286 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fa84012a150 con 0x7fa84c075dd0 2024-09-21T03:49:21.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.128+0000 7fa85b485740 1 -- 172.21.15.16:0/1211666299 shutdown_connections 2024-09-21T03:49:21.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.129+0000 7fa85b485740 1 -- 172.21.15.16:0/1211666299 wait complete. 2024-09-21T03:49:21.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.129+0000 7fa851400640 1 --2- 172.21.15.16:0/1822257286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa8340b65d0 0x7fa8340b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:21.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.130+0000 7fa85b485740 1 -- start start 2024-09-21T03:49:21.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.130+0000 7fa85b485740 1 Processor -- start 2024-09-21T03:49:21.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.131+0000 7fa851400640 1 --2- 172.21.15.16:0/1822257286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa8340b65d0 0x7fa8340b8ae0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa84c1161a0 tx=0x7fa83c005c50 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:21.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.132+0000 7fa85b485740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab22b60 0x56442aef6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.132+0000 7fa85b485740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 0x56442aef71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.132+0000 7fa85b485740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab021c0 0x56442af03060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.132+0000 7fa85b485740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56442acb17a0 con 0x56442ab22b60 2024-09-21T03:49:21.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.132+0000 7fa85b485740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56442adf5a10 con 0x56442ab021c0 2024-09-21T03:49:21.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.132+0000 7fa85b485740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56442aef9cf0 con 0x56442ab43840 2024-09-21T03:49:21.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.132+0000 7fa85b485740 10 client.0 _ll_register_callbacks cb 0x56442ae05be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:21.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.132+0000 7fa851e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab22b60 0x56442aef6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:21.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.133+0000 7fa852800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab021c0 0x56442af03060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:21.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.133+0000 7fa851e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab22b60 0x56442aef6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48746/0 (socket says 172.21.15.16:48746) 2024-09-21T03:49:21.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.133+0000 7fa851e00640 1 -- 172.21.15.16:0/3062073910 learned_addr learned my addr 172.21.15.16:0/3062073910 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:21.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.133+0000 7fa851400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 0x56442aef71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.133+0000 7fa852800640 1 -- 172.21.15.16:0/3062073910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 msgr2=0x56442aef71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.133+0000 7fa852800640 1 --2- 172.21.15.16:0/3062073910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 0x56442aef71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.133+0000 7fa852800640 1 -- 172.21.15.16:0/3062073910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab22b60 msgr2=0x56442aef6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.133+0000 7fa852800640 1 --2- 172.21.15.16:0/3062073910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab22b60 0x56442aef6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.133+0000 7fa852800640 1 -- 172.21.15.16:0/3062073910 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56442abbdb90 con 0x56442ab021c0 2024-09-21T03:49:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.134+0000 7fa851e00640 1 --2- 172.21.15.16:0/3062073910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab22b60 0x56442aef6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.134+0000 7fa82f400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:49:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.134+0000 7fa852800640 1 --2- 172.21.15.16:0/3062073910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab021c0 0x56442af03060 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c450 tx=0x7fa848009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=589698f1abcd6d6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:21.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.135+0000 7fa82f400640 1 -- 172.21.15.16:0/3062073910 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84801a070 con 0x56442ab021c0 2024-09-21T03:49:21.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.135+0000 7fa82f400640 1 -- 172.21.15.16:0/3062073910 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fa848015650 con 0x56442ab021c0 2024-09-21T03:49:21.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.135+0000 7fa85b485740 1 -- 172.21.15.16:0/3062073910 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8480078e0 con 0x56442ab021c0 2024-09-21T03:49:21.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.136+0000 7fa82f400640 1 -- 172.21.15.16:0/3062073910 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa848009c60 con 0x56442ab021c0 2024-09-21T03:49:21.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.136+0000 7fa85b485740 10 client.25919 fetch_fsmap learned FSMap version 30 2024-09-21T03:49:21.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.136+0000 7fa85b485740 1 -- 172.21.15.16:0/3062073910 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56442af05b50 con 0x56442ab021c0 2024-09-21T03:49:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.136+0000 7fa82f400640 1 -- 172.21.15.16:0/3062073910 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8480078e0 con 0x56442ab021c0 2024-09-21T03:49:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.138+0000 7fa82f400640 1 -- 172.21.15.16:0/3062073910 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa84801f870 con 0x56442ab021c0 2024-09-21T03:49:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.138+0000 7fa85b485740 10 client.25919 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:49:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.138+0000 7fa85b485740 10 client.25919 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:49:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.138+0000 7fa85b485740 20 client.25919 populate_metadata read hostname 'smithi016' 2024-09-21T03:49:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.138+0000 7fa85b485740 1 --2- 172.21.15.16:0/3062073910 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56442aad7570 0x56442af08fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.138+0000 7fa85b485740 4 client.25919 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.138+0000 7fa85b485740 1 -- 172.21.15.16:0/3062073910 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56442ae62bf0 con 0x56442aad7570 2024-09-21T03:49:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.138+0000 7fa851e00640 1 --2- 172.21.15.16:0/3062073910 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56442aad7570 0x56442af08fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.139+0000 7fa851e00640 1 --2- 172.21.15.16:0/3062073910 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56442aad7570 0x56442af08fc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2deb8c35ced6dc03 server_cookie=91194c5163602470 in_seq=0 out_seq=0 2024-09-21T03:49:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.139+0000 7fa82f400640 10 client.25919 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:21.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.140+0000 7fa82f400640 1 -- 172.21.15.16:0/3062073910 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56442ae62bf0 con 0x56442aad7570 2024-09-21T03:49:21.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.140+0000 7fa82f400640 10 client.25919 handle_command_reply: tid=0 2024-09-21T03:49:21.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.141+0000 7fa85b485740 1 client.25919 shutdown 2024-09-21T03:49:21.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.141+0000 7fa85b485740 1 -- 172.21.15.16:0/3062073910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab021c0 msgr2=0x56442af03060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:21.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.141+0000 7fa85b485740 1 --2- 172.21.15.16:0/3062073910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab021c0 0x56442af03060 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c450 tx=0x7fa848009790 comp rx=0 tx=0).stop 2024-09-21T03:49:21.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.142+0000 7fa85b485740 1 -- 172.21.15.16:0/3062073910 shutdown_connections 2024-09-21T03:49:21.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.142+0000 7fa85b485740 1 --2- 172.21.15.16:0/3062073910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab021c0 0x56442af03060 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.142+0000 7fa85b485740 1 --2- 172.21.15.16:0/3062073910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 0x56442aef71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.142+0000 7fa85b485740 1 --2- 172.21.15.16:0/3062073910 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56442aad7570 0x56442af08fc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.142+0000 7fa85b485740 1 --2- 172.21.15.16:0/3062073910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab22b60 0x56442aef6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.142+0000 7fa85b485740 1 -- 172.21.15.16:0/3062073910 >> 172.21.15.16:0/3062073910 conn(0x56442ab37990 msgr2=0x56442aeee6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:21.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.142+0000 7fa82f400640 0 client.25919 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:21.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.143+0000 7fa85b485740 1 -- 172.21.15.16:0/3062073910 shutdown_connections 2024-09-21T03:49:21.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.143+0000 7fa85b485740 1 -- 172.21.15.16:0/3062073910 wait complete. 2024-09-21T03:49:21.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.144+0000 7fa85b485740 20 client.25919 trim_cache size 0 max 16384 2024-09-21T03:49:21.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:20 smithi123 ceph-mon[34673]: pgmap v509: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 145 MiB/s wr, 780 op/s 2024-09-21T03:49:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.203+0000 7fa85b485740 1 Processor -- start 2024-09-21T03:49:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.203+0000 7fa85b485740 1 -- start start 2024-09-21T03:49:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.203+0000 7fa85b485740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab021c0 0x56442aee9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.203+0000 7fa85b485740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab22b60 0x56442aef88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.203+0000 7fa85b485740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 0x56442af02fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.203+0000 7fa85b485740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1b90a0 con 0x56442ab22b60 2024-09-21T03:49:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.203+0000 7fa85b485740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1b6e90 con 0x56442ab021c0 2024-09-21T03:49:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.203+0000 7fa85b485740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1b89d0 con 0x56442ab43840 2024-09-21T03:49:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.204+0000 7fa852800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 0x56442af02fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.204+0000 7fa852800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 0x56442af02fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:42814/0 (socket says 172.21.15.16:42814) 2024-09-21T03:49:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.204+0000 7fa852800640 1 -- 172.21.15.16:0/2543216732 learned_addr learned my addr 172.21.15.16:0/2543216732 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.204+0000 7fa851400640 1 --2- 172.21.15.16:0/2543216732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab22b60 0x56442aef88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.204+0000 7fa852800640 1 -- 172.21.15.16:0/2543216732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab021c0 msgr2=0x56442aee9550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:49:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.204+0000 7fa852800640 1 --2- 172.21.15.16:0/2543216732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab021c0 0x56442aee9550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.204+0000 7fa852800640 1 -- 172.21.15.16:0/2543216732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab22b60 msgr2=0x56442aef88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.204+0000 7fa852800640 1 --2- 172.21.15.16:0/2543216732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab22b60 0x56442aef88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.204+0000 7fa852800640 1 -- 172.21.15.16:0/2543216732 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56442aee9cf0 con 0x56442ab43840 2024-09-21T03:49:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.206+0000 7fa852800640 1 --2- 172.21.15.16:0/2543216732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 0x56442af02fd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa848009e00 tx=0x7fa8480070c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd04f688cd9ac164 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.206+0000 7fa83a600640 1 -- 172.21.15.16:0/2543216732 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84801a040 con 0x56442ab43840 2024-09-21T03:49:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.207+0000 7fa83a600640 1 -- 172.21.15.16:0/2543216732 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa84801b040 con 0x56442ab43840 2024-09-21T03:49:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.207+0000 7fa83a600640 1 -- 172.21.15.16:0/2543216732 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84801a040 con 0x56442ab43840 2024-09-21T03:49:21.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.210+0000 7fa85b485740 1 -- 172.21.15.16:0/2543216732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 msgr2=0x56442af02fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:21.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.210+0000 7fa85b485740 1 --2- 172.21.15.16:0/2543216732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 0x56442af02fd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa848009e00 tx=0x7fa8480070c0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.214+0000 7fa85b485740 1 -- 172.21.15.16:0/2543216732 shutdown_connections 2024-09-21T03:49:21.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.214+0000 7fa85b485740 1 --2- 172.21.15.16:0/2543216732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab43840 0x56442af02fd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.214+0000 7fa85b485740 1 --2- 172.21.15.16:0/2543216732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab22b60 0x56442aef88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.214+0000 7fa85b485740 1 --2- 172.21.15.16:0/2543216732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab021c0 0x56442aee9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.214+0000 7fa85b485740 1 -- 172.21.15.16:0/2543216732 >> 172.21.15.16:0/2543216732 conn(0x56442ab37990 msgr2=0x56442aef81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:21.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.215+0000 7fa85b485740 1 -- 172.21.15.16:0/2543216732 shutdown_connections 2024-09-21T03:49:21.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.215+0000 7fa85b485740 1 -- 172.21.15.16:0/2543216732 wait complete. 2024-09-21T03:49:21.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.216+0000 7fa85b485740 1 -- start start 2024-09-21T03:49:21.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.216+0000 7fa85b485740 1 Processor -- start 2024-09-21T03:49:21.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.217+0000 7fa85b485740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab021c0 0x56442af0d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.217+0000 7fa85b485740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab22b60 0x56442af08cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.217+0000 7fa85b485740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab43840 0x56442af09200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.217+0000 7fa85b485740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1b7c30 con 0x56442ab021c0 2024-09-21T03:49:21.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.217+0000 7fa85b485740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1b7560 con 0x56442ab43840 2024-09-21T03:49:21.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.217+0000 7fa85b485740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1b8300 con 0x56442ab22b60 2024-09-21T03:49:21.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.217+0000 7fa85b485740 10 client.0 _ll_register_callbacks cb 0x7fa848018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:21.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.217+0000 7fa852800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab43840 0x56442af09200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:21.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.218+0000 7fa852800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab43840 0x56442af09200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:33978/0 (socket says 172.21.15.16:33978) 2024-09-21T03:49:21.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.218+0000 7fa852800640 1 -- 172.21.15.16:0/1944379404 learned_addr learned my addr 172.21.15.16:0/1944379404 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:21.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.218+0000 7fa851e00640 1 --2- 172.21.15.16:0/1944379404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab021c0 0x56442af0d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:21.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.218+0000 7fa851400640 1 --2- 172.21.15.16:0/1944379404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab22b60 0x56442af08cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:21.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.219+0000 7fa852800640 1 -- 172.21.15.16:0/1944379404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab22b60 msgr2=0x56442af08cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:21.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.219+0000 7fa852800640 1 --2- 172.21.15.16:0/1944379404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab22b60 0x56442af08cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.219+0000 7fa852800640 1 -- 172.21.15.16:0/1944379404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab021c0 msgr2=0x56442af0d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:21.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.219+0000 7fa852800640 1 --2- 172.21.15.16:0/1944379404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab021c0 0x56442af0d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.219+0000 7fa852800640 1 -- 172.21.15.16:0/1944379404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56442af09940 con 0x56442ab43840 2024-09-21T03:49:21.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.219+0000 7fa851e00640 1 --2- 172.21.15.16:0/1944379404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab021c0 0x56442af0d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:49:21.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.219+0000 7fa852800640 1 --2- 172.21.15.16:0/1944379404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab43840 0x56442af09200 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa84801fd80 tx=0x7fa8480079e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2f1b329688ecb3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:21.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.219+0000 7fa82f400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:49:21.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.220+0000 7fa82f400640 1 -- 172.21.15.16:0/1944379404 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848035070 con 0x56442ab43840 2024-09-21T03:49:21.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.220+0000 7fa82f400640 1 -- 172.21.15.16:0/1944379404 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fa84801fe00 con 0x56442ab43840 2024-09-21T03:49:21.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.220+0000 7fa85b485740 1 -- 172.21.15.16:0/1944379404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa84801b040 con 0x56442ab43840 2024-09-21T03:49:21.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.221+0000 7fa82f400640 1 -- 172.21.15.16:0/1944379404 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa848041050 con 0x56442ab43840 2024-09-21T03:49:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.221+0000 7fa85b485740 10 client.25922 fetch_fsmap learned FSMap version 30 2024-09-21T03:49:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.221+0000 7fa85b485740 1 -- 172.21.15.16:0/1944379404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56442af73600 con 0x56442ab43840 2024-09-21T03:49:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.222+0000 7fa82f400640 1 -- 172.21.15.16:0/1944379404 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa84801b040 con 0x56442ab43840 2024-09-21T03:49:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.222+0000 7fa82f400640 1 -- 172.21.15.16:0/1944379404 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa84800a920 con 0x56442ab43840 2024-09-21T03:49:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.222+0000 7fa85b485740 10 client.25922 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:49:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.222+0000 7fa85b485740 10 client.25922 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:49:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.222+0000 7fa85b485740 20 client.25922 populate_metadata read hostname 'smithi016' 2024-09-21T03:49:21.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.222+0000 7fa85b485740 1 --2- 172.21.15.16:0/1944379404 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56442af73d20 0x56442af76140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:21.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.222+0000 7fa85b485740 4 client.25922 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}] 2024-09-21T03:49:21.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.222+0000 7fa85b485740 1 -- 172.21.15.16:0/1944379404 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}) -- 0x56442ae5e6a0 con 0x56442af73d20 2024-09-21T03:49:21.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.223+0000 7fa851e00640 1 --2- 172.21.15.16:0/1944379404 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56442af73d20 0x56442af76140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:21.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.224+0000 7fa851e00640 1 --2- 172.21.15.16:0/1944379404 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56442af73d20 0x56442af76140 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ce25c8bcecb15cf server_cookie=1c20ff68043c6a91 in_seq=0 out_seq=0 2024-09-21T03:49:21.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.225+0000 7fa82f400640 10 client.25922 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.263+0000 7fa82f400640 1 -- 172.21.15.16:0/1944379404 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56442ae5e6a0 con 0x56442af73d20 2024-09-21T03:49:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.263+0000 7fa82f400640 10 client.25922 handle_command_reply: tid=0 2024-09-21T03:49:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.263+0000 7fa85b485740 1 client.25922 shutdown 2024-09-21T03:49:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.264+0000 7fa85b485740 1 -- 172.21.15.16:0/1944379404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab43840 msgr2=0x56442af09200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.264+0000 7fa85b485740 1 --2- 172.21.15.16:0/1944379404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab43840 0x56442af09200 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa84801fd80 tx=0x7fa8480079e0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.264+0000 7fa85b485740 1 -- 172.21.15.16:0/1944379404 shutdown_connections 2024-09-21T03:49:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.264+0000 7fa85b485740 1 --2- 172.21.15.16:0/1944379404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442ab43840 0x56442af09200 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.264+0000 7fa85b485740 1 --2- 172.21.15.16:0/1944379404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442ab22b60 0x56442af08cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.264+0000 7fa85b485740 1 --2- 172.21.15.16:0/1944379404 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56442af73d20 0x56442af76140 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.264+0000 7fa85b485740 1 --2- 172.21.15.16:0/1944379404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442ab021c0 0x56442af0d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.264+0000 7fa85b485740 1 -- 172.21.15.16:0/1944379404 >> 172.21.15.16:0/1944379404 conn(0x56442ab37990 msgr2=0x56442aef5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.264+0000 7fa82f400640 0 client.25922 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.266+0000 7fa85b485740 1 -- 172.21.15.16:0/1944379404 shutdown_connections 2024-09-21T03:49:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.266+0000 7fa85b485740 1 -- 172.21.15.16:0/1944379404 wait complete. 2024-09-21T03:49:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.266+0000 7fa85b485740 20 client.25922 trim_cache size 0 max 16384 2024-09-21T03:49:21.268 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:49:21.268 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:49:21.268 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:49:21.268 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 14, 2024-09-21T03:49:21.268 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:49:21.268 INFO:teuthology.orchestra.run.smithi016.stdout: "ecfa781f": { 2024-09-21T03:49:21.268 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 14, 2024-09-21T03:49:21.268 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:49:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:49:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:49:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:49:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:49:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:49:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 147.8, 2024-09-21T03:49:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:49:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:49:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:49:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:49:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:49:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:49:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:49:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:49:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:49:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:49:21.270 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:49:21.270 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:49:21.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.267+0000 7fa853200640 1 -- 172.21.15.16:0/1822257286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa8340b65d0 msgr2=0x7fa8340b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:21.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.267+0000 7fa853200640 1 --2- 172.21.15.16:0/1822257286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa8340b65d0 0x7fa8340b8ae0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa84c1161a0 tx=0x7fa83c005c50 comp rx=0 tx=0).stop 2024-09-21T03:49:21.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.267+0000 7fa853200640 1 -- 172.21.15.16:0/1822257286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa84c075dd0 msgr2=0x7fa84c113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:21.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.267+0000 7fa853200640 1 --2- 172.21.15.16:0/1822257286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa84c075dd0 0x7fa84c113950 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa84002a9b0 tx=0x7fa840003d60 comp rx=0 tx=0).stop 2024-09-21T03:49:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.268+0000 7fa853200640 1 -- 172.21.15.16:0/1822257286 shutdown_connections 2024-09-21T03:49:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.268+0000 7fa853200640 1 --2- 172.21.15.16:0/1822257286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa8340b65d0 0x7fa8340b8ae0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.268+0000 7fa853200640 1 --2- 172.21.15.16:0/1822257286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa84c114290 0x7fa84c10e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.268+0000 7fa853200640 1 --2- 172.21.15.16:0/1822257286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa84c077380 0x7fa84c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.268+0000 7fa853200640 1 --2- 172.21.15.16:0/1822257286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa84c075dd0 0x7fa84c113950 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.268+0000 7fa853200640 1 -- 172.21.15.16:0/1822257286 >> 172.21.15.16:0/1822257286 conn(0x7fa84c0ff430 msgr2=0x7fa84c1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.268+0000 7fa853200640 1 -- 172.21.15.16:0/1822257286 shutdown_connections 2024-09-21T03:49:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:21.268+0000 7fa853200640 1 -- 172.21.15.16:0/1822257286 wait complete. 2024-09-21T03:49:23.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:22 smithi134 ceph-mon[35031]: pgmap v510: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 179 MiB/s wr, 954 op/s 2024-09-21T03:49:23.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:22 smithi123 ceph-mon[34673]: pgmap v510: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 179 MiB/s wr, 954 op/s 2024-09-21T03:49:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:22 smithi016 ceph-mon[30375]: pgmap v510: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 179 MiB/s wr, 954 op/s 2024-09-21T03:49:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.327+0000 7f67b5c00640 1 -- 172.21.15.16:0/3582255403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67b010fe40 msgr2=0x7f67b0112260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.327+0000 7f67b5c00640 1 --2- 172.21.15.16:0/3582255403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67b010fe40 0x7f67b0112260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f67a801acb0 tx=0x7f67a803b8e0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.328+0000 7f67b5c00640 1 -- 172.21.15.16:0/3582255403 shutdown_connections 2024-09-21T03:49:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.328+0000 7f67b5c00640 1 --2- 172.21.15.16:0/3582255403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67b010fe40 0x7f67b0112260 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.328+0000 7f67b5c00640 1 --2- 172.21.15.16:0/3582255403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67b0075cf0 0x7f67b0076100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.328+0000 7f67b5c00640 1 --2- 172.21.15.16:0/3582255403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b00770f0 0x7f67b00757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.328+0000 7f67b5c00640 1 -- 172.21.15.16:0/3582255403 >> 172.21.15.16:0/3582255403 conn(0x7f67b00ff020 msgr2=0x7f67b01014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.328+0000 7f67b5c00640 1 -- 172.21.15.16:0/3582255403 shutdown_connections 2024-09-21T03:49:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.328+0000 7f67b5c00640 1 -- 172.21.15.16:0/3582255403 wait complete. 2024-09-21T03:49:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.329+0000 7f67b5c00640 1 Processor -- start 2024-09-21T03:49:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.330+0000 7f67b5c00640 1 -- start start 2024-09-21T03:49:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.330+0000 7f67b5c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67b0075cf0 0x7f67b0197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.330+0000 7f67b5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67b00770f0 0x7f67b0198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.330+0000 7f67b5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b010fe40 0x7f67b0198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.330+0000 7f67b5c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67b010a110 con 0x7f67b010fe40 2024-09-21T03:49:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.330+0000 7f67b5c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f67b0109f90 con 0x7f67b00770f0 2024-09-21T03:49:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.330+0000 7f67b5c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67b010a290 con 0x7f67b0075cf0 2024-09-21T03:49:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.331+0000 7f67ae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67b00770f0 0x7f67b0198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.331+0000 7f67ae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67b00770f0 0x7f67b0198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:49544/0 (socket says 172.21.15.16:49544) 2024-09-21T03:49:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.331+0000 7f67ae000640 1 -- 172.21.15.16:0/790102357 learned_addr learned my addr 172.21.15.16:0/790102357 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.331+0000 7f67aea00640 1 --2- 172.21.15.16:0/790102357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67b0075cf0 0x7f67b0197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.331+0000 7f67ae000640 1 -- 172.21.15.16:0/790102357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67b0075cf0 msgr2=0x7f67b0197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.331+0000 7f67ae000640 1 --2- 172.21.15.16:0/790102357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67b0075cf0 0x7f67b0197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.331+0000 7f67ae000640 1 -- 172.21.15.16:0/790102357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b010fe40 msgr2=0x7f67b0198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.331+0000 7f67ae000640 1 --2- 172.21.15.16:0/790102357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b010fe40 0x7f67b0198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.331+0000 7f67ae000640 1 -- 172.21.15.16:0/790102357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b01996e0 con 0x7f67b00770f0 2024-09-21T03:49:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.332+0000 7f67ae000640 1 --2- 172.21.15.16:0/790102357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67b00770f0 0x7f67b0198450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f679800e9f0 tx=0x7f679800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.332+0000 7f67a7400640 1 -- 172.21.15.16:0/790102357 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679800cd80 con 0x7f67b00770f0 2024-09-21T03:49:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.332+0000 7f67b5c00640 1 -- 172.21.15.16:0/790102357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b01999d0 con 0x7f67b00770f0 2024-09-21T03:49:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.333+0000 7f67a7400640 1 -- 172.21.15.16:0/790102357 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67980108e0 con 0x7f67b00770f0 2024-09-21T03:49:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.333+0000 7f67a7400640 1 -- 172.21.15.16:0/790102357 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798018860 con 0x7f67b00770f0 2024-09-21T03:49:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.333+0000 7f67b5c00640 1 -- 172.21.15.16:0/790102357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b01a00a0 con 0x7f67b00770f0 2024-09-21T03:49:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.337+0000 7f67a7400640 1 -- 172.21.15.16:0/790102357 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6798018a00 con 0x7f67b00770f0 2024-09-21T03:49:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.337+0000 7f67bccd9740 1 Processor -- start 2024-09-21T03:49:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.337+0000 7f67bccd9740 1 -- start start 2024-09-21T03:49:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.338+0000 7f67bccd9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ef1b0 0x55bfb4b960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.338+0000 7f67bccd9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47c62f0 0x55bfb4b96740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.338+0000 7f67bccd9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47ce4d0 0x55bfb4b97290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.338+0000 7f67bccd9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bfb4b096b0 con 0x55bfb47ce4d0 2024-09-21T03:49:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.338+0000 7f67bccd9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bfb4af8ae0 con 0x55bfb47c62f0 2024-09-21T03:49:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.338+0000 7f67bccd9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bfb4afc8f0 con 0x55bfb47ef1b0 2024-09-21T03:49:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.338+0000 7f67af400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ef1b0 0x55bfb4b960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.338+0000 7f67af400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ef1b0 0x55bfb4b960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:47834/0 (socket says 172.21.15.16:47834) 2024-09-21T03:49:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.338+0000 7f67af400640 1 -- 172.21.15.16:0/1943254145 learned_addr learned my addr 172.21.15.16:0/1943254145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.339+0000 7f67ae000640 1 --2- 172.21.15.16:0/1943254145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47ce4d0 0x55bfb4b97290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.339+0000 7f67aea00640 1 --2- 172.21.15.16:0/1943254145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47c62f0 0x55bfb4b96740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.339+0000 7f67af400640 1 -- 172.21.15.16:0/1943254145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47c62f0 msgr2=0x55bfb4b96740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.339+0000 7f67af400640 1 --2- 172.21.15.16:0/1943254145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47c62f0 0x55bfb4b96740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.339+0000 7f67af400640 1 -- 172.21.15.16:0/1943254145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47ce4d0 msgr2=0x55bfb4b97290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.339+0000 7f67af400640 1 --2- 172.21.15.16:0/1943254145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47ce4d0 0x55bfb4b97290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.339+0000 7f67af400640 1 -- 172.21.15.16:0/1943254145 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfb4ac5780 con 0x55bfb47ef1b0 2024-09-21T03:49:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.340+0000 7f67ae000640 1 --2- 172.21.15.16:0/1943254145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47ce4d0 0x55bfb4b97290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.340+0000 7f67a7400640 1 --2- 172.21.15.16:0/790102357 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f67940b6550 0x7f67940b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.340+0000 7f67a7400640 1 -- 172.21.15.16:0/790102357 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f67980e7b90 con 0x7f67b00770f0 2024-09-21T03:49:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.341+0000 7f67af400640 1 --2- 172.21.15.16:0/1943254145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ef1b0 0x55bfb4b960d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f67a801a480 tx=0x7f67a801a4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcaaf7fe8656c060 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.341+0000 7f679fe00640 1 -- 172.21.15.16:0/1943254145 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a801a810 con 0x55bfb47ef1b0 2024-09-21T03:49:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.341+0000 7f679fe00640 1 -- 172.21.15.16:0/1943254145 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a8047040 con 0x55bfb47ef1b0 2024-09-21T03:49:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.341+0000 7f679fe00640 1 -- 172.21.15.16:0/1943254145 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a801a810 con 0x55bfb47ef1b0 2024-09-21T03:49:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.344+0000 7f67aea00640 1 --2- 172.21.15.16:0/790102357 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f67940b6550 0x7f67940b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.345+0000 7f67bccd9740 1 -- 172.21.15.16:0/1943254145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ef1b0 msgr2=0x55bfb4b960d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.345+0000 7f67aea00640 1 --2- 172.21.15.16:0/790102357 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f67940b6550 0x7f67940b8a60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f67a0005fd0 tx=0x7f67a0005c50 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.345+0000 7f67bccd9740 1 --2- 172.21.15.16:0/1943254145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ef1b0 0x55bfb4b960d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f67a801a480 tx=0x7f67a801a4b0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.347+0000 7f67bccd9740 1 -- 172.21.15.16:0/1943254145 shutdown_connections 2024-09-21T03:49:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.347+0000 7f67bccd9740 1 --2- 172.21.15.16:0/1943254145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47ce4d0 0x55bfb4b97290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.347+0000 7f67bccd9740 1 --2- 172.21.15.16:0/1943254145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47c62f0 0x55bfb4b96740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.347+0000 7f67bccd9740 1 --2- 172.21.15.16:0/1943254145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ef1b0 0x55bfb4b960d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.347+0000 7f67bccd9740 1 -- 172.21.15.16:0/1943254145 >> 172.21.15.16:0/1943254145 conn(0x55bfb47e3300 msgr2=0x55bfb4b91b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.347+0000 7f67bccd9740 1 -- 172.21.15.16:0/1943254145 shutdown_connections 2024-09-21T03:49:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.347+0000 7f67bccd9740 1 -- 172.21.15.16:0/1943254145 wait complete. 2024-09-21T03:49:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.348+0000 7f67bccd9740 1 -- start start 2024-09-21T03:49:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.348+0000 7f67bccd9740 1 Processor -- start 2024-09-21T03:49:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.349+0000 7f67bccd9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47c62f0 0x55bfb4b9f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.349+0000 7f67bccd9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ce4d0 0x55bfb4b9fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.349+0000 7f67bccd9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47ef1b0 0x55bfb4ba0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.349+0000 7f67bccd9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bfb4a30ab0 con 0x55bfb47c62f0 2024-09-21T03:49:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.349+0000 7f67bccd9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bfb4a9f570 con 0x55bfb47ef1b0 2024-09-21T03:49:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.349+0000 7f67bccd9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bfb4b97d50 con 0x55bfb47ce4d0 2024-09-21T03:49:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.349+0000 7f67bccd9740 10 client.0 _ll_register_callbacks cb 0x55bfb4aae100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.350+0000 7f67aea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47c62f0 0x55bfb4b9f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.350+0000 7f67ae000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ce4d0 0x55bfb4b9fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.350+0000 7f67ae000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ce4d0 0x55bfb4b9fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:47848/0 (socket says 172.21.15.16:47848) 2024-09-21T03:49:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.350+0000 7f67ae000640 1 -- 172.21.15.16:0/3878198462 learned_addr learned my addr 172.21.15.16:0/3878198462 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.350+0000 7f67af400640 1 --2- 172.21.15.16:0/3878198462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47ef1b0 0x55bfb4ba0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.351+0000 7f67af400640 1 -- 172.21.15.16:0/3878198462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ce4d0 msgr2=0x55bfb4b9fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.351+0000 7f67af400640 1 --2- 172.21.15.16:0/3878198462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ce4d0 0x55bfb4b9fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.351+0000 7f67af400640 1 -- 172.21.15.16:0/3878198462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47c62f0 msgr2=0x55bfb4b9f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.351+0000 7f67af400640 1 --2- 172.21.15.16:0/3878198462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47c62f0 0x55bfb4b9f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.351+0000 7f67af400640 1 -- 172.21.15.16:0/3878198462 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfb485a790 con 0x55bfb47ef1b0 2024-09-21T03:49:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.351+0000 7f67aea00640 1 --2- 172.21.15.16:0/3878198462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47c62f0 0x55bfb4b9f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.352+0000 7f67af400640 1 --2- 172.21.15.16:0/3878198462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47ef1b0 0x55bfb4ba0960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f67a803bed0 tx=0x7f67a8042890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7430193f32d48a48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.352+0000 7f679cc00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:49:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.352+0000 7f679cc00640 1 -- 172.21.15.16:0/3878198462 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8004040 con 0x55bfb47ef1b0 2024-09-21T03:49:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.352+0000 7f679cc00640 1 -- 172.21.15.16:0/3878198462 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f67a8019040 con 0x55bfb47ef1b0 2024-09-21T03:49:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.352+0000 7f67bccd9740 1 -- 172.21.15.16:0/3878198462 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67a8047040 con 0x55bfb47ef1b0 2024-09-21T03:49:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.353+0000 7f679cc00640 1 -- 172.21.15.16:0/3878198462 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f67a806a050 con 0x55bfb47ef1b0 2024-09-21T03:49:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.353+0000 7f67bccd9740 10 client.25934 fetch_fsmap learned FSMap version 30 2024-09-21T03:49:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.353+0000 7f67bccd9740 1 -- 172.21.15.16:0/3878198462 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bfb48d3620 con 0x55bfb47ef1b0 2024-09-21T03:49:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.353+0000 7f679cc00640 1 -- 172.21.15.16:0/3878198462 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a8047040 con 0x55bfb47ef1b0 2024-09-21T03:49:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.354+0000 7f679cc00640 1 -- 172.21.15.16:0/3878198462 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f67a8065030 con 0x55bfb47ef1b0 2024-09-21T03:49:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.354+0000 7f67bccd9740 10 client.25934 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:49:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.354+0000 7f67bccd9740 10 client.25934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.354+0000 7f67bccd9740 20 client.25934 populate_metadata read hostname 'smithi016' 2024-09-21T03:49:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.354+0000 7f67bccd9740 1 --2- 172.21.15.16:0/3878198462 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55bfb47add80 0x55bfb4baca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.354+0000 7f67bccd9740 4 client.25934 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.355+0000 7f67bccd9740 1 -- 172.21.15.16:0/3878198462 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfb4b0b600 con 0x55bfb47add80 2024-09-21T03:49:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.355+0000 7f67aea00640 1 --2- 172.21.15.16:0/3878198462 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55bfb47add80 0x55bfb4baca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.355+0000 7f67aea00640 1 --2- 172.21.15.16:0/3878198462 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55bfb47add80 0x55bfb4baca40 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2dff1cecd7b9782 server_cookie=1c87524b7e3acb3d in_seq=0 out_seq=0 2024-09-21T03:49:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.355+0000 7f679cc00640 10 client.25934 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.356+0000 7f679cc00640 1 -- 172.21.15.16:0/3878198462 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bfb4b0b600 con 0x55bfb47add80 2024-09-21T03:49:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.356+0000 7f679cc00640 10 client.25934 handle_command_reply: tid=0 2024-09-21T03:49:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.356+0000 7f67bccd9740 1 client.25934 shutdown 2024-09-21T03:49:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.357+0000 7f67bccd9740 1 -- 172.21.15.16:0/3878198462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47ef1b0 msgr2=0x55bfb4ba0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.357+0000 7f67bccd9740 1 --2- 172.21.15.16:0/3878198462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47ef1b0 0x55bfb4ba0960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f67a803bed0 tx=0x7f67a8042890 comp rx=0 tx=0).stop 2024-09-21T03:49:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.357+0000 7f67bccd9740 1 -- 172.21.15.16:0/3878198462 shutdown_connections 2024-09-21T03:49:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.357+0000 7f67bccd9740 1 --2- 172.21.15.16:0/3878198462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47ef1b0 0x55bfb4ba0960 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.357+0000 7f67bccd9740 1 --2- 172.21.15.16:0/3878198462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ce4d0 0x55bfb4b9fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.357+0000 7f67bccd9740 1 --2- 172.21.15.16:0/3878198462 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55bfb47add80 0x55bfb4baca40 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.357+0000 7f67bccd9740 1 --2- 172.21.15.16:0/3878198462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47c62f0 0x55bfb4b9f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.357+0000 7f67bccd9740 1 -- 172.21.15.16:0/3878198462 >> 172.21.15.16:0/3878198462 conn(0x55bfb47e3300 msgr2=0x55bfb4b900e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.358+0000 7f679cc00640 0 client.25934 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.358+0000 7f67bccd9740 1 -- 172.21.15.16:0/3878198462 shutdown_connections 2024-09-21T03:49:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.359+0000 7f67bccd9740 1 -- 172.21.15.16:0/3878198462 wait complete. 2024-09-21T03:49:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.359+0000 7f67bccd9740 20 client.25934 trim_cache size 0 max 16384 2024-09-21T03:49:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.390+0000 7f67bccd9740 1 Processor -- start 2024-09-21T03:49:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.390+0000 7f67bccd9740 1 -- start start 2024-09-21T03:49:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.391+0000 7f67bccd9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47c62f0 0x55bfb4b9a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.391+0000 7f67bccd9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47ce4d0 0x55bfb4b95d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.391+0000 7f67bccd9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47ef1b0 0x55bfb4b962d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.391+0000 7f67bccd9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67b01aa4a0 con 0x55bfb47ef1b0 2024-09-21T03:49:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.391+0000 7f67bccd9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f67b01abfe0 con 0x55bfb47ce4d0 2024-09-21T03:49:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.391+0000 7f67bccd9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67b01aab70 con 0x55bfb47c62f0 2024-09-21T03:49:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.392+0000 7f67af400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47ef1b0 0x55bfb4b962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.392+0000 7f67af400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47ef1b0 0x55bfb4b962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54916/0 (socket says 172.21.15.16:54916) 2024-09-21T03:49:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.392+0000 7f67af400640 1 -- 172.21.15.16:0/3717100028 learned_addr learned my addr 172.21.15.16:0/3717100028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.392+0000 7f67aea00640 1 --2- 172.21.15.16:0/3717100028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47c62f0 0x55bfb4b9a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.392+0000 7f67af400640 1 -- 172.21.15.16:0/3717100028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47c62f0 msgr2=0x55bfb4b9a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.392+0000 7f67af400640 1 --2- 172.21.15.16:0/3717100028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47c62f0 0x55bfb4b9a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.392+0000 7f67af400640 1 -- 172.21.15.16:0/3717100028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47ce4d0 msgr2=0x55bfb4b95d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:49:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.392+0000 7f67af400640 1 --2- 172.21.15.16:0/3717100028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47ce4d0 0x55bfb4b95d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.392+0000 7f67af400640 1 -- 172.21.15.16:0/3717100028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfb4ba0160 con 0x55bfb47ef1b0 2024-09-21T03:49:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.393+0000 7f67aea00640 1 --2- 172.21.15.16:0/3717100028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47c62f0 0x55bfb4b9a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.393+0000 7f67af400640 1 --2- 172.21.15.16:0/3717100028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47ef1b0 0x55bfb4b962d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f67a8000f80 tx=0x7f67a801a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b869993b57f2c9c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.393+0000 7f679f400640 1 -- 172.21.15.16:0/3717100028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8073070 con 0x55bfb47ef1b0 2024-09-21T03:49:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.394+0000 7f679f400640 1 -- 172.21.15.16:0/3717100028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a8047040 con 0x55bfb47ef1b0 2024-09-21T03:49:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.394+0000 7f679f400640 1 -- 172.21.15.16:0/3717100028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8004040 con 0x55bfb47ef1b0 2024-09-21T03:49:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.395+0000 7f67bccd9740 1 -- 172.21.15.16:0/3717100028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47ef1b0 msgr2=0x55bfb4b962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.395+0000 7f67bccd9740 1 --2- 172.21.15.16:0/3717100028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47ef1b0 0x55bfb4b962d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f67a8000f80 tx=0x7f67a801a5c0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.395+0000 7f67bccd9740 1 -- 172.21.15.16:0/3717100028 shutdown_connections 2024-09-21T03:49:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.395+0000 7f67bccd9740 1 --2- 172.21.15.16:0/3717100028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47ef1b0 0x55bfb4b962d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.395+0000 7f67bccd9740 1 --2- 172.21.15.16:0/3717100028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47ce4d0 0x55bfb4b95d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.395+0000 7f67bccd9740 1 --2- 172.21.15.16:0/3717100028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47c62f0 0x55bfb4b9a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.395+0000 7f67bccd9740 1 -- 172.21.15.16:0/3717100028 >> 172.21.15.16:0/3717100028 conn(0x55bfb47e3300 msgr2=0x55bfb4b9a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.396+0000 7f67bccd9740 1 -- 172.21.15.16:0/3717100028 shutdown_connections 2024-09-21T03:49:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.396+0000 7f67bccd9740 1 -- 172.21.15.16:0/3717100028 wait complete. 2024-09-21T03:49:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.396+0000 7f67bccd9740 1 -- start start 2024-09-21T03:49:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.396+0000 7f67bccd9740 1 Processor -- start 2024-09-21T03:49:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.397+0000 7f67bccd9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47c62f0 0x55bfb4ba7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.397+0000 7f67bccd9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47ce4d0 0x55bfb4ba5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.397+0000 7f67bccd9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ef1b0 0x55bfb4ba6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.398+0000 7f67ae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47ce4d0 0x55bfb4ba5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.398+0000 7f67ae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47ce4d0 0x55bfb4ba5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:49578/0 (socket says 172.21.15.16:49578) 2024-09-21T03:49:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.398+0000 7f67ae000640 1 -- 172.21.15.16:0/426107745 learned_addr learned my addr 172.21.15.16:0/426107745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.398+0000 7f67bccd9740 1 -- 172.21.15.16:0/426107745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67b01ab910 con 0x55bfb47c62f0 2024-09-21T03:49:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.398+0000 7f67bccd9740 1 -- 172.21.15.16:0/426107745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f67b01ab240 con 0x55bfb47ce4d0 2024-09-21T03:49:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.398+0000 7f67bccd9740 1 -- 172.21.15.16:0/426107745 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67b01a8480 con 0x55bfb47ef1b0 2024-09-21T03:49:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.398+0000 7f67aea00640 1 --2- 172.21.15.16:0/426107745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47c62f0 0x55bfb4ba7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.398+0000 7f67af400640 1 --2- 172.21.15.16:0/426107745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ef1b0 0x55bfb4ba6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.398+0000 7f67ae000640 1 -- 172.21.15.16:0/426107745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ef1b0 msgr2=0x55bfb4ba6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.398+0000 7f67ae000640 1 --2- 172.21.15.16:0/426107745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ef1b0 0x55bfb4ba6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.398+0000 7f67ae000640 1 -- 172.21.15.16:0/426107745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47c62f0 msgr2=0x55bfb4ba7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.398+0000 7f67ae000640 1 --2- 172.21.15.16:0/426107745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47c62f0 0x55bfb4ba7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.398+0000 7f67ae000640 1 -- 172.21.15.16:0/426107745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfb4ba6940 con 0x55bfb47ce4d0 2024-09-21T03:49:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.398+0000 7f67aea00640 1 --2- 172.21.15.16:0/426107745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47c62f0 0x55bfb4ba7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.398+0000 7f67ae000640 1 --2- 172.21.15.16:0/426107745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47ce4d0 0x55bfb4ba5df0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6798010740 tx=0x7f6798009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a162675a0157a0bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.398+0000 7f679cc00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:49:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.399+0000 7f67bccd9740 10 client.0 _ll_register_callbacks cb 0x7f67980eb4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.399+0000 7f67bccd9740 1 -- 172.21.15.16:0/426107745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67a8047040 con 0x55bfb47ce4d0 2024-09-21T03:49:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.399+0000 7f679cc00640 1 -- 172.21.15.16:0/426107745 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67980ed020 con 0x55bfb47ce4d0 2024-09-21T03:49:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.399+0000 7f679cc00640 1 -- 172.21.15.16:0/426107745 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f67980b6870 con 0x55bfb47ce4d0 2024-09-21T03:49:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.399+0000 7f679cc00640 1 -- 172.21.15.16:0/426107745 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f67980aaa10 con 0x55bfb47ce4d0 2024-09-21T03:49:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.399+0000 7f67bccd9740 10 client.25937 fetch_fsmap learned FSMap version 30 2024-09-21T03:49:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.399+0000 7f67bccd9740 1 -- 172.21.15.16:0/426107745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55bfb4ba57c0 con 0x55bfb47ce4d0 2024-09-21T03:49:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.400+0000 7f679cc00640 1 -- 172.21.15.16:0/426107745 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f67980d3070 con 0x55bfb47ce4d0 2024-09-21T03:49:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.400+0000 7f67bccd9740 10 client.25937 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:49:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.400+0000 7f67bccd9740 10 client.25937 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.400+0000 7f67bccd9740 20 client.25937 populate_metadata read hostname 'smithi016' 2024-09-21T03:49:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.400+0000 7f67bccd9740 1 --2- 172.21.15.16:0/426107745 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55bfb47831c0 0x55bfb4c2a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.400+0000 7f67bccd9740 4 client.25937 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:49:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.400+0000 7f67bccd9740 1 -- 172.21.15.16:0/426107745 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bfb4b070b0 con 0x55bfb47831c0 2024-09-21T03:49:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.400+0000 7f67aea00640 1 --2- 172.21.15.16:0/426107745 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55bfb47831c0 0x55bfb4c2a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.402+0000 7f67aea00640 1 --2- 172.21.15.16:0/426107745 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55bfb47831c0 0x55bfb4c2a960 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b4b62ef776c8bf9 server_cookie=43e24020e638771f in_seq=0 out_seq=0 2024-09-21T03:49:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.403+0000 7f679cc00640 10 client.25937 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.420+0000 7f679cc00640 1 -- 172.21.15.16:0/426107745 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bfb4b070b0 con 0x55bfb47831c0 2024-09-21T03:49:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.420+0000 7f679cc00640 10 client.25937 handle_command_reply: tid=0 2024-09-21T03:49:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.420+0000 7f67bccd9740 1 client.25937 shutdown 2024-09-21T03:49:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.420+0000 7f67bccd9740 1 -- 172.21.15.16:0/426107745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47ce4d0 msgr2=0x55bfb4ba5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.420+0000 7f67bccd9740 1 --2- 172.21.15.16:0/426107745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47ce4d0 0x55bfb4ba5df0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6798010740 tx=0x7f6798009310 comp rx=0 tx=0).stop 2024-09-21T03:49:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.421+0000 7f67bccd9740 1 -- 172.21.15.16:0/426107745 shutdown_connections 2024-09-21T03:49:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.421+0000 7f67bccd9740 1 --2- 172.21.15.16:0/426107745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bfb47ef1b0 0x55bfb4ba6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.421+0000 7f67bccd9740 1 --2- 172.21.15.16:0/426107745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bfb47ce4d0 0x55bfb4ba5df0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.421+0000 7f67bccd9740 1 --2- 172.21.15.16:0/426107745 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55bfb47831c0 0x55bfb4c2a960 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.421+0000 7f67bccd9740 1 --2- 172.21.15.16:0/426107745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfb47c62f0 0x55bfb4ba7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.421+0000 7f67bccd9740 1 -- 172.21.15.16:0/426107745 >> 172.21.15.16:0/426107745 conn(0x55bfb47e3300 msgr2=0x55bfb4b9d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.421+0000 7f679cc00640 0 client.25937 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.422+0000 7f67bccd9740 1 -- 172.21.15.16:0/426107745 shutdown_connections 2024-09-21T03:49:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.422+0000 7f67bccd9740 1 -- 172.21.15.16:0/426107745 wait complete. 2024-09-21T03:49:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.422+0000 7f67bccd9740 20 client.25937 trim_cache size 0 max 16384 2024-09-21T03:49:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:49:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:49:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:49:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:49:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.427+0000 7f67b5c00640 1 -- 172.21.15.16:0/790102357 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f67940b6550 msgr2=0x7f67940b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.427+0000 7f67b5c00640 1 --2- 172.21.15.16:0/790102357 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f67940b6550 0x7f67940b8a60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f67a0005fd0 tx=0x7f67a0005c50 comp rx=0 tx=0).stop 2024-09-21T03:49:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.427+0000 7f67b5c00640 1 -- 172.21.15.16:0/790102357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67b00770f0 msgr2=0x7f67b0198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.427+0000 7f67b5c00640 1 --2- 172.21.15.16:0/790102357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67b00770f0 0x7f67b0198450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f679800e9f0 tx=0x7f679800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.427+0000 7f67b5c00640 1 -- 172.21.15.16:0/790102357 shutdown_connections 2024-09-21T03:49:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.427+0000 7f67b5c00640 1 --2- 172.21.15.16:0/790102357 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f67940b6550 0x7f67940b8a60 secure :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f67a0005fd0 tx=0x7f67a0005c50 comp rx=0 tx=0).stop 2024-09-21T03:49:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.427+0000 7f67b5c00640 1 --2- 172.21.15.16:0/790102357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b010fe40 0x7f67b0198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.427+0000 7f67b5c00640 1 --2- 172.21.15.16:0/790102357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67b00770f0 0x7f67b0198450 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f679800e9f0 tx=0x7f679800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.427+0000 7f67b5c00640 1 --2- 172.21.15.16:0/790102357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67b0075cf0 0x7f67b0197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.427+0000 7f67b5c00640 1 -- 172.21.15.16:0/790102357 >> 172.21.15.16:0/790102357 conn(0x7f67b00ff020 msgr2=0x7f67b01030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.433+0000 7f67b5c00640 1 -- 172.21.15.16:0/790102357 shutdown_connections 2024-09-21T03:49:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:23.433+0000 7f67b5c00640 1 -- 172.21.15.16:0/790102357 wait complete. 2024-09-21T03:49:23.443 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ecfa781f', quiesce took 11.4 seconds. Will release after: 16.4 2024-09-21T03:49:25.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:24 smithi134 ceph-mon[35031]: pgmap v511: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 206 MiB/s wr, 1.09k op/s 2024-09-21T03:49:25.149 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:24 smithi016 ceph-mon[30375]: pgmap v511: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 206 MiB/s wr, 1.09k op/s 2024-09-21T03:49:25.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:24 smithi123 ceph-mon[34673]: pgmap v511: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 206 MiB/s wr, 1.09k op/s 2024-09-21T03:49:27.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:26 smithi134 ceph-mon[35031]: pgmap v512: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 225 MiB/s wr, 1.20k op/s 2024-09-21T03:49:27.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:26 smithi123 ceph-mon[34673]: pgmap v512: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 225 MiB/s wr, 1.20k op/s 2024-09-21T03:49:27.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:26 smithi016 ceph-mon[30375]: pgmap v512: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 225 MiB/s wr, 1.20k op/s 2024-09-21T03:49:29.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:28 smithi134 ceph-mon[35031]: pgmap v513: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 224 MiB/s wr, 1.24k op/s 2024-09-21T03:49:29.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:28 smithi123 ceph-mon[34673]: pgmap v513: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 224 MiB/s wr, 1.24k op/s 2024-09-21T03:49:29.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:28 smithi016 ceph-mon[30375]: pgmap v513: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 224 MiB/s wr, 1.24k op/s 2024-09-21T03:49:30.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:30 smithi016 ceph-mon[30375]: pgmap v514: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 216 MiB/s wr, 1.23k op/s 2024-09-21T03:49:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:30 smithi134 ceph-mon[35031]: pgmap v514: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 216 MiB/s wr, 1.23k op/s 2024-09-21T03:49:31.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:30 smithi123 ceph-mon[34673]: pgmap v514: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 216 MiB/s wr, 1.23k op/s 2024-09-21T03:49:32.815 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:32 smithi016 ceph-mon[30375]: pgmap v515: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 228 MiB/s wr, 1.36k op/s 2024-09-21T03:49:33.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:32 smithi134 ceph-mon[35031]: pgmap v515: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 228 MiB/s wr, 1.36k op/s 2024-09-21T03:49:33.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:32 smithi123 ceph-mon[34673]: pgmap v515: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 228 MiB/s wr, 1.36k op/s 2024-09-21T03:49:35.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:34 smithi134 ceph-mon[35031]: pgmap v516: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 234 MiB/s wr, 1.47k op/s 2024-09-21T03:49:35.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:34 smithi123 ceph-mon[34673]: pgmap v516: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 234 MiB/s wr, 1.47k op/s 2024-09-21T03:49:35.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:34 smithi016 ceph-mon[30375]: pgmap v516: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 234 MiB/s wr, 1.47k op/s 2024-09-21T03:49:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.805+0000 7fb84a000640 1 -- 172.21.15.16:0/4205101929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb844106950 msgr2=0x7fb844106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.805+0000 7fb84a000640 1 --2- 172.21.15.16:0/4205101929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb844106950 0x7fb844106d60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb830009900 tx=0x7fb83002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.805+0000 7fb84a000640 1 -- 172.21.15.16:0/4205101929 shutdown_connections 2024-09-21T03:49:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.805+0000 7fb84a000640 1 --2- 172.21.15.16:0/4205101929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8441072f0 0x7fb84410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.805+0000 7fb84a000640 1 --2- 172.21.15.16:0/4205101929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb844106950 0x7fb844106d60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.805+0000 7fb84a000640 1 --2- 172.21.15.16:0/4205101929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8441058d0 0x7fb844105d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.805+0000 7fb84a000640 1 -- 172.21.15.16:0/4205101929 >> 172.21.15.16:0/4205101929 conn(0x7fb844101230 msgr2=0x7fb8441036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.805+0000 7fb84a000640 1 -- 172.21.15.16:0/4205101929 shutdown_connections 2024-09-21T03:49:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.805+0000 7fb84a000640 1 -- 172.21.15.16:0/4205101929 wait complete. 2024-09-21T03:49:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.806+0000 7fb84a000640 1 Processor -- start 2024-09-21T03:49:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.806+0000 7fb84a000640 1 -- start start 2024-09-21T03:49:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.807+0000 7fb84a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8441058d0 0x7fb84419c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.807+0000 7fb84a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb844106950 0x7fb84419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.807+0000 7fb84a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8441072f0 0x7fb84419d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.807+0000 7fb84a000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb844111550 con 0x7fb844106950 2024-09-21T03:49:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.807+0000 7fb84a000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8441113d0 con 0x7fb8441058d0 2024-09-21T03:49:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.807+0000 7fb84a000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb8441116d0 con 0x7fb8441072f0 2024-09-21T03:49:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.807+0000 7fb843400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8441058d0 0x7fb84419c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.807+0000 7fb843400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8441058d0 0x7fb84419c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:59034/0 (socket says 172.21.15.16:59034) 2024-09-21T03:49:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.807+0000 7fb843400640 1 -- 172.21.15.16:0/3989459567 learned_addr learned my addr 172.21.15.16:0/3989459567 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.807+0000 7fb842a00640 1 --2- 172.21.15.16:0/3989459567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb844106950 0x7fb84419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.807+0000 7fb843e00640 1 --2- 172.21.15.16:0/3989459567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8441072f0 0x7fb84419d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.808+0000 7fb843400640 1 -- 172.21.15.16:0/3989459567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8441072f0 msgr2=0x7fb84419d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.808+0000 7fb843400640 1 --2- 172.21.15.16:0/3989459567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8441072f0 0x7fb84419d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.808+0000 7fb843400640 1 -- 172.21.15.16:0/3989459567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb844106950 msgr2=0x7fb84419c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.808+0000 7fb843400640 1 --2- 172.21.15.16:0/3989459567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb844106950 0x7fb84419c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.808+0000 7fb843400640 1 -- 172.21.15.16:0/3989459567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84419db80 con 0x7fb8441058d0 2024-09-21T03:49:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.808+0000 7fb843400640 1 --2- 172.21.15.16:0/3989459567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8441058d0 0x7fb84419c140 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb83400d950 tx=0x7fb83400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.808+0000 7fb83be00640 1 -- 172.21.15.16:0/3989459567 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834017030 con 0x7fb8441058d0 2024-09-21T03:49:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.808+0000 7fb83be00640 1 -- 172.21.15.16:0/3989459567 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb834004020 con 0x7fb8441058d0 2024-09-21T03:49:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.808+0000 7fb84a000640 1 -- 172.21.15.16:0/3989459567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84419de70 con 0x7fb8441058d0 2024-09-21T03:49:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.809+0000 7fb84a000640 1 -- 172.21.15.16:0/3989459567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8441a4560 con 0x7fb8441058d0 2024-09-21T03:49:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.809+0000 7fb83be00640 1 -- 172.21.15.16:0/3989459567 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8340047a0 con 0x7fb8441058d0 2024-09-21T03:49:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.811+0000 7fb85120a740 1 Processor -- start 2024-09-21T03:49:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.812+0000 7fb83be00640 1 -- 172.21.15.16:0/3989459567 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb834004940 con 0x7fb8441058d0 2024-09-21T03:49:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.812+0000 7fb85120a740 1 -- start start 2024-09-21T03:49:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.812+0000 7fb85120a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456991b0 0x55f545a400d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.812+0000 7fb85120a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456702f0 0x55f545a40740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.812+0000 7fb85120a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5456784d0 0x55f545a41290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.812+0000 7fb85120a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f5459b36b0 con 0x55f5456991b0 2024-09-21T03:49:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.812+0000 7fb85120a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f5459a2ae0 con 0x55f5456784d0 2024-09-21T03:49:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.812+0000 7fb85120a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f5459a68f0 con 0x55f5456702f0 2024-09-21T03:49:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.813+0000 7fb843e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456991b0 0x55f545a400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.813+0000 7fb842a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5456784d0 0x55f545a41290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.813+0000 7fb843e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456991b0 0x55f545a400d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57448/0 (socket says 172.21.15.16:57448) 2024-09-21T03:49:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.813+0000 7fb843e00640 1 -- 172.21.15.16:0/2308204217 learned_addr learned my addr 172.21.15.16:0/2308204217 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.813+0000 7fb843400640 1 --2- 172.21.15.16:0/2308204217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456702f0 0x55f545a40740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.813+0000 7fb843e00640 1 -- 172.21.15.16:0/2308204217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456702f0 msgr2=0x55f545a40740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.813+0000 7fb843e00640 1 --2- 172.21.15.16:0/2308204217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456702f0 0x55f545a40740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.813+0000 7fb843e00640 1 -- 172.21.15.16:0/2308204217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5456784d0 msgr2=0x55f545a41290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.813+0000 7fb843e00640 1 --2- 172.21.15.16:0/2308204217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5456784d0 0x55f545a41290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.813+0000 7fb843e00640 1 -- 172.21.15.16:0/2308204217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f54596f780 con 0x55f5456991b0 2024-09-21T03:49:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.814+0000 7fb843e00640 1 --2- 172.21.15.16:0/2308204217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456991b0 0x55f545a400d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009f70 tx=0x7fb83c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e4d6148fb2b475 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.814+0000 7fb83be00640 1 --2- 172.21.15.16:0/3989459567 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8280b6510 0x7fb8280b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.814+0000 7fb82fe00640 1 -- 172.21.15.16:0/2308204217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c016070 con 0x55f5456991b0 2024-09-21T03:49:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.814+0000 7fb83be00640 1 -- 172.21.15.16:0/3989459567 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb8340e70a0 con 0x7fb8441058d0 2024-09-21T03:49:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.814+0000 7fb82fe00640 1 -- 172.21.15.16:0/2308204217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb83c004d20 con 0x55f5456991b0 2024-09-21T03:49:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.814+0000 7fb82fe00640 1 -- 172.21.15.16:0/2308204217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0127d0 con 0x55f5456991b0 2024-09-21T03:49:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.814+0000 7fb843400640 1 --2- 172.21.15.16:0/3989459567 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8280b6510 0x7fb8280b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.815+0000 7fb85120a740 1 -- 172.21.15.16:0/2308204217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456991b0 msgr2=0x55f545a400d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.815+0000 7fb85120a740 1 --2- 172.21.15.16:0/2308204217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456991b0 0x55f545a400d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009f70 tx=0x7fb83c010800 comp rx=0 tx=0).stop 2024-09-21T03:49:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.815+0000 7fb843400640 1 --2- 172.21.15.16:0/3989459567 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8280b6510 0x7fb8280b8a20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb8340a93f0 tx=0x7fb834000bc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.816+0000 7fb85120a740 1 -- 172.21.15.16:0/2308204217 shutdown_connections 2024-09-21T03:49:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.816+0000 7fb85120a740 1 --2- 172.21.15.16:0/2308204217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5456784d0 0x55f545a41290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.816+0000 7fb85120a740 1 --2- 172.21.15.16:0/2308204217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456702f0 0x55f545a40740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.816+0000 7fb85120a740 1 --2- 172.21.15.16:0/2308204217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456991b0 0x55f545a400d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.816+0000 7fb85120a740 1 -- 172.21.15.16:0/2308204217 >> 172.21.15.16:0/2308204217 conn(0x55f54568d300 msgr2=0x55f545a3bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.816+0000 7fb85120a740 1 -- 172.21.15.16:0/2308204217 shutdown_connections 2024-09-21T03:49:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.816+0000 7fb85120a740 1 -- 172.21.15.16:0/2308204217 wait complete. 2024-09-21T03:49:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.817+0000 7fb85120a740 1 -- start start 2024-09-21T03:49:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.817+0000 7fb85120a740 1 Processor -- start 2024-09-21T03:49:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.817+0000 7fb85120a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456702f0 0x55f545a49780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.817+0000 7fb85120a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456784d0 0x55f545a49e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.818+0000 7fb85120a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5456991b0 0x55f545a4a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.818+0000 7fb85120a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f5458daab0 con 0x55f5456784d0 2024-09-21T03:49:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.818+0000 7fb85120a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f545949570 con 0x55f5456991b0 2024-09-21T03:49:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.818+0000 7fb85120a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f545a41db0 con 0x55f5456702f0 2024-09-21T03:49:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.818+0000 7fb85120a740 10 client.0 _ll_register_callbacks cb 0x55f545958100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.818+0000 7fb843e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456702f0 0x55f545a49780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.818+0000 7fb843e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456702f0 0x55f545a49780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:47504/0 (socket says 172.21.15.16:47504) 2024-09-21T03:49:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.818+0000 7fb843e00640 1 -- 172.21.15.16:0/1696376355 learned_addr learned my addr 172.21.15.16:0/1696376355 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.818+0000 7fb842a00640 1 --2- 172.21.15.16:0/1696376355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456784d0 0x55f545a49e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.818+0000 7fb843e00640 1 --2- 172.21.15.16:0/1696376355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5456991b0 0x55f545a4a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.819+0000 7fb842a00640 1 -- 172.21.15.16:0/1696376355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456702f0 msgr2=0x55f545a49780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.819+0000 7fb842a00640 1 --2- 172.21.15.16:0/1696376355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456702f0 0x55f545a49780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.819+0000 7fb842a00640 1 -- 172.21.15.16:0/1696376355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5456991b0 msgr2=0x55f545a4a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.819+0000 7fb842a00640 1 --2- 172.21.15.16:0/1696376355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5456991b0 0x55f545a4a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.819+0000 7fb842a00640 1 -- 172.21.15.16:0/1696376355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f545704790 con 0x55f5456784d0 2024-09-21T03:49:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.819+0000 7fb843e00640 1 --2- 172.21.15.16:0/1696376355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456702f0 0x55f545a49780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.819+0000 7fb842a00640 1 --2- 172.21.15.16:0/1696376355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456784d0 0x55f545a49e40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb83002a9e0 tx=0x7fb830004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba776e39041230e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.819+0000 7fb82cc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:49:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.819+0000 7fb82cc00640 1 -- 172.21.15.16:0/1696376355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830035040 con 0x55f5456784d0 2024-09-21T03:49:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.819+0000 7fb82cc00640 1 -- 172.21.15.16:0/1696376355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb830003b40 con 0x55f5456784d0 2024-09-21T03:49:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.820+0000 7fb85120a740 1 -- 172.21.15.16:0/1696376355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb83c004d20 con 0x55f5456784d0 2024-09-21T03:49:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.820+0000 7fb82cc00640 1 -- 172.21.15.16:0/1696376355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb83004c050 con 0x55f5456784d0 2024-09-21T03:49:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.820+0000 7fb85120a740 10 client.15876 fetch_fsmap learned FSMap version 30 2024-09-21T03:49:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.820+0000 7fb85120a740 1 -- 172.21.15.16:0/1696376355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f54577d620 con 0x55f5456784d0 2024-09-21T03:49:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.820+0000 7fb82cc00640 1 -- 172.21.15.16:0/1696376355 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb83c004d20 con 0x55f5456784d0 2024-09-21T03:49:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.820+0000 7fb82cc00640 1 -- 172.21.15.16:0/1696376355 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb83003c440 con 0x55f5456784d0 2024-09-21T03:49:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.821+0000 7fb85120a740 10 client.15876 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:49:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.821+0000 7fb85120a740 10 client.15876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.821+0000 7fb85120a740 20 client.15876 populate_metadata read hostname 'smithi016' 2024-09-21T03:49:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.821+0000 7fb85120a740 1 --2- 172.21.15.16:0/1696376355 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55f545657d80 0x55f545a56fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.821+0000 7fb85120a740 4 client.15876 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.821+0000 7fb85120a740 1 -- 172.21.15.16:0/1696376355 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5459b5600 con 0x55f545657d80 2024-09-21T03:49:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.821+0000 7fb843e00640 1 --2- 172.21.15.16:0/1696376355 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55f545657d80 0x55f545a56fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.821+0000 7fb843e00640 1 --2- 172.21.15.16:0/1696376355 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55f545657d80 0x55f545a56fc0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85f687b960b93b6e server_cookie=9824b05ec0e0d24 in_seq=0 out_seq=0 2024-09-21T03:49:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.821+0000 7fb82cc00640 10 client.15876 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.822+0000 7fb82cc00640 1 -- 172.21.15.16:0/1696376355 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f5459b5600 con 0x55f545657d80 2024-09-21T03:49:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.822+0000 7fb82cc00640 10 client.15876 handle_command_reply: tid=0 2024-09-21T03:49:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.822+0000 7fb85120a740 1 client.15876 shutdown 2024-09-21T03:49:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.823+0000 7fb85120a740 1 -- 172.21.15.16:0/1696376355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456784d0 msgr2=0x55f545a49e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.823+0000 7fb85120a740 1 --2- 172.21.15.16:0/1696376355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456784d0 0x55f545a49e40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb83002a9e0 tx=0x7fb830004580 comp rx=0 tx=0).stop 2024-09-21T03:49:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.823+0000 7fb85120a740 1 -- 172.21.15.16:0/1696376355 shutdown_connections 2024-09-21T03:49:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.823+0000 7fb85120a740 1 --2- 172.21.15.16:0/1696376355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5456991b0 0x55f545a4a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.823+0000 7fb85120a740 1 --2- 172.21.15.16:0/1696376355 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55f545657d80 0x55f545a56fc0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.823+0000 7fb85120a740 1 --2- 172.21.15.16:0/1696376355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456784d0 0x55f545a49e40 secure :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb83002a9e0 tx=0x7fb830004580 comp rx=0 tx=0).stop 2024-09-21T03:49:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.823+0000 7fb85120a740 1 --2- 172.21.15.16:0/1696376355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456702f0 0x55f545a49780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.823+0000 7fb85120a740 1 -- 172.21.15.16:0/1696376355 >> 172.21.15.16:0/1696376355 conn(0x55f54568d300 msgr2=0x55f545a3a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.823+0000 7fb82cc00640 0 client.15876 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.823+0000 7fb85120a740 1 -- 172.21.15.16:0/1696376355 shutdown_connections 2024-09-21T03:49:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.824+0000 7fb85120a740 1 -- 172.21.15.16:0/1696376355 wait complete. 2024-09-21T03:49:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.824+0000 7fb85120a740 20 client.15876 trim_cache size 0 max 16384 2024-09-21T03:49:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.848+0000 7fb85120a740 1 Processor -- start 2024-09-21T03:49:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.848+0000 7fb85120a740 1 -- start start 2024-09-21T03:49:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.848+0000 7fb85120a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456702f0 0x55f545a3d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.848+0000 7fb85120a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456991b0 0x55f545a3fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.848+0000 7fb85120a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f54562d1c0 0x55f545a40200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.848+0000 7fb85120a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb82400fc50 con 0x55f5456702f0 2024-09-21T03:49:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.848+0000 7fb85120a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb824011790 con 0x55f54562d1c0 2024-09-21T03:49:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.848+0000 7fb85120a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb824010320 con 0x55f5456991b0 2024-09-21T03:49:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.848+0000 7fb843e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456702f0 0x55f545a3d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.848+0000 7fb842a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456991b0 0x55f545a3fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.848+0000 7fb842a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456991b0 0x55f545a3fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:47514/0 (socket says 172.21.15.16:47514) 2024-09-21T03:49:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.848+0000 7fb843e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456702f0 0x55f545a3d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57476/0 (socket says 172.21.15.16:57476) 2024-09-21T03:49:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.848+0000 7fb842a00640 1 -- 172.21.15.16:0/2293666792 learned_addr learned my addr 172.21.15.16:0/2293666792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.848+0000 7fb843e00640 1 --2- 172.21.15.16:0/2293666792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f54562d1c0 0x55f545a40200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.849+0000 7fb842a00640 1 -- 172.21.15.16:0/2293666792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f54562d1c0 msgr2=0x55f545a40200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.849+0000 7fb842a00640 1 --2- 172.21.15.16:0/2293666792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f54562d1c0 0x55f545a40200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.849+0000 7fb842a00640 1 -- 172.21.15.16:0/2293666792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456702f0 msgr2=0x55f545a3d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.849+0000 7fb842a00640 1 --2- 172.21.15.16:0/2293666792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456702f0 0x55f545a3d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.849+0000 7fb842a00640 1 -- 172.21.15.16:0/2293666792 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f545a4a140 con 0x55f5456991b0 2024-09-21T03:49:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.849+0000 7fb843e00640 1 --2- 172.21.15.16:0/2293666792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f54562d1c0 0x55f545a40200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.849+0000 7fb842a00640 1 --2- 172.21.15.16:0/2293666792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456991b0 0x55f545a3fcc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb83003c810 tx=0x7fb830000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2213b706f05be1a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.849+0000 7fb82f400640 1 -- 172.21.15.16:0/2293666792 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830035070 con 0x55f5456991b0 2024-09-21T03:49:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.850+0000 7fb82f400640 1 -- 172.21.15.16:0/2293666792 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb830056070 con 0x55f5456991b0 2024-09-21T03:49:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.850+0000 7fb82f400640 1 -- 172.21.15.16:0/2293666792 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830003f50 con 0x55f5456991b0 2024-09-21T03:49:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.851+0000 7fb85120a740 1 -- 172.21.15.16:0/2293666792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456991b0 msgr2=0x55f545a3fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.851+0000 7fb85120a740 1 --2- 172.21.15.16:0/2293666792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456991b0 0x55f545a3fcc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb83003c810 tx=0x7fb830000bc0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.851+0000 7fb85120a740 1 -- 172.21.15.16:0/2293666792 shutdown_connections 2024-09-21T03:49:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.851+0000 7fb85120a740 1 --2- 172.21.15.16:0/2293666792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f54562d1c0 0x55f545a40200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.851+0000 7fb85120a740 1 --2- 172.21.15.16:0/2293666792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5456991b0 0x55f545a3fcc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.851+0000 7fb85120a740 1 --2- 172.21.15.16:0/2293666792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456702f0 0x55f545a3d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.851+0000 7fb85120a740 1 -- 172.21.15.16:0/2293666792 >> 172.21.15.16:0/2293666792 conn(0x55f54568d300 msgr2=0x55f545a40f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.851+0000 7fb85120a740 1 -- 172.21.15.16:0/2293666792 shutdown_connections 2024-09-21T03:49:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.851+0000 7fb85120a740 1 -- 172.21.15.16:0/2293666792 wait complete. 2024-09-21T03:49:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.852+0000 7fb85120a740 1 -- start start 2024-09-21T03:49:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.852+0000 7fb85120a740 1 Processor -- start 2024-09-21T03:49:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.853+0000 7fb85120a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f54562d1c0 0x55f545a51840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.853+0000 7fb85120a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456702f0 0x55f545a50610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.854+0000 7fb85120a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5456991b0 0x55f545a50d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.854+0000 7fb85120a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8240110c0 con 0x55f5456702f0 2024-09-21T03:49:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.854+0000 7fb85120a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8240109f0 con 0x55f5456991b0 2024-09-21T03:49:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.854+0000 7fb85120a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb82400dc30 con 0x55f54562d1c0 2024-09-21T03:49:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.854+0000 7fb85120a740 10 client.0 _ll_register_callbacks cb 0x7fb830009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.854+0000 7fb842a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456702f0 0x55f545a50610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.854+0000 7fb842a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456702f0 0x55f545a50610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57480/0 (socket says 172.21.15.16:57480) 2024-09-21T03:49:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.854+0000 7fb842a00640 1 -- 172.21.15.16:0/1859715721 learned_addr learned my addr 172.21.15.16:0/1859715721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.855+0000 7fb843e00640 1 --2- 172.21.15.16:0/1859715721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f54562d1c0 0x55f545a51840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.855+0000 7fb842a00640 1 -- 172.21.15.16:0/1859715721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f54562d1c0 msgr2=0x55f545a51840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.855+0000 7fb842a00640 1 --2- 172.21.15.16:0/1859715721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f54562d1c0 0x55f545a51840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.856+0000 7fb842a00640 1 -- 172.21.15.16:0/1859715721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5456991b0 msgr2=0x55f545a50d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.856+0000 7fb842a00640 1 --2- 172.21.15.16:0/1859715721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5456991b0 0x55f545a50d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.856+0000 7fb842a00640 1 -- 172.21.15.16:0/1859715721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f545a4f130 con 0x55f5456702f0 2024-09-21T03:49:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.856+0000 7fb842a00640 1 --2- 172.21.15.16:0/1859715721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456702f0 0x55f545a50610 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb83002a9e0 tx=0x7fb830005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d695d1e8b53190 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.856+0000 7fb82cc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:49:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.858+0000 7fb82cc00640 1 -- 172.21.15.16:0/1859715721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830035030 con 0x55f5456702f0 2024-09-21T03:49:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.858+0000 7fb82cc00640 1 -- 172.21.15.16:0/1859715721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb830003b40 con 0x55f5456702f0 2024-09-21T03:49:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.858+0000 7fb85120a740 1 -- 172.21.15.16:0/1859715721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb830056070 con 0x55f5456702f0 2024-09-21T03:49:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.859+0000 7fb82cc00640 1 -- 172.21.15.16:0/1859715721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb8300315a0 con 0x55f5456702f0 2024-09-21T03:49:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.859+0000 7fb85120a740 10 client.15882 fetch_fsmap learned FSMap version 30 2024-09-21T03:49:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.859+0000 7fb85120a740 1 -- 172.21.15.16:0/1859715721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f545aca410 con 0x55f5456702f0 2024-09-21T03:49:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.859+0000 7fb82cc00640 1 -- 172.21.15.16:0/1859715721 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb830056070 con 0x55f5456702f0 2024-09-21T03:49:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.860+0000 7fb82cc00640 1 -- 172.21.15.16:0/1859715721 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb830051560 con 0x55f5456702f0 2024-09-21T03:49:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.861+0000 7fb85120a740 10 client.15882 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:49:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.861+0000 7fb85120a740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.861+0000 7fb85120a740 20 client.15882 populate_metadata read hostname 'smithi016' 2024-09-21T03:49:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.861+0000 7fb85120a740 1 --2- 172.21.15.16:0/1859715721 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55f545acaca0 0x55f545acd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.861+0000 7fb85120a740 4 client.15882 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:49:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.861+0000 7fb85120a740 1 -- 172.21.15.16:0/1859715721 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f5459b10b0 con 0x55f545acaca0 2024-09-21T03:49:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.861+0000 7fb843e00640 1 --2- 172.21.15.16:0/1859715721 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55f545acaca0 0x55f545acd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.862+0000 7fb843e00640 1 --2- 172.21.15.16:0/1859715721 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55f545acaca0 0x55f545acd0c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d85ec87e8c021e7f server_cookie=72aed3e29cf7e4da in_seq=0 out_seq=0 2024-09-21T03:49:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.862+0000 7fb82cc00640 10 client.15882 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.874+0000 7fb82cc00640 1 -- 172.21.15.16:0/1859715721 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f5459b10b0 con 0x55f545acaca0 2024-09-21T03:49:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.874+0000 7fb82cc00640 10 client.15882 handle_command_reply: tid=0 2024-09-21T03:49:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.875+0000 7fb85120a740 1 client.15882 shutdown 2024-09-21T03:49:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.876+0000 7fb85120a740 1 -- 172.21.15.16:0/1859715721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456702f0 msgr2=0x55f545a50610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.876+0000 7fb85120a740 1 --2- 172.21.15.16:0/1859715721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456702f0 0x55f545a50610 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb83002a9e0 tx=0x7fb830005f00 comp rx=0 tx=0).stop 2024-09-21T03:49:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.876+0000 7fb85120a740 1 -- 172.21.15.16:0/1859715721 shutdown_connections 2024-09-21T03:49:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.876+0000 7fb85120a740 1 --2- 172.21.15.16:0/1859715721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5456991b0 0x55f545a50d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.876+0000 7fb85120a740 1 --2- 172.21.15.16:0/1859715721 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55f545acaca0 0x55f545acd0c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.876+0000 7fb85120a740 1 --2- 172.21.15.16:0/1859715721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5456702f0 0x55f545a50610 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.876+0000 7fb85120a740 1 --2- 172.21.15.16:0/1859715721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f54562d1c0 0x55f545a51840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.876+0000 7fb85120a740 1 -- 172.21.15.16:0/1859715721 >> 172.21.15.16:0/1859715721 conn(0x55f54568d300 msgr2=0x55f545a40f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.876+0000 7fb82cc00640 0 client.15882 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.876+0000 7fb85120a740 1 -- 172.21.15.16:0/1859715721 shutdown_connections 2024-09-21T03:49:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.878+0000 7fb85120a740 1 -- 172.21.15.16:0/1859715721 wait complete. 2024-09-21T03:49:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.878+0000 7fb85120a740 20 client.15882 trim_cache size 0 max 16384 2024-09-21T03:49:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:49:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:49:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:49:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:49:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.879+0000 7fb84a000640 1 -- 172.21.15.16:0/3989459567 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8280b6510 msgr2=0x7fb8280b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.879+0000 7fb84a000640 1 --2- 172.21.15.16:0/3989459567 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8280b6510 0x7fb8280b8a20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb8340a93f0 tx=0x7fb834000bc0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.879+0000 7fb84a000640 1 -- 172.21.15.16:0/3989459567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8441058d0 msgr2=0x7fb84419c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.879+0000 7fb84a000640 1 --2- 172.21.15.16:0/3989459567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8441058d0 0x7fb84419c140 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb83400d950 tx=0x7fb83400de10 comp rx=0 tx=0).stop 2024-09-21T03:49:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.879+0000 7fb84a000640 1 -- 172.21.15.16:0/3989459567 shutdown_connections 2024-09-21T03:49:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.879+0000 7fb84a000640 1 --2- 172.21.15.16:0/3989459567 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb8280b6510 0x7fb8280b8a20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.879+0000 7fb84a000640 1 --2- 172.21.15.16:0/3989459567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8441072f0 0x7fb84419d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.879+0000 7fb84a000640 1 --2- 172.21.15.16:0/3989459567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb844106950 0x7fb84419c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.879+0000 7fb84a000640 1 --2- 172.21.15.16:0/3989459567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8441058d0 0x7fb84419c140 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.879+0000 7fb84a000640 1 -- 172.21.15.16:0/3989459567 >> 172.21.15.16:0/3989459567 conn(0x7fb844101230 msgr2=0x7fb844109c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.879+0000 7fb84a000640 1 -- 172.21.15.16:0/3989459567 shutdown_connections 2024-09-21T03:49:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:36.879+0000 7fb84a000640 1 -- 172.21.15.16:0/3989459567 wait complete. 2024-09-21T03:49:37.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:36 smithi134 ceph-mon[35031]: pgmap v517: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 240 MiB/s wr, 1.59k op/s 2024-09-21T03:49:37.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:36 smithi123 ceph-mon[34673]: pgmap v517: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 240 MiB/s wr, 1.59k op/s 2024-09-21T03:49:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:36 smithi016 ceph-mon[30375]: pgmap v517: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 240 MiB/s wr, 1.59k op/s 2024-09-21T03:49:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:38 smithi134 ceph-mon[35031]: pgmap v518: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 235 MiB/s wr, 1.64k op/s 2024-09-21T03:49:39.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:38 smithi123 ceph-mon[34673]: pgmap v518: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 235 MiB/s wr, 1.64k op/s 2024-09-21T03:49:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:38 smithi016 ceph-mon[30375]: pgmap v518: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 235 MiB/s wr, 1.64k op/s 2024-09-21T03:49:39.843 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:49:40.665 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:41.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:40 smithi123 ceph-mon[34673]: pgmap v519: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 222 MiB/s wr, 1.62k op/s 2024-09-21T03:49:41.169 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:40 smithi016 ceph-mon[30375]: pgmap v519: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 222 MiB/s wr, 1.62k op/s 2024-09-21T03:49:41.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:40 smithi134 ceph-mon[35031]: pgmap v519: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 222 MiB/s wr, 1.62k op/s 2024-09-21T03:49:41.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.594+0000 7f1139800640 1 -- 172.21.15.16:0/930392087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1134076100 msgr2=0x7f113410fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.594+0000 7f1139800640 1 --2- 172.21.15.16:0/930392087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1134076100 0x7f113410fb90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f111c009900 tx=0x7f111c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.594+0000 7f1139800640 1 -- 172.21.15.16:0/930392087 shutdown_connections 2024-09-21T03:49:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.594+0000 7f1139800640 1 --2- 172.21.15.16:0/930392087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11341100d0 0x7f1134112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.594+0000 7f1139800640 1 --2- 172.21.15.16:0/930392087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1134076100 0x7f113410fb90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.594+0000 7f1139800640 1 --2- 172.21.15.16:0/930392087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11340757b0 0x7f1134075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.594+0000 7f1139800640 1 -- 172.21.15.16:0/930392087 >> 172.21.15.16:0/930392087 conn(0x7f11340ff5e0 msgr2=0x7f1134101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.594+0000 7f1139800640 1 -- 172.21.15.16:0/930392087 shutdown_connections 2024-09-21T03:49:41.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.595+0000 7f1139800640 1 -- 172.21.15.16:0/930392087 wait complete. 2024-09-21T03:49:41.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.596+0000 7f1139800640 1 Processor -- start 2024-09-21T03:49:41.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.596+0000 7f1139800640 1 -- start start 2024-09-21T03:49:41.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.596+0000 7f1139800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11340757b0 0x7f113419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:41.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.596+0000 7f1139800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1134076100 0x7f113419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:41.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.596+0000 7f1139800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11341100d0 0x7f11341a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:41.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.596+0000 7f1139800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1134115c80 con 0x7f1134076100 2024-09-21T03:49:41.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.596+0000 7f1139800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1134115b00 con 0x7f11340757b0 2024-09-21T03:49:41.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.596+0000 7f1139800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1134115e00 con 0x7f11341100d0 2024-09-21T03:49:41.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.597+0000 7f1132a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11340757b0 0x7f113419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:41.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.597+0000 7f112be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1134076100 0x7f113419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:41.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.597+0000 7f1132a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11340757b0 0x7f113419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:59102/0 (socket says 172.21.15.16:59102) 2024-09-21T03:49:41.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.597+0000 7f1132a00640 1 -- 172.21.15.16:0/3003316913 learned_addr learned my addr 172.21.15.16:0/3003316913 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:41.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.597+0000 7f1133400640 1 --2- 172.21.15.16:0/3003316913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11341100d0 0x7f11341a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:41.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.597+0000 7f1132a00640 1 -- 172.21.15.16:0/3003316913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11341100d0 msgr2=0x7f11341a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:41.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.597+0000 7f1132a00640 1 --2- 172.21.15.16:0/3003316913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11341100d0 0x7f11341a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.597+0000 7f1132a00640 1 -- 172.21.15.16:0/3003316913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1134076100 msgr2=0x7f113419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:41.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.597+0000 7f1132a00640 1 --2- 172.21.15.16:0/3003316913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1134076100 0x7f113419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.597+0000 7f1132a00640 1 -- 172.21.15.16:0/3003316913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11341a25e0 con 0x7f11340757b0 2024-09-21T03:49:41.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.597+0000 7f1132a00640 1 --2- 172.21.15.16:0/3003316913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11340757b0 0x7f113419c670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f112400ea60 tx=0x7f112400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:41.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.598+0000 7f112b400640 1 -- 172.21.15.16:0/3003316913 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112400cde0 con 0x7f11340757b0 2024-09-21T03:49:41.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.598+0000 7f112b400640 1 -- 172.21.15.16:0/3003316913 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11240035f0 con 0x7f11340757b0 2024-09-21T03:49:41.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.598+0000 7f112b400640 1 -- 172.21.15.16:0/3003316913 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124010690 con 0x7f11340757b0 2024-09-21T03:49:41.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.598+0000 7f1139800640 1 -- 172.21.15.16:0/3003316913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11341a28d0 con 0x7f11340757b0 2024-09-21T03:49:41.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.598+0000 7f1139800640 1 -- 172.21.15.16:0/3003316913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11341a9030 con 0x7f11340757b0 2024-09-21T03:49:41.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.599+0000 7f1139800640 1 -- 172.21.15.16:0/3003316913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1134075bc0 con 0x7f11340757b0 2024-09-21T03:49:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.601+0000 7f112b400640 1 -- 172.21.15.16:0/3003316913 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f11240037a0 con 0x7f11340757b0 2024-09-21T03:49:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.603+0000 7f112b400640 1 --2- 172.21.15.16:0/3003316913 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f11180b6510 0x7f11180b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.603+0000 7f112b400640 1 -- 172.21.15.16:0/3003316913 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f11240e79a0 con 0x7f11340757b0 2024-09-21T03:49:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.603+0000 7f112be00640 1 --2- 172.21.15.16:0/3003316913 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f11180b6510 0x7f11180b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:41.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.604+0000 7f112be00640 1 --2- 172.21.15.16:0/3003316913 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f11180b6510 0x7f11180b8a20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f111c000c00 tx=0x7f111c031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.607+0000 7f112b400640 1 -- 172.21.15.16:0/3003316913 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f11240ade20 con 0x7f11340757b0 2024-09-21T03:49:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.850+0000 7f1139800640 1 -- 172.21.15.16:0/3003316913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f113419d7a0 con 0x7f11340757b0 2024-09-21T03:49:41.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.851+0000 7f112b400640 1 -- 172.21.15.16:0/3003316913 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f11240a7e20 con 0x7f11340757b0 2024-09-21T03:49:41.854 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:49:41.854 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:49:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:49:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.853+0000 7f1139800640 1 -- 172.21.15.16:0/3003316913 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f11180b6510 msgr2=0x7f11180b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.853+0000 7f1139800640 1 --2- 172.21.15.16:0/3003316913 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f11180b6510 0x7f11180b8a20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f111c000c00 tx=0x7f111c031400 comp rx=0 tx=0).stop 2024-09-21T03:49:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.853+0000 7f1139800640 1 -- 172.21.15.16:0/3003316913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11340757b0 msgr2=0x7f113419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.853+0000 7f1139800640 1 --2- 172.21.15.16:0/3003316913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11340757b0 0x7f113419c670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f112400ea60 tx=0x7f112400ef20 comp rx=0 tx=0).stop 2024-09-21T03:49:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.853+0000 7f1139800640 1 -- 172.21.15.16:0/3003316913 shutdown_connections 2024-09-21T03:49:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.853+0000 7f1139800640 1 --2- 172.21.15.16:0/3003316913 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f11180b6510 0x7f11180b8a20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.853+0000 7f1139800640 1 --2- 172.21.15.16:0/3003316913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11341100d0 0x7f11341a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.853+0000 7f1139800640 1 --2- 172.21.15.16:0/3003316913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1134076100 0x7f113419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.853+0000 7f1139800640 1 --2- 172.21.15.16:0/3003316913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11340757b0 0x7f113419c670 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.853+0000 7f1139800640 1 -- 172.21.15.16:0/3003316913 >> 172.21.15.16:0/3003316913 conn(0x7f11340ff5e0 msgr2=0x7f1134109c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.854+0000 7f1139800640 1 -- 172.21.15.16:0/3003316913 shutdown_connections 2024-09-21T03:49:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:41.854+0000 7f1139800640 1 -- 172.21.15.16:0/3003316913 wait complete. 2024-09-21T03:49:42.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:41 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3003316913' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:42.253 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:41 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3003316913' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:42.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:41 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3003316913' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:49:42.768 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db --set-id ecfa781f --release --await' 2024-09-21T03:49:42.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db --set-id ecfa781f --release --await 2024-09-21T03:49:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:49:43.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:42 smithi016 ceph-mon[30375]: pgmap v520: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 231 MiB/s wr, 1.77k op/s 2024-09-21T03:49:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:42 smithi134 ceph-mon[35031]: pgmap v520: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 231 MiB/s wr, 1.77k op/s 2024-09-21T03:49:43.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:42 smithi123 ceph-mon[34673]: pgmap v520: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 231 MiB/s wr, 1.77k op/s 2024-09-21T03:49:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.982+0000 7f9afc400640 1 -- 172.21.15.16:0/978765816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9af4114440 msgr2=0x7f9af4116910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.982+0000 7f9afc400640 1 --2- 172.21.15.16:0/978765816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9af4114440 0x7f9af4116910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009ca0 tx=0x7f9aec02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.982+0000 7f9afc400640 1 -- 172.21.15.16:0/978765816 shutdown_connections 2024-09-21T03:49:43.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.982+0000 7f9afc400640 1 --2- 172.21.15.16:0/978765816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9af4114440 0x7f9af4116910 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.982+0000 7f9afc400640 1 --2- 172.21.15.16:0/978765816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9af4075dd0 0x7f9af40761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.982+0000 7f9afc400640 1 --2- 172.21.15.16:0/978765816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9af4077380 0x7f9af40757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.982+0000 7f9afc400640 1 -- 172.21.15.16:0/978765816 >> 172.21.15.16:0/978765816 conn(0x7f9af40ff5e0 msgr2=0x7f9af4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:43.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.982+0000 7f9afc400640 1 -- 172.21.15.16:0/978765816 shutdown_connections 2024-09-21T03:49:43.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.982+0000 7f9afc400640 1 -- 172.21.15.16:0/978765816 wait complete. 2024-09-21T03:49:43.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.983+0000 7f9afc400640 1 Processor -- start 2024-09-21T03:49:43.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.984+0000 7f9afc400640 1 -- start start 2024-09-21T03:49:43.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.984+0000 7f9afc400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9af4075dd0 0x7f9af41a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.984+0000 7f9afc400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9af4077380 0x7f9af41a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.984+0000 7f9afc400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9af4114440 0x7f9af41a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.984+0000 7f9afc400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9af411a0c0 con 0x7f9af4114440 2024-09-21T03:49:43.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.984+0000 7f9afc400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4119f40 con 0x7f9af4077380 2024-09-21T03:49:43.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.984+0000 7f9afc400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9af411a240 con 0x7f9af4075dd0 2024-09-21T03:49:43.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.984+0000 7f9afa400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9af4114440 0x7f9af41a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.984+0000 7f9af9a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9af4075dd0 0x7f9af41a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.984+0000 7f9afa400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9af4114440 0x7f9af41a6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39460/0 (socket says 172.21.15.16:39460) 2024-09-21T03:49:43.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.984+0000 7f9af9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9af4077380 0x7f9af41a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.985+0000 7f9afa400640 1 -- 172.21.15.16:0/945979577 learned_addr learned my addr 172.21.15.16:0/945979577 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.985+0000 7f9af9a00640 1 -- 172.21.15.16:0/945979577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9af4077380 msgr2=0x7f9af41a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.985+0000 7f9af9a00640 1 --2- 172.21.15.16:0/945979577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9af4077380 0x7f9af41a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.985+0000 7f9af9a00640 1 -- 172.21.15.16:0/945979577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9af4114440 msgr2=0x7f9af41a6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.985+0000 7f9af9a00640 1 --2- 172.21.15.16:0/945979577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9af4114440 0x7f9af41a6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.985+0000 7f9af9a00640 1 -- 172.21.15.16:0/945979577 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af41a69a0 con 0x7f9af4075dd0 2024-09-21T03:49:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.985+0000 7f9af9000640 1 --2- 172.21.15.16:0/945979577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9af4077380 0x7f9af41a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.985+0000 7f9afa400640 1 --2- 172.21.15.16:0/945979577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9af4114440 0x7f9af41a6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.985+0000 7f9af9a00640 1 --2- 172.21.15.16:0/945979577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9af4075dd0 0x7f9af41a0a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800e9f0 tx=0x7f9ae800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.986+0000 7f9af2000640 1 -- 172.21.15.16:0/945979577 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae800ccc0 con 0x7f9af4075dd0 2024-09-21T03:49:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.986+0000 7f9af2000640 1 -- 172.21.15.16:0/945979577 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ae80108e0 con 0x7f9af4075dd0 2024-09-21T03:49:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.986+0000 7f9afc400640 1 -- 172.21.15.16:0/945979577 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af41a6c90 con 0x7f9af4075dd0 2024-09-21T03:49:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.986+0000 7f9af2000640 1 -- 172.21.15.16:0/945979577 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8018880 con 0x7f9af4075dd0 2024-09-21T03:49:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.986+0000 7f9afc400640 1 -- 172.21.15.16:0/945979577 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af41ad3f0 con 0x7f9af4075dd0 2024-09-21T03:49:43.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.989+0000 7f9b0306e740 1 Processor -- start 2024-09-21T03:49:43.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.989+0000 7f9b0306e740 1 -- start start 2024-09-21T03:49:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.989+0000 7f9b0306e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f75520 0x557b0a31c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.989+0000 7f9b0306e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f4c660 0x557b0a31c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.990+0000 7f9b0306e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f54840 0x557b0a31cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.990+0000 7f9b0306e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557b0a293290 con 0x557b09f54840 2024-09-21T03:49:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.990+0000 7f9b0306e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557b0a2826c0 con 0x557b09f75520 2024-09-21T03:49:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.990+0000 7f9b0306e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557b0a2864d0 con 0x557b09f4c660 2024-09-21T03:49:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.990+0000 7f9af2000640 1 -- 172.21.15.16:0/945979577 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9ae8010430 con 0x7f9af4075dd0 2024-09-21T03:49:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.990+0000 7f9afa400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f75520 0x557b0a31c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.990+0000 7f9af9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f54840 0x557b0a31cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.990+0000 7f9afa400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f75520 0x557b0a31c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:47708/0 (socket says 172.21.15.16:47708) 2024-09-21T03:49:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.990+0000 7f9af9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f54840 0x557b0a31cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39474/0 (socket says 172.21.15.16:39474) 2024-09-21T03:49:43.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.990+0000 7f9afa400640 1 -- 172.21.15.16:0/1452300335 learned_addr learned my addr 172.21.15.16:0/1452300335 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:43.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.990+0000 7f9af9a00640 1 --2- 172.21.15.16:0/1452300335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f4c660 0x557b0a31c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.990+0000 7f9afa400640 1 -- 172.21.15.16:0/1452300335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f4c660 msgr2=0x557b0a31c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:43.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.990+0000 7f9afa400640 1 --2- 172.21.15.16:0/1452300335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f4c660 0x557b0a31c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.990+0000 7f9afa400640 1 -- 172.21.15.16:0/1452300335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f54840 msgr2=0x557b0a31cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:43.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.990+0000 7f9afa400640 1 --2- 172.21.15.16:0/1452300335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f54840 0x557b0a31cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.990+0000 7f9afa400640 1 -- 172.21.15.16:0/1452300335 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b0a20f460 con 0x557b09f75520 2024-09-21T03:49:43.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.991+0000 7f9af9a00640 1 --2- 172.21.15.16:0/1452300335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f4c660 0x557b0a31c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:43.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.991+0000 7f9afa400640 1 --2- 172.21.15.16:0/1452300335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f75520 0x557b0a31c1c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009880 tx=0x7f9aec0098b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e060fd9a7dd2d4ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:43.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.991+0000 7f9ae6a00640 1 -- 172.21.15.16:0/1452300335 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec031650 con 0x557b09f75520 2024-09-21T03:49:43.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.991+0000 7f9ae6a00640 1 -- 172.21.15.16:0/1452300335 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aec036040 con 0x557b09f75520 2024-09-21T03:49:43.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.991+0000 7f9ae6a00640 1 -- 172.21.15.16:0/1452300335 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec004430 con 0x557b09f75520 2024-09-21T03:49:43.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.992+0000 7f9af2000640 1 --2- 172.21.15.16:0/945979577 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9adc0b6590 0x7f9adc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.992+0000 7f9af2000640 1 -- 172.21.15.16:0/945979577 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f9ae80e7a00 con 0x7f9af4075dd0 2024-09-21T03:49:43.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.992+0000 7f9af9a00640 1 --2- 172.21.15.16:0/945979577 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9adc0b6590 0x7f9adc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:43.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.993+0000 7f9b0306e740 1 -- 172.21.15.16:0/1452300335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f75520 msgr2=0x557b0a31c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:43.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.993+0000 7f9b0306e740 1 --2- 172.21.15.16:0/1452300335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f75520 0x557b0a31c1c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009880 tx=0x7f9aec0098b0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.993+0000 7f9b0306e740 1 -- 172.21.15.16:0/1452300335 shutdown_connections 2024-09-21T03:49:43.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.993+0000 7f9b0306e740 1 --2- 172.21.15.16:0/1452300335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f54840 0x557b0a31cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.993+0000 7f9b0306e740 1 --2- 172.21.15.16:0/1452300335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f4c660 0x557b0a31c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.993+0000 7f9b0306e740 1 --2- 172.21.15.16:0/1452300335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f75520 0x557b0a31c1c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.993+0000 7f9b0306e740 1 -- 172.21.15.16:0/1452300335 >> 172.21.15.16:0/1452300335 conn(0x557b09f69910 msgr2=0x557b0a31a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.993+0000 7f9af9a00640 1 --2- 172.21.15.16:0/945979577 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9adc0b6590 0x7f9adc0b8aa0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80ad7c0 tx=0x7f9ae800ce60 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.993+0000 7f9b0306e740 1 -- 172.21.15.16:0/1452300335 shutdown_connections 2024-09-21T03:49:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.993+0000 7f9b0306e740 1 -- 172.21.15.16:0/1452300335 wait complete. 2024-09-21T03:49:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.994+0000 7f9b0306e740 1 -- start start 2024-09-21T03:49:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.994+0000 7f9b0306e740 1 Processor -- start 2024-09-21T03:49:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.995+0000 7f9b0306e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f4c660 0x557b0a327cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.995+0000 7f9b0306e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f54840 0x557b0a328230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.995+0000 7f9b0306e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f75520 0x557b0a333ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.995+0000 7f9b0306e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557b0a0e3bb0 con 0x557b09f4c660 2024-09-21T03:49:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.995+0000 7f9b0306e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557b0a21d6c0 con 0x557b09f75520 2024-09-21T03:49:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.995+0000 7f9b0306e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557b0a32a7c0 con 0x557b09f54840 2024-09-21T03:49:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.995+0000 7f9b0306e740 10 client.0 _ll_register_callbacks cb 0x557b0a238490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:43.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.995+0000 7f9afa400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f4c660 0x557b0a327cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:44.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.996+0000 7f9afa400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f4c660 0x557b0a327cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39482/0 (socket says 172.21.15.16:39482) 2024-09-21T03:49:44.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.996+0000 7f9afa400640 1 -- 172.21.15.16:0/2759116473 learned_addr learned my addr 172.21.15.16:0/2759116473 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:44.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.996+0000 7f9af9000640 1 --2- 172.21.15.16:0/2759116473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f54840 0x557b0a328230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:44.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.996+0000 7f9afa400640 1 --2- 172.21.15.16:0/2759116473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f75520 0x557b0a333ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:44.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.996+0000 7f9af9000640 1 -- 172.21.15.16:0/2759116473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f75520 msgr2=0x557b0a333ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:44.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.996+0000 7f9af9000640 1 --2- 172.21.15.16:0/2759116473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f75520 0x557b0a333ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.996+0000 7f9af9000640 1 -- 172.21.15.16:0/2759116473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f4c660 msgr2=0x557b0a327cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:44.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.996+0000 7f9af9000640 1 --2- 172.21.15.16:0/2759116473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f4c660 0x557b0a327cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.996+0000 7f9af9000640 1 -- 172.21.15.16:0/2759116473 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b09fe0b00 con 0x557b09f54840 2024-09-21T03:49:44.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.996+0000 7f9afa400640 1 --2- 172.21.15.16:0/2759116473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f75520 0x557b0a333ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:49:44.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.997+0000 7f9af9000640 1 --2- 172.21.15.16:0/2759116473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f54840 0x557b0a328230 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9af406bd40 tx=0x7f9ae000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1dc65e911de6392 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:44.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.997+0000 7f9adb400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:49:44.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.997+0000 7f9adb400640 1 -- 172.21.15.16:0/2759116473 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0012110 con 0x557b09f54840 2024-09-21T03:49:44.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.997+0000 7f9b0306e740 1 -- 172.21.15.16:0/2759116473 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aec036040 con 0x557b09f54840 2024-09-21T03:49:44.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.997+0000 7f9adb400640 1 -- 172.21.15.16:0/2759116473 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f9ae0004050 con 0x557b09f54840 2024-09-21T03:49:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.997+0000 7f9adb400640 1 -- 172.21.15.16:0/2759116473 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9ae0025050 con 0x557b09f54840 2024-09-21T03:49:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.997+0000 7f9b0306e740 10 client.25924 fetch_fsmap learned FSMap version 30 2024-09-21T03:49:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.997+0000 7f9b0306e740 1 -- 172.21.15.16:0/2759116473 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557b0a057960 con 0x557b09f54840 2024-09-21T03:49:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.998+0000 7f9adb400640 1 -- 172.21.15.16:0/2759116473 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aec036040 con 0x557b09f54840 2024-09-21T03:49:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.998+0000 7f9adb400640 1 -- 172.21.15.16:0/2759116473 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9ae0010430 con 0x557b09f54840 2024-09-21T03:49:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.998+0000 7f9b0306e740 10 client.25924 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:49:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.998+0000 7f9b0306e740 10 client.25924 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:49:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.998+0000 7f9b0306e740 20 client.25924 populate_metadata read hostname 'smithi016' 2024-09-21T03:49:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.998+0000 7f9b0306e740 1 --2- 172.21.15.16:0/2759116473 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557b09f33ea0 0x557b0a339b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.998+0000 7f9b0306e740 4 client.25924 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.998+0000 7f9b0306e740 1 -- 172.21.15.16:0/2759116473 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b0a2951e0 con 0x557b09f33ea0 2024-09-21T03:49:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.999+0000 7f9afa400640 1 --2- 172.21.15.16:0/2759116473 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557b09f33ea0 0x557b0a339b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.999+0000 7f9afa400640 1 --2- 172.21.15.16:0/2759116473 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557b09f33ea0 0x557b0a339b80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f89e331f51e7b6c3 server_cookie=5bea28d6e4973807 in_seq=0 out_seq=0 2024-09-21T03:49:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:43.999+0000 7f9adb400640 10 client.25924 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:44.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.000+0000 7f9adb400640 1 -- 172.21.15.16:0/2759116473 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557b0a2951e0 con 0x557b09f33ea0 2024-09-21T03:49:44.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.000+0000 7f9adb400640 10 client.25924 handle_command_reply: tid=0 2024-09-21T03:49:44.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.000+0000 7f9b0306e740 1 client.25924 shutdown 2024-09-21T03:49:44.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.000+0000 7f9b0306e740 1 -- 172.21.15.16:0/2759116473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f54840 msgr2=0x557b0a328230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:44.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.000+0000 7f9b0306e740 1 --2- 172.21.15.16:0/2759116473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f54840 0x557b0a328230 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9af406bd40 tx=0x7f9ae000f790 comp rx=0 tx=0).stop 2024-09-21T03:49:44.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.000+0000 7f9b0306e740 1 -- 172.21.15.16:0/2759116473 shutdown_connections 2024-09-21T03:49:44.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.000+0000 7f9b0306e740 1 --2- 172.21.15.16:0/2759116473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f75520 0x557b0a333ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.000+0000 7f9b0306e740 1 --2- 172.21.15.16:0/2759116473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f54840 0x557b0a328230 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.000+0000 7f9b0306e740 1 --2- 172.21.15.16:0/2759116473 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557b09f33ea0 0x557b0a339b80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.000+0000 7f9b0306e740 1 --2- 172.21.15.16:0/2759116473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f4c660 0x557b0a327cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.000+0000 7f9b0306e740 1 -- 172.21.15.16:0/2759116473 >> 172.21.15.16:0/2759116473 conn(0x557b09f69910 msgr2=0x557b0a31f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:44.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.000+0000 7f9adb400640 0 client.25924 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:44.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.001+0000 7f9b0306e740 1 -- 172.21.15.16:0/2759116473 shutdown_connections 2024-09-21T03:49:44.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.001+0000 7f9b0306e740 1 -- 172.21.15.16:0/2759116473 wait complete. 2024-09-21T03:49:44.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.001+0000 7f9b0306e740 20 client.25924 trim_cache size 0 max 16384 2024-09-21T03:49:44.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.019+0000 7f9b0306e740 1 Processor -- start 2024-09-21T03:49:44.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.019+0000 7f9b0306e740 1 -- start start 2024-09-21T03:49:44.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.020+0000 7f9b0306e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f4c660 0x557b0a319010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:44.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.020+0000 7f9b0306e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f54840 0x557b0a329720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:44.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.020+0000 7f9b0306e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f75520 0x557b0a324ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:44.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.020+0000 7f9b0306e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41b7130 con 0x557b09f4c660 2024-09-21T03:49:44.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.020+0000 7f9b0306e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41b8c70 con 0x557b09f54840 2024-09-21T03:49:44.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.020+0000 7f9b0306e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41b7800 con 0x557b09f75520 2024-09-21T03:49:44.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.020+0000 7f9afa400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f4c660 0x557b0a319010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:44.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.020+0000 7f9afa400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f4c660 0x557b0a319010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39494/0 (socket says 172.21.15.16:39494) 2024-09-21T03:49:44.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.020+0000 7f9af9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f54840 0x557b0a329720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:44.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.020+0000 7f9afa400640 1 -- 172.21.15.16:0/4292251757 learned_addr learned my addr 172.21.15.16:0/4292251757 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:44.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.020+0000 7f9afa400640 1 --2- 172.21.15.16:0/4292251757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f75520 0x557b0a324ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:44.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.021+0000 7f9af9000640 1 -- 172.21.15.16:0/4292251757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f75520 msgr2=0x557b0a324ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:44.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.021+0000 7f9af9000640 1 --2- 172.21.15.16:0/4292251757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f75520 0x557b0a324ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.021+0000 7f9af9000640 1 -- 172.21.15.16:0/4292251757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f4c660 msgr2=0x557b0a319010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:44.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.021+0000 7f9af9000640 1 --2- 172.21.15.16:0/4292251757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f4c660 0x557b0a319010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.021+0000 7f9af9000640 1 -- 172.21.15.16:0/4292251757 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b0a31ae70 con 0x557b09f54840 2024-09-21T03:49:44.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.021+0000 7f9afa400640 1 --2- 172.21.15.16:0/4292251757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f4c660 0x557b0a319010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:44.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.021+0000 7f9af9000640 1 --2- 172.21.15.16:0/4292251757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f54840 0x557b0a329720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0010850 tx=0x7f9ae0012320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15fa547073925237 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:44.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.021+0000 7f9ae6000640 1 -- 172.21.15.16:0/4292251757 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae002e070 con 0x557b09f54840 2024-09-21T03:49:44.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.021+0000 7f9ae6000640 1 -- 172.21.15.16:0/4292251757 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ae0010430 con 0x557b09f54840 2024-09-21T03:49:44.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.021+0000 7f9ae6000640 1 -- 172.21.15.16:0/4292251757 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0010ac0 con 0x557b09f54840 2024-09-21T03:49:44.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.022+0000 7f9b0306e740 1 -- 172.21.15.16:0/4292251757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f54840 msgr2=0x557b0a329720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:44.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.022+0000 7f9b0306e740 1 --2- 172.21.15.16:0/4292251757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f54840 0x557b0a329720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0010850 tx=0x7f9ae0012320 comp rx=0 tx=0).stop 2024-09-21T03:49:44.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.022+0000 7f9b0306e740 1 -- 172.21.15.16:0/4292251757 shutdown_connections 2024-09-21T03:49:44.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.022+0000 7f9b0306e740 1 --2- 172.21.15.16:0/4292251757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f75520 0x557b0a324ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.022+0000 7f9b0306e740 1 --2- 172.21.15.16:0/4292251757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f54840 0x557b0a329720 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.022+0000 7f9b0306e740 1 --2- 172.21.15.16:0/4292251757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f4c660 0x557b0a319010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.022+0000 7f9b0306e740 1 -- 172.21.15.16:0/4292251757 >> 172.21.15.16:0/4292251757 conn(0x557b09f69910 msgr2=0x557b0a329010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:44.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.022+0000 7f9b0306e740 1 -- 172.21.15.16:0/4292251757 shutdown_connections 2024-09-21T03:49:44.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.022+0000 7f9b0306e740 1 -- 172.21.15.16:0/4292251757 wait complete. 2024-09-21T03:49:44.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.023+0000 7f9b0306e740 1 -- start start 2024-09-21T03:49:44.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.023+0000 7f9b0306e740 1 Processor -- start 2024-09-21T03:49:44.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.024+0000 7f9b0306e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f4c660 0x557b0a32ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:44.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.024+0000 7f9b0306e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f54840 0x557b0a32b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:44.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.024+0000 7f9b0306e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f75520 0x557b0a3a4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:44.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.024+0000 7f9b0306e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41b85a0 con 0x557b09f75520 2024-09-21T03:49:44.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.024+0000 7f9b0306e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41b7ed0 con 0x557b09f4c660 2024-09-21T03:49:44.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.024+0000 7f9b0306e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41b5110 con 0x557b09f54840 2024-09-21T03:49:44.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.024+0000 7f9b0306e740 10 client.0 _ll_register_callbacks cb 0x7f9ae0009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:44.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.024+0000 7f9afa400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f75520 0x557b0a3a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:44.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.024+0000 7f9af9000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f54840 0x557b0a32b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:44.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.024+0000 7f9afa400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f75520 0x557b0a3a4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39498/0 (socket says 172.21.15.16:39498) 2024-09-21T03:49:44.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.024+0000 7f9afa400640 1 -- 172.21.15.16:0/3137930219 learned_addr learned my addr 172.21.15.16:0/3137930219 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:44.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.024+0000 7f9afa400640 1 --2- 172.21.15.16:0/3137930219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f4c660 0x557b0a32ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:44.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.024+0000 7f9af9000640 1 -- 172.21.15.16:0/3137930219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f4c660 msgr2=0x557b0a32ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:44.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.024+0000 7f9af9000640 1 --2- 172.21.15.16:0/3137930219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f4c660 0x557b0a32ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.024+0000 7f9af9000640 1 -- 172.21.15.16:0/3137930219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f75520 msgr2=0x557b0a3a4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:44.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.024+0000 7f9af9000640 1 --2- 172.21.15.16:0/3137930219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f75520 0x557b0a3a4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.024+0000 7f9af9000640 1 -- 172.21.15.16:0/3137930219 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b0a32bb30 con 0x557b09f54840 2024-09-21T03:49:44.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.025+0000 7f9af9000640 1 --2- 172.21.15.16:0/3137930219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f54840 0x557b0a32b340 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00059f0 tx=0x7f9ae00106e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72f0220a1ad6b308 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:44.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.025+0000 7f9adb400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:49:44.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.025+0000 7f9adb400640 1 -- 172.21.15.16:0/3137930219 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0037060 con 0x557b09f54840 2024-09-21T03:49:44.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.025+0000 7f9adb400640 1 -- 172.21.15.16:0/3137930219 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f9ae0022040 con 0x557b09f54840 2024-09-21T03:49:44.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.025+0000 7f9b0306e740 1 -- 172.21.15.16:0/3137930219 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557b0a3236a0 con 0x557b09f54840 2024-09-21T03:49:44.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.025+0000 7f9adb400640 1 -- 172.21.15.16:0/3137930219 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9ae003e050 con 0x557b09f54840 2024-09-21T03:49:44.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.025+0000 7f9b0306e740 10 client.25930 fetch_fsmap learned FSMap version 30 2024-09-21T03:49:44.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.025+0000 7f9b0306e740 1 -- 172.21.15.16:0/3137930219 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557b0a3a5a70 con 0x557b09f54840 2024-09-21T03:49:44.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.026+0000 7f9adb400640 1 -- 172.21.15.16:0/3137930219 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557b0a3236a0 con 0x557b09f54840 2024-09-21T03:49:44.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.026+0000 7f9adb400640 1 -- 172.21.15.16:0/3137930219 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9ae00035f0 con 0x557b09f54840 2024-09-21T03:49:44.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.026+0000 7f9b0306e740 10 client.25930 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:49:44.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.026+0000 7f9b0306e740 10 client.25930 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:49:44.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.026+0000 7f9b0306e740 20 client.25930 populate_metadata read hostname 'smithi016' 2024-09-21T03:49:44.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.026+0000 7f9b0306e740 1 --2- 172.21.15.16:0/3137930219 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557b09f09250 0x557b0a3a8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:44.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.026+0000 7f9b0306e740 4 client.25930 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ecfa781f", "release": true, "await": true}] 2024-09-21T03:49:44.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.026+0000 7f9b0306e740 1 -- 172.21.15.16:0/3137930219 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ecfa781f", "release": true, "await": true}) -- 0x557b0a3227b0 con 0x557b09f09250 2024-09-21T03:49:44.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.027+0000 7f9afa400640 1 --2- 172.21.15.16:0/3137930219 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557b09f09250 0x557b0a3a8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:44.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.027+0000 7f9afa400640 1 --2- 172.21.15.16:0/3137930219 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557b09f09250 0x557b0a3a8130 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19e2fd3ea3350f2 server_cookie=5580ef7631d03a0f in_seq=0 out_seq=0 2024-09-21T03:49:44.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.027+0000 7f9adb400640 10 client.25930 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:44.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.033+0000 7f9adb400640 1 -- 172.21.15.16:0/3137930219 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557b0a3227b0 con 0x557b09f09250 2024-09-21T03:49:44.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.033+0000 7f9adb400640 10 client.25930 handle_command_reply: tid=0 2024-09-21T03:49:44.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.033+0000 7f9b0306e740 1 client.25930 shutdown 2024-09-21T03:49:44.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.033+0000 7f9b0306e740 1 -- 172.21.15.16:0/3137930219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f54840 msgr2=0x557b0a32b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:44.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.033+0000 7f9b0306e740 1 --2- 172.21.15.16:0/3137930219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f54840 0x557b0a32b340 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00059f0 tx=0x7f9ae00106e0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.033+0000 7f9b0306e740 1 -- 172.21.15.16:0/3137930219 shutdown_connections 2024-09-21T03:49:44.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.033+0000 7f9b0306e740 1 --2- 172.21.15.16:0/3137930219 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557b09f09250 0x557b0a3a8130 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.033+0000 7f9b0306e740 1 --2- 172.21.15.16:0/3137930219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b09f75520 0x557b0a3a4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.033+0000 7f9b0306e740 1 --2- 172.21.15.16:0/3137930219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b09f54840 0x557b0a32b340 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.033+0000 7f9b0306e740 1 --2- 172.21.15.16:0/3137930219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b09f4c660 0x557b0a32ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.033+0000 7f9b0306e740 1 -- 172.21.15.16:0/3137930219 >> 172.21.15.16:0/3137930219 conn(0x557b09f69910 msgr2=0x557b0a318de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:44.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.033+0000 7f9adb400640 0 client.25930 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:44.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.034+0000 7f9b0306e740 1 -- 172.21.15.16:0/3137930219 shutdown_connections 2024-09-21T03:49:44.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.034+0000 7f9b0306e740 1 -- 172.21.15.16:0/3137930219 wait complete. 2024-09-21T03:49:44.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.034+0000 7f9b0306e740 20 client.25930 trim_cache size 0 max 16384 2024-09-21T03:49:44.038 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:49:44.038 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:49:44.038 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:49:44.038 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 16, 2024-09-21T03:49:44.038 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:49:44.038 INFO:teuthology.orchestra.run.smithi016.stdout: "ecfa781f": { 2024-09-21T03:49:44.038 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 16, 2024-09-21T03:49:44.038 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:49:44.038 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:49:44.038 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:49:44.039 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:49:44.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:49:44.039 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:49:44.039 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 147.8, 2024-09-21T03:49:44.039 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:49:44.039 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:49:44.039 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:49:44.039 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:49:44.039 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:49:44.039 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:49:44.040 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:49:44.040 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:49:44.040 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:49:44.040 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:49:44.040 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:49:44.040 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:49:44.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.034+0000 7f9afc400640 1 -- 172.21.15.16:0/945979577 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9adc0b6590 msgr2=0x7f9adc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.034+0000 7f9afc400640 1 --2- 172.21.15.16:0/945979577 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9adc0b6590 0x7f9adc0b8aa0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80ad7c0 tx=0x7f9ae800ce60 comp rx=0 tx=0).stop 2024-09-21T03:49:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.035+0000 7f9afc400640 1 -- 172.21.15.16:0/945979577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9af4075dd0 msgr2=0x7f9af41a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.035+0000 7f9afc400640 1 --2- 172.21.15.16:0/945979577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9af4075dd0 0x7f9af41a0a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800e9f0 tx=0x7f9ae800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.035+0000 7f9afc400640 1 -- 172.21.15.16:0/945979577 shutdown_connections 2024-09-21T03:49:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.035+0000 7f9afc400640 1 --2- 172.21.15.16:0/945979577 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9adc0b6590 0x7f9adc0b8aa0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.035+0000 7f9afc400640 1 --2- 172.21.15.16:0/945979577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9af4114440 0x7f9af41a6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.035+0000 7f9afc400640 1 --2- 172.21.15.16:0/945979577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9af4077380 0x7f9af41a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.035+0000 7f9afc400640 1 --2- 172.21.15.16:0/945979577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9af4075dd0 0x7f9af41a0a30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.035+0000 7f9afc400640 1 -- 172.21.15.16:0/945979577 >> 172.21.15.16:0/945979577 conn(0x7f9af40ff5e0 msgr2=0x7f9af4110340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.035+0000 7f9afc400640 1 -- 172.21.15.16:0/945979577 shutdown_connections 2024-09-21T03:49:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:49:44.035+0000 7f9afc400640 1 -- 172.21.15.16:0/945979577 wait complete. 2024-09-21T03:49:45.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:44 smithi016 ceph-mon[30375]: pgmap v521: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 221 MiB/s wr, 1.81k op/s 2024-09-21T03:49:45.217 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ecfa781f', total seconds elapsed: 33.2 2024-09-21T03:49:45.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:44 smithi134 ceph-mon[35031]: pgmap v521: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 221 MiB/s wr, 1.81k op/s 2024-09-21T03:49:45.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:44 smithi123 ceph-mon[34673]: pgmap v521: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 221 MiB/s wr, 1.81k op/s 2024-09-21T03:49:47.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:46 smithi134 ceph-mon[35031]: pgmap v522: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 192 MiB/s wr, 1.67k op/s 2024-09-21T03:49:47.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:46 smithi123 ceph-mon[34673]: pgmap v522: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 192 MiB/s wr, 1.67k op/s 2024-09-21T03:49:47.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:46 smithi016 ceph-mon[30375]: pgmap v522: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 192 MiB/s wr, 1.67k op/s 2024-09-21T03:49:49.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:48 smithi134 ceph-mon[35031]: pgmap v523: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 154 MiB/s wr, 1.43k op/s 2024-09-21T03:49:49.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:48 smithi123 ceph-mon[34673]: pgmap v523: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 154 MiB/s wr, 1.43k op/s 2024-09-21T03:49:49.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:48 smithi016 ceph-mon[30375]: pgmap v523: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 154 MiB/s wr, 1.43k op/s 2024-09-21T03:49:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.229+0000 7f6bd6600640 1 -- 172.21.15.16:0/2418407382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd0104560 msgr2=0x7f6bd0104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.229+0000 7f6bd6600640 1 --2- 172.21.15.16:0/2418407382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd0104560 0x7f6bd0104970 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009900 tx=0x7f6bbc02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.229+0000 7f6bd6600640 1 -- 172.21.15.16:0/2418407382 shutdown_connections 2024-09-21T03:49:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.229+0000 7f6bd6600640 1 --2- 172.21.15.16:0/2418407382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd0104f00 0x7f6bd010b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.229+0000 7f6bd6600640 1 --2- 172.21.15.16:0/2418407382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd0104560 0x7f6bd0104970 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.229+0000 7f6bd6600640 1 --2- 172.21.15.16:0/2418407382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd01034e0 0x7f6bd0103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.229+0000 7f6bd6600640 1 -- 172.21.15.16:0/2418407382 >> 172.21.15.16:0/2418407382 conn(0x7f6bd00fee20 msgr2=0x7f6bd01012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.230+0000 7f6bd6600640 1 -- 172.21.15.16:0/2418407382 shutdown_connections 2024-09-21T03:49:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.230+0000 7f6bd6600640 1 -- 172.21.15.16:0/2418407382 wait complete. 2024-09-21T03:49:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.231+0000 7f6bd6600640 1 Processor -- start 2024-09-21T03:49:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.232+0000 7f6bd6600640 1 -- start start 2024-09-21T03:49:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.233+0000 7f6bd6600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd01034e0 0x7f6bd0199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.233+0000 7f6bd6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd0104560 0x7f6bd019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.233+0000 7f6bd6600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd0104f00 0x7f6bd019af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.233+0000 7f6bd6600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd010f180 con 0x7f6bd0104560 2024-09-21T03:49:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.233+0000 7f6bd6600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd010f000 con 0x7f6bd0104f00 2024-09-21T03:49:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.233+0000 7f6bd6600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd010f300 con 0x7f6bd01034e0 2024-09-21T03:49:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.233+0000 7f6bd5200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd01034e0 0x7f6bd0199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.233+0000 7f6bd5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd0104f00 0x7f6bd019af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.233+0000 7f6bd5200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd01034e0 0x7f6bd0199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52056/0 (socket says 172.21.15.16:52056) 2024-09-21T03:49:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.233+0000 7f6bd5200640 1 -- 172.21.15.16:0/3261820600 learned_addr learned my addr 172.21.15.16:0/3261820600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.233+0000 7f6bcfe00640 1 --2- 172.21.15.16:0/3261820600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd0104560 0x7f6bd019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.234+0000 7f6bd5c00640 1 -- 172.21.15.16:0/3261820600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd01034e0 msgr2=0x7f6bd0199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.234+0000 7f6bd5c00640 1 --2- 172.21.15.16:0/3261820600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd01034e0 0x7f6bd0199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.234+0000 7f6bd5c00640 1 -- 172.21.15.16:0/3261820600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd0104560 msgr2=0x7f6bd019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.234+0000 7f6bd5c00640 1 --2- 172.21.15.16:0/3261820600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd0104560 0x7f6bd019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.234+0000 7f6bd5c00640 1 -- 172.21.15.16:0/3261820600 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd019b780 con 0x7f6bd0104f00 2024-09-21T03:49:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.234+0000 7f6bd5200640 1 --2- 172.21.15.16:0/3261820600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd01034e0 0x7f6bd0199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:49:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.234+0000 7f6bd5c00640 1 --2- 172.21.15.16:0/3261820600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd0104f00 0x7f6bd019af60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800e970 tx=0x7f6bc800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.234+0000 7f6bcd600640 1 -- 172.21.15.16:0/3261820600 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc800ccc0 con 0x7f6bd0104f00 2024-09-21T03:49:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.234+0000 7f6bcd600640 1 -- 172.21.15.16:0/3261820600 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bc8005020 con 0x7f6bd0104f00 2024-09-21T03:49:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.234+0000 7f6bcd600640 1 -- 172.21.15.16:0/3261820600 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8004a00 con 0x7f6bd0104f00 2024-09-21T03:49:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.235+0000 7f6bd6600640 1 -- 172.21.15.16:0/3261820600 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd019ba70 con 0x7f6bd0104f00 2024-09-21T03:49:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.235+0000 7f6bd6600640 1 -- 172.21.15.16:0/3261820600 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd01a2160 con 0x7f6bd0104f00 2024-09-21T03:49:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.240+0000 7f6bdee31740 1 Processor -- start 2024-09-21T03:49:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.240+0000 7f6bcd600640 1 -- 172.21.15.16:0/3261820600 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6bc8004ba0 con 0x7f6bd0104f00 2024-09-21T03:49:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.241+0000 7f6bdee31740 1 -- start start 2024-09-21T03:49:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.241+0000 7f6bdee31740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caee31b0 0x5643cb28a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.241+0000 7f6bdee31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caeba2f0 0x5643cb28a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.241+0000 7f6bdee31740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caec24d0 0x5643cb28b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.241+0000 7f6bdee31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5643cb1fd6b0 con 0x5643caeba2f0 2024-09-21T03:49:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.241+0000 7f6bdee31740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5643cb1ecae0 con 0x5643caee31b0 2024-09-21T03:49:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.241+0000 7f6bdee31740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5643cb1f08f0 con 0x5643caec24d0 2024-09-21T03:49:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.242+0000 7f6bcfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caee31b0 0x5643cb28a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.242+0000 7f6bcfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caee31b0 0x5643cb28a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:47758/0 (socket says 172.21.15.16:47758) 2024-09-21T03:49:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.242+0000 7f6bcfe00640 1 -- 172.21.15.16:0/52040834 learned_addr learned my addr 172.21.15.16:0/52040834 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.242+0000 7f6bd5200640 1 --2- 172.21.15.16:0/52040834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caeba2f0 0x5643cb28a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.242+0000 7f6bcfe00640 1 -- 172.21.15.16:0/52040834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caec24d0 msgr2=0x5643cb28b470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:49:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.242+0000 7f6bcfe00640 1 --2- 172.21.15.16:0/52040834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caec24d0 0x5643cb28b470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.242+0000 7f6bcfe00640 1 -- 172.21.15.16:0/52040834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caeba2f0 msgr2=0x5643cb28a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.242+0000 7f6bcfe00640 1 --2- 172.21.15.16:0/52040834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caeba2f0 0x5643cb28a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.242+0000 7f6bcfe00640 1 -- 172.21.15.16:0/52040834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643cb1b9780 con 0x5643caee31b0 2024-09-21T03:49:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.242+0000 7f6bd5200640 1 --2- 172.21.15.16:0/52040834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caeba2f0 0x5643cb28a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.243+0000 7f6bcfe00640 1 --2- 172.21.15.16:0/52040834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caee31b0 0x5643cb28a2b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009230 tx=0x7f6bbc005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9956c74cbb146c17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.243+0000 7f6bcd600640 1 --2- 172.21.15.16:0/3261820600 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6bb80b65d0 0x7f6bb80b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.243+0000 7f6bc2000640 1 -- 172.21.15.16:0/52040834 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc040070 con 0x5643caee31b0 2024-09-21T03:49:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.243+0000 7f6bcd600640 1 -- 172.21.15.16:0/3261820600 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f6bc80e7550 con 0x7f6bd0104f00 2024-09-21T03:49:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.243+0000 7f6bc2000640 1 -- 172.21.15.16:0/52040834 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bbc003aa0 con 0x5643caee31b0 2024-09-21T03:49:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.243+0000 7f6bc2000640 1 -- 172.21.15.16:0/52040834 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc035040 con 0x5643caee31b0 2024-09-21T03:49:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.243+0000 7f6bd5200640 1 --2- 172.21.15.16:0/3261820600 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6bb80b65d0 0x7f6bb80b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.245+0000 7f6bd5200640 1 --2- 172.21.15.16:0/3261820600 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6bb80b65d0 0x7f6bb80b8ae0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4011a90 tx=0x7f6bc4009500 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.245+0000 7f6bdee31740 1 -- 172.21.15.16:0/52040834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caee31b0 msgr2=0x5643cb28a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.245+0000 7f6bdee31740 1 --2- 172.21.15.16:0/52040834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caee31b0 0x5643cb28a2b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009230 tx=0x7f6bbc005fd0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.246+0000 7f6bdee31740 1 -- 172.21.15.16:0/52040834 shutdown_connections 2024-09-21T03:49:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.246+0000 7f6bdee31740 1 --2- 172.21.15.16:0/52040834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caec24d0 0x5643cb28b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.246+0000 7f6bdee31740 1 --2- 172.21.15.16:0/52040834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caeba2f0 0x5643cb28a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.246+0000 7f6bdee31740 1 --2- 172.21.15.16:0/52040834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caee31b0 0x5643cb28a2b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.246+0000 7f6bdee31740 1 -- 172.21.15.16:0/52040834 >> 172.21.15.16:0/52040834 conn(0x5643caed7300 msgr2=0x5643cb285d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.246+0000 7f6bdee31740 1 -- 172.21.15.16:0/52040834 shutdown_connections 2024-09-21T03:49:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.246+0000 7f6bdee31740 1 -- 172.21.15.16:0/52040834 wait complete. 2024-09-21T03:49:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.247+0000 7f6bdee31740 1 -- start start 2024-09-21T03:49:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.248+0000 7f6bdee31740 1 Processor -- start 2024-09-21T03:49:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.249+0000 7f6bdee31740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caeba2f0 0x5643cb2a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.249+0000 7f6bdee31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caec24d0 0x5643cb2a9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.249+0000 7f6bdee31740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caee31b0 0x5643cb2a9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.249+0000 7f6bdee31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5643cb124ab0 con 0x5643caec24d0 2024-09-21T03:49:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.249+0000 7f6bdee31740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5643cb193570 con 0x5643caeba2f0 2024-09-21T03:49:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.249+0000 7f6bdee31740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5643cb28bf30 con 0x5643caee31b0 2024-09-21T03:49:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.249+0000 7f6bdee31740 10 client.0 _ll_register_callbacks cb 0x5643cb1a2100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.249+0000 7f6bd5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caeba2f0 0x5643cb2a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.249+0000 7f6bd5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caeba2f0 0x5643cb2a8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:47768/0 (socket says 172.21.15.16:47768) 2024-09-21T03:49:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.249+0000 7f6bd5200640 1 -- 172.21.15.16:0/733194606 learned_addr learned my addr 172.21.15.16:0/733194606 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.249+0000 7f6bd5c00640 1 --2- 172.21.15.16:0/733194606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caee31b0 0x5643cb2a9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.249+0000 7f6bcfe00640 1 --2- 172.21.15.16:0/733194606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caec24d0 0x5643cb2a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.250+0000 7f6bd5200640 1 -- 172.21.15.16:0/733194606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caee31b0 msgr2=0x5643cb2a9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.250+0000 7f6bd5200640 1 --2- 172.21.15.16:0/733194606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caee31b0 0x5643cb2a9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.250+0000 7f6bd5200640 1 -- 172.21.15.16:0/733194606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caec24d0 msgr2=0x5643cb2a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.250+0000 7f6bd5200640 1 --2- 172.21.15.16:0/733194606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caec24d0 0x5643cb2a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.250+0000 7f6bd5200640 1 -- 172.21.15.16:0/733194606 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643caf4e790 con 0x5643caeba2f0 2024-09-21T03:49:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.250+0000 7f6bcfe00640 1 --2- 172.21.15.16:0/733194606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caec24d0 0x5643cb2a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.250+0000 7f6bd5200640 1 --2- 172.21.15.16:0/733194606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caeba2f0 0x5643cb2a8d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4011870 tx=0x7f6bc4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53cccf0f0729e3e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.250+0000 7f6bb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:49:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.251+0000 7f6bb6a00640 1 -- 172.21.15.16:0/733194606 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc402c060 con 0x5643caeba2f0 2024-09-21T03:49:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.251+0000 7f6bb6a00640 1 -- 172.21.15.16:0/733194606 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f6bc4017030 con 0x5643caeba2f0 2024-09-21T03:49:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.251+0000 7f6bdee31740 1 -- 172.21.15.16:0/733194606 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bbc003aa0 con 0x5643caeba2f0 2024-09-21T03:49:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.251+0000 7f6bb6a00640 1 -- 172.21.15.16:0/733194606 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6bc4027410 con 0x5643caeba2f0 2024-09-21T03:49:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.251+0000 7f6bdee31740 10 client.25985 fetch_fsmap learned FSMap version 30 2024-09-21T03:49:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.251+0000 7f6bdee31740 1 -- 172.21.15.16:0/733194606 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5643cafc7620 con 0x5643caeba2f0 2024-09-21T03:49:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.252+0000 7f6bb6a00640 1 -- 172.21.15.16:0/733194606 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bbc003aa0 con 0x5643caeba2f0 2024-09-21T03:49:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.253+0000 7f6bb6a00640 1 -- 172.21.15.16:0/733194606 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6bc401c5b0 con 0x5643caeba2f0 2024-09-21T03:49:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.256+0000 7f6bdee31740 10 client.25985 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:49:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.256+0000 7f6bdee31740 10 client.25985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.256+0000 7f6bdee31740 20 client.25985 populate_metadata read hostname 'smithi016' 2024-09-21T03:49:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.256+0000 7f6bdee31740 1 --2- 172.21.15.16:0/733194606 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5643caea1d80 0x5643cb2b6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.256+0000 7f6bdee31740 4 client.25985 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:49:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.256+0000 7f6bdee31740 1 -- 172.21.15.16:0/733194606 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643cb1ff600 con 0x5643caea1d80 2024-09-21T03:49:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.256+0000 7f6bcfe00640 1 --2- 172.21.15.16:0/733194606 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5643caea1d80 0x5643cb2b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.257+0000 7f6bb6a00640 10 client.25985 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.257+0000 7f6bcfe00640 1 --2- 172.21.15.16:0/733194606 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5643caea1d80 0x5643cb2b6590 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0a5eb4c17613e11 server_cookie=5c46f3d77b4b280b in_seq=0 out_seq=0 2024-09-21T03:49:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.258+0000 7f6bb6a00640 1 -- 172.21.15.16:0/733194606 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5643cb1ff600 con 0x5643caea1d80 2024-09-21T03:49:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.258+0000 7f6bb6a00640 10 client.25985 handle_command_reply: tid=0 2024-09-21T03:49:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.262+0000 7f6bdee31740 1 client.25985 shutdown 2024-09-21T03:49:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.263+0000 7f6bdee31740 1 -- 172.21.15.16:0/733194606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caeba2f0 msgr2=0x5643cb2a8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.263+0000 7f6bdee31740 1 --2- 172.21.15.16:0/733194606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caeba2f0 0x5643cb2a8d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4011870 tx=0x7f6bc4004ab0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.272+0000 7f6bdee31740 1 -- 172.21.15.16:0/733194606 shutdown_connections 2024-09-21T03:49:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.273+0000 7f6bdee31740 1 --2- 172.21.15.16:0/733194606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caee31b0 0x5643cb2a9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.273+0000 7f6bdee31740 1 --2- 172.21.15.16:0/733194606 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5643caea1d80 0x5643cb2b6590 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.273+0000 7f6bdee31740 1 --2- 172.21.15.16:0/733194606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caec24d0 0x5643cb2a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.273+0000 7f6bb6a00640 0 client.25985 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.273+0000 7f6bdee31740 1 --2- 172.21.15.16:0/733194606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caeba2f0 0x5643cb2a8d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.273+0000 7f6bdee31740 1 -- 172.21.15.16:0/733194606 >> 172.21.15.16:0/733194606 conn(0x5643caed7300 msgr2=0x5643cb284360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.274+0000 7f6bdee31740 1 -- 172.21.15.16:0/733194606 shutdown_connections 2024-09-21T03:49:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.274+0000 7f6bdee31740 1 -- 172.21.15.16:0/733194606 wait complete. 2024-09-21T03:49:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.274+0000 7f6bdee31740 20 client.25985 trim_cache size 0 max 16384 2024-09-21T03:49:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.305+0000 7f6bdee31740 1 Processor -- start 2024-09-21T03:49:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.305+0000 7f6bdee31740 1 -- start start 2024-09-21T03:49:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.306+0000 7f6bdee31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caea1d80 0x5643cb289f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.307+0000 7f6bdee31740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caeba2f0 0x5643cb28a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.309+0000 7f6bdee31740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caec24d0 0x5643cb2a3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.309+0000 7f6bdee31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd0076f40 con 0x5643caea1d80 2024-09-21T03:49:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.309+0000 7f6bdee31740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd01a35b0 con 0x5643caec24d0 2024-09-21T03:49:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.309+0000 7f6bdee31740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd0077610 con 0x5643caeba2f0 2024-09-21T03:49:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.309+0000 7f6bcfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caeba2f0 0x5643cb28a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.309+0000 7f6bcfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caeba2f0 0x5643cb28a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52090/0 (socket says 172.21.15.16:52090) 2024-09-21T03:49:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.309+0000 7f6bcfe00640 1 -- 172.21.15.16:0/3646089099 learned_addr learned my addr 172.21.15.16:0/3646089099 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.310+0000 7f6bd5c00640 1 --2- 172.21.15.16:0/3646089099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caec24d0 0x5643cb2a3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.310+0000 7f6bcfe00640 1 -- 172.21.15.16:0/3646089099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caec24d0 msgr2=0x5643cb2a3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.310+0000 7f6bcfe00640 1 --2- 172.21.15.16:0/3646089099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caec24d0 0x5643cb2a3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.310+0000 7f6bcfe00640 1 -- 172.21.15.16:0/3646089099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caea1d80 msgr2=0x5643cb289f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.310+0000 7f6bcfe00640 1 --2- 172.21.15.16:0/3646089099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caea1d80 0x5643cb289f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.310+0000 7f6bcfe00640 1 -- 172.21.15.16:0/3646089099 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643cb2aa330 con 0x5643caeba2f0 2024-09-21T03:49:50.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.310+0000 7f6bd5c00640 1 --2- 172.21.15.16:0/3646089099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caec24d0 0x5643cb2a3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:49:50.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.311+0000 7f6bcfe00640 1 --2- 172.21.15.16:0/3646089099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caeba2f0 0x5643cb28a4d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0096c0 tx=0x7f6bbc0315a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96176dcbdff78678 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:50.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.315+0000 7f6bc1600640 1 -- 172.21.15.16:0/3646089099 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc035030 con 0x5643caeba2f0 2024-09-21T03:49:50.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.315+0000 7f6bc1600640 1 -- 172.21.15.16:0/3646089099 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5643cb1ff600 con 0x5643caeba2f0 2024-09-21T03:49:50.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.315+0000 7f6bc1600640 1 -- 172.21.15.16:0/3646089099 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc040040 con 0x5643caeba2f0 2024-09-21T03:49:50.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.320+0000 7f6bdee31740 1 -- 172.21.15.16:0/3646089099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caeba2f0 msgr2=0x5643cb28a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:50.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.320+0000 7f6bdee31740 1 --2- 172.21.15.16:0/3646089099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caeba2f0 0x5643cb28a4d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0096c0 tx=0x7f6bbc0315a0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.321+0000 7f6bdee31740 1 -- 172.21.15.16:0/3646089099 shutdown_connections 2024-09-21T03:49:50.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.321+0000 7f6bdee31740 1 --2- 172.21.15.16:0/3646089099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caec24d0 0x5643cb2a3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.321+0000 7f6bdee31740 1 --2- 172.21.15.16:0/3646089099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caeba2f0 0x5643cb28a4d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.321+0000 7f6bdee31740 1 --2- 172.21.15.16:0/3646089099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caea1d80 0x5643cb289f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.321+0000 7f6bdee31740 1 -- 172.21.15.16:0/3646089099 >> 172.21.15.16:0/3646089099 conn(0x5643caed7300 msgr2=0x5643cb2a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:50.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.321+0000 7f6bdee31740 1 -- 172.21.15.16:0/3646089099 shutdown_connections 2024-09-21T03:49:50.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.321+0000 7f6bdee31740 1 -- 172.21.15.16:0/3646089099 wait complete. 2024-09-21T03:49:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.325+0000 7f6bdee31740 1 -- start start 2024-09-21T03:49:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.326+0000 7f6bdee31740 1 Processor -- start 2024-09-21T03:49:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.327+0000 7f6bdee31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caea1d80 0x5643cb290d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.327+0000 7f6bdee31740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caec24d0 0x5643cb291370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.327+0000 7f6bdee31740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caee31b0 0x5643cb28f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.327+0000 7f6bdee31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd01a2ee0 con 0x5643caea1d80 2024-09-21T03:49:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.327+0000 7f6bdee31740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd01a2810 con 0x5643caee31b0 2024-09-21T03:49:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.327+0000 7f6bdee31740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd0074f20 con 0x5643caec24d0 2024-09-21T03:49:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.327+0000 7f6bdee31740 10 client.0 _ll_register_callbacks cb 0x7f6bc4011050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:49:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.329+0000 7f6bd5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caee31b0 0x5643cb28f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.329+0000 7f6bd5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caee31b0 0x5643cb28f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:47786/0 (socket says 172.21.15.16:47786) 2024-09-21T03:49:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.329+0000 7f6bd5c00640 1 -- 172.21.15.16:0/814480663 learned_addr learned my addr 172.21.15.16:0/814480663 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:49:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.329+0000 7f6bcfe00640 1 --2- 172.21.15.16:0/814480663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caec24d0 0x5643cb291370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.330+0000 7f6bd5200640 1 --2- 172.21.15.16:0/814480663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caea1d80 0x5643cb290d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.331+0000 7f6bd5200640 1 -- 172.21.15.16:0/814480663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caec24d0 msgr2=0x5643cb291370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.331+0000 7f6bd5200640 1 --2- 172.21.15.16:0/814480663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caec24d0 0x5643cb291370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.331+0000 7f6bd5200640 1 -- 172.21.15.16:0/814480663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caee31b0 msgr2=0x5643cb28f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:49:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.331+0000 7f6bd5200640 1 --2- 172.21.15.16:0/814480663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caee31b0 0x5643cb28f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.331+0000 7f6bd5200640 1 -- 172.21.15.16:0/814480663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643cb2b0f60 con 0x5643caea1d80 2024-09-21T03:49:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.333+0000 7f6bd5200640 1 --2- 172.21.15.16:0/814480663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caea1d80 0x5643cb290d90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4018040 tx=0x7f6bc401c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19599f28305dae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:49:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.333+0000 7f6bb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:49:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.334+0000 7f6bb6a00640 1 -- 172.21.15.16:0/814480663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4036080 con 0x5643caea1d80 2024-09-21T03:49:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.334+0000 7f6bb6a00640 1 -- 172.21.15.16:0/814480663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f6bc4003d80 con 0x5643caea1d80 2024-09-21T03:49:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.334+0000 7f6bdee31740 1 -- 172.21.15.16:0/814480663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5643cb1ff600 con 0x5643caea1d80 2024-09-21T03:49:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.335+0000 7f6bb6a00640 1 -- 172.21.15.16:0/814480663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6bc400fb10 con 0x5643caea1d80 2024-09-21T03:49:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.335+0000 7f6bdee31740 10 client.15921 fetch_fsmap learned FSMap version 30 2024-09-21T03:49:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.335+0000 7f6bdee31740 1 -- 172.21.15.16:0/814480663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5643cb31d1b0 con 0x5643caea1d80 2024-09-21T03:49:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.337+0000 7f6bb6a00640 1 -- 172.21.15.16:0/814480663 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5643cb1ff600 con 0x5643caea1d80 2024-09-21T03:49:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.340+0000 7f6bb6a00640 1 -- 172.21.15.16:0/814480663 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6bc40275b0 con 0x5643caea1d80 2024-09-21T03:49:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.343+0000 7f6bdee31740 10 client.15921 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:49:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.344+0000 7f6bdee31740 10 client.15921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:49:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.344+0000 7f6bdee31740 20 client.15921 populate_metadata read hostname 'smithi016' 2024-09-21T03:49:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.344+0000 7f6bdee31740 1 --2- 172.21.15.16:0/814480663 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5643cae771c0 0x5643cb31fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:49:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.344+0000 7f6bdee31740 4 client.15921 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:49:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.344+0000 7f6bdee31740 1 -- 172.21.15.16:0/814480663 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5643cb1fb0b0 con 0x5643cae771c0 2024-09-21T03:49:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.345+0000 7f6bcfe00640 1 --2- 172.21.15.16:0/814480663 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5643cae771c0 0x5643cb31fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:49:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.347+0000 7f6bcfe00640 1 --2- 172.21.15.16:0/814480663 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5643cae771c0 0x5643cb31fb50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f860f165679088c0 server_cookie=ca2ef646f99ce567 in_seq=0 out_seq=0 2024-09-21T03:49:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.347+0000 7f6bb6a00640 10 client.15921 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.368+0000 7f6bb6a00640 1 -- 172.21.15.16:0/814480663 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5643cb1fb0b0 con 0x5643cae771c0 2024-09-21T03:49:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.368+0000 7f6bb6a00640 10 client.15921 handle_command_reply: tid=0 2024-09-21T03:49:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.368+0000 7f6bdee31740 1 client.15921 shutdown 2024-09-21T03:49:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.370+0000 7f6bdee31740 1 -- 172.21.15.16:0/814480663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caea1d80 msgr2=0x5643cb290d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.370+0000 7f6bdee31740 1 --2- 172.21.15.16:0/814480663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caea1d80 0x5643cb290d90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4018040 tx=0x7f6bc401c6a0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.370+0000 7f6bdee31740 1 -- 172.21.15.16:0/814480663 shutdown_connections 2024-09-21T03:49:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.370+0000 7f6bdee31740 1 --2- 172.21.15.16:0/814480663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643caee31b0 0x5643cb28f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.370+0000 7f6bdee31740 1 --2- 172.21.15.16:0/814480663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643caec24d0 0x5643cb291370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.370+0000 7f6bdee31740 1 --2- 172.21.15.16:0/814480663 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5643cae771c0 0x5643cb31fb50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.370+0000 7f6bdee31740 1 --2- 172.21.15.16:0/814480663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643caea1d80 0x5643cb290d90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.370+0000 7f6bdee31740 1 -- 172.21.15.16:0/814480663 >> 172.21.15.16:0/814480663 conn(0x5643caed7300 msgr2=0x5643cb289ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.371+0000 7f6bb6a00640 0 client.15921 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:49:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.371+0000 7f6bdee31740 1 -- 172.21.15.16:0/814480663 shutdown_connections 2024-09-21T03:49:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.371+0000 7f6bdee31740 1 -- 172.21.15.16:0/814480663 wait complete. 2024-09-21T03:49:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.371+0000 7f6bdee31740 20 client.15921 trim_cache size 0 max 16384 2024-09-21T03:49:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:49:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:49:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:49:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:49:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.373+0000 7f6bd6600640 1 -- 172.21.15.16:0/3261820600 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6bb80b65d0 msgr2=0x7f6bb80b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.373+0000 7f6bd6600640 1 --2- 172.21.15.16:0/3261820600 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6bb80b65d0 0x7f6bb80b8ae0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4011a90 tx=0x7f6bc4009500 comp rx=0 tx=0).stop 2024-09-21T03:49:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.373+0000 7f6bd6600640 1 -- 172.21.15.16:0/3261820600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd0104f00 msgr2=0x7f6bd019af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:49:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.373+0000 7f6bd6600640 1 --2- 172.21.15.16:0/3261820600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd0104f00 0x7f6bd019af60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800e970 tx=0x7f6bc800ee30 comp rx=0 tx=0).stop 2024-09-21T03:49:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.373+0000 7f6bd6600640 1 -- 172.21.15.16:0/3261820600 shutdown_connections 2024-09-21T03:49:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.373+0000 7f6bd6600640 1 --2- 172.21.15.16:0/3261820600 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6bb80b65d0 0x7f6bb80b8ae0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.373+0000 7f6bd6600640 1 --2- 172.21.15.16:0/3261820600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd0104f00 0x7f6bd019af60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.373+0000 7f6bd6600640 1 --2- 172.21.15.16:0/3261820600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd0104560 0x7f6bd019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.373+0000 7f6bd6600640 1 --2- 172.21.15.16:0/3261820600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd01034e0 0x7f6bd0199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:49:50.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.373+0000 7f6bd6600640 1 -- 172.21.15.16:0/3261820600 >> 172.21.15.16:0/3261820600 conn(0x7f6bd00fee20 msgr2=0x7f6bd0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:49:50.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.373+0000 7f6bd6600640 1 -- 172.21.15.16:0/3261820600 shutdown_connections 2024-09-21T03:49:50.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:49:50.373+0000 7f6bd6600640 1 -- 172.21.15.16:0/3261820600 wait complete. 2024-09-21T03:49:51.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:50 smithi134 ceph-mon[35031]: pgmap v524: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 127 MiB/s wr, 1.26k op/s 2024-09-21T03:49:51.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:50 smithi123 ceph-mon[34673]: pgmap v524: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 127 MiB/s wr, 1.26k op/s 2024-09-21T03:49:51.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:50 smithi016 ceph-mon[30375]: pgmap v524: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 127 MiB/s wr, 1.26k op/s 2024-09-21T03:49:53.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:53 smithi134 ceph-mon[35031]: pgmap v525: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 136 MiB/s wr, 1.38k op/s 2024-09-21T03:49:53.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:53 smithi123 ceph-mon[34673]: pgmap v525: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 136 MiB/s wr, 1.38k op/s 2024-09-21T03:49:53.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:53 smithi016 ceph-mon[30375]: pgmap v525: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 136 MiB/s wr, 1.38k op/s 2024-09-21T03:49:55.288 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:55 smithi016 ceph-mon[30375]: pgmap v526: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 138 MiB/s wr, 1.37k op/s 2024-09-21T03:49:55.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:55 smithi134 ceph-mon[35031]: pgmap v526: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 138 MiB/s wr, 1.37k op/s 2024-09-21T03:49:55.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:55 smithi123 ceph-mon[34673]: pgmap v526: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 138 MiB/s wr, 1.37k op/s 2024-09-21T03:49:57.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:57 smithi134 ceph-mon[35031]: pgmap v527: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 152 MiB/s wr, 1.39k op/s 2024-09-21T03:49:57.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:57 smithi123 ceph-mon[34673]: pgmap v527: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 152 MiB/s wr, 1.39k op/s 2024-09-21T03:49:57.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:57 smithi016 ceph-mon[30375]: pgmap v527: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 152 MiB/s wr, 1.39k op/s 2024-09-21T03:49:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:49:59 smithi134 ceph-mon[35031]: pgmap v528: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 178 MiB/s wr, 1.50k op/s 2024-09-21T03:49:59.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:49:59 smithi123 ceph-mon[34673]: pgmap v528: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 178 MiB/s wr, 1.50k op/s 2024-09-21T03:49:59.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:49:59 smithi016 ceph-mon[30375]: pgmap v528: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 178 MiB/s wr, 1.50k op/s 2024-09-21T03:50:00.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:00 smithi134 ceph-mon[35031]: overall HEALTH_OK 2024-09-21T03:50:00.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:00 smithi123 ceph-mon[34673]: overall HEALTH_OK 2024-09-21T03:50:00.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:00 smithi016 ceph-mon[30375]: overall HEALTH_OK 2024-09-21T03:50:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:01 smithi016 ceph-mon[30375]: pgmap v529: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 201 MiB/s wr, 1.60k op/s 2024-09-21T03:50:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:01 smithi134 ceph-mon[35031]: pgmap v529: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 201 MiB/s wr, 1.60k op/s 2024-09-21T03:50:01.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:01 smithi123 ceph-mon[34673]: pgmap v529: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 201 MiB/s wr, 1.60k op/s 2024-09-21T03:50:03.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:03 smithi134 ceph-mon[35031]: pgmap v530: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 224 MiB/s wr, 1.67k op/s 2024-09-21T03:50:03.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:03 smithi123 ceph-mon[34673]: pgmap v530: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 224 MiB/s wr, 1.67k op/s 2024-09-21T03:50:03.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:03 smithi016 ceph-mon[30375]: pgmap v530: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 224 MiB/s wr, 1.67k op/s 2024-09-21T03:50:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.733+0000 7fcb4ae00640 1 -- 172.21.15.16:0/917834020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb4410b510 msgr2=0x7fcb44111eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.733+0000 7fcb4ae00640 1 --2- 172.21.15.16:0/917834020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb4410b510 0x7fcb44111eb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009ca0 tx=0x7fcb3c02a8f0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.734+0000 7fcb4ae00640 1 -- 172.21.15.16:0/917834020 shutdown_connections 2024-09-21T03:50:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.734+0000 7fcb4ae00640 1 --2- 172.21.15.16:0/917834020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb4410b510 0x7fcb44111eb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.735+0000 7fcb4ae00640 1 --2- 172.21.15.16:0/917834020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb4410ab70 0x7fcb4410af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.735+0000 7fcb4ae00640 1 --2- 172.21.15.16:0/917834020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb44109af0 0x7fcb44109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.735+0000 7fcb4ae00640 1 -- 172.21.15.16:0/917834020 >> 172.21.15.16:0/917834020 conn(0x7fcb44078e30 msgr2=0x7fcb4407b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.736+0000 7fcb4ae00640 1 -- 172.21.15.16:0/917834020 shutdown_connections 2024-09-21T03:50:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.737+0000 7fcb4ae00640 1 -- 172.21.15.16:0/917834020 wait complete. 2024-09-21T03:50:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.739+0000 7fcb4ae00640 1 Processor -- start 2024-09-21T03:50:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.740+0000 7fcb4ae00640 1 -- start start 2024-09-21T03:50:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.742+0000 7fcb4ae00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb44109af0 0x7fcb44080f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.742+0000 7fcb4ae00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb4410ab70 0x7fcb440815c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.742+0000 7fcb49000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb4410ab70 0x7fcb440815c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.743+0000 7fcb49000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb4410ab70 0x7fcb440815c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44346/0 (socket says 172.21.15.16:44346) 2024-09-21T03:50:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.743+0000 7fcb4ae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb4410b510 0x7fcb44082170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.746+0000 7fcb49000640 1 -- 172.21.15.16:0/1820227219 learned_addr learned my addr 172.21.15.16:0/1820227219 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:03.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.746+0000 7fcb4ae00640 1 -- 172.21.15.16:0/1820227219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb44115880 con 0x7fcb4410b510 2024-09-21T03:50:03.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.746+0000 7fcb4ae00640 1 -- 172.21.15.16:0/1820227219 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcb44115700 con 0x7fcb4410ab70 2024-09-21T03:50:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.746+0000 7fcb4ae00640 1 -- 172.21.15.16:0/1820227219 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcb44115a00 con 0x7fcb44109af0 2024-09-21T03:50:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.747+0000 7fcb49000640 1 -- 172.21.15.16:0/1820227219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb44109af0 msgr2=0x7fcb44080f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:50:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.747+0000 7fcb49000640 1 --2- 172.21.15.16:0/1820227219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb44109af0 0x7fcb44080f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.747+0000 7fcb49000640 1 -- 172.21.15.16:0/1820227219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb4410b510 msgr2=0x7fcb44082170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.747+0000 7fcb49000640 1 --2- 172.21.15.16:0/1820227219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb4410b510 0x7fcb44082170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.747+0000 7fcb49000640 1 -- 172.21.15.16:0/1820227219 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb44082990 con 0x7fcb4410ab70 2024-09-21T03:50:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.748+0000 7fcb49000640 1 --2- 172.21.15.16:0/1820227219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb4410ab70 0x7fcb440815c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcb3800b4a0 tx=0x7fcb3800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.748+0000 7fcb42000640 1 -- 172.21.15.16:0/1820227219 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38014070 con 0x7fcb4410ab70 2024-09-21T03:50:03.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.749+0000 7fcb4ae00640 1 -- 172.21.15.16:0/1820227219 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb44082c80 con 0x7fcb4410ab70 2024-09-21T03:50:03.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.749+0000 7fcb4ae00640 1 -- 172.21.15.16:0/1820227219 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb440833c0 con 0x7fcb4410ab70 2024-09-21T03:50:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.749+0000 7fcb42000640 1 -- 172.21.15.16:0/1820227219 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb38010ac0 con 0x7fcb4410ab70 2024-09-21T03:50:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.749+0000 7fcb42000640 1 -- 172.21.15.16:0/1820227219 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38017c00 con 0x7fcb4410ab70 2024-09-21T03:50:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.760+0000 7fcb42000640 1 -- 172.21.15.16:0/1820227219 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fcb38017070 con 0x7fcb4410ab70 2024-09-21T03:50:03.766 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.764+0000 7fcb42000640 1 --2- 172.21.15.16:0/1820227219 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcb2c0b6610 0x7fcb2c0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.764+0000 7fcb42000640 1 -- 172.21.15.16:0/1820227219 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fcb380e7900 con 0x7fcb4410ab70 2024-09-21T03:50:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.765+0000 7fcb5348c740 1 Processor -- start 2024-09-21T03:50:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.766+0000 7fcb5348c740 1 -- start start 2024-09-21T03:50:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.766+0000 7fcb5348c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0fa51b0 0x5580d134c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.766+0000 7fcb5348c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0f7c2f0 0x5580d134c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.766+0000 7fcb5348c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f844d0 0x5580d134d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.766+0000 7fcb5348c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5580d12bf6b0 con 0x5580d0fa51b0 2024-09-21T03:50:03.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.766+0000 7fcb5348c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5580d12aeae0 con 0x5580d0f844d0 2024-09-21T03:50:03.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.766+0000 7fcb5348c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5580d12b28f0 con 0x5580d0f7c2f0 2024-09-21T03:50:03.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.767+0000 7fcb4a400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f844d0 0x5580d134d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.767+0000 7fcb4a400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f844d0 0x5580d134d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44356/0 (socket says 172.21.15.16:44356) 2024-09-21T03:50:03.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.767+0000 7fcb4a400640 1 -- 172.21.15.16:0/2314034990 learned_addr learned my addr 172.21.15.16:0/2314034990 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:03.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.768+0000 7fcb4a400640 1 -- 172.21.15.16:0/2314034990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0f7c2f0 msgr2=0x5580d134c920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:50:03.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.768+0000 7fcb4a400640 1 --2- 172.21.15.16:0/2314034990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0f7c2f0 0x5580d134c920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.768+0000 7fcb4a400640 1 -- 172.21.15.16:0/2314034990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0fa51b0 msgr2=0x5580d134c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.768+0000 7fcb4a400640 1 --2- 172.21.15.16:0/2314034990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0fa51b0 0x5580d134c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.768+0000 7fcb4a400640 1 -- 172.21.15.16:0/2314034990 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580d127b780 con 0x5580d0f844d0 2024-09-21T03:50:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.770+0000 7fcb4a400640 1 --2- 172.21.15.16:0/2314034990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f844d0 0x5580d134d470 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c031b50 tx=0x7fcb3c033000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93d87686013b37ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.770+0000 7fcb36a00640 1 -- 172.21.15.16:0/2314034990 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c031480 con 0x5580d0f844d0 2024-09-21T03:50:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.771+0000 7fcb36a00640 1 -- 172.21.15.16:0/2314034990 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb3c036070 con 0x5580d0f844d0 2024-09-21T03:50:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.771+0000 7fcb36a00640 1 -- 172.21.15.16:0/2314034990 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c031480 con 0x5580d0f844d0 2024-09-21T03:50:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.772+0000 7fcb49a00640 1 --2- 172.21.15.16:0/1820227219 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcb2c0b6610 0x7fcb2c0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.772+0000 7fcb5348c740 1 -- 172.21.15.16:0/2314034990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f844d0 msgr2=0x5580d134d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.772+0000 7fcb5348c740 1 --2- 172.21.15.16:0/2314034990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f844d0 0x5580d134d470 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c031b50 tx=0x7fcb3c033000 comp rx=0 tx=0).stop 2024-09-21T03:50:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.773+0000 7fcb5348c740 1 -- 172.21.15.16:0/2314034990 shutdown_connections 2024-09-21T03:50:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.773+0000 7fcb5348c740 1 --2- 172.21.15.16:0/2314034990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f844d0 0x5580d134d470 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.773+0000 7fcb5348c740 1 --2- 172.21.15.16:0/2314034990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0f7c2f0 0x5580d134c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.773+0000 7fcb5348c740 1 --2- 172.21.15.16:0/2314034990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0fa51b0 0x5580d134c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.773+0000 7fcb5348c740 1 -- 172.21.15.16:0/2314034990 >> 172.21.15.16:0/2314034990 conn(0x5580d0f99300 msgr2=0x5580d1347d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.780 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.777+0000 7fcb5348c740 1 -- 172.21.15.16:0/2314034990 shutdown_connections 2024-09-21T03:50:03.781 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.777+0000 7fcb5348c740 1 -- 172.21.15.16:0/2314034990 wait complete. 2024-09-21T03:50:03.781 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.778+0000 7fcb5348c740 1 -- start start 2024-09-21T03:50:03.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.778+0000 7fcb5348c740 1 Processor -- start 2024-09-21T03:50:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.785+0000 7fcb5348c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0f7c2f0 0x5580d13559a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.785+0000 7fcb5348c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f844d0 0x5580d1356060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.785+0000 7fcb5348c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0fa51b0 0x5580d1356bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.786+0000 7fcb5348c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5580d11e6ab0 con 0x5580d0fa51b0 2024-09-21T03:50:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.786+0000 7fcb5348c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5580d1255570 con 0x5580d0f844d0 2024-09-21T03:50:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.786+0000 7fcb5348c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5580d134df30 con 0x5580d0f7c2f0 2024-09-21T03:50:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.786+0000 7fcb5348c740 10 client.0 _ll_register_callbacks cb 0x5580d1264100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.787+0000 7fcb49000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f844d0 0x5580d1356060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.787+0000 7fcb49000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f844d0 0x5580d1356060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44360/0 (socket says 172.21.15.16:44360) 2024-09-21T03:50:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.787+0000 7fcb49000640 1 -- 172.21.15.16:0/1981514365 learned_addr learned my addr 172.21.15.16:0/1981514365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.788+0000 7fcb49000640 1 -- 172.21.15.16:0/1981514365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0f7c2f0 msgr2=0x5580d13559a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:50:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.788+0000 7fcb49000640 1 --2- 172.21.15.16:0/1981514365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0f7c2f0 0x5580d13559a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.788+0000 7fcb49000640 1 -- 172.21.15.16:0/1981514365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0fa51b0 msgr2=0x5580d1356bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.788+0000 7fcb49000640 1 --2- 172.21.15.16:0/1981514365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0fa51b0 0x5580d1356bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.788+0000 7fcb49000640 1 -- 172.21.15.16:0/1981514365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580d1010790 con 0x5580d0f844d0 2024-09-21T03:50:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.789+0000 7fcb49000640 1 --2- 172.21.15.16:0/1981514365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f844d0 0x5580d1356060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcb380ad380 tx=0x7fcb3800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4343a9f411884820 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.789+0000 7fcb2b400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:50:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.790+0000 7fcb2b400640 1 -- 172.21.15.16:0/1981514365 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3801e070 con 0x5580d0f844d0 2024-09-21T03:50:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.790+0000 7fcb2b400640 1 -- 172.21.15.16:0/1981514365 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fcb38010610 con 0x5580d0f844d0 2024-09-21T03:50:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.790+0000 7fcb5348c740 1 -- 172.21.15.16:0/1981514365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb3c036070 con 0x5580d0f844d0 2024-09-21T03:50:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.791+0000 7fcb2b400640 1 -- 172.21.15.16:0/1981514365 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcb380b2160 con 0x5580d0f844d0 2024-09-21T03:50:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.792+0000 7fcb5348c740 10 client.26000 fetch_fsmap learned FSMap version 30 2024-09-21T03:50:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.792+0000 7fcb5348c740 1 -- 172.21.15.16:0/1981514365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5580d1089620 con 0x5580d0f844d0 2024-09-21T03:50:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.793+0000 7fcb2b400640 1 -- 172.21.15.16:0/1981514365 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb3c036070 con 0x5580d0f844d0 2024-09-21T03:50:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.794+0000 7fcb2b400640 1 -- 172.21.15.16:0/1981514365 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcb380a8fd0 con 0x5580d0f844d0 2024-09-21T03:50:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.794+0000 7fcb5348c740 10 client.26000 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:50:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.794+0000 7fcb5348c740 10 client.26000 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.794+0000 7fcb5348c740 20 client.26000 populate_metadata read hostname 'smithi016' 2024-09-21T03:50:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.794+0000 7fcb5348c740 1 --2- 172.21.15.16:0/1981514365 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5580d0f63d80 0x5580d13632e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.794+0000 7fcb5348c740 4 client.26000 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.794+0000 7fcb5348c740 1 -- 172.21.15.16:0/1981514365 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580d12c1600 con 0x5580d0f63d80 2024-09-21T03:50:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.804+0000 7fcb49a00640 1 --2- 172.21.15.16:0/1981514365 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5580d0f63d80 0x5580d13632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.804+0000 7fcb49a00640 1 --2- 172.21.15.16:0/1820227219 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcb2c0b6610 0x7fcb2c0b8b20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcb30005e40 tx=0x7fcb30007a90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.813+0000 7fcb2b400640 10 client.26000 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.813+0000 7fcb49a00640 1 --2- 172.21.15.16:0/1981514365 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5580d0f63d80 0x5580d13632e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b87801cebba03d3f server_cookie=7e2ce1447040f4f8 in_seq=0 out_seq=0 2024-09-21T03:50:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.817+0000 7fcb2b400640 1 -- 172.21.15.16:0/1981514365 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5580d12c1600 con 0x5580d0f63d80 2024-09-21T03:50:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.817+0000 7fcb2b400640 10 client.26000 handle_command_reply: tid=0 2024-09-21T03:50:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.818+0000 7fcb5348c740 1 client.26000 shutdown 2024-09-21T03:50:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.819+0000 7fcb5348c740 1 -- 172.21.15.16:0/1981514365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f844d0 msgr2=0x5580d1356060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.819+0000 7fcb5348c740 1 --2- 172.21.15.16:0/1981514365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f844d0 0x5580d1356060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcb380ad380 tx=0x7fcb3800bf90 comp rx=0 tx=0).stop 2024-09-21T03:50:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.822+0000 7fcb5348c740 1 -- 172.21.15.16:0/1981514365 shutdown_connections 2024-09-21T03:50:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.822+0000 7fcb5348c740 1 --2- 172.21.15.16:0/1981514365 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5580d0f63d80 0x5580d13632e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.822+0000 7fcb5348c740 1 --2- 172.21.15.16:0/1981514365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0fa51b0 0x5580d1356bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.822+0000 7fcb5348c740 1 --2- 172.21.15.16:0/1981514365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f844d0 0x5580d1356060 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.822+0000 7fcb5348c740 1 --2- 172.21.15.16:0/1981514365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0f7c2f0 0x5580d13559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.822+0000 7fcb2b400640 0 client.26000 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.822+0000 7fcb5348c740 1 -- 172.21.15.16:0/1981514365 >> 172.21.15.16:0/1981514365 conn(0x5580d0f99300 msgr2=0x5580d1346360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.823+0000 7fcb5348c740 1 -- 172.21.15.16:0/1981514365 shutdown_connections 2024-09-21T03:50:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.823+0000 7fcb5348c740 1 -- 172.21.15.16:0/1981514365 wait complete. 2024-09-21T03:50:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.824+0000 7fcb5348c740 20 client.26000 trim_cache size 0 max 16384 2024-09-21T03:50:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.874+0000 7fcb5348c740 1 Processor -- start 2024-09-21T03:50:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.874+0000 7fcb5348c740 1 -- start start 2024-09-21T03:50:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.875+0000 7fcb5348c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0f7c2f0 0x5580d1353eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.875+0000 7fcb5348c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0f844d0 0x5580d135da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.875+0000 7fcb5348c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0fa51b0 0x5580d13543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.875+0000 7fcb5348c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2400fbb0 con 0x5580d0f844d0 2024-09-21T03:50:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.875+0000 7fcb5348c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcb240116f0 con 0x5580d0fa51b0 2024-09-21T03:50:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.875+0000 7fcb5348c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcb24010280 con 0x5580d0f7c2f0 2024-09-21T03:50:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.876+0000 7fcb4a400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0fa51b0 0x5580d13543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.876+0000 7fcb4a400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0fa51b0 0x5580d13543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44368/0 (socket says 172.21.15.16:44368) 2024-09-21T03:50:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.876+0000 7fcb4a400640 1 -- 172.21.15.16:0/4054992728 learned_addr learned my addr 172.21.15.16:0/4054992728 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.876+0000 7fcb49000640 1 --2- 172.21.15.16:0/4054992728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0f844d0 0x5580d135da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.877+0000 7fcb4a400640 1 -- 172.21.15.16:0/4054992728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0f7c2f0 msgr2=0x5580d1353eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.877+0000 7fcb4a400640 1 --2- 172.21.15.16:0/4054992728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0f7c2f0 0x5580d1353eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.877+0000 7fcb4a400640 1 -- 172.21.15.16:0/4054992728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0f844d0 msgr2=0x5580d135da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.877+0000 7fcb4a400640 1 --2- 172.21.15.16:0/4054992728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0f844d0 0x5580d135da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.877+0000 7fcb4a400640 1 -- 172.21.15.16:0/4054992728 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580d1354930 con 0x5580d0fa51b0 2024-09-21T03:50:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.878+0000 7fcb4a400640 1 --2- 172.21.15.16:0/4054992728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0fa51b0 0x5580d13543f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c02ae10 tx=0x7fcb3c001040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab4dee5b658275db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.879+0000 7fcb36000640 1 -- 172.21.15.16:0/4054992728 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c051070 con 0x5580d0fa51b0 2024-09-21T03:50:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.879+0000 7fcb36000640 1 -- 172.21.15.16:0/4054992728 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb3c036040 con 0x5580d0fa51b0 2024-09-21T03:50:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.879+0000 7fcb36000640 1 -- 172.21.15.16:0/4054992728 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c004040 con 0x5580d0fa51b0 2024-09-21T03:50:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.880+0000 7fcb5348c740 1 -- 172.21.15.16:0/4054992728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0fa51b0 msgr2=0x5580d13543f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.881+0000 7fcb5348c740 1 --2- 172.21.15.16:0/4054992728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0fa51b0 0x5580d13543f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c02ae10 tx=0x7fcb3c001040 comp rx=0 tx=0).stop 2024-09-21T03:50:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.881+0000 7fcb5348c740 1 -- 172.21.15.16:0/4054992728 shutdown_connections 2024-09-21T03:50:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.881+0000 7fcb5348c740 1 --2- 172.21.15.16:0/4054992728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0fa51b0 0x5580d13543f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.881+0000 7fcb5348c740 1 --2- 172.21.15.16:0/4054992728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0f844d0 0x5580d135da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.881+0000 7fcb5348c740 1 --2- 172.21.15.16:0/4054992728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0f7c2f0 0x5580d1353eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.881+0000 7fcb5348c740 1 -- 172.21.15.16:0/4054992728 >> 172.21.15.16:0/4054992728 conn(0x5580d0f99300 msgr2=0x5580d134d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.884+0000 7fcb5348c740 1 -- 172.21.15.16:0/4054992728 shutdown_connections 2024-09-21T03:50:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.884+0000 7fcb5348c740 1 -- 172.21.15.16:0/4054992728 wait complete. 2024-09-21T03:50:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.885+0000 7fcb5348c740 1 -- start start 2024-09-21T03:50:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.885+0000 7fcb5348c740 1 Processor -- start 2024-09-21T03:50:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.886+0000 7fcb5348c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f7c2f0 0x5580d135d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.886+0000 7fcb5348c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0f844d0 0x5580d135c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.886+0000 7fcb5348c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0fa51b0 0x5580d135c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.886+0000 7fcb5348c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb24011020 con 0x5580d0f844d0 2024-09-21T03:50:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.886+0000 7fcb5348c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcb24010950 con 0x5580d0f7c2f0 2024-09-21T03:50:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.886+0000 7fcb5348c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2400db90 con 0x5580d0fa51b0 2024-09-21T03:50:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.886+0000 7fcb5348c740 10 client.0 _ll_register_callbacks cb 0x7fcb380a9cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.887+0000 7fcb49a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f7c2f0 0x5580d135d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.887+0000 7fcb4a400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0fa51b0 0x5580d135c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.887+0000 7fcb4a400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0fa51b0 0x5580d135c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39686/0 (socket says 172.21.15.16:39686) 2024-09-21T03:50:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.887+0000 7fcb4a400640 1 -- 172.21.15.16:0/1394300100 learned_addr learned my addr 172.21.15.16:0/1394300100 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.888+0000 7fcb49000640 1 --2- 172.21.15.16:0/1394300100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0f844d0 0x5580d135c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.889+0000 7fcb4a400640 1 -- 172.21.15.16:0/1394300100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f7c2f0 msgr2=0x5580d135d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.889+0000 7fcb4a400640 1 --2- 172.21.15.16:0/1394300100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f7c2f0 0x5580d135d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.889+0000 7fcb4a400640 1 -- 172.21.15.16:0/1394300100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0f844d0 msgr2=0x5580d135c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.889+0000 7fcb4a400640 1 --2- 172.21.15.16:0/1394300100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0f844d0 0x5580d135c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.889+0000 7fcb4a400640 1 -- 172.21.15.16:0/1394300100 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580d135cc70 con 0x5580d0fa51b0 2024-09-21T03:50:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.889+0000 7fcb49a00640 1 --2- 172.21.15.16:0/1394300100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f7c2f0 0x5580d135d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.890+0000 7fcb4a400640 1 --2- 172.21.15.16:0/1394300100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0fa51b0 0x5580d135c700 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0094e0 tx=0x7fcb3c0317b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7f625eaa292076a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.890+0000 7fcb2b400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:50:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.890+0000 7fcb2b400640 1 -- 172.21.15.16:0/1394300100 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c059060 con 0x5580d0fa51b0 2024-09-21T03:50:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.891+0000 7fcb2b400640 1 -- 172.21.15.16:0/1394300100 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fcb3c003d80 con 0x5580d0fa51b0 2024-09-21T03:50:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.891+0000 7fcb5348c740 1 -- 172.21.15.16:0/1394300100 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb3c036040 con 0x5580d0fa51b0 2024-09-21T03:50:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.891+0000 7fcb2b400640 1 -- 172.21.15.16:0/1394300100 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcb3c051070 con 0x5580d0fa51b0 2024-09-21T03:50:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.891+0000 7fcb5348c740 10 client.25951 fetch_fsmap learned FSMap version 30 2024-09-21T03:50:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.891+0000 7fcb5348c740 1 -- 172.21.15.16:0/1394300100 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5580d135baa0 con 0x5580d0fa51b0 2024-09-21T03:50:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.892+0000 7fcb2b400640 1 -- 172.21.15.16:0/1394300100 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb3c036040 con 0x5580d0fa51b0 2024-09-21T03:50:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.893+0000 7fcb2b400640 1 -- 172.21.15.16:0/1394300100 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcb3c004040 con 0x5580d0fa51b0 2024-09-21T03:50:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.893+0000 7fcb5348c740 10 client.25951 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:50:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.893+0000 7fcb5348c740 10 client.25951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.893+0000 7fcb5348c740 20 client.25951 populate_metadata read hostname 'smithi016' 2024-09-21T03:50:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.894+0000 7fcb5348c740 1 --2- 172.21.15.16:0/1394300100 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5580d0f391c0 0x5580d13d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.894+0000 7fcb5348c740 4 client.25951 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:50:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.894+0000 7fcb5348c740 1 -- 172.21.15.16:0/1394300100 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5580d12bd0b0 con 0x5580d0f391c0 2024-09-21T03:50:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.894+0000 7fcb49a00640 1 --2- 172.21.15.16:0/1394300100 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5580d0f391c0 0x5580d13d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.895+0000 7fcb2b400640 10 client.25951 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.895+0000 7fcb49a00640 1 --2- 172.21.15.16:0/1394300100 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5580d0f391c0 0x5580d13d0300 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b7ca19118e949a server_cookie=247e60b03802ab9b in_seq=0 out_seq=0 2024-09-21T03:50:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.919+0000 7fcb2b400640 1 -- 172.21.15.16:0/1394300100 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5580d12bd0b0 con 0x5580d0f391c0 2024-09-21T03:50:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.919+0000 7fcb2b400640 10 client.25951 handle_command_reply: tid=0 2024-09-21T03:50:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.919+0000 7fcb5348c740 1 client.25951 shutdown 2024-09-21T03:50:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.919+0000 7fcb5348c740 1 -- 172.21.15.16:0/1394300100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0fa51b0 msgr2=0x5580d135c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.919+0000 7fcb5348c740 1 --2- 172.21.15.16:0/1394300100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0fa51b0 0x5580d135c700 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0094e0 tx=0x7fcb3c0317b0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.919+0000 7fcb5348c740 1 -- 172.21.15.16:0/1394300100 shutdown_connections 2024-09-21T03:50:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.919+0000 7fcb5348c740 1 --2- 172.21.15.16:0/1394300100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580d0fa51b0 0x5580d135c700 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.919+0000 7fcb5348c740 1 --2- 172.21.15.16:0/1394300100 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5580d0f391c0 0x5580d13d0300 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.919+0000 7fcb5348c740 1 --2- 172.21.15.16:0/1394300100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580d0f844d0 0x5580d135c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.919+0000 7fcb5348c740 1 --2- 172.21.15.16:0/1394300100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580d0f7c2f0 0x5580d135d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.919+0000 7fcb5348c740 1 -- 172.21.15.16:0/1394300100 >> 172.21.15.16:0/1394300100 conn(0x5580d0f99300 msgr2=0x5580d134d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.919+0000 7fcb2b400640 0 client.25951 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.920+0000 7fcb5348c740 1 -- 172.21.15.16:0/1394300100 shutdown_connections 2024-09-21T03:50:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.921+0000 7fcb5348c740 1 -- 172.21.15.16:0/1394300100 wait complete. 2024-09-21T03:50:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.921+0000 7fcb5348c740 20 client.25951 trim_cache size 0 max 16384 2024-09-21T03:50:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:50:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:50:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:50:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:50:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.923+0000 7fcb4ae00640 1 -- 172.21.15.16:0/1820227219 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcb2c0b6610 msgr2=0x7fcb2c0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.923+0000 7fcb4ae00640 1 --2- 172.21.15.16:0/1820227219 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcb2c0b6610 0x7fcb2c0b8b20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcb30005e40 tx=0x7fcb30007a90 comp rx=0 tx=0).stop 2024-09-21T03:50:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.923+0000 7fcb4ae00640 1 -- 172.21.15.16:0/1820227219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb4410ab70 msgr2=0x7fcb440815c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.923+0000 7fcb4ae00640 1 --2- 172.21.15.16:0/1820227219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb4410ab70 0x7fcb440815c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcb3800b4a0 tx=0x7fcb3800b960 comp rx=0 tx=0).stop 2024-09-21T03:50:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.923+0000 7fcb4ae00640 1 -- 172.21.15.16:0/1820227219 shutdown_connections 2024-09-21T03:50:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.923+0000 7fcb4ae00640 1 --2- 172.21.15.16:0/1820227219 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcb2c0b6610 0x7fcb2c0b8b20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.923+0000 7fcb4ae00640 1 --2- 172.21.15.16:0/1820227219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb4410b510 0x7fcb44082170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.923+0000 7fcb4ae00640 1 --2- 172.21.15.16:0/1820227219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb4410ab70 0x7fcb440815c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.923+0000 7fcb4ae00640 1 --2- 172.21.15.16:0/1820227219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb44109af0 0x7fcb44080f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.923+0000 7fcb4ae00640 1 -- 172.21.15.16:0/1820227219 >> 172.21.15.16:0/1820227219 conn(0x7fcb44078e30 msgr2=0x7fcb4410de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.924+0000 7fcb4ae00640 1 -- 172.21.15.16:0/1820227219 shutdown_connections 2024-09-21T03:50:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:03.924+0000 7fcb4ae00640 1 -- 172.21.15.16:0/1820227219 wait complete. 2024-09-21T03:50:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:05 smithi134 ceph-mon[35031]: pgmap v531: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 224 MiB/s wr, 1.55k op/s 2024-09-21T03:50:05.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:05 smithi123 ceph-mon[34673]: pgmap v531: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 224 MiB/s wr, 1.55k op/s 2024-09-21T03:50:05.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:05 smithi016 ceph-mon[30375]: pgmap v531: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 224 MiB/s wr, 1.55k op/s 2024-09-21T03:50:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:07 smithi134 ceph-mon[35031]: pgmap v532: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 232 MiB/s wr, 1.46k op/s 2024-09-21T03:50:07.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:07 smithi123 ceph-mon[34673]: pgmap v532: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 232 MiB/s wr, 1.46k op/s 2024-09-21T03:50:07.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:07 smithi016 ceph-mon[30375]: pgmap v532: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 232 MiB/s wr, 1.46k op/s 2024-09-21T03:50:09.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:09 smithi134 ceph-mon[35031]: pgmap v533: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 231 MiB/s wr, 1.33k op/s 2024-09-21T03:50:09.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:09 smithi123 ceph-mon[34673]: pgmap v533: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 231 MiB/s wr, 1.33k op/s 2024-09-21T03:50:09.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:09 smithi016 ceph-mon[30375]: pgmap v533: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 231 MiB/s wr, 1.33k op/s 2024-09-21T03:50:11.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:11 smithi123 ceph-mon[34673]: pgmap v534: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 222 MiB/s wr, 1.19k op/s 2024-09-21T03:50:11.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:11 smithi016 ceph-mon[30375]: pgmap v534: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 222 MiB/s wr, 1.19k op/s 2024-09-21T03:50:11.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:11 smithi134 ceph-mon[35031]: pgmap v534: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 222 MiB/s wr, 1.19k op/s 2024-09-21T03:50:13.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:13 smithi123 ceph-mon[34673]: pgmap v535: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 225 MiB/s wr, 1.14k op/s 2024-09-21T03:50:13.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:13 smithi016 ceph-mon[30375]: pgmap v535: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 225 MiB/s wr, 1.14k op/s 2024-09-21T03:50:13.415 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:13 smithi134 ceph-mon[35031]: pgmap v535: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 225 MiB/s wr, 1.14k op/s 2024-09-21T03:50:15.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:15 smithi123 ceph-mon[34673]: pgmap v536: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 231 MiB/s wr, 1.08k op/s 2024-09-21T03:50:15.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:15 smithi016 ceph-mon[30375]: pgmap v536: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 231 MiB/s wr, 1.08k op/s 2024-09-21T03:50:15.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:15 smithi134 ceph-mon[35031]: pgmap v536: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 231 MiB/s wr, 1.08k op/s 2024-09-21T03:50:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.288+0000 7f665ac00640 1 -- 172.21.15.16:0/4092133677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66540770f0 msgr2=0x7f66540757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.288+0000 7f665ac00640 1 --2- 172.21.15.16:0/4092133677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66540770f0 0x7f66540757b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6644009900 tx=0x7f664402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.288+0000 7f665ac00640 1 -- 172.21.15.16:0/4092133677 shutdown_connections 2024-09-21T03:50:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.288+0000 7f665ac00640 1 --2- 172.21.15.16:0/4092133677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f665410fe20 0x7f6654112240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.288+0000 7f665ac00640 1 --2- 172.21.15.16:0/4092133677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6654075cf0 0x7f6654076100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.288+0000 7f665ac00640 1 --2- 172.21.15.16:0/4092133677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66540770f0 0x7f66540757b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.288+0000 7f665ac00640 1 -- 172.21.15.16:0/4092133677 >> 172.21.15.16:0/4092133677 conn(0x7f66540fefe0 msgr2=0x7f6654101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.289+0000 7f665ac00640 1 -- 172.21.15.16:0/4092133677 shutdown_connections 2024-09-21T03:50:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.289+0000 7f665ac00640 1 -- 172.21.15.16:0/4092133677 wait complete. 2024-09-21T03:50:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.290+0000 7f665ac00640 1 Processor -- start 2024-09-21T03:50:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.290+0000 7f665ac00640 1 -- start start 2024-09-21T03:50:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.290+0000 7f665ac00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6654075cf0 0x7f66541a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.290+0000 7f665ac00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66540770f0 0x7f66541a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.290+0000 7f665ac00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f665410fe20 0x7f66541a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.290+0000 7f665ac00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66541158c0 con 0x7f665410fe20 2024-09-21T03:50:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.290+0000 7f665ac00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6654115740 con 0x7f66540770f0 2024-09-21T03:50:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.290+0000 7f665ac00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6654115a40 con 0x7f6654075cf0 2024-09-21T03:50:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.290+0000 7f6653e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6654075cf0 0x7f66541a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.290+0000 7f6653e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6654075cf0 0x7f66541a0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:36282/0 (socket says 172.21.15.16:36282) 2024-09-21T03:50:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.290+0000 7f6653400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66540770f0 0x7f66541a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.290+0000 7f6653e00640 1 -- 172.21.15.16:0/446494203 learned_addr learned my addr 172.21.15.16:0/446494203 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.291+0000 7f6658c00640 1 --2- 172.21.15.16:0/446494203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f665410fe20 0x7f66541a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.291+0000 7f6653e00640 1 -- 172.21.15.16:0/446494203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66540770f0 msgr2=0x7f66541a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.291+0000 7f6653e00640 1 --2- 172.21.15.16:0/446494203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66540770f0 0x7f66541a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.291+0000 7f6653e00640 1 -- 172.21.15.16:0/446494203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f665410fe20 msgr2=0x7f66541a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.291+0000 7f6653e00640 1 --2- 172.21.15.16:0/446494203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f665410fe20 0x7f66541a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.291+0000 7f6653e00640 1 -- 172.21.15.16:0/446494203 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66541a1f80 con 0x7f6654075cf0 2024-09-21T03:50:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.291+0000 7f6653e00640 1 --2- 172.21.15.16:0/446494203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6654075cf0 0x7f66541a0540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6644006010 tx=0x7f66440042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.291+0000 7f6650c00640 1 -- 172.21.15.16:0/446494203 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664403f030 con 0x7f6654075cf0 2024-09-21T03:50:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.292+0000 7f665ac00640 1 -- 172.21.15.16:0/446494203 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66541a2210 con 0x7f6654075cf0 2024-09-21T03:50:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.292+0000 7f6650c00640 1 -- 172.21.15.16:0/446494203 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6644031910 con 0x7f6654075cf0 2024-09-21T03:50:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.292+0000 7f6650c00640 1 -- 172.21.15.16:0/446494203 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644035070 con 0x7f6654075cf0 2024-09-21T03:50:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.292+0000 7f665ac00640 1 -- 172.21.15.16:0/446494203 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66541a8960 con 0x7f6654075cf0 2024-09-21T03:50:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.295+0000 7f6650c00640 1 -- 172.21.15.16:0/446494203 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6644042030 con 0x7f6654075cf0 2024-09-21T03:50:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.295+0000 7f6661dec740 1 Processor -- start 2024-09-21T03:50:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.296+0000 7f6661dec740 1 -- start start 2024-09-21T03:50:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.296+0000 7f6661dec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba23a1b0 0x563fba5e10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.296+0000 7f6661dec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2112f0 0x563fba5e1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.296+0000 7f6661dec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba2194d0 0x563fba5e2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.296+0000 7f6661dec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563fba5546b0 con 0x563fba2194d0 2024-09-21T03:50:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.296+0000 7f6661dec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563fba543ae0 con 0x563fba2112f0 2024-09-21T03:50:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.296+0000 7f6661dec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563fba5478f0 con 0x563fba23a1b0 2024-09-21T03:50:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.296+0000 7f6658c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba23a1b0 0x563fba5e10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.296+0000 7f6658c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba23a1b0 0x563fba5e10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:36288/0 (socket says 172.21.15.16:36288) 2024-09-21T03:50:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.296+0000 7f6653e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2112f0 0x563fba5e1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.296+0000 7f6658c00640 1 -- 172.21.15.16:0/941174841 learned_addr learned my addr 172.21.15.16:0/941174841 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.297+0000 7f6653400640 1 --2- 172.21.15.16:0/941174841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba2194d0 0x563fba5e2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.297+0000 7f6653e00640 1 -- 172.21.15.16:0/941174841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba23a1b0 msgr2=0x563fba5e10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.297+0000 7f6653e00640 1 --2- 172.21.15.16:0/941174841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba23a1b0 0x563fba5e10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.297+0000 7f6653e00640 1 -- 172.21.15.16:0/941174841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba2194d0 msgr2=0x563fba5e2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.297+0000 7f6653e00640 1 --2- 172.21.15.16:0/941174841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba2194d0 0x563fba5e2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.297+0000 7f6653e00640 1 -- 172.21.15.16:0/941174841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fba510780 con 0x563fba2112f0 2024-09-21T03:50:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.297+0000 7f6650c00640 1 --2- 172.21.15.16:0/446494203 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f66380b6590 0x7f66380b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.297+0000 7f6653e00640 1 --2- 172.21.15.16:0/941174841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2112f0 0x563fba5e1740 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f66441052f0 tx=0x7f664410be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=138ec4bbb5f1680f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.297+0000 7f6650c00640 1 -- 172.21.15.16:0/446494203 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f6644107c60 con 0x7f6654075cf0 2024-09-21T03:50:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.297+0000 7f6649200640 1 -- 172.21.15.16:0/941174841 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66440ca030 con 0x563fba2112f0 2024-09-21T03:50:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.298+0000 7f6649200640 1 -- 172.21.15.16:0/941174841 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66440cd4f0 con 0x563fba2112f0 2024-09-21T03:50:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.298+0000 7f6649200640 1 -- 172.21.15.16:0/941174841 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66440cd150 con 0x563fba2112f0 2024-09-21T03:50:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.298+0000 7f6658c00640 1 --2- 172.21.15.16:0/446494203 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f66380b6590 0x7f66380b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.299+0000 7f6661dec740 1 -- 172.21.15.16:0/941174841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2112f0 msgr2=0x563fba5e1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.299+0000 7f6661dec740 1 --2- 172.21.15.16:0/941174841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2112f0 0x563fba5e1740 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f66441052f0 tx=0x7f664410be20 comp rx=0 tx=0).stop 2024-09-21T03:50:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.299+0000 7f6661dec740 1 -- 172.21.15.16:0/941174841 shutdown_connections 2024-09-21T03:50:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.299+0000 7f6661dec740 1 --2- 172.21.15.16:0/941174841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba2194d0 0x563fba5e2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.299+0000 7f6661dec740 1 --2- 172.21.15.16:0/941174841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2112f0 0x563fba5e1740 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.299+0000 7f6661dec740 1 --2- 172.21.15.16:0/941174841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba23a1b0 0x563fba5e10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.299+0000 7f6661dec740 1 -- 172.21.15.16:0/941174841 >> 172.21.15.16:0/941174841 conn(0x563fba22e300 msgr2=0x563fba5dcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.299+0000 7f6661dec740 1 -- 172.21.15.16:0/941174841 shutdown_connections 2024-09-21T03:50:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.299+0000 7f6658c00640 1 --2- 172.21.15.16:0/446494203 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f66380b6590 0x7f66380b8aa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f664c003bc0 tx=0x7f664c009500 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.300+0000 7f6661dec740 1 -- 172.21.15.16:0/941174841 wait complete. 2024-09-21T03:50:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.300+0000 7f6661dec740 1 -- start start 2024-09-21T03:50:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.300+0000 7f6661dec740 1 Processor -- start 2024-09-21T03:50:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.301+0000 7f6661dec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba2112f0 0x563fba5ea540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.301+0000 7f6661dec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2194d0 0x563fba5eac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.301+0000 7f6661dec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba23a1b0 0x563fba5eb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.301+0000 7f6661dec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563fba47bab0 con 0x563fba23a1b0 2024-09-21T03:50:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.301+0000 7f6661dec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563fba4ea570 con 0x563fba2194d0 2024-09-21T03:50:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.301+0000 7f6661dec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563fba5e2b70 con 0x563fba2112f0 2024-09-21T03:50:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.301+0000 7f6661dec740 10 client.0 _ll_register_callbacks cb 0x563fba4f9100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.301+0000 7f6658c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba23a1b0 0x563fba5eb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.301+0000 7f6653e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba2112f0 0x563fba5ea540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.301+0000 7f6658c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba23a1b0 0x563fba5eb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58948/0 (socket says 172.21.15.16:58948) 2024-09-21T03:50:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.301+0000 7f6653e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba2112f0 0x563fba5ea540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:36304/0 (socket says 172.21.15.16:36304) 2024-09-21T03:50:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.301+0000 7f6658c00640 1 -- 172.21.15.16:0/1377791883 learned_addr learned my addr 172.21.15.16:0/1377791883 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.302+0000 7f6653400640 1 --2- 172.21.15.16:0/1377791883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2194d0 0x563fba5eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.302+0000 7f6658c00640 1 -- 172.21.15.16:0/1377791883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba2112f0 msgr2=0x563fba5ea540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.302+0000 7f6658c00640 1 --2- 172.21.15.16:0/1377791883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba2112f0 0x563fba5ea540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.302+0000 7f6658c00640 1 -- 172.21.15.16:0/1377791883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2194d0 msgr2=0x563fba5eac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.302+0000 7f6658c00640 1 --2- 172.21.15.16:0/1377791883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2194d0 0x563fba5eac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.302+0000 7f6658c00640 1 -- 172.21.15.16:0/1377791883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fba2a5790 con 0x563fba23a1b0 2024-09-21T03:50:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.302+0000 7f6653400640 1 --2- 172.21.15.16:0/1377791883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2194d0 0x563fba5eac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.302+0000 7f6658c00640 1 --2- 172.21.15.16:0/1377791883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba23a1b0 0x563fba5eb770 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f664c01fa00 tx=0x7f664c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a933e27475ff3e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.302+0000 7f6641600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:50:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.303+0000 7f6641600640 1 -- 172.21.15.16:0/1377791883 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c02c060 con 0x563fba23a1b0 2024-09-21T03:50:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.303+0000 7f6661dec740 1 -- 172.21.15.16:0/1377791883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66440cd4f0 con 0x563fba23a1b0 2024-09-21T03:50:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.303+0000 7f6641600640 1 -- 172.21.15.16:0/1377791883 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f664c017030 con 0x563fba23a1b0 2024-09-21T03:50:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.303+0000 7f6641600640 1 -- 172.21.15.16:0/1377791883 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f664c033050 con 0x563fba23a1b0 2024-09-21T03:50:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.304+0000 7f6661dec740 10 client.15939 fetch_fsmap learned FSMap version 30 2024-09-21T03:50:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.304+0000 7f6661dec740 1 -- 172.21.15.16:0/1377791883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563fba31e620 con 0x563fba23a1b0 2024-09-21T03:50:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.304+0000 7f6641600640 1 -- 172.21.15.16:0/1377791883 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66440cd4f0 con 0x563fba23a1b0 2024-09-21T03:50:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.305+0000 7f6641600640 1 -- 172.21.15.16:0/1377791883 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f664c0279f0 con 0x563fba23a1b0 2024-09-21T03:50:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.305+0000 7f6661dec740 10 client.15939 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:50:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.305+0000 7f6661dec740 10 client.15939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.305+0000 7f6661dec740 20 client.15939 populate_metadata read hostname 'smithi016' 2024-09-21T03:50:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.305+0000 7f6661dec740 1 --2- 172.21.15.16:0/1377791883 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x563fba1f8d80 0x563fba5f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.305+0000 7f6661dec740 4 client.15939 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.305+0000 7f6661dec740 1 -- 172.21.15.16:0/1377791883 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fba556600 con 0x563fba1f8d80 2024-09-21T03:50:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.305+0000 7f6653e00640 1 --2- 172.21.15.16:0/1377791883 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x563fba1f8d80 0x563fba5f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.306+0000 7f6653e00640 1 --2- 172.21.15.16:0/1377791883 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x563fba1f8d80 0x563fba5f7ce0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19c57f6b834b33a9 server_cookie=b231087e9987f409 in_seq=0 out_seq=0 2024-09-21T03:50:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.306+0000 7f6641600640 10 client.15939 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.307+0000 7f6641600640 1 -- 172.21.15.16:0/1377791883 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563fba556600 con 0x563fba1f8d80 2024-09-21T03:50:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.307+0000 7f6641600640 10 client.15939 handle_command_reply: tid=0 2024-09-21T03:50:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.307+0000 7f6661dec740 1 client.15939 shutdown 2024-09-21T03:50:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.308+0000 7f6661dec740 1 -- 172.21.15.16:0/1377791883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba23a1b0 msgr2=0x563fba5eb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.308+0000 7f6661dec740 1 --2- 172.21.15.16:0/1377791883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba23a1b0 0x563fba5eb770 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f664c01fa00 tx=0x7f664c004ab0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.309+0000 7f6661dec740 1 -- 172.21.15.16:0/1377791883 shutdown_connections 2024-09-21T03:50:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.309+0000 7f6661dec740 1 --2- 172.21.15.16:0/1377791883 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x563fba1f8d80 0x563fba5f7ce0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.309+0000 7f6641600640 0 client.15939 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.309+0000 7f6661dec740 1 --2- 172.21.15.16:0/1377791883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba23a1b0 0x563fba5eb770 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.309+0000 7f6661dec740 1 --2- 172.21.15.16:0/1377791883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2194d0 0x563fba5eac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.309+0000 7f6661dec740 1 --2- 172.21.15.16:0/1377791883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba2112f0 0x563fba5ea540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.309+0000 7f6661dec740 1 -- 172.21.15.16:0/1377791883 >> 172.21.15.16:0/1377791883 conn(0x563fba22e300 msgr2=0x563fba5db0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.309+0000 7f6661dec740 1 -- 172.21.15.16:0/1377791883 shutdown_connections 2024-09-21T03:50:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.309+0000 7f6661dec740 1 -- 172.21.15.16:0/1377791883 wait complete. 2024-09-21T03:50:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.310+0000 7f6661dec740 20 client.15939 trim_cache size 0 max 16384 2024-09-21T03:50:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.336+0000 7f6661dec740 1 Processor -- start 2024-09-21T03:50:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.336+0000 7f6661dec740 1 -- start start 2024-09-21T03:50:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.336+0000 7f6661dec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba2112f0 0x563fba5e9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.336+0000 7f6661dec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2194d0 0x563fba5e0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.336+0000 7f6661dec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba23a1b0 0x563fba5db850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.336+0000 7f6661dec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66541b2d60 con 0x563fba23a1b0 2024-09-21T03:50:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.336+0000 7f6661dec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66541b48a0 con 0x563fba2194d0 2024-09-21T03:50:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.336+0000 7f6661dec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66541b3430 con 0x563fba2112f0 2024-09-21T03:50:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.337+0000 7f6653400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2194d0 0x563fba5e0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.337+0000 7f6653400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2194d0 0x563fba5e0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:50296/0 (socket says 172.21.15.16:50296) 2024-09-21T03:50:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.337+0000 7f6653400640 1 -- 172.21.15.16:0/3814101375 learned_addr learned my addr 172.21.15.16:0/3814101375 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.337+0000 7f6658c00640 1 --2- 172.21.15.16:0/3814101375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba23a1b0 0x563fba5db850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.337+0000 7f6653e00640 1 --2- 172.21.15.16:0/3814101375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba2112f0 0x563fba5e9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.338+0000 7f6653400640 1 -- 172.21.15.16:0/3814101375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba2112f0 msgr2=0x563fba5e9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.338+0000 7f6653400640 1 --2- 172.21.15.16:0/3814101375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba2112f0 0x563fba5e9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.338+0000 7f6653400640 1 -- 172.21.15.16:0/3814101375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba23a1b0 msgr2=0x563fba5db850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.338+0000 7f6653400640 1 --2- 172.21.15.16:0/3814101375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba23a1b0 0x563fba5db850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.338+0000 7f6653400640 1 -- 172.21.15.16:0/3814101375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fba5dcbb0 con 0x563fba2194d0 2024-09-21T03:50:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.338+0000 7f6653e00640 1 --2- 172.21.15.16:0/3814101375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba2112f0 0x563fba5e9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.338+0000 7f6658c00640 1 --2- 172.21.15.16:0/3814101375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba23a1b0 0x563fba5db850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.338+0000 7f6653400640 1 --2- 172.21.15.16:0/3814101375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2194d0 0x563fba5e0cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f663c009c50 tx=0x7f663c010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=828a8ce4a227f8ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.338+0000 7f6643e00640 1 -- 172.21.15.16:0/3814101375 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c004030 con 0x563fba2194d0 2024-09-21T03:50:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.338+0000 7f6643e00640 1 -- 172.21.15.16:0/3814101375 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f663c0035f0 con 0x563fba2194d0 2024-09-21T03:50:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.338+0000 7f6643e00640 1 -- 172.21.15.16:0/3814101375 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c00f6a0 con 0x563fba2194d0 2024-09-21T03:50:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.339+0000 7f6661dec740 1 -- 172.21.15.16:0/3814101375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2194d0 msgr2=0x563fba5e0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.339+0000 7f6661dec740 1 --2- 172.21.15.16:0/3814101375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2194d0 0x563fba5e0cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f663c009c50 tx=0x7f663c010800 comp rx=0 tx=0).stop 2024-09-21T03:50:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.339+0000 7f6661dec740 1 -- 172.21.15.16:0/3814101375 shutdown_connections 2024-09-21T03:50:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.339+0000 7f6661dec740 1 --2- 172.21.15.16:0/3814101375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba23a1b0 0x563fba5db850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.339+0000 7f6661dec740 1 --2- 172.21.15.16:0/3814101375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2194d0 0x563fba5e0cc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.339+0000 7f6661dec740 1 --2- 172.21.15.16:0/3814101375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba2112f0 0x563fba5e9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.339+0000 7f6661dec740 1 -- 172.21.15.16:0/3814101375 >> 172.21.15.16:0/3814101375 conn(0x563fba22e300 msgr2=0x563fba5dc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.340+0000 7f6661dec740 1 -- 172.21.15.16:0/3814101375 shutdown_connections 2024-09-21T03:50:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.340+0000 7f6661dec740 1 -- 172.21.15.16:0/3814101375 wait complete. 2024-09-21T03:50:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.340+0000 7f6661dec740 1 -- start start 2024-09-21T03:50:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.340+0000 7f6661dec740 1 Processor -- start 2024-09-21T03:50:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.341+0000 7f6661dec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2112f0 0x563fba5f1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.341+0000 7f6661dec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba2194d0 0x563fba5f18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.341+0000 7f6661dec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba23a1b0 0x563fba5f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.341+0000 7f6661dec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66541b41d0 con 0x563fba2194d0 2024-09-21T03:50:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.341+0000 7f6661dec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66541b3b00 con 0x563fba2112f0 2024-09-21T03:50:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.341+0000 7f6661dec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66541b0d40 con 0x563fba23a1b0 2024-09-21T03:50:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.341+0000 7f6661dec740 10 client.0 _ll_register_callbacks cb 0x7f664c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.341+0000 7f6653e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2112f0 0x563fba5f1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.341+0000 7f6658c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba23a1b0 0x563fba5f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.341+0000 7f6653e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2112f0 0x563fba5f1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:50310/0 (socket says 172.21.15.16:50310) 2024-09-21T03:50:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.341+0000 7f6653e00640 1 -- 172.21.15.16:0/2259489942 learned_addr learned my addr 172.21.15.16:0/2259489942 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.342+0000 7f6653e00640 1 -- 172.21.15.16:0/2259489942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba23a1b0 msgr2=0x563fba5f1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.342+0000 7f6653e00640 1 --2- 172.21.15.16:0/2259489942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba23a1b0 0x563fba5f1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.342+0000 7f6653e00640 1 -- 172.21.15.16:0/2259489942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba2194d0 msgr2=0x563fba5f18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.342+0000 7f6653e00640 1 --2- 172.21.15.16:0/2259489942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba2194d0 0x563fba5f18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.342+0000 7f6653e00640 1 -- 172.21.15.16:0/2259489942 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fba5eff70 con 0x563fba2112f0 2024-09-21T03:50:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.342+0000 7f6658c00640 1 --2- 172.21.15.16:0/2259489942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba23a1b0 0x563fba5f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.342+0000 7f6653e00640 1 --2- 172.21.15.16:0/2259489942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2112f0 0x563fba5f1380 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6644105350 tx=0x7f66440f10d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e1ad551ceebf66b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.342+0000 7f6641600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:50:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.342+0000 7f6641600640 1 -- 172.21.15.16:0/2259489942 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644049040 con 0x563fba2112f0 2024-09-21T03:50:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.342+0000 7f6641600640 1 -- 172.21.15.16:0/2259489942 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f66441053d0 con 0x563fba2112f0 2024-09-21T03:50:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.342+0000 7f6661dec740 1 -- 172.21.15.16:0/2259489942 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f663c0035f0 con 0x563fba2112f0 2024-09-21T03:50:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.343+0000 7f6641600640 1 -- 172.21.15.16:0/2259489942 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f66440cd330 con 0x563fba2112f0 2024-09-21T03:50:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.343+0000 7f6661dec740 10 client.26021 fetch_fsmap learned FSMap version 30 2024-09-21T03:50:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.343+0000 7f6661dec740 1 -- 172.21.15.16:0/2259489942 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563fba662a20 con 0x563fba2112f0 2024-09-21T03:50:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.343+0000 7f6641600640 1 -- 172.21.15.16:0/2259489942 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f663c0035f0 con 0x563fba2112f0 2024-09-21T03:50:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.343+0000 7f6641600640 1 -- 172.21.15.16:0/2259489942 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f664407a030 con 0x563fba2112f0 2024-09-21T03:50:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.344+0000 7f6661dec740 10 client.26021 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:50:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.344+0000 7f6661dec740 10 client.26021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.344+0000 7f6661dec740 20 client.26021 populate_metadata read hostname 'smithi016' 2024-09-21T03:50:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.344+0000 7f6661dec740 1 --2- 172.21.15.16:0/2259489942 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x563fba1ce1c0 0x563fba6651e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.344+0000 7f6661dec740 4 client.26021 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:50:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.344+0000 7f6661dec740 1 -- 172.21.15.16:0/2259489942 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563fba5520b0 con 0x563fba1ce1c0 2024-09-21T03:50:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.344+0000 7f6653400640 1 --2- 172.21.15.16:0/2259489942 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x563fba1ce1c0 0x563fba6651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.345+0000 7f6653400640 1 --2- 172.21.15.16:0/2259489942 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x563fba1ce1c0 0x563fba6651e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a988caec018a1b5 server_cookie=f3e55e7fdfac9d50 in_seq=0 out_seq=0 2024-09-21T03:50:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.345+0000 7f6641600640 10 client.26021 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.374+0000 7f6641600640 1 -- 172.21.15.16:0/2259489942 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563fba5520b0 con 0x563fba1ce1c0 2024-09-21T03:50:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.374+0000 7f6641600640 10 client.26021 handle_command_reply: tid=0 2024-09-21T03:50:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.374+0000 7f6661dec740 1 client.26021 shutdown 2024-09-21T03:50:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.374+0000 7f6661dec740 1 -- 172.21.15.16:0/2259489942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2112f0 msgr2=0x563fba5f1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.374+0000 7f6661dec740 1 --2- 172.21.15.16:0/2259489942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2112f0 0x563fba5f1380 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6644105350 tx=0x7f66440f10d0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.375+0000 7f6661dec740 1 -- 172.21.15.16:0/2259489942 shutdown_connections 2024-09-21T03:50:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.375+0000 7f6661dec740 1 --2- 172.21.15.16:0/2259489942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fba23a1b0 0x563fba5f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.375+0000 7f6661dec740 1 --2- 172.21.15.16:0/2259489942 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x563fba1ce1c0 0x563fba6651e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.375+0000 7f6661dec740 1 --2- 172.21.15.16:0/2259489942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fba2194d0 0x563fba5f18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.375+0000 7f6661dec740 1 --2- 172.21.15.16:0/2259489942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fba2112f0 0x563fba5f1380 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.375+0000 7f6661dec740 1 -- 172.21.15.16:0/2259489942 >> 172.21.15.16:0/2259489942 conn(0x563fba22e300 msgr2=0x563fba5dc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.375+0000 7f6641600640 0 client.26021 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.375+0000 7f6661dec740 1 -- 172.21.15.16:0/2259489942 shutdown_connections 2024-09-21T03:50:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.375+0000 7f6661dec740 1 -- 172.21.15.16:0/2259489942 wait complete. 2024-09-21T03:50:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.376+0000 7f6661dec740 20 client.26021 trim_cache size 0 max 16384 2024-09-21T03:50:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:50:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:50:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:50:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:50:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.377+0000 7f665ac00640 1 -- 172.21.15.16:0/446494203 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f66380b6590 msgr2=0x7f66380b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.377+0000 7f665ac00640 1 --2- 172.21.15.16:0/446494203 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f66380b6590 0x7f66380b8aa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f664c003bc0 tx=0x7f664c009500 comp rx=0 tx=0).stop 2024-09-21T03:50:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.377+0000 7f665ac00640 1 -- 172.21.15.16:0/446494203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6654075cf0 msgr2=0x7f66541a0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.377+0000 7f665ac00640 1 --2- 172.21.15.16:0/446494203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6654075cf0 0x7f66541a0540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6644006010 tx=0x7f66440042f0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.377+0000 7f665ac00640 1 -- 172.21.15.16:0/446494203 shutdown_connections 2024-09-21T03:50:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.377+0000 7f665ac00640 1 --2- 172.21.15.16:0/446494203 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f66380b6590 0x7f66380b8aa0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.377+0000 7f665ac00640 1 --2- 172.21.15.16:0/446494203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f665410fe20 0x7f66541a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.377+0000 7f665ac00640 1 --2- 172.21.15.16:0/446494203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66540770f0 0x7f66541a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.377+0000 7f665ac00640 1 --2- 172.21.15.16:0/446494203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6654075cf0 0x7f66541a0540 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.377+0000 7f665ac00640 1 -- 172.21.15.16:0/446494203 >> 172.21.15.16:0/446494203 conn(0x7f66540fefe0 msgr2=0x7f6654103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.378+0000 7f665ac00640 1 -- 172.21.15.16:0/446494203 shutdown_connections 2024-09-21T03:50:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:17.378+0000 7f665ac00640 1 -- 172.21.15.16:0/446494203 wait complete. 2024-09-21T03:50:17.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:17 smithi016 ceph-mon[30375]: pgmap v537: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 245 MiB/s wr, 1.07k op/s 2024-09-21T03:50:17.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:17 smithi134 ceph-mon[35031]: pgmap v537: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 245 MiB/s wr, 1.07k op/s 2024-09-21T03:50:17.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:17 smithi123 ceph-mon[34673]: pgmap v537: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 245 MiB/s wr, 1.07k op/s 2024-09-21T03:50:19.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:19 smithi134 ceph-mon[35031]: pgmap v538: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 236 MiB/s wr, 990 op/s 2024-09-21T03:50:19.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:19 smithi123 ceph-mon[34673]: pgmap v538: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 236 MiB/s wr, 990 op/s 2024-09-21T03:50:19.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:19 smithi016 ceph-mon[30375]: pgmap v538: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 236 MiB/s wr, 990 op/s 2024-09-21T03:50:20.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:20 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:50:20.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:20 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:50:20.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:20 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:50:21.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:21 smithi134 ceph-mon[35031]: pgmap v539: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 217 MiB/s wr, 905 op/s 2024-09-21T03:50:21.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:21 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:50:21.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:21 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:50:21.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:21 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:50:21.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:21 smithi123 ceph-mon[34673]: pgmap v539: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 217 MiB/s wr, 905 op/s 2024-09-21T03:50:21.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:21 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:50:21.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:21 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:50:21.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:21 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:50:21.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:21 smithi016 ceph-mon[30375]: pgmap v539: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 217 MiB/s wr, 905 op/s 2024-09-21T03:50:21.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:21 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:50:21.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:21 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:50:21.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:21 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:50:23.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:23 smithi134 ceph-mon[35031]: pgmap v540: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 226 MiB/s wr, 956 op/s 2024-09-21T03:50:23.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:23 smithi123 ceph-mon[34673]: pgmap v540: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 226 MiB/s wr, 956 op/s 2024-09-21T03:50:23.669 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:23 smithi016 ceph-mon[30375]: pgmap v540: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 226 MiB/s wr, 956 op/s 2024-09-21T03:50:25.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:25 smithi134 ceph-mon[35031]: pgmap v541: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 231 MiB/s wr, 1.01k op/s 2024-09-21T03:50:25.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:25 smithi123 ceph-mon[34673]: pgmap v541: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 231 MiB/s wr, 1.01k op/s 2024-09-21T03:50:25.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:25 smithi016 ceph-mon[30375]: pgmap v541: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 231 MiB/s wr, 1.01k op/s 2024-09-21T03:50:26.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:26 smithi134 ceph-mon[35031]: pgmap v542: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 233 MiB/s wr, 1.05k op/s 2024-09-21T03:50:26.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:26 smithi123 ceph-mon[34673]: pgmap v542: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 233 MiB/s wr, 1.05k op/s 2024-09-21T03:50:26.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:26 smithi016 ceph-mon[30375]: pgmap v542: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 233 MiB/s wr, 1.05k op/s 2024-09-21T03:50:28.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:28 smithi134 ceph-mon[35031]: pgmap v543: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 228 MiB/s wr, 1.05k op/s 2024-09-21T03:50:28.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:28 smithi123 ceph-mon[34673]: pgmap v543: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 228 MiB/s wr, 1.05k op/s 2024-09-21T03:50:28.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:28 smithi016 ceph-mon[30375]: pgmap v543: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 228 MiB/s wr, 1.05k op/s 2024-09-21T03:50:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.798+0000 7f6315c00640 1 -- 172.21.15.16:0/2946990079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f631010a5a0 msgr2=0x7f631010ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.798+0000 7f6315c00640 1 --2- 172.21.15.16:0/2946990079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f631010a5a0 0x7f631010ca00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6308009900 tx=0x7f630802b6a0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.799+0000 7f6315c00640 1 -- 172.21.15.16:0/2946990079 shutdown_connections 2024-09-21T03:50:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.799+0000 7f6315c00640 1 --2- 172.21.15.16:0/2946990079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f631010fc60 0x7f6310112170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.799+0000 7f6315c00640 1 --2- 172.21.15.16:0/2946990079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f631010d040 0x7f631010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.799+0000 7f6315c00640 1 --2- 172.21.15.16:0/2946990079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f631010a5a0 0x7f631010ca00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.800+0000 7f6315c00640 1 -- 172.21.15.16:0/2946990079 >> 172.21.15.16:0/2946990079 conn(0x7f6310070640 msgr2=0x7f6310072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.800+0000 7f6315c00640 1 -- 172.21.15.16:0/2946990079 shutdown_connections 2024-09-21T03:50:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.800+0000 7f6315c00640 1 -- 172.21.15.16:0/2946990079 wait complete. 2024-09-21T03:50:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.801+0000 7f6315c00640 1 Processor -- start 2024-09-21T03:50:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.802+0000 7f6315c00640 1 -- start start 2024-09-21T03:50:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.802+0000 7f6315c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f631010a5a0 0x7f63101a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.802+0000 7f6315c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f631010d040 0x7f63101a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.803+0000 7f630fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f631010a5a0 0x7f63101a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.803+0000 7f630fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f631010a5a0 0x7f63101a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39154/0 (socket says 172.21.15.16:39154) 2024-09-21T03:50:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.803+0000 7f6315c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f631010fc60 0x7f63101a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.803+0000 7f6315c00640 1 -- 172.21.15.16:0/1368270882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6310115790 con 0x7f631010d040 2024-09-21T03:50:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.803+0000 7f630fe00640 1 -- 172.21.15.16:0/1368270882 learned_addr learned my addr 172.21.15.16:0/1368270882 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.803+0000 7f6315c00640 1 -- 172.21.15.16:0/1368270882 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6310115610 con 0x7f631010fc60 2024-09-21T03:50:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.803+0000 7f6315c00640 1 -- 172.21.15.16:0/1368270882 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6310115910 con 0x7f631010a5a0 2024-09-21T03:50:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.803+0000 7f630f400640 1 --2- 172.21.15.16:0/1368270882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f631010d040 0x7f63101a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.803+0000 7f6315200640 1 --2- 172.21.15.16:0/1368270882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f631010fc60 0x7f63101a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.803+0000 7f630fe00640 1 -- 172.21.15.16:0/1368270882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f631010fc60 msgr2=0x7f63101a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.803+0000 7f630fe00640 1 --2- 172.21.15.16:0/1368270882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f631010fc60 0x7f63101a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.803+0000 7f630fe00640 1 -- 172.21.15.16:0/1368270882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f631010d040 msgr2=0x7f63101a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.803+0000 7f630fe00640 1 --2- 172.21.15.16:0/1368270882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f631010d040 0x7f63101a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.803+0000 7f630fe00640 1 -- 172.21.15.16:0/1368270882 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63101a1de0 con 0x7f631010a5a0 2024-09-21T03:50:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.803+0000 7f630f400640 1 --2- 172.21.15.16:0/1368270882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f631010d040 0x7f63101a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.804+0000 7f630fe00640 1 --2- 172.21.15.16:0/1368270882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f631010a5a0 0x7f63101a03a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f630802bc90 tx=0x7f6308032630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.804+0000 7f630cc00640 1 -- 172.21.15.16:0/1368270882 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308050070 con 0x7f631010a5a0 2024-09-21T03:50:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.804+0000 7f6315c00640 1 -- 172.21.15.16:0/1368270882 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63101a2070 con 0x7f631010a5a0 2024-09-21T03:50:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.804+0000 7f6315c00640 1 -- 172.21.15.16:0/1368270882 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63101a87c0 con 0x7f631010a5a0 2024-09-21T03:50:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.804+0000 7f630cc00640 1 -- 172.21.15.16:0/1368270882 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6308040e30 con 0x7f631010a5a0 2024-09-21T03:50:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.804+0000 7f630cc00640 1 -- 172.21.15.16:0/1368270882 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308004040 con 0x7f631010a5a0 2024-09-21T03:50:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.808+0000 7f631e281740 1 Processor -- start 2024-09-21T03:50:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.808+0000 7f631e281740 1 -- start start 2024-09-21T03:50:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.809+0000 7f631e281740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5637271b0 0x55e563adf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.809+0000 7f631e281740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5636fe2f0 0x55e563adf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.809+0000 7f631e281740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637064d0 0x55e563ae0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.809+0000 7f631e281740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e563a416b0 con 0x55e5637064d0 2024-09-21T03:50:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.809+0000 7f631e281740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e563a30ae0 con 0x55e5637271b0 2024-09-21T03:50:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.809+0000 7f631e281740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e563a348f0 con 0x55e5636fe2f0 2024-09-21T03:50:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.809+0000 7f630cc00640 1 -- 172.21.15.16:0/1368270882 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6308053070 con 0x7f631010a5a0 2024-09-21T03:50:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.810+0000 7f630fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5636fe2f0 0x55e563adf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.810+0000 7f630f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637064d0 0x55e563ae0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.810+0000 7f630fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5636fe2f0 0x55e563adf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39166/0 (socket says 172.21.15.16:39166) 2024-09-21T03:50:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.810+0000 7f630fe00640 1 -- 172.21.15.16:0/3523834018 learned_addr learned my addr 172.21.15.16:0/3523834018 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.810+0000 7f6315200640 1 --2- 172.21.15.16:0/3523834018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5637271b0 0x55e563adf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.811+0000 7f630fe00640 1 -- 172.21.15.16:0/3523834018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5637271b0 msgr2=0x55e563adf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.811+0000 7f630fe00640 1 --2- 172.21.15.16:0/3523834018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5637271b0 0x55e563adf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.811+0000 7f630fe00640 1 -- 172.21.15.16:0/3523834018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637064d0 msgr2=0x55e563ae0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.811+0000 7f630fe00640 1 --2- 172.21.15.16:0/3523834018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637064d0 0x55e563ae0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.811+0000 7f630fe00640 1 -- 172.21.15.16:0/3523834018 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5639fd780 con 0x55e5636fe2f0 2024-09-21T03:50:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.812+0000 7f630fe00640 1 --2- 172.21.15.16:0/3523834018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5636fe2f0 0x55e563adf930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f630810dc20 tx=0x7f630810dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b61d8ee344ac8042 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.813+0000 7f6305600640 1 -- 172.21.15.16:0/3523834018 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308116070 con 0x55e5636fe2f0 2024-09-21T03:50:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.813+0000 7f6305600640 1 -- 172.21.15.16:0/3523834018 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f630810fd30 con 0x55e5636fe2f0 2024-09-21T03:50:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.813+0000 7f6305600640 1 -- 172.21.15.16:0/3523834018 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308116070 con 0x55e5636fe2f0 2024-09-21T03:50:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.813+0000 7f630f400640 1 --2- 172.21.15.16:0/3523834018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637064d0 0x55e563ae0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.813+0000 7f630cc00640 1 --2- 172.21.15.16:0/1368270882 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f62f40b65f0 0x7f62f40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.814+0000 7f630cc00640 1 -- 172.21.15.16:0/1368270882 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f6308107710 con 0x7f631010a5a0 2024-09-21T03:50:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.815+0000 7f6315200640 1 --2- 172.21.15.16:0/1368270882 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f62f40b65f0 0x7f62f40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.815+0000 7f631e281740 1 -- 172.21.15.16:0/3523834018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5636fe2f0 msgr2=0x55e563adf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.815+0000 7f631e281740 1 --2- 172.21.15.16:0/3523834018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5636fe2f0 0x55e563adf930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f630810dc20 tx=0x7f630810dd10 comp rx=0 tx=0).stop 2024-09-21T03:50:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.818+0000 7f631e281740 1 -- 172.21.15.16:0/3523834018 shutdown_connections 2024-09-21T03:50:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.818+0000 7f631e281740 1 --2- 172.21.15.16:0/3523834018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637064d0 0x55e563ae0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.818+0000 7f631e281740 1 --2- 172.21.15.16:0/3523834018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5636fe2f0 0x55e563adf930 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.818+0000 7f631e281740 1 --2- 172.21.15.16:0/3523834018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5637271b0 0x55e563adf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.818+0000 7f631e281740 1 -- 172.21.15.16:0/3523834018 >> 172.21.15.16:0/3523834018 conn(0x55e56371b300 msgr2=0x55e563ac9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.819+0000 7f6315200640 1 --2- 172.21.15.16:0/1368270882 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f62f40b65f0 0x7f62f40b8b00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00b710 tx=0x7f62fc00b2e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.819+0000 7f631e281740 1 -- 172.21.15.16:0/3523834018 shutdown_connections 2024-09-21T03:50:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.822+0000 7f631e281740 1 -- 172.21.15.16:0/3523834018 wait complete. 2024-09-21T03:50:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.823+0000 7f631e281740 1 -- start start 2024-09-21T03:50:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.823+0000 7f631e281740 1 Processor -- start 2024-09-21T03:50:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.825+0000 7f631e281740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5636fe2f0 0x55e563ae8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.825+0000 7f631e281740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637064d0 0x55e563ae8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.825+0000 7f631e281740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5637271b0 0x55e563ae9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.825+0000 7f631e281740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e563968ab0 con 0x55e5637064d0 2024-09-21T03:50:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.826+0000 7f630f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637064d0 0x55e563ae8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.826+0000 7f6315200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5637271b0 0x55e563ae9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.826+0000 7f630f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637064d0 0x55e563ae8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36218/0 (socket says 172.21.15.16:36218) 2024-09-21T03:50:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.826+0000 7f630f400640 1 -- 172.21.15.16:0/951385439 learned_addr learned my addr 172.21.15.16:0/951385439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.826+0000 7f631e281740 1 -- 172.21.15.16:0/951385439 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e5639d7570 con 0x55e5636fe2f0 2024-09-21T03:50:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.826+0000 7f631e281740 1 -- 172.21.15.16:0/951385439 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e563ae0d60 con 0x55e5637271b0 2024-09-21T03:50:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.826+0000 7f630fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5636fe2f0 0x55e563ae8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.826+0000 7f631e281740 10 client.0 _ll_register_callbacks cb 0x55e5639e6100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.827+0000 7f6315200640 1 -- 172.21.15.16:0/951385439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5636fe2f0 msgr2=0x55e563ae8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.827+0000 7f6315200640 1 --2- 172.21.15.16:0/951385439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5636fe2f0 0x55e563ae8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.827+0000 7f6315200640 1 -- 172.21.15.16:0/951385439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637064d0 msgr2=0x55e563ae8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.827+0000 7f6315200640 1 --2- 172.21.15.16:0/951385439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637064d0 0x55e563ae8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.827+0000 7f6315200640 1 -- 172.21.15.16:0/951385439 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e563792790 con 0x55e5637271b0 2024-09-21T03:50:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.827+0000 7f630f400640 1 --2- 172.21.15.16:0/951385439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637064d0 0x55e563ae8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.830+0000 7f6315200640 1 --2- 172.21.15.16:0/951385439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5637271b0 0x55e563ae9830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00bfa0 tx=0x7f62fc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74ad31aada7c9c5b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.830+0000 7f62fa000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:50:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.838+0000 7f62fa000640 1 -- 172.21.15.16:0/951385439 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc02c060 con 0x55e5637271b0 2024-09-21T03:50:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.841+0000 7f62fa000640 1 -- 172.21.15.16:0/951385439 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f62fc017030 con 0x55e5637271b0 2024-09-21T03:50:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.842+0000 7f631e281740 1 -- 172.21.15.16:0/951385439 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f630810fd30 con 0x55e5637271b0 2024-09-21T03:50:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.846+0000 7f62fa000640 1 -- 172.21.15.16:0/951385439 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f62fc01fc50 con 0x55e5637271b0 2024-09-21T03:50:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.846+0000 7f631e281740 10 client.25981 fetch_fsmap learned FSMap version 30 2024-09-21T03:50:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.846+0000 7f631e281740 1 -- 172.21.15.16:0/951385439 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e56380b620 con 0x55e5637271b0 2024-09-21T03:50:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.847+0000 7f62fa000640 1 -- 172.21.15.16:0/951385439 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f630810fd30 con 0x55e5637271b0 2024-09-21T03:50:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.848+0000 7f62fa000640 1 -- 172.21.15.16:0/951385439 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f62fc011440 con 0x55e5637271b0 2024-09-21T03:50:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.848+0000 7f631e281740 10 client.25981 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:50:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.848+0000 7f631e281740 10 client.25981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.848+0000 7f631e281740 20 client.25981 populate_metadata read hostname 'smithi016' 2024-09-21T03:50:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.848+0000 7f631e281740 1 --2- 172.21.15.16:0/951385439 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e5636e5d80 0x55e563af5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.849+0000 7f631e281740 4 client.25981 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.850+0000 7f631e281740 1 -- 172.21.15.16:0/951385439 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e563a43600 con 0x55e5636e5d80 2024-09-21T03:50:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.852+0000 7f630fe00640 1 --2- 172.21.15.16:0/951385439 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e5636e5d80 0x55e563af5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.852+0000 7f630fe00640 1 --2- 172.21.15.16:0/951385439 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e5636e5d80 0x55e563af5dc0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4615ffc6043f8fb9 server_cookie=72a7535244030b33 in_seq=0 out_seq=0 2024-09-21T03:50:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.852+0000 7f62fa000640 10 client.25981 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.853+0000 7f62fa000640 1 -- 172.21.15.16:0/951385439 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e563a43600 con 0x55e5636e5d80 2024-09-21T03:50:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.853+0000 7f62fa000640 10 client.25981 handle_command_reply: tid=0 2024-09-21T03:50:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.853+0000 7f631e281740 1 client.25981 shutdown 2024-09-21T03:50:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.854+0000 7f631e281740 1 -- 172.21.15.16:0/951385439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5637271b0 msgr2=0x55e563ae9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.854+0000 7f631e281740 1 --2- 172.21.15.16:0/951385439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5637271b0 0x55e563ae9830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00bfa0 tx=0x7f62fc004730 comp rx=0 tx=0).stop 2024-09-21T03:50:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.854+0000 7f631e281740 1 -- 172.21.15.16:0/951385439 shutdown_connections 2024-09-21T03:50:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.854+0000 7f631e281740 1 --2- 172.21.15.16:0/951385439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5637271b0 0x55e563ae9830 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.854+0000 7f631e281740 1 --2- 172.21.15.16:0/951385439 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e5636e5d80 0x55e563af5dc0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.855+0000 7f631e281740 1 --2- 172.21.15.16:0/951385439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637064d0 0x55e563ae8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.855+0000 7f631e281740 1 --2- 172.21.15.16:0/951385439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5636fe2f0 0x55e563ae8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.855+0000 7f631e281740 1 -- 172.21.15.16:0/951385439 >> 172.21.15.16:0/951385439 conn(0x55e56371b300 msgr2=0x55e563ac8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.855+0000 7f62fa000640 0 client.25981 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.855+0000 7f631e281740 1 -- 172.21.15.16:0/951385439 shutdown_connections 2024-09-21T03:50:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.855+0000 7f631e281740 1 -- 172.21.15.16:0/951385439 wait complete. 2024-09-21T03:50:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.856+0000 7f631e281740 20 client.25981 trim_cache size 0 max 16384 2024-09-21T03:50:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.900+0000 7f631e281740 1 Processor -- start 2024-09-21T03:50:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.901+0000 7f631e281740 1 -- start start 2024-09-21T03:50:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.902+0000 7f631e281740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5636fe2f0 0x55e563ac4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.902+0000 7f631e281740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5637064d0 0x55e563ae7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.902+0000 7f631e281740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637271b0 0x55e563ae7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.902+0000 7f631e281740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62f000fcc0 con 0x55e5637271b0 2024-09-21T03:50:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.902+0000 7f631e281740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0011800 con 0x55e5636fe2f0 2024-09-21T03:50:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.902+0000 7f631e281740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0010390 con 0x55e5637064d0 2024-09-21T03:50:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.903+0000 7f6315200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637271b0 0x55e563ae7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.903+0000 7f630fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5636fe2f0 0x55e563ac4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.903+0000 7f630fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5636fe2f0 0x55e563ac4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34654/0 (socket says 172.21.15.16:34654) 2024-09-21T03:50:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.903+0000 7f630fe00640 1 -- 172.21.15.16:0/3936720424 learned_addr learned my addr 172.21.15.16:0/3936720424 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.904+0000 7f6315200640 1 -- 172.21.15.16:0/3936720424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5637064d0 msgr2=0x55e563ae7800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:50:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.904+0000 7f6315200640 1 --2- 172.21.15.16:0/3936720424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5637064d0 0x55e563ae7800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.904+0000 7f6315200640 1 -- 172.21.15.16:0/3936720424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5636fe2f0 msgr2=0x55e563ac4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:30.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:30 smithi123 ceph-mon[34673]: pgmap v544: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 222 MiB/s wr, 1.04k op/s 2024-09-21T03:50:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.905+0000 7f6315200640 1 --2- 172.21.15.16:0/3936720424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5636fe2f0 0x55e563ac4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.905+0000 7f6315200640 1 -- 172.21.15.16:0/3936720424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e563ae6810 con 0x55e5637271b0 2024-09-21T03:50:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.906+0000 7f6315200640 1 --2- 172.21.15.16:0/3936720424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637271b0 0x55e563ae7d40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f62fc004010 tx=0x7f62fc01f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a139b0f0a03ae996 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.906+0000 7f6304c00640 1 -- 172.21.15.16:0/3936720424 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc036080 con 0x55e5637271b0 2024-09-21T03:50:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.906+0000 7f6304c00640 1 -- 172.21.15.16:0/3936720424 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62fc011440 con 0x55e5637271b0 2024-09-21T03:50:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.906+0000 7f6304c00640 1 -- 172.21.15.16:0/3936720424 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc031420 con 0x55e5637271b0 2024-09-21T03:50:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.907+0000 7f631e281740 1 -- 172.21.15.16:0/3936720424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637271b0 msgr2=0x55e563ae7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.908+0000 7f631e281740 1 --2- 172.21.15.16:0/3936720424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637271b0 0x55e563ae7d40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f62fc004010 tx=0x7f62fc01f880 comp rx=0 tx=0).stop 2024-09-21T03:50:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.908+0000 7f631e281740 1 -- 172.21.15.16:0/3936720424 shutdown_connections 2024-09-21T03:50:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.908+0000 7f631e281740 1 --2- 172.21.15.16:0/3936720424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5637271b0 0x55e563ae7d40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.908+0000 7f631e281740 1 --2- 172.21.15.16:0/3936720424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5637064d0 0x55e563ae7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.908+0000 7f631e281740 1 --2- 172.21.15.16:0/3936720424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5636fe2f0 0x55e563ac4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.908+0000 7f631e281740 1 -- 172.21.15.16:0/3936720424 >> 172.21.15.16:0/3936720424 conn(0x55e56371b300 msgr2=0x55e563ae6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.909+0000 7f631e281740 1 -- 172.21.15.16:0/3936720424 shutdown_connections 2024-09-21T03:50:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.909+0000 7f631e281740 1 -- 172.21.15.16:0/3936720424 wait complete. 2024-09-21T03:50:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.909+0000 7f631e281740 1 -- start start 2024-09-21T03:50:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.910+0000 7f631e281740 1 Processor -- start 2024-09-21T03:50:30.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:30 smithi016 ceph-mon[30375]: pgmap v544: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 222 MiB/s wr, 1.04k op/s 2024-09-21T03:50:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.911+0000 7f631e281740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5636fe2f0 0x55e563af0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.911+0000 7f631e281740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5637064d0 0x55e563aeee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.911+0000 7f631e281740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5637271b0 0x55e563aef380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.911+0000 7f630f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5637064d0 0x55e563aeee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.912+0000 7f630f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5637064d0 0x55e563aeee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34658/0 (socket says 172.21.15.16:34658) 2024-09-21T03:50:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.915+0000 7f6315200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5637271b0 0x55e563aef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.915+0000 7f6315200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5637271b0 0x55e563aef380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39178/0 (socket says 172.21.15.16:39178) 2024-09-21T03:50:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.915+0000 7f6315200640 1 -- 172.21.15.16:0/1066022862 learned_addr learned my addr 172.21.15.16:0/1066022862 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.915+0000 7f630fe00640 1 --2- 172.21.15.16:0/1066022862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5636fe2f0 0x55e563af0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.916+0000 7f631e281740 1 -- 172.21.15.16:0/1066022862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0011130 con 0x55e5636fe2f0 2024-09-21T03:50:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.917+0000 7f631e281740 1 -- 172.21.15.16:0/1066022862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0010a60 con 0x55e5637064d0 2024-09-21T03:50:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.917+0000 7f631e281740 1 -- 172.21.15.16:0/1066022862 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62f000dca0 con 0x55e5637271b0 2024-09-21T03:50:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.917+0000 7f631e281740 10 client.0 _ll_register_callbacks cb 0x7f62fc004390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.918+0000 7f6315200640 1 -- 172.21.15.16:0/1066022862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5637064d0 msgr2=0x55e563aeee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.919+0000 7f6315200640 1 --2- 172.21.15.16:0/1066022862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5637064d0 0x55e563aeee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.919+0000 7f6315200640 1 -- 172.21.15.16:0/1066022862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5636fe2f0 msgr2=0x55e563af0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.919+0000 7f6315200640 1 --2- 172.21.15.16:0/1066022862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5636fe2f0 0x55e563af0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.919+0000 7f6315200640 1 -- 172.21.15.16:0/1066022862 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e563aef8c0 con 0x55e5637271b0 2024-09-21T03:50:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.919+0000 7f630f400640 1 --2- 172.21.15.16:0/1066022862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5637064d0 0x55e563aeee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.919+0000 7f62fa000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:50:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.920+0000 7f6315200640 1 --2- 172.21.15.16:0/1066022862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5637271b0 0x55e563aef380 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f62fc003610 tx=0x7f62fc0108f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ccab4ff0d376f8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.920+0000 7f62fa000640 1 -- 172.21.15.16:0/1066022862 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc03f060 con 0x55e5637271b0 2024-09-21T03:50:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.920+0000 7f62fa000640 1 -- 172.21.15.16:0/1066022862 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f62fc005030 con 0x55e5637271b0 2024-09-21T03:50:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.921+0000 7f631e281740 1 -- 172.21.15.16:0/1066022862 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62fc011440 con 0x55e5637271b0 2024-09-21T03:50:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.923+0000 7f62fa000640 1 -- 172.21.15.16:0/1066022862 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f62fc046050 con 0x55e5637271b0 2024-09-21T03:50:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.923+0000 7f631e281740 10 client.25984 fetch_fsmap learned FSMap version 30 2024-09-21T03:50:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.923+0000 7f631e281740 1 -- 172.21.15.16:0/1066022862 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e563aee6a0 con 0x55e5637271b0 2024-09-21T03:50:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.924+0000 7f62fa000640 1 -- 172.21.15.16:0/1066022862 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62fc011440 con 0x55e5637271b0 2024-09-21T03:50:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.924+0000 7f62fa000640 1 -- 172.21.15.16:0/1066022862 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f62fc004a40 con 0x55e5637271b0 2024-09-21T03:50:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.924+0000 7f631e281740 10 client.25984 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:50:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.924+0000 7f631e281740 10 client.25984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.924+0000 7f631e281740 20 client.25984 populate_metadata read hostname 'smithi016' 2024-09-21T03:50:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.924+0000 7f631e281740 1 --2- 172.21.15.16:0/1066022862 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e5636bb1c0 0x55e563b62dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.924+0000 7f631e281740 4 client.25984 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:50:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.924+0000 7f631e281740 1 -- 172.21.15.16:0/1066022862 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e563a3f0b0 con 0x55e5636bb1c0 2024-09-21T03:50:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.926+0000 7f630fe00640 1 --2- 172.21.15.16:0/1066022862 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e5636bb1c0 0x55e563b62dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.928+0000 7f62fa000640 10 client.25984 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.928+0000 7f630fe00640 1 --2- 172.21.15.16:0/1066022862 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e5636bb1c0 0x55e563b62dc0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b08b5056526d388 server_cookie=a437c39481363c10 in_seq=0 out_seq=0 2024-09-21T03:50:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.946+0000 7f62fa000640 1 -- 172.21.15.16:0/1066022862 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e563a3f0b0 con 0x55e5636bb1c0 2024-09-21T03:50:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.946+0000 7f62fa000640 10 client.25984 handle_command_reply: tid=0 2024-09-21T03:50:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.946+0000 7f631e281740 1 client.25984 shutdown 2024-09-21T03:50:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.946+0000 7f631e281740 1 -- 172.21.15.16:0/1066022862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5637271b0 msgr2=0x55e563aef380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.946+0000 7f631e281740 1 --2- 172.21.15.16:0/1066022862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5637271b0 0x55e563aef380 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f62fc003610 tx=0x7f62fc0108f0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.947+0000 7f631e281740 1 -- 172.21.15.16:0/1066022862 shutdown_connections 2024-09-21T03:50:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.947+0000 7f631e281740 1 --2- 172.21.15.16:0/1066022862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5637271b0 0x55e563aef380 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.947+0000 7f631e281740 1 --2- 172.21.15.16:0/1066022862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5637064d0 0x55e563aeee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.947+0000 7f631e281740 1 --2- 172.21.15.16:0/1066022862 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e5636bb1c0 0x55e563b62dc0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.947+0000 7f631e281740 1 --2- 172.21.15.16:0/1066022862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5636fe2f0 0x55e563af0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.947+0000 7f631e281740 1 -- 172.21.15.16:0/1066022862 >> 172.21.15.16:0/1066022862 conn(0x55e56371b300 msgr2=0x55e563ac4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.947+0000 7f62fa000640 0 client.25984 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.947+0000 7f631e281740 1 -- 172.21.15.16:0/1066022862 shutdown_connections 2024-09-21T03:50:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.947+0000 7f631e281740 1 -- 172.21.15.16:0/1066022862 wait complete. 2024-09-21T03:50:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.947+0000 7f631e281740 20 client.25984 trim_cache size 0 max 16384 2024-09-21T03:50:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:50:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:50:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:50:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:50:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.948+0000 7f6315c00640 1 -- 172.21.15.16:0/1368270882 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f62f40b65f0 msgr2=0x7f62f40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.948+0000 7f6315c00640 1 --2- 172.21.15.16:0/1368270882 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f62f40b65f0 0x7f62f40b8b00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00b710 tx=0x7f62fc00b2e0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.948+0000 7f6315c00640 1 -- 172.21.15.16:0/1368270882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f631010a5a0 msgr2=0x7f63101a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.948+0000 7f6315c00640 1 --2- 172.21.15.16:0/1368270882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f631010a5a0 0x7f63101a03a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f630802bc90 tx=0x7f6308032630 comp rx=0 tx=0).stop 2024-09-21T03:50:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.950+0000 7f6315c00640 1 -- 172.21.15.16:0/1368270882 shutdown_connections 2024-09-21T03:50:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.950+0000 7f6315c00640 1 --2- 172.21.15.16:0/1368270882 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f62f40b65f0 0x7f62f40b8b00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.950+0000 7f6315c00640 1 --2- 172.21.15.16:0/1368270882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f631010fc60 0x7f63101a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.950+0000 7f6315c00640 1 --2- 172.21.15.16:0/1368270882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f631010d040 0x7f63101a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.950+0000 7f6315c00640 1 --2- 172.21.15.16:0/1368270882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f631010a5a0 0x7f63101a03a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.950+0000 7f6315c00640 1 -- 172.21.15.16:0/1368270882 >> 172.21.15.16:0/1368270882 conn(0x7f6310070640 msgr2=0x7f631010b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.950+0000 7f6315c00640 1 -- 172.21.15.16:0/1368270882 shutdown_connections 2024-09-21T03:50:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:30.950+0000 7f6315c00640 1 -- 172.21.15.16:0/1368270882 wait complete. 2024-09-21T03:50:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:30 smithi134 ceph-mon[35031]: pgmap v544: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 222 MiB/s wr, 1.04k op/s 2024-09-21T03:50:32.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:32 smithi123 ceph-mon[34673]: pgmap v545: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 240 MiB/s wr, 1.11k op/s 2024-09-21T03:50:32.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:32 smithi016 ceph-mon[30375]: pgmap v545: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 240 MiB/s wr, 1.11k op/s 2024-09-21T03:50:33.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:32 smithi134 ceph-mon[35031]: pgmap v545: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 240 MiB/s wr, 1.11k op/s 2024-09-21T03:50:34.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:34 smithi123 ceph-mon[34673]: pgmap v546: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 242 MiB/s wr, 1.11k op/s 2024-09-21T03:50:34.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:34 smithi016 ceph-mon[30375]: pgmap v546: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 242 MiB/s wr, 1.11k op/s 2024-09-21T03:50:35.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:34 smithi134 ceph-mon[35031]: pgmap v546: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 242 MiB/s wr, 1.11k op/s 2024-09-21T03:50:36.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:36 smithi123 ceph-mon[34673]: pgmap v547: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 252 MiB/s wr, 1.14k op/s 2024-09-21T03:50:36.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:36 smithi016 ceph-mon[30375]: pgmap v547: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 252 MiB/s wr, 1.14k op/s 2024-09-21T03:50:37.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:36 smithi134 ceph-mon[35031]: pgmap v547: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 252 MiB/s wr, 1.14k op/s 2024-09-21T03:50:38.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:38 smithi123 ceph-mon[34673]: pgmap v548: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 244 MiB/s wr, 1.09k op/s 2024-09-21T03:50:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:38 smithi016 ceph-mon[30375]: pgmap v548: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 244 MiB/s wr, 1.09k op/s 2024-09-21T03:50:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:38 smithi134 ceph-mon[35031]: pgmap v548: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 244 MiB/s wr, 1.09k op/s 2024-09-21T03:50:40.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:40 smithi123 ceph-mon[34673]: pgmap v549: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 233 MiB/s wr, 1.03k op/s 2024-09-21T03:50:40.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:40 smithi016 ceph-mon[30375]: pgmap v549: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 233 MiB/s wr, 1.03k op/s 2024-09-21T03:50:41.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:40 smithi134 ceph-mon[35031]: pgmap v549: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 233 MiB/s wr, 1.03k op/s 2024-09-21T03:50:42.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:42 smithi016 ceph-mon[30375]: pgmap v550: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 237 MiB/s wr, 1.04k op/s 2024-09-21T03:50:43.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:42 smithi134 ceph-mon[35031]: pgmap v550: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 237 MiB/s wr, 1.04k op/s 2024-09-21T03:50:43.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:42 smithi123 ceph-mon[34673]: pgmap v550: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 237 MiB/s wr, 1.04k op/s 2024-09-21T03:50:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.353+0000 7f5d2aa00640 1 -- 172.21.15.16:0/84242512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d2410a3c0 msgr2=0x7f5d2410a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.353+0000 7f5d2aa00640 1 --2- 172.21.15.16:0/84242512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d2410a3c0 0x7f5d2410a7d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009be0 tx=0x7f5d1802a6c0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.354+0000 7f5d2aa00640 1 -- 172.21.15.16:0/84242512 shutdown_connections 2024-09-21T03:50:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.354+0000 7f5d2aa00640 1 --2- 172.21.15.16:0/84242512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d24111c40 0x7f5d24114060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.354+0000 7f5d2aa00640 1 --2- 172.21.15.16:0/84242512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2410ad10 0x7f5d24111700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.354+0000 7f5d2aa00640 1 --2- 172.21.15.16:0/84242512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d2410a3c0 0x7f5d2410a7d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.354+0000 7f5d2aa00640 1 -- 172.21.15.16:0/84242512 >> 172.21.15.16:0/84242512 conn(0x7f5d24070640 msgr2=0x7f5d24072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.360+0000 7f5d2aa00640 1 -- 172.21.15.16:0/84242512 shutdown_connections 2024-09-21T03:50:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.361+0000 7f5d2aa00640 1 -- 172.21.15.16:0/84242512 wait complete. 2024-09-21T03:50:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.363+0000 7f5d2aa00640 1 Processor -- start 2024-09-21T03:50:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.367+0000 7f5d2aa00640 1 -- start start 2024-09-21T03:50:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.367+0000 7f5d2aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d2410a3c0 0x7f5d2410cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.367+0000 7f5d2aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2410ad10 0x7f5d2410b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.367+0000 7f5d2aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d24111c40 0x7f5d2410bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.367+0000 7f5d2aa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d24117960 con 0x7f5d2410ad10 2024-09-21T03:50:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.367+0000 7f5d2aa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d241177e0 con 0x7f5d24111c40 2024-09-21T03:50:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.367+0000 7f5d2aa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d24117ae0 con 0x7f5d2410a3c0 2024-09-21T03:50:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.368+0000 7f5d2a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d24111c40 0x7f5d2410bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.368+0000 7f5d2a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d24111c40 0x7f5d2410bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:59798/0 (socket says 172.21.15.16:59798) 2024-09-21T03:50:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.368+0000 7f5d2a000640 1 -- 172.21.15.16:0/1812056250 learned_addr learned my addr 172.21.15.16:0/1812056250 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.369+0000 7f5d28c00640 1 --2- 172.21.15.16:0/1812056250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2410ad10 0x7f5d2410b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.371+0000 7f5d2a000640 1 -- 172.21.15.16:0/1812056250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d2410a3c0 msgr2=0x7f5d2410cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.369+0000 7f5d29600640 1 --2- 172.21.15.16:0/1812056250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d2410a3c0 0x7f5d2410cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.371+0000 7f5d2a000640 1 --2- 172.21.15.16:0/1812056250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d2410a3c0 0x7f5d2410cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.371+0000 7f5d2a000640 1 -- 172.21.15.16:0/1812056250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2410ad10 msgr2=0x7f5d2410b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.372+0000 7f5d2a000640 1 --2- 172.21.15.16:0/1812056250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2410ad10 0x7f5d2410b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.372+0000 7f5d2a000640 1 -- 172.21.15.16:0/1812056250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d241a62f0 con 0x7f5d24111c40 2024-09-21T03:50:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.372+0000 7f5d29600640 1 --2- 172.21.15.16:0/1812056250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d2410a3c0 0x7f5d2410cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.372+0000 7f5d28c00640 1 --2- 172.21.15.16:0/1812056250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2410ad10 0x7f5d2410b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.372+0000 7f5d2a000640 1 --2- 172.21.15.16:0/1812056250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d24111c40 0x7f5d2410bb30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00e970 tx=0x7f5d1c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.372+0000 7f5d22000640 1 -- 172.21.15.16:0/1812056250 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00ccc0 con 0x7f5d24111c40 2024-09-21T03:50:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.372+0000 7f5d2aa00640 1 -- 172.21.15.16:0/1812056250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d241a65e0 con 0x7f5d24111c40 2024-09-21T03:50:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.372+0000 7f5d22000640 1 -- 172.21.15.16:0/1812056250 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d1c0054d0 con 0x7f5d24111c40 2024-09-21T03:50:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.372+0000 7f5d22000640 1 -- 172.21.15.16:0/1812056250 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c010430 con 0x7f5d24111c40 2024-09-21T03:50:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.373+0000 7f5d2aa00640 1 -- 172.21.15.16:0/1812056250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d241a6b30 con 0x7f5d24111c40 2024-09-21T03:50:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.384+0000 7f5d32fdd740 1 Processor -- start 2024-09-21T03:50:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.387+0000 7f5d22000640 1 -- 172.21.15.16:0/1812056250 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5d1c005020 con 0x7f5d24111c40 2024-09-21T03:50:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.391+0000 7f5d22000640 1 --2- 172.21.15.16:0/1812056250 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d0c0b65c0 0x7f5d0c0b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.391+0000 7f5d32fdd740 1 -- start start 2024-09-21T03:50:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.391+0000 7f5d32fdd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 0x55985ecb42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.391+0000 7f5d32fdd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecb4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.391+0000 7f5d32fdd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e8ec4d0 0x55985ecb5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.393+0000 7f5d28c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecb4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.393+0000 7f5d28c00640 1 --2- 172.21.15.16:0/1812056250 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d0c0b65c0 0x7f5d0c0b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.394+0000 7f5d32fdd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55985ec276b0 con 0x55985e8e42f0 2024-09-21T03:50:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.394+0000 7f5d32fdd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55985ec16ae0 con 0x55985e90d1b0 2024-09-21T03:50:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.394+0000 7f5d32fdd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55985ec1a8f0 con 0x55985e8ec4d0 2024-09-21T03:50:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.394+0000 7f5d29600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 0x55985ecb42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.394+0000 7f5d29600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 0x55985ecb42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:59808/0 (socket says 172.21.15.16:59808) 2024-09-21T03:50:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.394+0000 7f5d29600640 1 -- 172.21.15.16:0/12493138 learned_addr learned my addr 172.21.15.16:0/12493138 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.395+0000 7f5d29600640 1 -- 172.21.15.16:0/12493138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e8ec4d0 msgr2=0x55985ecb5470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:50:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.395+0000 7f5d29600640 1 --2- 172.21.15.16:0/12493138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e8ec4d0 0x55985ecb5470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.395+0000 7f5d29600640 1 -- 172.21.15.16:0/12493138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 msgr2=0x55985ecb4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.395+0000 7f5d29600640 1 --2- 172.21.15.16:0/12493138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecb4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.395+0000 7f5d29600640 1 -- 172.21.15.16:0/12493138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55985ebe3780 con 0x55985e90d1b0 2024-09-21T03:50:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.396+0000 7f5d22000640 1 -- 172.21.15.16:0/1812056250 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f5d1c0e7c10 con 0x7f5d24111c40 2024-09-21T03:50:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.396+0000 7f5d29600640 1 --2- 172.21.15.16:0/12493138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 0x55985ecb42b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d18000f80 tx=0x7f5d180303a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10d1459d731e0aee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.396+0000 7f5d16600640 1 -- 172.21.15.16:0/12493138 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18036070 con 0x55985e90d1b0 2024-09-21T03:50:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.397+0000 7f5d16600640 1 -- 172.21.15.16:0/12493138 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d18037040 con 0x55985e90d1b0 2024-09-21T03:50:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.397+0000 7f5d16600640 1 -- 172.21.15.16:0/12493138 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d180044d0 con 0x55985e90d1b0 2024-09-21T03:50:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.398+0000 7f5d32fdd740 1 -- 172.21.15.16:0/12493138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 msgr2=0x55985ecb42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.398+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/12493138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 0x55985ecb42b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d18000f80 tx=0x7f5d180303a0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.398+0000 7f5d28c00640 1 --2- 172.21.15.16:0/1812056250 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d0c0b65c0 0x7f5d0c0b8ad0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000ce40 tx=0x7f5d1000c390 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.398+0000 7f5d32fdd740 1 -- 172.21.15.16:0/12493138 shutdown_connections 2024-09-21T03:50:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.398+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/12493138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e8ec4d0 0x55985ecb5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.398+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/12493138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecb4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.398+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/12493138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 0x55985ecb42b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.398+0000 7f5d32fdd740 1 -- 172.21.15.16:0/12493138 >> 172.21.15.16:0/12493138 conn(0x55985e901300 msgr2=0x55985ecafd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.399+0000 7f5d32fdd740 1 -- 172.21.15.16:0/12493138 shutdown_connections 2024-09-21T03:50:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.399+0000 7f5d32fdd740 1 -- 172.21.15.16:0/12493138 wait complete. 2024-09-21T03:50:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.401+0000 7f5d32fdd740 1 -- start start 2024-09-21T03:50:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.401+0000 7f5d32fdd740 1 Processor -- start 2024-09-21T03:50:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.403+0000 7f5d32fdd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecbd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.403+0000 7f5d32fdd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e8ec4d0 0x55985ecbe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.403+0000 7f5d32fdd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 0x55985ecbeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.403+0000 7f5d32fdd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55985eb4eab0 con 0x55985e8e42f0 2024-09-21T03:50:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.403+0000 7f5d32fdd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55985ebbd570 con 0x55985e90d1b0 2024-09-21T03:50:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.403+0000 7f5d32fdd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55985ecb5f30 con 0x55985e8ec4d0 2024-09-21T03:50:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.403+0000 7f5d32fdd740 10 client.0 _ll_register_callbacks cb 0x55985ebcc100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.404+0000 7f5d28c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e8ec4d0 0x55985ecbe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.404+0000 7f5d28c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e8ec4d0 0x55985ecbe030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:46770/0 (socket says 172.21.15.16:46770) 2024-09-21T03:50:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.404+0000 7f5d28c00640 1 -- 172.21.15.16:0/2491355212 learned_addr learned my addr 172.21.15.16:0/2491355212 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.404+0000 7f5d29600640 1 --2- 172.21.15.16:0/2491355212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecbd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.405+0000 7f5d2a000640 1 --2- 172.21.15.16:0/2491355212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 0x55985ecbeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.405+0000 7f5d29600640 1 -- 172.21.15.16:0/2491355212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e8ec4d0 msgr2=0x55985ecbe030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.405+0000 7f5d29600640 1 --2- 172.21.15.16:0/2491355212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e8ec4d0 0x55985ecbe030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.405+0000 7f5d29600640 1 -- 172.21.15.16:0/2491355212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 msgr2=0x55985ecbeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.405+0000 7f5d29600640 1 --2- 172.21.15.16:0/2491355212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 0x55985ecbeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.405+0000 7f5d29600640 1 -- 172.21.15.16:0/2491355212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55985e978790 con 0x55985e8e42f0 2024-09-21T03:50:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.406+0000 7f5d29600640 1 --2- 172.21.15.16:0/2491355212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecbd970 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009be0 tx=0x7f5d18030ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d6533d4a8e4444 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.406+0000 7f5d0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:50:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.406+0000 7f5d0aa00640 1 -- 172.21.15.16:0/2491355212 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1804a070 con 0x55985e8e42f0 2024-09-21T03:50:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.406+0000 7f5d0aa00640 1 -- 172.21.15.16:0/2491355212 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f5d18003d80 con 0x55985e8e42f0 2024-09-21T03:50:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.406+0000 7f5d32fdd740 1 -- 172.21.15.16:0/2491355212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d18037040 con 0x55985e8e42f0 2024-09-21T03:50:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.407+0000 7f5d0aa00640 1 -- 172.21.15.16:0/2491355212 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d18058050 con 0x55985e8e42f0 2024-09-21T03:50:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.407+0000 7f5d32fdd740 10 client.15960 fetch_fsmap learned FSMap version 30 2024-09-21T03:50:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.407+0000 7f5d32fdd740 1 -- 172.21.15.16:0/2491355212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55985e9f1620 con 0x55985e8e42f0 2024-09-21T03:50:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.407+0000 7f5d0aa00640 1 -- 172.21.15.16:0/2491355212 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d18037040 con 0x55985e8e42f0 2024-09-21T03:50:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.408+0000 7f5d0aa00640 1 -- 172.21.15.16:0/2491355212 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5d1803d890 con 0x55985e8e42f0 2024-09-21T03:50:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.408+0000 7f5d32fdd740 10 client.15960 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:50:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.408+0000 7f5d32fdd740 10 client.15960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.408+0000 7f5d32fdd740 20 client.15960 populate_metadata read hostname 'smithi016' 2024-09-21T03:50:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.408+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/2491355212 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55985e8cbd80 0x55985eccac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.408+0000 7f5d32fdd740 4 client.15960 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.408+0000 7f5d32fdd740 1 -- 172.21.15.16:0/2491355212 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55985ec29600 con 0x55985e8cbd80 2024-09-21T03:50:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.409+0000 7f5d28c00640 1 --2- 172.21.15.16:0/2491355212 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55985e8cbd80 0x55985eccac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.411+0000 7f5d28c00640 1 --2- 172.21.15.16:0/2491355212 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55985e8cbd80 0x55985eccac80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a282c32a63ad9c06 server_cookie=a8c83a6dbfc9a678 in_seq=0 out_seq=0 2024-09-21T03:50:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.411+0000 7f5d0aa00640 10 client.15960 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.412+0000 7f5d0aa00640 1 -- 172.21.15.16:0/2491355212 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55985ec29600 con 0x55985e8cbd80 2024-09-21T03:50:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.412+0000 7f5d0aa00640 10 client.15960 handle_command_reply: tid=0 2024-09-21T03:50:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.412+0000 7f5d32fdd740 1 client.15960 shutdown 2024-09-21T03:50:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.413+0000 7f5d32fdd740 1 -- 172.21.15.16:0/2491355212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 msgr2=0x55985ecbd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.413+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/2491355212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecbd970 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009be0 tx=0x7f5d18030ca0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.413+0000 7f5d32fdd740 1 -- 172.21.15.16:0/2491355212 shutdown_connections 2024-09-21T03:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.413+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/2491355212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 0x55985ecbeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.413+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/2491355212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e8ec4d0 0x55985ecbe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.413+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/2491355212 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55985e8cbd80 0x55985eccac80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.413+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/2491355212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecbd970 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.413+0000 7f5d32fdd740 1 -- 172.21.15.16:0/2491355212 >> 172.21.15.16:0/2491355212 conn(0x55985e901300 msgr2=0x55985ecb4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.413+0000 7f5d0aa00640 0 client.15960 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.414+0000 7f5d32fdd740 1 -- 172.21.15.16:0/2491355212 shutdown_connections 2024-09-21T03:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.414+0000 7f5d32fdd740 1 -- 172.21.15.16:0/2491355212 wait complete. 2024-09-21T03:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.414+0000 7f5d32fdd740 20 client.15960 trim_cache size 0 max 16384 2024-09-21T03:50:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.436+0000 7f5d32fdd740 1 Processor -- start 2024-09-21T03:50:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.436+0000 7f5d32fdd740 1 -- start start 2024-09-21T03:50:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.437+0000 7f5d32fdd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecb4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.437+0000 7f5d32fdd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e8ec4d0 0x55985ecae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.437+0000 7f5d32fdd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 0x55985ecb7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.437+0000 7f5d32fdd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0400fbf0 con 0x55985e8e42f0 2024-09-21T03:50:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.437+0000 7f5d32fdd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d04011730 con 0x55985e90d1b0 2024-09-21T03:50:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.437+0000 7f5d32fdd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d040102c0 con 0x55985e8ec4d0 2024-09-21T03:50:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.437+0000 7f5d28c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e8ec4d0 0x55985ecae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.437+0000 7f5d28c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e8ec4d0 0x55985ecae360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:46782/0 (socket says 172.21.15.16:46782) 2024-09-21T03:50:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.437+0000 7f5d2a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 0x55985ecb7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.437+0000 7f5d28c00640 1 -- 172.21.15.16:0/538820520 learned_addr learned my addr 172.21.15.16:0/538820520 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.437+0000 7f5d29600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecb4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.438+0000 7f5d28c00640 1 -- 172.21.15.16:0/538820520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 msgr2=0x55985ecb7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.438+0000 7f5d28c00640 1 --2- 172.21.15.16:0/538820520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 0x55985ecb7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.438+0000 7f5d28c00640 1 -- 172.21.15.16:0/538820520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 msgr2=0x55985ecb4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.438+0000 7f5d28c00640 1 --2- 172.21.15.16:0/538820520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecb4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.438+0000 7f5d28c00640 1 -- 172.21.15.16:0/538820520 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55985ecb0470 con 0x55985e8ec4d0 2024-09-21T03:50:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.438+0000 7f5d2a000640 1 --2- 172.21.15.16:0/538820520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 0x55985ecb7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:50:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.438+0000 7f5d28c00640 1 --2- 172.21.15.16:0/538820520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e8ec4d0 0x55985ecae360 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5d10011c40 tx=0x7f5d10014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5112eb661dee973e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.438+0000 7f5d15c00640 1 -- 172.21.15.16:0/538820520 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10005020 con 0x55985e8ec4d0 2024-09-21T03:50:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.438+0000 7f5d15c00640 1 -- 172.21.15.16:0/538820520 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d10016070 con 0x55985e8ec4d0 2024-09-21T03:50:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.438+0000 7f5d15c00640 1 -- 172.21.15.16:0/538820520 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1001ce60 con 0x55985e8ec4d0 2024-09-21T03:50:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.440+0000 7f5d32fdd740 1 -- 172.21.15.16:0/538820520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e8ec4d0 msgr2=0x55985ecae360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.440+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/538820520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e8ec4d0 0x55985ecae360 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5d10011c40 tx=0x7f5d10014000 comp rx=0 tx=0).stop 2024-09-21T03:50:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.440+0000 7f5d32fdd740 1 -- 172.21.15.16:0/538820520 shutdown_connections 2024-09-21T03:50:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.440+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/538820520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e90d1b0 0x55985ecb7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.440+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/538820520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e8ec4d0 0x55985ecae360 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.440+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/538820520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecb4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.440+0000 7f5d32fdd740 1 -- 172.21.15.16:0/538820520 >> 172.21.15.16:0/538820520 conn(0x55985e901300 msgr2=0x55985ecb7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.440+0000 7f5d32fdd740 1 -- 172.21.15.16:0/538820520 shutdown_connections 2024-09-21T03:50:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.440+0000 7f5d32fdd740 1 -- 172.21.15.16:0/538820520 wait complete. 2024-09-21T03:50:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.441+0000 7f5d32fdd740 1 -- start start 2024-09-21T03:50:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.441+0000 7f5d32fdd740 1 Processor -- start 2024-09-21T03:50:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.441+0000 7f5d32fdd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecc5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.442+0000 7f5d32fdd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e8ec4d0 0x55985ecc12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.442+0000 7f5d32fdd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e90d1b0 0x55985ecc2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.442+0000 7f5d32fdd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d04011060 con 0x55985e8e42f0 2024-09-21T03:50:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.442+0000 7f5d32fdd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d04010990 con 0x55985e8ec4d0 2024-09-21T03:50:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.442+0000 7f5d32fdd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0400dbd0 con 0x55985e90d1b0 2024-09-21T03:50:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.442+0000 7f5d32fdd740 10 client.0 _ll_register_callbacks cb 0x7f5d1c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.442+0000 7f5d29600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecc5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.443+0000 7f5d28c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e8ec4d0 0x55985ecc12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.443+0000 7f5d28c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e8ec4d0 0x55985ecc12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:59836/0 (socket says 172.21.15.16:59836) 2024-09-21T03:50:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.443+0000 7f5d28c00640 1 -- 172.21.15.16:0/2380882267 learned_addr learned my addr 172.21.15.16:0/2380882267 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.445+0000 7f5d29600640 1 -- 172.21.15.16:0/2380882267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e90d1b0 msgr2=0x55985ecc2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.445+0000 7f5d29600640 1 --2- 172.21.15.16:0/2380882267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e90d1b0 0x55985ecc2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.445+0000 7f5d29600640 1 -- 172.21.15.16:0/2380882267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e8ec4d0 msgr2=0x55985ecc12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.445+0000 7f5d29600640 1 --2- 172.21.15.16:0/2380882267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e8ec4d0 0x55985ecc12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.445+0000 7f5d29600640 1 -- 172.21.15.16:0/2380882267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55985ecc1c80 con 0x55985e8e42f0 2024-09-21T03:50:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.446+0000 7f5d29600640 1 --2- 172.21.15.16:0/2380882267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecc5400 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5d1803de00 tx=0x7f5d18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68a77181efb83f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.446+0000 7f5d0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:50:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.447+0000 7f5d0aa00640 1 -- 172.21.15.16:0/2380882267 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1804a070 con 0x55985e8e42f0 2024-09-21T03:50:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.448+0000 7f5d0aa00640 1 -- 172.21.15.16:0/2380882267 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f5d1803d690 con 0x55985e8e42f0 2024-09-21T03:50:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.448+0000 7f5d32fdd740 1 -- 172.21.15.16:0/2380882267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d10016070 con 0x55985e8e42f0 2024-09-21T03:50:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.448+0000 7f5d0aa00640 1 -- 172.21.15.16:0/2380882267 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d18036050 con 0x55985e8e42f0 2024-09-21T03:50:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.450+0000 7f5d32fdd740 10 client.15966 fetch_fsmap learned FSMap version 30 2024-09-21T03:50:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.450+0000 7f5d32fdd740 1 -- 172.21.15.16:0/2380882267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55985ed35b00 con 0x55985e8e42f0 2024-09-21T03:50:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.452+0000 7f5d0aa00640 1 -- 172.21.15.16:0/2380882267 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d10016070 con 0x55985e8e42f0 2024-09-21T03:50:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.452+0000 7f5d0aa00640 1 -- 172.21.15.16:0/2380882267 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5d1804fcd0 con 0x55985e8e42f0 2024-09-21T03:50:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.453+0000 7f5d32fdd740 10 client.15966 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:50:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.453+0000 7f5d32fdd740 10 client.15966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.454+0000 7f5d32fdd740 20 client.15966 populate_metadata read hostname 'smithi016' 2024-09-21T03:50:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.454+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/2380882267 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55985e8a11c0 0x55985ed384f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.454+0000 7f5d32fdd740 4 client.15966 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:50:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.454+0000 7f5d32fdd740 1 -- 172.21.15.16:0/2380882267 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55985ec250b0 con 0x55985e8a11c0 2024-09-21T03:50:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.455+0000 7f5d28c00640 1 --2- 172.21.15.16:0/2380882267 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55985e8a11c0 0x55985ed384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.455+0000 7f5d28c00640 1 --2- 172.21.15.16:0/2380882267 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55985e8a11c0 0x55985ed384f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3cdf2365c7db379 server_cookie=2fa8aa3915dd13a9 in_seq=0 out_seq=0 2024-09-21T03:50:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.456+0000 7f5d0aa00640 10 client.15966 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.470+0000 7f5d0aa00640 1 -- 172.21.15.16:0/2380882267 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55985ec250b0 con 0x55985e8a11c0 2024-09-21T03:50:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.470+0000 7f5d0aa00640 10 client.15966 handle_command_reply: tid=0 2024-09-21T03:50:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.470+0000 7f5d32fdd740 1 client.15966 shutdown 2024-09-21T03:50:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.474+0000 7f5d32fdd740 1 -- 172.21.15.16:0/2380882267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 msgr2=0x55985ecc5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.474+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/2380882267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecc5400 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5d1803de00 tx=0x7f5d18004a40 comp rx=0 tx=0).stop 2024-09-21T03:50:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.474+0000 7f5d32fdd740 1 -- 172.21.15.16:0/2380882267 shutdown_connections 2024-09-21T03:50:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.474+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/2380882267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55985e90d1b0 0x55985ecc2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.474+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/2380882267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55985e8ec4d0 0x55985ecc12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.474+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/2380882267 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55985e8a11c0 0x55985ed384f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.474+0000 7f5d32fdd740 1 --2- 172.21.15.16:0/2380882267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55985e8e42f0 0x55985ecc5400 secure :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5d1803de00 tx=0x7f5d18004a40 comp rx=0 tx=0).stop 2024-09-21T03:50:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.474+0000 7f5d32fdd740 1 -- 172.21.15.16:0/2380882267 >> 172.21.15.16:0/2380882267 conn(0x55985e901300 msgr2=0x55985ecb4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.474+0000 7f5d0aa00640 0 client.15966 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.475+0000 7f5d32fdd740 1 -- 172.21.15.16:0/2380882267 shutdown_connections 2024-09-21T03:50:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.475+0000 7f5d32fdd740 1 -- 172.21.15.16:0/2380882267 wait complete. 2024-09-21T03:50:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.476+0000 7f5d32fdd740 20 client.15966 trim_cache size 0 max 16384 2024-09-21T03:50:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:50:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:50:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:50:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:50:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.479+0000 7f5d2aa00640 1 -- 172.21.15.16:0/1812056250 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d0c0b65c0 msgr2=0x7f5d0c0b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.479+0000 7f5d2aa00640 1 --2- 172.21.15.16:0/1812056250 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d0c0b65c0 0x7f5d0c0b8ad0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000ce40 tx=0x7f5d1000c390 comp rx=0 tx=0).stop 2024-09-21T03:50:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.479+0000 7f5d2aa00640 1 -- 172.21.15.16:0/1812056250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d24111c40 msgr2=0x7f5d2410bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.479+0000 7f5d2aa00640 1 --2- 172.21.15.16:0/1812056250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d24111c40 0x7f5d2410bb30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00e970 tx=0x7f5d1c00ee30 comp rx=0 tx=0).stop 2024-09-21T03:50:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.479+0000 7f5d2aa00640 1 -- 172.21.15.16:0/1812056250 shutdown_connections 2024-09-21T03:50:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.479+0000 7f5d2aa00640 1 --2- 172.21.15.16:0/1812056250 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d0c0b65c0 0x7f5d0c0b8ad0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.480+0000 7f5d2aa00640 1 --2- 172.21.15.16:0/1812056250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d24111c40 0x7f5d2410bb30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.480+0000 7f5d2aa00640 1 --2- 172.21.15.16:0/1812056250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2410ad10 0x7f5d2410b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.480+0000 7f5d2aa00640 1 --2- 172.21.15.16:0/1812056250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d2410a3c0 0x7f5d2410cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.480+0000 7f5d2aa00640 1 -- 172.21.15.16:0/1812056250 >> 172.21.15.16:0/1812056250 conn(0x7f5d24070640 msgr2=0x7f5d2410d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.480+0000 7f5d2aa00640 1 -- 172.21.15.16:0/1812056250 shutdown_connections 2024-09-21T03:50:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:44.480+0000 7f5d2aa00640 1 -- 172.21.15.16:0/1812056250 wait complete. 2024-09-21T03:50:45.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:44 smithi134 ceph-mon[35031]: pgmap v551: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 240 MiB/s wr, 1.05k op/s 2024-09-21T03:50:45.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:44 smithi123 ceph-mon[34673]: pgmap v551: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 240 MiB/s wr, 1.05k op/s 2024-09-21T03:50:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:44 smithi016 ceph-mon[30375]: pgmap v551: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 240 MiB/s wr, 1.05k op/s 2024-09-21T03:50:47.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:46 smithi134 ceph-mon[35031]: pgmap v552: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 246 MiB/s wr, 1.08k op/s 2024-09-21T03:50:47.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:46 smithi123 ceph-mon[34673]: pgmap v552: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 246 MiB/s wr, 1.08k op/s 2024-09-21T03:50:47.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:46 smithi016 ceph-mon[30375]: pgmap v552: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 246 MiB/s wr, 1.08k op/s 2024-09-21T03:50:48.958 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:48 smithi123 ceph-mon[34673]: pgmap v553: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 233 MiB/s wr, 1.03k op/s 2024-09-21T03:50:49.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:48 smithi134 ceph-mon[35031]: pgmap v553: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 233 MiB/s wr, 1.03k op/s 2024-09-21T03:50:49.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:48 smithi016 ceph-mon[30375]: pgmap v553: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 233 MiB/s wr, 1.03k op/s 2024-09-21T03:50:50.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:50 smithi123 ceph-mon[34673]: pgmap v554: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 229 MiB/s wr, 1.01k op/s 2024-09-21T03:50:51.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:50 smithi134 ceph-mon[35031]: pgmap v554: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 229 MiB/s wr, 1.01k op/s 2024-09-21T03:50:51.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:50 smithi016 ceph-mon[30375]: pgmap v554: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 229 MiB/s wr, 1.01k op/s 2024-09-21T03:50:53.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:52 smithi134 ceph-mon[35031]: pgmap v555: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 241 MiB/s wr, 1.07k op/s 2024-09-21T03:50:53.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:52 smithi123 ceph-mon[34673]: pgmap v555: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 241 MiB/s wr, 1.07k op/s 2024-09-21T03:50:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:52 smithi016 ceph-mon[30375]: pgmap v555: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 241 MiB/s wr, 1.07k op/s 2024-09-21T03:50:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:54 smithi134 ceph-mon[35031]: pgmap v556: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 245 MiB/s wr, 1.11k op/s 2024-09-21T03:50:55.158 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:54 smithi123 ceph-mon[34673]: pgmap v556: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 245 MiB/s wr, 1.11k op/s 2024-09-21T03:50:55.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:54 smithi016 ceph-mon[30375]: pgmap v556: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 245 MiB/s wr, 1.11k op/s 2024-09-21T03:50:57.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:56 smithi134 ceph-mon[35031]: pgmap v557: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 247 MiB/s wr, 1.12k op/s 2024-09-21T03:50:57.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:56 smithi123 ceph-mon[34673]: pgmap v557: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 247 MiB/s wr, 1.12k op/s 2024-09-21T03:50:57.168 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:56 smithi016 ceph-mon[30375]: pgmap v557: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 247 MiB/s wr, 1.12k op/s 2024-09-21T03:50:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.831+0000 7f2be7e00640 1 -- 172.21.15.16:0/2006106107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2be806b710 msgr2=0x7f2be8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.831+0000 7f2be7e00640 1 --2- 172.21.15.16:0/2006106107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2be806b710 0x7f2be8103f70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009900 tx=0x7f2bd402a400 comp rx=0 tx=0).stop 2024-09-21T03:50:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.832+0000 7f2be7e00640 1 -- 172.21.15.16:0/2006106107 shutdown_connections 2024-09-21T03:50:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.832+0000 7f2be7e00640 1 --2- 172.21.15.16:0/2006106107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2be8107070 0x7f2be8107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.832+0000 7f2be7e00640 1 --2- 172.21.15.16:0/2006106107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be81044b0 0x7f2be8106930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.832+0000 7f2be7e00640 1 --2- 172.21.15.16:0/2006106107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2be806b710 0x7f2be8103f70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.832+0000 7f2be7e00640 1 -- 172.21.15.16:0/2006106107 >> 172.21.15.16:0/2006106107 conn(0x7f2be80fdd80 msgr2=0x7f2be8100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.834+0000 7f2be7e00640 1 -- 172.21.15.16:0/2006106107 shutdown_connections 2024-09-21T03:50:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.834+0000 7f2be7e00640 1 -- 172.21.15.16:0/2006106107 wait complete. 2024-09-21T03:50:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.836+0000 7f2be7e00640 1 Processor -- start 2024-09-21T03:50:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.837+0000 7f2be7e00640 1 -- start start 2024-09-21T03:50:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.840+0000 7f2be7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2be806b710 0x7f2be81a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.840+0000 7f2be7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2be81044b0 0x7f2be81a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.840+0000 7f2be7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be8107070 0x7f2be81a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.840+0000 7f2be7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81157e0 con 0x7f2be81044b0 2024-09-21T03:50:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.840+0000 7f2be7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2be8115660 con 0x7f2be806b710 2024-09-21T03:50:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.840+0000 7f2be7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2be8115960 con 0x7f2be8107070 2024-09-21T03:50:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.842+0000 7f2be6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2be806b710 0x7f2be81a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.842+0000 7f2be6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2be806b710 0x7f2be81a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:51856/0 (socket says 172.21.15.16:51856) 2024-09-21T03:50:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.842+0000 7f2be6a00640 1 -- 172.21.15.16:0/1677411466 learned_addr learned my addr 172.21.15.16:0/1677411466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.842+0000 7f2be7400640 1 --2- 172.21.15.16:0/1677411466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be8107070 0x7f2be81a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.842+0000 7f2be6000640 1 --2- 172.21.15.16:0/1677411466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2be81044b0 0x7f2be81a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.842+0000 7f2be7400640 1 -- 172.21.15.16:0/1677411466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2be806b710 msgr2=0x7f2be81a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.842+0000 7f2be7400640 1 --2- 172.21.15.16:0/1677411466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2be806b710 0x7f2be81a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.842+0000 7f2be7400640 1 -- 172.21.15.16:0/1677411466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2be81044b0 msgr2=0x7f2be81a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.842+0000 7f2be7400640 1 --2- 172.21.15.16:0/1677411466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2be81044b0 0x7f2be81a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.842+0000 7f2be7400640 1 -- 172.21.15.16:0/1677411466 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be81a1e50 con 0x7f2be8107070 2024-09-21T03:50:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.843+0000 7f2be6000640 1 --2- 172.21.15.16:0/1677411466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2be81044b0 0x7f2be81a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.843+0000 7f2be7400640 1 --2- 172.21.15.16:0/1677411466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be8107070 0x7f2be81a1630 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800e970 tx=0x7f2bd800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.843+0000 7f2beca00640 1 -- 172.21.15.16:0/1677411466 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd800ccc0 con 0x7f2be8107070 2024-09-21T03:50:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.843+0000 7f2beca00640 1 -- 172.21.15.16:0/1677411466 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bd8005020 con 0x7f2be8107070 2024-09-21T03:50:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.843+0000 7f2beca00640 1 -- 172.21.15.16:0/1677411466 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8004a00 con 0x7f2be8107070 2024-09-21T03:50:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.843+0000 7f2be7e00640 1 -- 172.21.15.16:0/1677411466 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be81a2140 con 0x7f2be8107070 2024-09-21T03:50:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.844+0000 7f2be7e00640 1 -- 172.21.15.16:0/1677411466 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be81a8830 con 0x7f2be8107070 2024-09-21T03:50:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.850+0000 7f2bf4fc9740 1 Processor -- start 2024-09-21T03:50:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.851+0000 7f2beca00640 1 -- 172.21.15.16:0/1677411466 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2bd8004ba0 con 0x7f2be8107070 2024-09-21T03:50:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.851+0000 7f2bf4fc9740 1 -- start start 2024-09-21T03:50:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.852+0000 7f2bf4fc9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e88956d1b0 0x55e8899142b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.852+0000 7f2bf4fc9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8895442f0 0x55e889914920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.852+0000 7f2bf4fc9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88954c4d0 0x55e889915470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.852+0000 7f2bf4fc9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e8898876b0 con 0x55e88954c4d0 2024-09-21T03:50:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.852+0000 7f2bf4fc9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e889876ae0 con 0x55e8895442f0 2024-09-21T03:50:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.852+0000 7f2bf4fc9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e88987a8f0 con 0x55e88956d1b0 2024-09-21T03:50:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.852+0000 7f2be6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e88956d1b0 0x55e8899142b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.852+0000 7f2be6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e88956d1b0 0x55e8899142b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:53946/0 (socket says 172.21.15.16:53946) 2024-09-21T03:50:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.852+0000 7f2be6000640 1 -- 172.21.15.16:0/2562157103 learned_addr learned my addr 172.21.15.16:0/2562157103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.853+0000 7f2be6a00640 1 --2- 172.21.15.16:0/2562157103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8895442f0 0x55e889914920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.853+0000 7f2be6000640 1 --2- 172.21.15.16:0/2562157103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88954c4d0 0x55e889915470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.853+0000 7f2be6000640 1 -- 172.21.15.16:0/2562157103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8895442f0 msgr2=0x55e889914920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.853+0000 7f2beca00640 1 --2- 172.21.15.16:0/1677411466 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2bcc0b6590 0x7f2bcc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.853+0000 7f2be6000640 1 --2- 172.21.15.16:0/2562157103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8895442f0 0x55e889914920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.853+0000 7f2be6000640 1 -- 172.21.15.16:0/2562157103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88954c4d0 msgr2=0x55e889915470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.853+0000 7f2be6000640 1 --2- 172.21.15.16:0/2562157103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88954c4d0 0x55e889915470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.853+0000 7f2be6000640 1 -- 172.21.15.16:0/2562157103 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e889843780 con 0x55e88956d1b0 2024-09-21T03:50:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.854+0000 7f2be6000640 1 --2- 172.21.15.16:0/2562157103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e88956d1b0 0x55e8899142b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2be000a7d0 tx=0x7f2be0009a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43ca227d08153b44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.854+0000 7f2be7400640 1 --2- 172.21.15.16:0/1677411466 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2bcc0b6590 0x7f2bcc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.854+0000 7f2bd3e00640 1 -- 172.21.15.16:0/2562157103 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be001d070 con 0x55e88956d1b0 2024-09-21T03:50:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.854+0000 7f2beca00640 1 -- 172.21.15.16:0/1677411466 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f2bd80e7710 con 0x7f2be8107070 2024-09-21T03:50:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.854+0000 7f2bd3e00640 1 -- 172.21.15.16:0/2562157103 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be000ae10 con 0x55e88956d1b0 2024-09-21T03:50:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.854+0000 7f2bd3e00640 1 -- 172.21.15.16:0/2562157103 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0018ad0 con 0x55e88956d1b0 2024-09-21T03:50:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.856+0000 7f2be7400640 1 --2- 172.21.15.16:0/1677411466 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2bcc0b6590 0x7f2bcc0b8aa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80aa760 tx=0x7f2bd807a820 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.856+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/2562157103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e88956d1b0 msgr2=0x55e8899142b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.856+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/2562157103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e88956d1b0 0x55e8899142b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2be000a7d0 tx=0x7f2be0009a80 comp rx=0 tx=0).stop 2024-09-21T03:50:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.856+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/2562157103 shutdown_connections 2024-09-21T03:50:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.856+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/2562157103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88954c4d0 0x55e889915470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.856+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/2562157103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8895442f0 0x55e889914920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.856+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/2562157103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e88956d1b0 0x55e8899142b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.856+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/2562157103 >> 172.21.15.16:0/2562157103 conn(0x55e889561300 msgr2=0x55e88990fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.857+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/2562157103 shutdown_connections 2024-09-21T03:50:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.857+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/2562157103 wait complete. 2024-09-21T03:50:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.857+0000 7f2bf4fc9740 1 -- start start 2024-09-21T03:50:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.858+0000 7f2bf4fc9740 1 Processor -- start 2024-09-21T03:50:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.859+0000 7f2bf4fc9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 0x55e88991d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.859+0000 7f2bf4fc9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88954c4d0 0x55e88991e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.859+0000 7f2bf4fc9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88956d1b0 0x55e88991ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.859+0000 7f2bf4fc9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e8897aeab0 con 0x55e88954c4d0 2024-09-21T03:50:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.859+0000 7f2bf4fc9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e88981d570 con 0x55e88956d1b0 2024-09-21T03:50:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.859+0000 7f2bf4fc9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e889915f90 con 0x55e8895442f0 2024-09-21T03:50:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.859+0000 7f2bf4fc9740 10 client.0 _ll_register_callbacks cb 0x55e88982c100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.859+0000 7f2be6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 0x55e88991d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.859+0000 7f2be6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 0x55e88991d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:53950/0 (socket says 172.21.15.16:53950) 2024-09-21T03:50:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.859+0000 7f2be6000640 1 -- 172.21.15.16:0/4278921525 learned_addr learned my addr 172.21.15.16:0/4278921525 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.859+0000 7f2be6a00640 1 --2- 172.21.15.16:0/4278921525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88954c4d0 0x55e88991e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.860+0000 7f2be6000640 1 --2- 172.21.15.16:0/4278921525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88956d1b0 0x55e88991ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.860+0000 7f2be6000640 1 -- 172.21.15.16:0/4278921525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88956d1b0 msgr2=0x55e88991ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.860+0000 7f2be6000640 1 --2- 172.21.15.16:0/4278921525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88956d1b0 0x55e88991ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.860+0000 7f2be6000640 1 -- 172.21.15.16:0/4278921525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88954c4d0 msgr2=0x55e88991e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.860+0000 7f2be6000640 1 --2- 172.21.15.16:0/4278921525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88954c4d0 0x55e88991e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.860+0000 7f2be6000640 1 -- 172.21.15.16:0/4278921525 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8895d8790 con 0x55e8895442f0 2024-09-21T03:50:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.860+0000 7f2be6a00640 1 --2- 172.21.15.16:0/4278921525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88954c4d0 0x55e88991e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:50:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.860+0000 7f2be6000640 1 --2- 172.21.15.16:0/4278921525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 0x55e88991d9a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2be00184f0 tx=0x7f2be00093d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=991cf2171242283f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.860+0000 7f2bd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:50:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.860+0000 7f2bd0c00640 1 -- 172.21.15.16:0/4278921525 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be001d040 con 0x55e8895442f0 2024-09-21T03:50:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.860+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/4278921525 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be000ae10 con 0x55e8895442f0 2024-09-21T03:50:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.861+0000 7f2bd0c00640 1 -- 172.21.15.16:0/4278921525 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f2be000f9f0 con 0x55e8895442f0 2024-09-21T03:50:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.861+0000 7f2bd0c00640 1 -- 172.21.15.16:0/4278921525 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2be0033050 con 0x55e8895442f0 2024-09-21T03:50:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.861+0000 7f2bf4fc9740 10 client.26002 fetch_fsmap learned FSMap version 30 2024-09-21T03:50:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.861+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/4278921525 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e889651620 con 0x55e8895442f0 2024-09-21T03:50:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.861+0000 7f2bd0c00640 1 -- 172.21.15.16:0/4278921525 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be000ae10 con 0x55e8895442f0 2024-09-21T03:50:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.862+0000 7f2bd0c00640 1 -- 172.21.15.16:0/4278921525 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2be0025c50 con 0x55e8895442f0 2024-09-21T03:50:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.862+0000 7f2bf4fc9740 10 client.26002 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:50:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.862+0000 7f2bf4fc9740 10 client.26002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.862+0000 7f2bf4fc9740 20 client.26002 populate_metadata read hostname 'smithi016' 2024-09-21T03:50:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.862+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/4278921525 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e88952bd80 0x55e88993bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.862+0000 7f2bf4fc9740 4 client.26002 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:50:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.862+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/4278921525 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e889889600 con 0x55e88952bd80 2024-09-21T03:50:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.862+0000 7f2be6a00640 1 --2- 172.21.15.16:0/4278921525 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e88952bd80 0x55e88993bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.863+0000 7f2be6a00640 1 --2- 172.21.15.16:0/4278921525 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e88952bd80 0x55e88993bdd0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2868a16238d47913 server_cookie=bd02653edc85e0d in_seq=0 out_seq=0 2024-09-21T03:50:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.863+0000 7f2bd0c00640 10 client.26002 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.864+0000 7f2bd0c00640 1 -- 172.21.15.16:0/4278921525 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e889889600 con 0x55e88952bd80 2024-09-21T03:50:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.864+0000 7f2bd0c00640 10 client.26002 handle_command_reply: tid=0 2024-09-21T03:50:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.864+0000 7f2bf4fc9740 1 client.26002 shutdown 2024-09-21T03:50:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.864+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/4278921525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 msgr2=0x55e88991d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.864+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/4278921525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 0x55e88991d9a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2be00184f0 tx=0x7f2be00093d0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.864+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/4278921525 shutdown_connections 2024-09-21T03:50:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.864+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/4278921525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88956d1b0 0x55e88991ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.864+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/4278921525 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e88952bd80 0x55e88993bdd0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.864+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/4278921525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88954c4d0 0x55e88991e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.864+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/4278921525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 0x55e88991d9a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.864+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/4278921525 >> 172.21.15.16:0/4278921525 conn(0x55e889561300 msgr2=0x55e88990e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.864+0000 7f2bd0c00640 0 client.26002 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.865+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/4278921525 shutdown_connections 2024-09-21T03:50:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.866+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/4278921525 wait complete. 2024-09-21T03:50:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.866+0000 7f2bf4fc9740 20 client.26002 trim_cache size 0 max 16384 2024-09-21T03:50:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.899+0000 7f2bf4fc9740 1 Processor -- start 2024-09-21T03:50:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.899+0000 7f2bf4fc9740 1 -- start start 2024-09-21T03:50:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.900+0000 7f2bf4fc9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 0x55e88991c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.900+0000 7f2bf4fc9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88954c4d0 0x55e889915210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.900+0000 7f2bf4fc9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88956d1b0 0x55e88990f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.900+0000 7f2bf4fc9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81aa450 con 0x55e88956d1b0 2024-09-21T03:50:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.900+0000 7f2bf4fc9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81abf90 con 0x55e88954c4d0 2024-09-21T03:50:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.900+0000 7f2bf4fc9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81aab20 con 0x55e8895442f0 2024-09-21T03:50:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.905+0000 7f2be6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88954c4d0 0x55e889915210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.905+0000 7f2be6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88954c4d0 0x55e889915210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:51886/0 (socket says 172.21.15.16:51886) 2024-09-21T03:50:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.905+0000 7f2be6a00640 1 -- 172.21.15.16:0/500856292 learned_addr learned my addr 172.21.15.16:0/500856292 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.905+0000 7f2be6a00640 1 -- 172.21.15.16:0/500856292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 msgr2=0x55e88991c600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:50:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.905+0000 7f2be6a00640 1 --2- 172.21.15.16:0/500856292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 0x55e88991c600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.905+0000 7f2be6a00640 1 -- 172.21.15.16:0/500856292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88956d1b0 msgr2=0x55e88990f370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:50:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.905+0000 7f2be6a00640 1 --2- 172.21.15.16:0/500856292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88956d1b0 0x55e88990f370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.905+0000 7f2be6a00640 1 -- 172.21.15.16:0/500856292 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e88990f8b0 con 0x55e88954c4d0 2024-09-21T03:50:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.906+0000 7f2be6a00640 1 --2- 172.21.15.16:0/500856292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88954c4d0 0x55e889915210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009230 tx=0x7f2bd4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5057709bf7e7ba1a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.906+0000 7f2bd3400640 1 -- 172.21.15.16:0/500856292 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4048020 con 0x55e88954c4d0 2024-09-21T03:50:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.906+0000 7f2bd3400640 1 -- 172.21.15.16:0/500856292 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bd4003d60 con 0x55e88954c4d0 2024-09-21T03:50:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.907+0000 7f2bd3400640 1 -- 172.21.15.16:0/500856292 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4035040 con 0x55e88954c4d0 2024-09-21T03:50:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.908+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/500856292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88954c4d0 msgr2=0x55e889915210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.908+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/500856292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88954c4d0 0x55e889915210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009230 tx=0x7f2bd4004040 comp rx=0 tx=0).stop 2024-09-21T03:50:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.908+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/500856292 shutdown_connections 2024-09-21T03:50:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.908+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/500856292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88956d1b0 0x55e88990f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.908+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/500856292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88954c4d0 0x55e889915210 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.908+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/500856292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 0x55e88991c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.908+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/500856292 >> 172.21.15.16:0/500856292 conn(0x55e889561300 msgr2=0x55e88991bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.908+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/500856292 shutdown_connections 2024-09-21T03:50:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.908+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/500856292 wait complete. 2024-09-21T03:50:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.909+0000 7f2bf4fc9740 1 -- start start 2024-09-21T03:50:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.909+0000 7f2bf4fc9740 1 Processor -- start 2024-09-21T03:50:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.910+0000 7f2bf4fc9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 0x55e889922980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.910+0000 7f2bf4fc9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88954c4d0 0x55e889923040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.911+0000 7f2be6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 0x55e889922980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.911+0000 7f2be6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 0x55e889922980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:53958/0 (socket says 172.21.15.16:53958) 2024-09-21T03:50:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.911+0000 7f2be6000640 1 -- 172.21.15.16:0/2291795539 learned_addr learned my addr 172.21.15.16:0/2291795539 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:50:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.911+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/2291795539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88956d1b0 0x55e889924c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.911+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/2291795539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81ab8c0 con 0x55e88954c4d0 2024-09-21T03:50:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.911+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/2291795539 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81ab1f0 con 0x55e88956d1b0 2024-09-21T03:50:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.911+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/2291795539 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81107b0 con 0x55e8895442f0 2024-09-21T03:50:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.911+0000 7f2bf4fc9740 10 client.0 _ll_register_callbacks cb 0x7f2be001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:50:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.911+0000 7f2be6a00640 1 --2- 172.21.15.16:0/2291795539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88954c4d0 0x55e889923040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.913+0000 7f2be6a00640 1 -- 172.21.15.16:0/2291795539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 msgr2=0x55e889922980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:50:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.913+0000 7f2be6a00640 1 --2- 172.21.15.16:0/2291795539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 0x55e889922980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.913+0000 7f2be6a00640 1 -- 172.21.15.16:0/2291795539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88956d1b0 msgr2=0x55e889924c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:50:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.913+0000 7f2be6a00640 1 --2- 172.21.15.16:0/2291795539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88956d1b0 0x55e889924c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.913+0000 7f2be6a00640 1 -- 172.21.15.16:0/2291795539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e889925380 con 0x55e88954c4d0 2024-09-21T03:50:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.914+0000 7f2bd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:50:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.914+0000 7f2be6a00640 1 --2- 172.21.15.16:0/2291795539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88954c4d0 0x55e889923040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40057c0 tx=0x7f2bd4031e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5f94f3a8da57a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:50:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.915+0000 7f2bd0c00640 1 -- 172.21.15.16:0/2291795539 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4048030 con 0x55e88954c4d0 2024-09-21T03:50:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.915+0000 7f2bd0c00640 1 -- 172.21.15.16:0/2291795539 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f2bd40036b0 con 0x55e88954c4d0 2024-09-21T03:50:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.917+0000 7f2be6000640 1 --2- 172.21.15.16:0/2291795539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 0x55e889922980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:50:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.917+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/2291795539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd4003d60 con 0x55e88954c4d0 2024-09-21T03:50:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.917+0000 7f2bd0c00640 1 -- 172.21.15.16:0/2291795539 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2bd4035070 con 0x55e88954c4d0 2024-09-21T03:50:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.918+0000 7f2bf4fc9740 10 client.15981 fetch_fsmap learned FSMap version 30 2024-09-21T03:50:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.918+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/2291795539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e8899245c0 con 0x55e88954c4d0 2024-09-21T03:50:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.918+0000 7f2bd0c00640 1 -- 172.21.15.16:0/2291795539 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bd4003d60 con 0x55e88954c4d0 2024-09-21T03:50:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.918+0000 7f2bd0c00640 1 -- 172.21.15.16:0/2291795539 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2bd403bbe0 con 0x55e88954c4d0 2024-09-21T03:50:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.919+0000 7f2bf4fc9740 10 client.15981 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:50:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.919+0000 7f2bf4fc9740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:50:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.919+0000 7f2bf4fc9740 20 client.15981 populate_metadata read hostname 'smithi016' 2024-09-21T03:50:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.919+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/2291795539 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e8895011c0 0x55e8899a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:50:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.919+0000 7f2bf4fc9740 4 client.15981 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:50:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.919+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/2291795539 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8898850b0 con 0x55e8895011c0 2024-09-21T03:50:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.919+0000 7f2be6000640 1 --2- 172.21.15.16:0/2291795539 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e8895011c0 0x55e8899a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:50:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.920+0000 7f2bd0c00640 10 client.15981 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.920+0000 7f2be6000640 1 --2- 172.21.15.16:0/2291795539 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e8895011c0 0x55e8899a2ba0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70d5f22c98c68259 server_cookie=bc85d3e4fa203dc3 in_seq=0 out_seq=0 2024-09-21T03:50:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.947+0000 7f2bd0c00640 1 -- 172.21.15.16:0/2291795539 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e8898850b0 con 0x55e8895011c0 2024-09-21T03:50:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.947+0000 7f2bd0c00640 10 client.15981 handle_command_reply: tid=0 2024-09-21T03:50:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.947+0000 7f2bf4fc9740 1 client.15981 shutdown 2024-09-21T03:50:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.953+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/2291795539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88954c4d0 msgr2=0x55e889923040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.953+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/2291795539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88954c4d0 0x55e889923040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40057c0 tx=0x7f2bd4031e00 comp rx=0 tx=0).stop 2024-09-21T03:50:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.954+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/2291795539 shutdown_connections 2024-09-21T03:50:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.954+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/2291795539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88956d1b0 0x55e889924c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.954+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/2291795539 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e8895011c0 0x55e8899a2ba0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.954+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/2291795539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e88954c4d0 0x55e889923040 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.954+0000 7f2bf4fc9740 1 --2- 172.21.15.16:0/2291795539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8895442f0 0x55e889922980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.954+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/2291795539 >> 172.21.15.16:0/2291795539 conn(0x55e889561300 msgr2=0x55e889918400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.954+0000 7f2bd0c00640 0 client.15981 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:50:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.957+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/2291795539 shutdown_connections 2024-09-21T03:50:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.957+0000 7f2bf4fc9740 1 -- 172.21.15.16:0/2291795539 wait complete. 2024-09-21T03:50:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.958+0000 7f2bf4fc9740 20 client.15981 trim_cache size 0 max 16384 2024-09-21T03:50:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:50:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:50:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:50:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:50:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.964+0000 7f2be7e00640 1 -- 172.21.15.16:0/1677411466 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2bcc0b6590 msgr2=0x7f2bcc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.964+0000 7f2be7e00640 1 --2- 172.21.15.16:0/1677411466 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2bcc0b6590 0x7f2bcc0b8aa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80aa760 tx=0x7f2bd807a820 comp rx=0 tx=0).stop 2024-09-21T03:50:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.964+0000 7f2be7e00640 1 -- 172.21.15.16:0/1677411466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be8107070 msgr2=0x7f2be81a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:50:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.964+0000 7f2be7e00640 1 --2- 172.21.15.16:0/1677411466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be8107070 0x7f2be81a1630 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800e970 tx=0x7f2bd800ee30 comp rx=0 tx=0).stop 2024-09-21T03:50:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.967+0000 7f2be7e00640 1 -- 172.21.15.16:0/1677411466 shutdown_connections 2024-09-21T03:50:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.967+0000 7f2be7e00640 1 --2- 172.21.15.16:0/1677411466 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f2bcc0b6590 0x7f2bcc0b8aa0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.967+0000 7f2be7e00640 1 --2- 172.21.15.16:0/1677411466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be8107070 0x7f2be81a1630 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.967+0000 7f2be7e00640 1 --2- 172.21.15.16:0/1677411466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2be81044b0 0x7f2be81a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.967+0000 7f2be7e00640 1 --2- 172.21.15.16:0/1677411466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2be806b710 0x7f2be81a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:50:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.967+0000 7f2be7e00640 1 -- 172.21.15.16:0/1677411466 >> 172.21.15.16:0/1677411466 conn(0x7f2be80fdd80 msgr2=0x7f2be81053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:50:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.967+0000 7f2be7e00640 1 -- 172.21.15.16:0/1677411466 shutdown_connections 2024-09-21T03:50:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:50:57.967+0000 7f2be7e00640 1 -- 172.21.15.16:0/1677411466 wait complete. 2024-09-21T03:50:59.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:50:58 smithi134 ceph-mon[35031]: pgmap v558: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 239 MiB/s wr, 1.07k op/s 2024-09-21T03:50:59.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:50:58 smithi123 ceph-mon[34673]: pgmap v558: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 239 MiB/s wr, 1.07k op/s 2024-09-21T03:50:59.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:50:58 smithi016 ceph-mon[30375]: pgmap v558: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 239 MiB/s wr, 1.07k op/s 2024-09-21T03:51:00.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:00 smithi123 ceph-mon[34673]: pgmap v559: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 238 MiB/s wr, 1.07k op/s 2024-09-21T03:51:01.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:00 smithi134 ceph-mon[35031]: pgmap v559: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 238 MiB/s wr, 1.07k op/s 2024-09-21T03:51:01.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:00 smithi016 ceph-mon[30375]: pgmap v559: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 238 MiB/s wr, 1.07k op/s 2024-09-21T03:51:03.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:02 smithi134 ceph-mon[35031]: pgmap v560: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 244 MiB/s wr, 1.10k op/s 2024-09-21T03:51:03.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:02 smithi123 ceph-mon[34673]: pgmap v560: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 244 MiB/s wr, 1.10k op/s 2024-09-21T03:51:03.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:02 smithi016 ceph-mon[30375]: pgmap v560: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 244 MiB/s wr, 1.10k op/s 2024-09-21T03:51:05.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:04 smithi134 ceph-mon[35031]: pgmap v561: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 240 MiB/s wr, 1.09k op/s 2024-09-21T03:51:05.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:04 smithi123 ceph-mon[34673]: pgmap v561: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 240 MiB/s wr, 1.09k op/s 2024-09-21T03:51:05.170 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:04 smithi016 ceph-mon[30375]: pgmap v561: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 240 MiB/s wr, 1.09k op/s 2024-09-21T03:51:07.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:06 smithi134 ceph-mon[35031]: pgmap v562: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 247 MiB/s wr, 1.11k op/s 2024-09-21T03:51:07.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:06 smithi123 ceph-mon[34673]: pgmap v562: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 247 MiB/s wr, 1.11k op/s 2024-09-21T03:51:07.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:06 smithi016 ceph-mon[30375]: pgmap v562: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 247 MiB/s wr, 1.11k op/s 2024-09-21T03:51:09.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:08 smithi134 ceph-mon[35031]: pgmap v563: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 238 MiB/s wr, 1.07k op/s 2024-09-21T03:51:09.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:08 smithi123 ceph-mon[34673]: pgmap v563: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 238 MiB/s wr, 1.07k op/s 2024-09-21T03:51:09.168 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:08 smithi016 ceph-mon[30375]: pgmap v563: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 238 MiB/s wr, 1.07k op/s 2024-09-21T03:51:11.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:10 smithi134 ceph-mon[35031]: pgmap v564: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 233 MiB/s wr, 1.05k op/s 2024-09-21T03:51:11.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:10 smithi123 ceph-mon[34673]: pgmap v564: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 233 MiB/s wr, 1.05k op/s 2024-09-21T03:51:11.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:10 smithi016 ceph-mon[30375]: pgmap v564: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 233 MiB/s wr, 1.05k op/s 2024-09-21T03:51:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.497+0000 7fcddc400640 1 -- 172.21.15.16:0/3140844940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd4109480 msgr2=0x7fcdd410b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.497+0000 7fcddc400640 1 --2- 172.21.15.16:0/3140844940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd4109480 0x7fcdd410b920 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009ca0 tx=0x7fcdcc02a8b0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.500+0000 7fcddc400640 1 -- 172.21.15.16:0/3140844940 shutdown_connections 2024-09-21T03:51:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.500+0000 7fcddc400640 1 --2- 172.21.15.16:0/3140844940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd4109480 0x7fcdd410b920 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.500+0000 7fcddc400640 1 --2- 172.21.15.16:0/3140844940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdd4106b20 0x7fcdd4108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.500+0000 7fcddc400640 1 --2- 172.21.15.16:0/3140844940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdd4101a90 0x7fcdd4101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.500+0000 7fcddc400640 1 -- 172.21.15.16:0/3140844940 >> 172.21.15.16:0/3140844940 conn(0x7fcdd40fd690 msgr2=0x7fcdd40ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.502+0000 7fcddc400640 1 -- 172.21.15.16:0/3140844940 shutdown_connections 2024-09-21T03:51:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.502+0000 7fcddc400640 1 -- 172.21.15.16:0/3140844940 wait complete. 2024-09-21T03:51:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.503+0000 7fcddc400640 1 Processor -- start 2024-09-21T03:51:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.504+0000 7fcddc400640 1 -- start start 2024-09-21T03:51:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.504+0000 7fcddc400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdd4101a90 0x7fcdd4104160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.504+0000 7fcddc400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdd4106b20 0x7fcdd4102a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.504+0000 7fcddc400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd4109480 0x7fcdd4102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.504+0000 7fcddc400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd410f320 con 0x7fcdd4106b20 2024-09-21T03:51:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.504+0000 7fcddc400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd410f1a0 con 0x7fcdd4101a90 2024-09-21T03:51:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.504+0000 7fcddc400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd410f4a0 con 0x7fcdd4109480 2024-09-21T03:51:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.505+0000 7fcdda400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd4109480 0x7fcdd4102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.505+0000 7fcdda400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd4109480 0x7fcdd4102f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39228/0 (socket says 172.21.15.16:39228) 2024-09-21T03:51:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.505+0000 7fcdda400640 1 -- 172.21.15.16:0/734716786 learned_addr learned my addr 172.21.15.16:0/734716786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.506+0000 7fcdd9a00640 1 --2- 172.21.15.16:0/734716786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdd4101a90 0x7fcdd4104160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.506+0000 7fcdda400640 1 -- 172.21.15.16:0/734716786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdd4101a90 msgr2=0x7fcdd4104160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.506+0000 7fcdda400640 1 --2- 172.21.15.16:0/734716786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdd4101a90 0x7fcdd4104160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.506+0000 7fcdda400640 1 -- 172.21.15.16:0/734716786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdd4106b20 msgr2=0x7fcdd4102a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.506+0000 7fcdda400640 1 --2- 172.21.15.16:0/734716786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdd4106b20 0x7fcdd4102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.506+0000 7fcdda400640 1 -- 172.21.15.16:0/734716786 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd419dd60 con 0x7fcdd4109480 2024-09-21T03:51:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.506+0000 7fcdda400640 1 --2- 172.21.15.16:0/734716786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd4109480 0x7fcdd4102f60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0307a0 tx=0x7fcdcc003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.507+0000 7fcdd2000640 1 -- 172.21.15.16:0/734716786 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc004240 con 0x7fcdd4109480 2024-09-21T03:51:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.507+0000 7fcddc400640 1 -- 172.21.15.16:0/734716786 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd4103560 con 0x7fcdd4109480 2024-09-21T03:51:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.507+0000 7fcdd2000640 1 -- 172.21.15.16:0/734716786 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdcc040070 con 0x7fcdd4109480 2024-09-21T03:51:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.507+0000 7fcdd2000640 1 -- 172.21.15.16:0/734716786 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc045c10 con 0x7fcdd4109480 2024-09-21T03:51:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.507+0000 7fcddc400640 1 -- 172.21.15.16:0/734716786 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd41a8960 con 0x7fcdd4109480 2024-09-21T03:51:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.511+0000 7fcdd2000640 1 -- 172.21.15.16:0/734716786 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fcdcc045470 con 0x7fcdd4109480 2024-09-21T03:51:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.512+0000 7fcde34bb740 1 Processor -- start 2024-09-21T03:51:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.513+0000 7fcde34bb740 1 -- start start 2024-09-21T03:51:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.514+0000 7fcdd2000640 1 --2- 172.21.15.16:0/734716786 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcdbc0b65d0 0x7fcdbc0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.514+0000 7fcdd2000640 1 -- 172.21.15.16:0/734716786 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fcdcc1087e0 con 0x7fcdd4109480 2024-09-21T03:51:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.514+0000 7fcde34bb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x56266ff82100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.514+0000 7fcde34bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbb22f0 0x56266ff82770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.514+0000 7fcde34bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fbba4d0 0x56266ff832c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.514+0000 7fcde34bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56266fef56b0 con 0x56266fbb22f0 2024-09-21T03:51:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.514+0000 7fcde34bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56266fee4ae0 con 0x56266fbba4d0 2024-09-21T03:51:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.514+0000 7fcde34bb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56266fee88f0 con 0x56266fbdb1b0 2024-09-21T03:51:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.514+0000 7fcdd9a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x56266ff82100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.514+0000 7fcdd9a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x56266ff82100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39234/0 (socket says 172.21.15.16:39234) 2024-09-21T03:51:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.514+0000 7fcdd9a00640 1 -- 172.21.15.16:0/3551188201 learned_addr learned my addr 172.21.15.16:0/3551188201 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.515+0000 7fcdd9000640 1 --2- 172.21.15.16:0/734716786 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcdbc0b65d0 0x7fcdbc0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.515+0000 7fcdd9a00640 1 -- 172.21.15.16:0/3551188201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fbba4d0 msgr2=0x56266ff832c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:51:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.515+0000 7fcdd9a00640 1 --2- 172.21.15.16:0/3551188201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fbba4d0 0x56266ff832c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.515+0000 7fcdd9a00640 1 -- 172.21.15.16:0/3551188201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbb22f0 msgr2=0x56266ff82770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.515+0000 7fcdd9a00640 1 --2- 172.21.15.16:0/3551188201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbb22f0 0x56266ff82770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.515+0000 7fcdd9a00640 1 -- 172.21.15.16:0/3551188201 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56266feb1780 con 0x56266fbdb1b0 2024-09-21T03:51:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.515+0000 7fcdd9a00640 1 --2- 172.21.15.16:0/3551188201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x56266ff82100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009bc0 tx=0x7fcdc800e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f6809fb71304fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.516+0000 7fcdc6a00640 1 -- 172.21.15.16:0/3551188201 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8016070 con 0x56266fbdb1b0 2024-09-21T03:51:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.516+0000 7fcdd9000640 1 --2- 172.21.15.16:0/734716786 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcdbc0b65d0 0x7fcdbc0b8ae0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0005d00 tx=0x7fcdc0005ef0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.516+0000 7fcdc6a00640 1 -- 172.21.15.16:0/3551188201 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdc8011920 con 0x56266fbdb1b0 2024-09-21T03:51:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.516+0000 7fcdc6a00640 1 -- 172.21.15.16:0/3551188201 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc80107f0 con 0x56266fbdb1b0 2024-09-21T03:51:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.517+0000 7fcde34bb740 1 -- 172.21.15.16:0/3551188201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 msgr2=0x56266ff82100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.517+0000 7fcde34bb740 1 --2- 172.21.15.16:0/3551188201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x56266ff82100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009bc0 tx=0x7fcdc800e6c0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.522+0000 7fcde34bb740 1 -- 172.21.15.16:0/3551188201 shutdown_connections 2024-09-21T03:51:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.522+0000 7fcde34bb740 1 --2- 172.21.15.16:0/3551188201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fbba4d0 0x56266ff832c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.522+0000 7fcde34bb740 1 --2- 172.21.15.16:0/3551188201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbb22f0 0x56266ff82770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.522+0000 7fcde34bb740 1 --2- 172.21.15.16:0/3551188201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x56266ff82100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.522+0000 7fcde34bb740 1 -- 172.21.15.16:0/3551188201 >> 172.21.15.16:0/3551188201 conn(0x56266fbcf300 msgr2=0x56266ff7db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.522+0000 7fcde34bb740 1 -- 172.21.15.16:0/3551188201 shutdown_connections 2024-09-21T03:51:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.522+0000 7fcde34bb740 1 -- 172.21.15.16:0/3551188201 wait complete. 2024-09-21T03:51:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.523+0000 7fcde34bb740 1 -- start start 2024-09-21T03:51:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.524+0000 7fcde34bb740 1 Processor -- start 2024-09-21T03:51:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.526+0000 7fcde34bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbb22f0 0x56266ff9c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.527+0000 7fcde34bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fbba4d0 0x56266ff9cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.527+0000 7fcde34bb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x56266ff9da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.527+0000 7fcde34bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56266fe1cab0 con 0x56266fbb22f0 2024-09-21T03:51:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.527+0000 7fcde34bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56266fe8b570 con 0x56266fbba4d0 2024-09-21T03:51:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.527+0000 7fcde34bb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56266ff83de0 con 0x56266fbdb1b0 2024-09-21T03:51:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.527+0000 7fcdda400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x56266ff9da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.527+0000 7fcdda400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x56266ff9da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39242/0 (socket says 172.21.15.16:39242) 2024-09-21T03:51:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.527+0000 7fcdda400640 1 -- 172.21.15.16:0/2254404151 learned_addr learned my addr 172.21.15.16:0/2254404151 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.528+0000 7fcdd9000640 1 --2- 172.21.15.16:0/2254404151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fbba4d0 0x56266ff9cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.528+0000 7fcde34bb740 10 client.0 _ll_register_callbacks cb 0x56266fe9a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.528+0000 7fcdd9a00640 1 --2- 172.21.15.16:0/2254404151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbb22f0 0x56266ff9c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.528+0000 7fcdda400640 1 -- 172.21.15.16:0/2254404151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fbba4d0 msgr2=0x56266ff9cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.528+0000 7fcdda400640 1 --2- 172.21.15.16:0/2254404151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fbba4d0 0x56266ff9cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.529+0000 7fcdda400640 1 -- 172.21.15.16:0/2254404151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbb22f0 msgr2=0x56266ff9c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.529+0000 7fcdda400640 1 --2- 172.21.15.16:0/2254404151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbb22f0 0x56266ff9c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.529+0000 7fcdda400640 1 -- 172.21.15.16:0/2254404151 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56266fc46790 con 0x56266fbdb1b0 2024-09-21T03:51:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.529+0000 7fcdd9a00640 1 --2- 172.21.15.16:0/2254404151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbb22f0 0x56266ff9c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.529+0000 7fcdda400640 1 --2- 172.21.15.16:0/2254404151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x56266ff9da70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc105f80 tx=0x7fcdcc030fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aa92d2c870593d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.530+0000 7fcdbb400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:51:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.530+0000 7fcdbb400640 1 -- 172.21.15.16:0/2254404151 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0ce0d0 con 0x56266fbdb1b0 2024-09-21T03:51:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.530+0000 7fcdbb400640 1 -- 172.21.15.16:0/2254404151 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fcdcc0d7040 con 0x56266fbdb1b0 2024-09-21T03:51:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.530+0000 7fcde34bb740 1 -- 172.21.15.16:0/2254404151 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc8011920 con 0x56266fbdb1b0 2024-09-21T03:51:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.531+0000 7fcdbb400640 1 -- 172.21.15.16:0/2254404151 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcdcc0e9050 con 0x56266fbdb1b0 2024-09-21T03:51:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.532+0000 7fcde34bb740 10 client.26014 fetch_fsmap learned FSMap version 30 2024-09-21T03:51:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.532+0000 7fcde34bb740 1 -- 172.21.15.16:0/2254404151 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56266fcbf620 con 0x56266fbdb1b0 2024-09-21T03:51:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.532+0000 7fcdbb400640 1 -- 172.21.15.16:0/2254404151 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdc8011920 con 0x56266fbdb1b0 2024-09-21T03:51:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.532+0000 7fcdbb400640 1 -- 172.21.15.16:0/2254404151 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcdcc04d030 con 0x56266fbdb1b0 2024-09-21T03:51:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.532+0000 7fcde34bb740 10 client.26014 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:51:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.533+0000 7fcde34bb740 10 client.26014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.533+0000 7fcde34bb740 20 client.26014 populate_metadata read hostname 'smithi016' 2024-09-21T03:51:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.533+0000 7fcde34bb740 1 --2- 172.21.15.16:0/2254404151 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56266fb99d80 0x56266ffa9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.533+0000 7fcde34bb740 4 client.26014 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.533+0000 7fcde34bb740 1 -- 172.21.15.16:0/2254404151 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56266fef7600 con 0x56266fb99d80 2024-09-21T03:51:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.534+0000 7fcdd9a00640 1 --2- 172.21.15.16:0/2254404151 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56266fb99d80 0x56266ffa9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.535+0000 7fcdd9a00640 1 --2- 172.21.15.16:0/2254404151 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56266fb99d80 0x56266ffa9fb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dd7ced58f39ac20 server_cookie=61d3115648e3b172 in_seq=0 out_seq=0 2024-09-21T03:51:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.535+0000 7fcdbb400640 10 client.26014 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.536+0000 7fcdbb400640 1 -- 172.21.15.16:0/2254404151 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56266fef7600 con 0x56266fb99d80 2024-09-21T03:51:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.536+0000 7fcdbb400640 10 client.26014 handle_command_reply: tid=0 2024-09-21T03:51:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.536+0000 7fcde34bb740 1 client.26014 shutdown 2024-09-21T03:51:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.538+0000 7fcde34bb740 1 -- 172.21.15.16:0/2254404151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 msgr2=0x56266ff9da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.538+0000 7fcde34bb740 1 --2- 172.21.15.16:0/2254404151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x56266ff9da70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc105f80 tx=0x7fcdcc030fc0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.538+0000 7fcde34bb740 1 -- 172.21.15.16:0/2254404151 shutdown_connections 2024-09-21T03:51:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.538+0000 7fcde34bb740 1 --2- 172.21.15.16:0/2254404151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x56266ff9da70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.539+0000 7fcde34bb740 1 --2- 172.21.15.16:0/2254404151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fbba4d0 0x56266ff9cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.539+0000 7fcde34bb740 1 --2- 172.21.15.16:0/2254404151 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56266fb99d80 0x56266ffa9fb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.539+0000 7fcde34bb740 1 --2- 172.21.15.16:0/2254404151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbb22f0 0x56266ff9c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.539+0000 7fcdbb400640 0 client.26014 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.539+0000 7fcde34bb740 1 -- 172.21.15.16:0/2254404151 >> 172.21.15.16:0/2254404151 conn(0x56266fbcf300 msgr2=0x56266ff7c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.539+0000 7fcde34bb740 1 -- 172.21.15.16:0/2254404151 shutdown_connections 2024-09-21T03:51:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.540+0000 7fcde34bb740 1 -- 172.21.15.16:0/2254404151 wait complete. 2024-09-21T03:51:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.541+0000 7fcde34bb740 20 client.26014 trim_cache size 0 max 16384 2024-09-21T03:51:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.559+0000 7fcde34bb740 1 Processor -- start 2024-09-21T03:51:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.559+0000 7fcde34bb740 1 -- start start 2024-09-21T03:51:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.559+0000 7fcde34bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbb22f0 0x56266ff86b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.559+0000 7fcde34bb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbba4d0 0x56266ff82400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.559+0000 7fcde34bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fbdb1b0 0x56266ff886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.560+0000 7fcdd9000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbba4d0 0x56266ff82400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.560+0000 7fcde34bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41aa520 con 0x56266fbb22f0 2024-09-21T03:51:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.560+0000 7fcde34bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41ac060 con 0x56266fbdb1b0 2024-09-21T03:51:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.560+0000 7fcde34bb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41aabf0 con 0x56266fbba4d0 2024-09-21T03:51:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.560+0000 7fcdd9000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbba4d0 0x56266ff82400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39250/0 (socket says 172.21.15.16:39250) 2024-09-21T03:51:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.560+0000 7fcdd9000640 1 -- 172.21.15.16:0/2741604094 learned_addr learned my addr 172.21.15.16:0/2741604094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.560+0000 7fcdda400640 1 --2- 172.21.15.16:0/2741604094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fbdb1b0 0x56266ff886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.561+0000 7fcdd9000640 1 -- 172.21.15.16:0/2741604094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fbdb1b0 msgr2=0x56266ff886b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.561+0000 7fcdd9000640 1 --2- 172.21.15.16:0/2741604094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fbdb1b0 0x56266ff886b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.561+0000 7fcdd9000640 1 -- 172.21.15.16:0/2741604094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbb22f0 msgr2=0x56266ff86b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:51:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.561+0000 7fcdd9000640 1 --2- 172.21.15.16:0/2741604094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbb22f0 0x56266ff86b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.561+0000 7fcdd9000640 1 -- 172.21.15.16:0/2741604094 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56266ff899c0 con 0x56266fbba4d0 2024-09-21T03:51:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.562+0000 7fcdd9000640 1 --2- 172.21.15.16:0/2741604094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbba4d0 0x56266ff82400 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000ef80 tx=0x7fcdc000af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ca310c338b848a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.562+0000 7fcdc6000640 1 -- 172.21.15.16:0/2741604094 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc001d630 con 0x56266fbba4d0 2024-09-21T03:51:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.563+0000 7fcdc6000640 1 -- 172.21.15.16:0/2741604094 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdc0021030 con 0x56266fbba4d0 2024-09-21T03:51:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.563+0000 7fcdc6000640 1 -- 172.21.15.16:0/2741604094 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc001cad0 con 0x56266fbba4d0 2024-09-21T03:51:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.564+0000 7fcde34bb740 1 -- 172.21.15.16:0/2741604094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbba4d0 msgr2=0x56266ff82400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.564+0000 7fcde34bb740 1 --2- 172.21.15.16:0/2741604094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbba4d0 0x56266ff82400 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000ef80 tx=0x7fcdc000af90 comp rx=0 tx=0).stop 2024-09-21T03:51:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.565+0000 7fcde34bb740 1 -- 172.21.15.16:0/2741604094 shutdown_connections 2024-09-21T03:51:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.565+0000 7fcde34bb740 1 --2- 172.21.15.16:0/2741604094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fbdb1b0 0x56266ff886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.565+0000 7fcde34bb740 1 --2- 172.21.15.16:0/2741604094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbba4d0 0x56266ff82400 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.565+0000 7fcde34bb740 1 --2- 172.21.15.16:0/2741604094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbb22f0 0x56266ff86b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.565+0000 7fcde34bb740 1 -- 172.21.15.16:0/2741604094 >> 172.21.15.16:0/2741604094 conn(0x56266fbcf300 msgr2=0x56266ff81cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.571+0000 7fcde34bb740 1 -- 172.21.15.16:0/2741604094 shutdown_connections 2024-09-21T03:51:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.571+0000 7fcde34bb740 1 -- 172.21.15.16:0/2741604094 wait complete. 2024-09-21T03:51:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.571+0000 7fcde34bb740 1 -- start start 2024-09-21T03:51:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.572+0000 7fcde34bb740 1 Processor -- start 2024-09-21T03:51:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.573+0000 7fcde34bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbba4d0 0x56266ffa4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.573+0000 7fcde34bb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x5626700109b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.574+0000 7fcde34bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fb6f1c0 0x5626700130d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.575+0000 7fcdd9000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x5626700109b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.575+0000 7fcdd9000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x5626700109b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39254/0 (socket says 172.21.15.16:39254) 2024-09-21T03:51:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.575+0000 7fcdd9000640 1 -- 172.21.15.16:0/524974357 learned_addr learned my addr 172.21.15.16:0/524974357 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.575+0000 7fcdda400640 1 --2- 172.21.15.16:0/524974357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fb6f1c0 0x5626700130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.575+0000 7fcde34bb740 1 -- 172.21.15.16:0/524974357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41ab990 con 0x56266fbba4d0 2024-09-21T03:51:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.575+0000 7fcdd9a00640 1 --2- 172.21.15.16:0/524974357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbba4d0 0x56266ffa4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.576+0000 7fcde34bb740 1 -- 172.21.15.16:0/524974357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41ab2c0 con 0x56266fb6f1c0 2024-09-21T03:51:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.576+0000 7fcde34bb740 1 -- 172.21.15.16:0/524974357 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41a60e0 con 0x56266fbdb1b0 2024-09-21T03:51:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.576+0000 7fcde34bb740 10 client.0 _ll_register_callbacks cb 0x7fcdcc009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.577+0000 7fcdd9a00640 1 -- 172.21.15.16:0/524974357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 msgr2=0x5626700109b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.577+0000 7fcdd9a00640 1 --2- 172.21.15.16:0/524974357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x5626700109b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.577+0000 7fcdd9a00640 1 -- 172.21.15.16:0/524974357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fb6f1c0 msgr2=0x5626700130d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.577+0000 7fcdd9a00640 1 --2- 172.21.15.16:0/524974357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fb6f1c0 0x5626700130d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.577+0000 7fcdd9a00640 1 -- 172.21.15.16:0/524974357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562670013860 con 0x56266fbba4d0 2024-09-21T03:51:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.577+0000 7fcdd9000640 1 --2- 172.21.15.16:0/524974357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x5626700109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.578+0000 7fcdda400640 1 --2- 172.21.15.16:0/524974357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fb6f1c0 0x5626700130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.578+0000 7fcdbb400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:51:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.578+0000 7fcdd9a00640 1 --2- 172.21.15.16:0/524974357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbba4d0 0x56266ffa4330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x56266ffaa710 tx=0x7fcdc8005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4c0eefb9a2970f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.580+0000 7fcdbb400640 1 -- 172.21.15.16:0/524974357 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8029040 con 0x56266fbba4d0 2024-09-21T03:51:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.580+0000 7fcdbb400640 1 -- 172.21.15.16:0/524974357 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fcdc80055c0 con 0x56266fbba4d0 2024-09-21T03:51:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.580+0000 7fcde34bb740 1 -- 172.21.15.16:0/524974357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc0021030 con 0x56266fbba4d0 2024-09-21T03:51:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.581+0000 7fcdbb400640 1 -- 172.21.15.16:0/524974357 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcdc8034050 con 0x56266fbba4d0 2024-09-21T03:51:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.581+0000 7fcde34bb740 10 client.15990 fetch_fsmap learned FSMap version 30 2024-09-21T03:51:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.581+0000 7fcde34bb740 1 -- 172.21.15.16:0/524974357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562670014b60 con 0x56266fbba4d0 2024-09-21T03:51:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.582+0000 7fcdbb400640 1 -- 172.21.15.16:0/524974357 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdc0021030 con 0x56266fbba4d0 2024-09-21T03:51:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.582+0000 7fcdbb400640 1 -- 172.21.15.16:0/524974357 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcdc800f040 con 0x56266fbba4d0 2024-09-21T03:51:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.583+0000 7fcde34bb740 10 client.15990 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:51:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.583+0000 7fcde34bb740 10 client.15990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.583+0000 7fcde34bb740 20 client.15990 populate_metadata read hostname 'smithi016' 2024-09-21T03:51:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.583+0000 7fcde34bb740 1 --2- 172.21.15.16:0/524974357 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562670015190 0x5626700175b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.583+0000 7fcde34bb740 4 client.15990 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:51:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.583+0000 7fcde34bb740 1 -- 172.21.15.16:0/524974357 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56266fef30b0 con 0x562670015190 2024-09-21T03:51:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.583+0000 7fcdd9000640 1 --2- 172.21.15.16:0/524974357 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562670015190 0x5626700175b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.585+0000 7fcdd9000640 1 --2- 172.21.15.16:0/524974357 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562670015190 0x5626700175b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68aba43516a649ee server_cookie=d6a3b2ad4fc3db4f in_seq=0 out_seq=0 2024-09-21T03:51:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.585+0000 7fcdbb400640 10 client.15990 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.613+0000 7fcdbb400640 1 -- 172.21.15.16:0/524974357 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56266fef30b0 con 0x562670015190 2024-09-21T03:51:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.613+0000 7fcdbb400640 10 client.15990 handle_command_reply: tid=0 2024-09-21T03:51:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.613+0000 7fcde34bb740 1 client.15990 shutdown 2024-09-21T03:51:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.614+0000 7fcde34bb740 1 -- 172.21.15.16:0/524974357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbba4d0 msgr2=0x56266ffa4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.614+0000 7fcde34bb740 1 --2- 172.21.15.16:0/524974357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbba4d0 0x56266ffa4330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x56266ffaa710 tx=0x7fcdc8005250 comp rx=0 tx=0).stop 2024-09-21T03:51:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.614+0000 7fcde34bb740 1 -- 172.21.15.16:0/524974357 shutdown_connections 2024-09-21T03:51:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.614+0000 7fcde34bb740 1 --2- 172.21.15.16:0/524974357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56266fb6f1c0 0x5626700130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.614+0000 7fcde34bb740 1 --2- 172.21.15.16:0/524974357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56266fbdb1b0 0x5626700109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.614+0000 7fcde34bb740 1 --2- 172.21.15.16:0/524974357 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562670015190 0x5626700175b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.614+0000 7fcde34bb740 1 --2- 172.21.15.16:0/524974357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56266fbba4d0 0x56266ffa4330 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.614+0000 7fcdbb400640 0 client.15990 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.614+0000 7fcde34bb740 1 -- 172.21.15.16:0/524974357 >> 172.21.15.16:0/524974357 conn(0x56266fbcf300 msgr2=0x56266ff86950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.615+0000 7fcde34bb740 1 -- 172.21.15.16:0/524974357 shutdown_connections 2024-09-21T03:51:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.615+0000 7fcde34bb740 1 -- 172.21.15.16:0/524974357 wait complete. 2024-09-21T03:51:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.615+0000 7fcde34bb740 20 client.15990 trim_cache size 0 max 16384 2024-09-21T03:51:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:51:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:51:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:51:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:51:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.616+0000 7fcddc400640 1 -- 172.21.15.16:0/734716786 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcdbc0b65d0 msgr2=0x7fcdbc0b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.616+0000 7fcddc400640 1 --2- 172.21.15.16:0/734716786 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcdbc0b65d0 0x7fcdbc0b8ae0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0005d00 tx=0x7fcdc0005ef0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.616+0000 7fcddc400640 1 -- 172.21.15.16:0/734716786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd4109480 msgr2=0x7fcdd4102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.616+0000 7fcddc400640 1 --2- 172.21.15.16:0/734716786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd4109480 0x7fcdd4102f60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0307a0 tx=0x7fcdcc003b80 comp rx=0 tx=0).stop 2024-09-21T03:51:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.617+0000 7fcddc400640 1 -- 172.21.15.16:0/734716786 shutdown_connections 2024-09-21T03:51:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.617+0000 7fcddc400640 1 --2- 172.21.15.16:0/734716786 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fcdbc0b65d0 0x7fcdbc0b8ae0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.617+0000 7fcddc400640 1 --2- 172.21.15.16:0/734716786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd4109480 0x7fcdd4102f60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.617+0000 7fcddc400640 1 --2- 172.21.15.16:0/734716786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdd4106b20 0x7fcdd4102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.617+0000 7fcddc400640 1 --2- 172.21.15.16:0/734716786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdd4101a90 0x7fcdd4104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.617+0000 7fcddc400640 1 -- 172.21.15.16:0/734716786 >> 172.21.15.16:0/734716786 conn(0x7fcdd40fd690 msgr2=0x7fcdd41078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.617+0000 7fcddc400640 1 -- 172.21.15.16:0/734716786 shutdown_connections 2024-09-21T03:51:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:11.617+0000 7fcddc400640 1 -- 172.21.15.16:0/734716786 wait complete. 2024-09-21T03:51:13.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:12 smithi134 ceph-mon[35031]: pgmap v565: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 232 MiB/s wr, 1.05k op/s 2024-09-21T03:51:13.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:12 smithi123 ceph-mon[34673]: pgmap v565: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 232 MiB/s wr, 1.05k op/s 2024-09-21T03:51:13.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:12 smithi016 ceph-mon[30375]: pgmap v565: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 232 MiB/s wr, 1.05k op/s 2024-09-21T03:51:15.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:14 smithi134 ceph-mon[35031]: pgmap v566: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 239 MiB/s wr, 1.08k op/s 2024-09-21T03:51:15.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:14 smithi123 ceph-mon[34673]: pgmap v566: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 239 MiB/s wr, 1.08k op/s 2024-09-21T03:51:15.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:14 smithi016 ceph-mon[30375]: pgmap v566: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 239 MiB/s wr, 1.08k op/s 2024-09-21T03:51:17.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:16 smithi123 ceph-mon[34673]: pgmap v567: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 246 MiB/s wr, 1.11k op/s 2024-09-21T03:51:17.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:16 smithi016 ceph-mon[30375]: pgmap v567: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 246 MiB/s wr, 1.11k op/s 2024-09-21T03:51:17.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:16 smithi134 ceph-mon[35031]: pgmap v567: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 246 MiB/s wr, 1.11k op/s 2024-09-21T03:51:19.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:18 smithi123 ceph-mon[34673]: pgmap v568: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 233 MiB/s wr, 1.06k op/s 2024-09-21T03:51:19.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:18 smithi016 ceph-mon[30375]: pgmap v568: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 233 MiB/s wr, 1.06k op/s 2024-09-21T03:51:19.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:18 smithi134 ceph-mon[35031]: pgmap v568: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 233 MiB/s wr, 1.06k op/s 2024-09-21T03:51:21.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:20 smithi123 ceph-mon[34673]: pgmap v569: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 231 MiB/s wr, 1.06k op/s 2024-09-21T03:51:21.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:20 smithi016 ceph-mon[30375]: pgmap v569: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 231 MiB/s wr, 1.06k op/s 2024-09-21T03:51:21.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:20 smithi134 ceph-mon[35031]: pgmap v569: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 231 MiB/s wr, 1.06k op/s 2024-09-21T03:51:22.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:21 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:51:22.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:21 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:51:22.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:21 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:51:23.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:22 smithi134 ceph-mon[35031]: pgmap v570: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 241 MiB/s wr, 1.11k op/s 2024-09-21T03:51:23.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:51:23.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:51:23.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:51:23.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:51:23.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:51:23.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:51:23.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:51:23.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:22 smithi123 ceph-mon[34673]: pgmap v570: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 241 MiB/s wr, 1.11k op/s 2024-09-21T03:51:23.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:51:23.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:51:23.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:51:23.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:51:23.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:51:23.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:51:23.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:51:23.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:22 smithi016 ceph-mon[30375]: pgmap v570: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 241 MiB/s wr, 1.11k op/s 2024-09-21T03:51:23.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:51:23.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:51:23.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:51:23.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:51:23.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:51:23.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:51:23.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:51:24.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:23 smithi134 ceph-mon[35031]: Adjusting osd_memory_target on smithi016 to 177.5M 2024-09-21T03:51:24.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:23 smithi134 ceph-mon[35031]: Unable to set osd_memory_target on smithi016 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-21T03:51:24.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:23 smithi123 ceph-mon[34673]: Adjusting osd_memory_target on smithi016 to 177.5M 2024-09-21T03:51:24.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:23 smithi123 ceph-mon[34673]: Unable to set osd_memory_target on smithi016 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-21T03:51:24.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:23 smithi016 ceph-mon[30375]: Adjusting osd_memory_target on smithi016 to 177.5M 2024-09-21T03:51:24.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:23 smithi016 ceph-mon[30375]: Unable to set osd_memory_target on smithi016 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-21T03:51:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.925+0000 7f0439a00640 1 -- 172.21.15.16:0/1334844135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0434077c20 msgr2=0x7f0434078090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.925+0000 7f0439a00640 1 --2- 172.21.15.16:0/1334844135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0434077c20 0x7f0434078090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0424009900 tx=0x7f042402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.925+0000 7f0439a00640 1 -- 172.21.15.16:0/1334844135 shutdown_connections 2024-09-21T03:51:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.925+0000 7f0439a00640 1 --2- 172.21.15.16:0/1334844135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0434079640 0x7f0434079fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.925+0000 7f0439a00640 1 --2- 172.21.15.16:0/1334844135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0434078ca0 0x7f04340790b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.925+0000 7f0439a00640 1 --2- 172.21.15.16:0/1334844135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0434077c20 0x7f0434078090 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.925+0000 7f0439a00640 1 -- 172.21.15.16:0/1334844135 >> 172.21.15.16:0/1334844135 conn(0x7f0434101230 msgr2=0x7f04341036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.925+0000 7f0439a00640 1 -- 172.21.15.16:0/1334844135 shutdown_connections 2024-09-21T03:51:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.925+0000 7f0439a00640 1 -- 172.21.15.16:0/1334844135 wait complete. 2024-09-21T03:51:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.927+0000 7f0439a00640 1 Processor -- start 2024-09-21T03:51:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.928+0000 7f0439a00640 1 -- start start 2024-09-21T03:51:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.928+0000 7f0439a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0434077c20 0x7f04341109b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.928+0000 7f0439a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0434078ca0 0x7f0434111020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.928+0000 7f0439a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0434079640 0x7f0434111bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.928+0000 7f0439a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0434115990 con 0x7f0434077c20 2024-09-21T03:51:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.928+0000 7f0439a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0434115810 con 0x7f0434079640 2024-09-21T03:51:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.928+0000 7f0439a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0434115b10 con 0x7f0434078ca0 2024-09-21T03:51:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.928+0000 7f0432000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0434078ca0 0x7f0434111020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.929+0000 7f0432000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0434078ca0 0x7f0434111020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:47848/0 (socket says 172.21.15.16:47848) 2024-09-21T03:51:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.929+0000 7f0432000640 1 -- 172.21.15.16:0/420079565 learned_addr learned my addr 172.21.15.16:0/420079565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.929+0000 7f0432a00640 1 --2- 172.21.15.16:0/420079565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0434077c20 0x7f04341109b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.929+0000 7f0433400640 1 --2- 172.21.15.16:0/420079565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0434079640 0x7f0434111bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.929+0000 7f0432000640 1 -- 172.21.15.16:0/420079565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0434079640 msgr2=0x7f0434111bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.929+0000 7f0432000640 1 --2- 172.21.15.16:0/420079565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0434079640 0x7f0434111bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.929+0000 7f0432000640 1 -- 172.21.15.16:0/420079565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0434077c20 msgr2=0x7f04341109b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.929+0000 7f0432000640 1 --2- 172.21.15.16:0/420079565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0434077c20 0x7f04341109b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.929+0000 7f0432000640 1 -- 172.21.15.16:0/420079565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04341123f0 con 0x7f0434078ca0 2024-09-21T03:51:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.929+0000 7f0432000640 1 --2- 172.21.15.16:0/420079565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0434078ca0 0x7f0434111020 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f042000d950 tx=0x7f042000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.933+0000 7f042b400640 1 -- 172.21.15.16:0/420079565 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0420017030 con 0x7f0434078ca0 2024-09-21T03:51:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.933+0000 7f042b400640 1 -- 172.21.15.16:0/420079565 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0420004020 con 0x7f0434078ca0 2024-09-21T03:51:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.933+0000 7f042b400640 1 -- 172.21.15.16:0/420079565 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04200047a0 con 0x7f0434078ca0 2024-09-21T03:51:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.933+0000 7f0439a00640 1 -- 172.21.15.16:0/420079565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04341126e0 con 0x7f0434078ca0 2024-09-21T03:51:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.933+0000 7f0439a00640 1 -- 172.21.15.16:0/420079565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0434112da0 con 0x7f0434078ca0 2024-09-21T03:51:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.938+0000 7f0440a66740 1 Processor -- start 2024-09-21T03:51:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.939+0000 7f042b400640 1 -- 172.21.15.16:0/420079565 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0420009a30 con 0x7f0434078ca0 2024-09-21T03:51:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.939+0000 7f0440a66740 1 -- start start 2024-09-21T03:51:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.939+0000 7f0440a66740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3331b0 0x55cefa6da0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.939+0000 7f0440a66740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa30a2f0 0x55cefa6da740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.939+0000 7f0440a66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa3124d0 0x55cefa6db290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.939+0000 7f0440a66740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cefa64d6b0 con 0x55cefa30a2f0 2024-09-21T03:51:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.939+0000 7f0440a66740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cefa63cae0 con 0x55cefa3331b0 2024-09-21T03:51:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.939+0000 7f0440a66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cefa6408f0 con 0x55cefa3124d0 2024-09-21T03:51:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.940+0000 7f0432a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa30a2f0 0x55cefa6da740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.940+0000 7f0432000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa3124d0 0x55cefa6db290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.940+0000 7f0433400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3331b0 0x55cefa6da0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.940+0000 7f0432000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa3124d0 0x55cefa6db290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:47856/0 (socket says 172.21.15.16:47856) 2024-09-21T03:51:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.940+0000 7f0432000640 1 -- 172.21.15.16:0/1422780474 learned_addr learned my addr 172.21.15.16:0/1422780474 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.940+0000 7f0433400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3331b0 0x55cefa6da0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:53072/0 (socket says 172.21.15.16:53072) 2024-09-21T03:51:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.940+0000 7f0432000640 1 -- 172.21.15.16:0/1422780474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3331b0 msgr2=0x55cefa6da0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.940+0000 7f0432000640 1 --2- 172.21.15.16:0/1422780474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3331b0 0x55cefa6da0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.940+0000 7f0432000640 1 -- 172.21.15.16:0/1422780474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa30a2f0 msgr2=0x55cefa6da740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.940+0000 7f0432000640 1 --2- 172.21.15.16:0/1422780474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa30a2f0 0x55cefa6da740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.940+0000 7f0432000640 1 -- 172.21.15.16:0/1422780474 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cefa609780 con 0x55cefa3124d0 2024-09-21T03:51:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.941+0000 7f0432000640 1 --2- 172.21.15.16:0/1422780474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa3124d0 0x55cefa6db290 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f04200eaf70 tx=0x7f04200ef570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31a2eab4b732ebe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.941+0000 7f041fe00640 1 -- 172.21.15.16:0/1422780474 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04200fa070 con 0x55cefa3124d0 2024-09-21T03:51:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.942+0000 7f041fe00640 1 -- 172.21.15.16:0/1422780474 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04200e92d0 con 0x55cefa3124d0 2024-09-21T03:51:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.942+0000 7f041fe00640 1 -- 172.21.15.16:0/1422780474 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04200f5750 con 0x55cefa3124d0 2024-09-21T03:51:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.942+0000 7f042b400640 1 --2- 172.21.15.16:0/420079565 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04180b6610 0x7f04180b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.942+0000 7f042b400640 1 -- 172.21.15.16:0/420079565 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f04200e70e0 con 0x7f0434078ca0 2024-09-21T03:51:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.943+0000 7f0440a66740 1 -- 172.21.15.16:0/1422780474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa3124d0 msgr2=0x55cefa6db290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.943+0000 7f0440a66740 1 --2- 172.21.15.16:0/1422780474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa3124d0 0x55cefa6db290 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f04200eaf70 tx=0x7f04200ef570 comp rx=0 tx=0).stop 2024-09-21T03:51:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.944+0000 7f0433400640 1 --2- 172.21.15.16:0/420079565 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04180b6610 0x7f04180b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.944+0000 7f0440a66740 1 -- 172.21.15.16:0/1422780474 shutdown_connections 2024-09-21T03:51:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.944+0000 7f0440a66740 1 --2- 172.21.15.16:0/1422780474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa3124d0 0x55cefa6db290 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.944+0000 7f0440a66740 1 --2- 172.21.15.16:0/1422780474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa30a2f0 0x55cefa6da740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.944+0000 7f0440a66740 1 --2- 172.21.15.16:0/1422780474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3331b0 0x55cefa6da0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.944+0000 7f0440a66740 1 -- 172.21.15.16:0/1422780474 >> 172.21.15.16:0/1422780474 conn(0x55cefa327300 msgr2=0x55cefa6d5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.944+0000 7f0440a66740 1 -- 172.21.15.16:0/1422780474 shutdown_connections 2024-09-21T03:51:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.945+0000 7f0440a66740 1 -- 172.21.15.16:0/1422780474 wait complete. 2024-09-21T03:51:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.946+0000 7f0440a66740 1 -- start start 2024-09-21T03:51:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.946+0000 7f0433400640 1 --2- 172.21.15.16:0/420079565 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04180b6610 0x7f04180b8b20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f042c005fd0 tx=0x7f042c009500 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.947+0000 7f0440a66740 1 Processor -- start 2024-09-21T03:51:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.948+0000 7f0440a66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa30a2f0 0x55cefa6e3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.948+0000 7f0440a66740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3124d0 0x55cefa6e3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.948+0000 7f0440a66740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa3331b0 0x55cefa6e4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.948+0000 7f0440a66740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cefa574ab0 con 0x55cefa3331b0 2024-09-21T03:51:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.948+0000 7f0440a66740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cefa5e3570 con 0x55cefa3124d0 2024-09-21T03:51:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.948+0000 7f0440a66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cefa6dbdb0 con 0x55cefa30a2f0 2024-09-21T03:51:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.948+0000 7f0440a66740 10 client.0 _ll_register_callbacks cb 0x55cefa5f2100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.949+0000 7f0432000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3124d0 0x55cefa6e3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.949+0000 7f0432000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3124d0 0x55cefa6e3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:53078/0 (socket says 172.21.15.16:53078) 2024-09-21T03:51:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.949+0000 7f0432000640 1 -- 172.21.15.16:0/1913617506 learned_addr learned my addr 172.21.15.16:0/1913617506 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.949+0000 7f0432a00640 1 --2- 172.21.15.16:0/1913617506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa30a2f0 0x55cefa6e3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.950+0000 7f0433400640 1 --2- 172.21.15.16:0/1913617506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa3331b0 0x55cefa6e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.950+0000 7f0432000640 1 -- 172.21.15.16:0/1913617506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa30a2f0 msgr2=0x55cefa6e3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.950+0000 7f0432000640 1 --2- 172.21.15.16:0/1913617506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa30a2f0 0x55cefa6e3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.950+0000 7f0432000640 1 -- 172.21.15.16:0/1913617506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa3331b0 msgr2=0x55cefa6e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.950+0000 7f0432000640 1 --2- 172.21.15.16:0/1913617506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa3331b0 0x55cefa6e4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.950+0000 7f0432000640 1 -- 172.21.15.16:0/1913617506 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cefa39e790 con 0x55cefa3124d0 2024-09-21T03:51:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.950+0000 7f0433400640 1 --2- 172.21.15.16:0/1913617506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa3331b0 0x55cefa6e4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.951+0000 7f0432000640 1 --2- 172.21.15.16:0/1913617506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3124d0 0x55cefa6e3ed0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0420005fd0 tx=0x7f042000df40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48f3089c73a9e42a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.951+0000 7f041cc00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:51:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.951+0000 7f041cc00640 1 -- 172.21.15.16:0/1913617506 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04200a94b0 con 0x55cefa3124d0 2024-09-21T03:51:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.951+0000 7f041cc00640 1 -- 172.21.15.16:0/1913617506 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f04200b1610 con 0x55cefa3124d0 2024-09-21T03:51:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.951+0000 7f0440a66740 1 -- 172.21.15.16:0/1913617506 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04200e92d0 con 0x55cefa3124d0 2024-09-21T03:51:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.952+0000 7f041cc00640 1 -- 172.21.15.16:0/1913617506 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f04200ac5d0 con 0x55cefa3124d0 2024-09-21T03:51:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.952+0000 7f0440a66740 10 client.26093 fetch_fsmap learned FSMap version 30 2024-09-21T03:51:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.952+0000 7f0440a66740 1 -- 172.21.15.16:0/1913617506 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cefa417620 con 0x55cefa3124d0 2024-09-21T03:51:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.952+0000 7f041cc00640 1 -- 172.21.15.16:0/1913617506 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04200e92d0 con 0x55cefa3124d0 2024-09-21T03:51:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.953+0000 7f041cc00640 1 -- 172.21.15.16:0/1913617506 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f04200a90d0 con 0x55cefa3124d0 2024-09-21T03:51:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.953+0000 7f0440a66740 10 client.26093 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:51:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.953+0000 7f0440a66740 10 client.26093 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.953+0000 7f0440a66740 20 client.26093 populate_metadata read hostname 'smithi016' 2024-09-21T03:51:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.953+0000 7f0440a66740 1 --2- 172.21.15.16:0/1913617506 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cefa2f1d80 0x55cefa6f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.953+0000 7f0440a66740 4 client.26093 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.953+0000 7f0440a66740 1 -- 172.21.15.16:0/1913617506 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cefa64f600 con 0x55cefa2f1d80 2024-09-21T03:51:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.954+0000 7f0432a00640 1 --2- 172.21.15.16:0/1913617506 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cefa2f1d80 0x55cefa6f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.954+0000 7f0432a00640 1 --2- 172.21.15.16:0/1913617506 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cefa2f1d80 0x55cefa6f0b20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3bd6b21e2962363 server_cookie=3c9aad352b59336e in_seq=0 out_seq=0 2024-09-21T03:51:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.954+0000 7f041cc00640 10 client.26093 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.955+0000 7f041cc00640 1 -- 172.21.15.16:0/1913617506 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cefa64f600 con 0x55cefa2f1d80 2024-09-21T03:51:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.955+0000 7f041cc00640 10 client.26093 handle_command_reply: tid=0 2024-09-21T03:51:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.956+0000 7f0440a66740 1 client.26093 shutdown 2024-09-21T03:51:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.956+0000 7f0440a66740 1 -- 172.21.15.16:0/1913617506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3124d0 msgr2=0x55cefa6e3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.956+0000 7f0440a66740 1 --2- 172.21.15.16:0/1913617506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3124d0 0x55cefa6e3ed0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0420005fd0 tx=0x7f042000df40 comp rx=0 tx=0).stop 2024-09-21T03:51:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.956+0000 7f0440a66740 1 -- 172.21.15.16:0/1913617506 shutdown_connections 2024-09-21T03:51:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.957+0000 7f0440a66740 1 --2- 172.21.15.16:0/1913617506 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cefa2f1d80 0x55cefa6f0b20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.957+0000 7f0440a66740 1 --2- 172.21.15.16:0/1913617506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa3331b0 0x55cefa6e4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.957+0000 7f0440a66740 1 --2- 172.21.15.16:0/1913617506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3124d0 0x55cefa6e3ed0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.957+0000 7f0440a66740 1 --2- 172.21.15.16:0/1913617506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa30a2f0 0x55cefa6e3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.957+0000 7f0440a66740 1 -- 172.21.15.16:0/1913617506 >> 172.21.15.16:0/1913617506 conn(0x55cefa327300 msgr2=0x55cefa6d40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.957+0000 7f041cc00640 0 client.26093 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.957+0000 7f0440a66740 1 -- 172.21.15.16:0/1913617506 shutdown_connections 2024-09-21T03:51:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.957+0000 7f0440a66740 1 -- 172.21.15.16:0/1913617506 wait complete. 2024-09-21T03:51:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.957+0000 7f0440a66740 20 client.26093 trim_cache size 0 max 16384 2024-09-21T03:51:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.993+0000 7f0440a66740 1 Processor -- start 2024-09-21T03:51:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.993+0000 7f0440a66740 1 -- start start 2024-09-21T03:51:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.994+0000 7f0440a66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa30a2f0 0x55cefa6e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.994+0000 7f0440a66740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa3124d0 0x55cefa6eb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.994+0000 7f0440a66740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3331b0 0x55cefa6d9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.994+0000 7f0440a66740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04341bb510 con 0x55cefa3124d0 2024-09-21T03:51:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.994+0000 7f0440a66740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04341bd050 con 0x55cefa3331b0 2024-09-21T03:51:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.994+0000 7f0440a66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04341bbbe0 con 0x55cefa30a2f0 2024-09-21T03:51:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.994+0000 7f0432a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa30a2f0 0x55cefa6e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.995+0000 7f0433400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3331b0 0x55cefa6d9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.995+0000 7f0433400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3331b0 0x55cefa6d9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:53082/0 (socket says 172.21.15.16:53082) 2024-09-21T03:51:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.995+0000 7f0433400640 1 -- 172.21.15.16:0/1814851530 learned_addr learned my addr 172.21.15.16:0/1814851530 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.996+0000 7f0432000640 1 --2- 172.21.15.16:0/1814851530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa3124d0 0x55cefa6eb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.997+0000 7f0432a00640 1 -- 172.21.15.16:0/1814851530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3331b0 msgr2=0x55cefa6d9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.997+0000 7f0432a00640 1 --2- 172.21.15.16:0/1814851530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3331b0 0x55cefa6d9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.997+0000 7f0432a00640 1 -- 172.21.15.16:0/1814851530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa3124d0 msgr2=0x55cefa6eb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.997+0000 7f0432a00640 1 --2- 172.21.15.16:0/1814851530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa3124d0 0x55cefa6eb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.997+0000 7f0432a00640 1 -- 172.21.15.16:0/1814851530 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cefa6cfd50 con 0x55cefa30a2f0 2024-09-21T03:51:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.997+0000 7f0432a00640 1 --2- 172.21.15.16:0/1814851530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa30a2f0 0x55cefa6e2a10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f042402a9e0 tx=0x7f0424005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb792132cd03d14c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:24.998+0000 7f041f400640 1 -- 172.21.15.16:0/1814851530 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424035040 con 0x55cefa30a2f0 2024-09-21T03:51:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.001+0000 7f041f400640 1 -- 172.21.15.16:0/1814851530 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0424003ca0 con 0x55cefa30a2f0 2024-09-21T03:51:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.001+0000 7f041f400640 1 -- 172.21.15.16:0/1814851530 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424031d90 con 0x55cefa30a2f0 2024-09-21T03:51:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.003+0000 7f0440a66740 1 -- 172.21.15.16:0/1814851530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa30a2f0 msgr2=0x55cefa6e2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.004+0000 7f0440a66740 1 --2- 172.21.15.16:0/1814851530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa30a2f0 0x55cefa6e2a10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f042402a9e0 tx=0x7f0424005850 comp rx=0 tx=0).stop 2024-09-21T03:51:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.005+0000 7f0440a66740 1 -- 172.21.15.16:0/1814851530 shutdown_connections 2024-09-21T03:51:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.005+0000 7f0440a66740 1 --2- 172.21.15.16:0/1814851530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa3331b0 0x55cefa6d9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.005+0000 7f0440a66740 1 --2- 172.21.15.16:0/1814851530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa3124d0 0x55cefa6eb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.005+0000 7f0440a66740 1 --2- 172.21.15.16:0/1814851530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa30a2f0 0x55cefa6e2a10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.005+0000 7f0440a66740 1 -- 172.21.15.16:0/1814851530 >> 172.21.15.16:0/1814851530 conn(0x55cefa327300 msgr2=0x55cefa6e5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.006+0000 7f0440a66740 1 -- 172.21.15.16:0/1814851530 shutdown_connections 2024-09-21T03:51:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.006+0000 7f0440a66740 1 -- 172.21.15.16:0/1814851530 wait complete. 2024-09-21T03:51:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.007+0000 7f0440a66740 1 -- start start 2024-09-21T03:51:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.007+0000 7f0440a66740 1 Processor -- start 2024-09-21T03:51:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.008+0000 7f0440a66740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa30a2f0 0x55cefa6e8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.008+0000 7f0440a66740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa3124d0 0x55cefa6e9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.008+0000 7f0440a66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa3331b0 0x55cefa6e9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.010+0000 7f0440a66740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04341bc980 con 0x55cefa3124d0 2024-09-21T03:51:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.010+0000 7f0433400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa3331b0 0x55cefa6e9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.010+0000 7f0432a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa30a2f0 0x55cefa6e8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.010+0000 7f0433400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa3331b0 0x55cefa6e9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:47890/0 (socket says 172.21.15.16:47890) 2024-09-21T03:51:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.010+0000 7f0433400640 1 -- 172.21.15.16:0/1659063751 learned_addr learned my addr 172.21.15.16:0/1659063751 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.010+0000 7f0432000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa3124d0 0x55cefa6e9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.010+0000 7f0440a66740 1 -- 172.21.15.16:0/1659063751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04341bc2b0 con 0x55cefa30a2f0 2024-09-21T03:51:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.010+0000 7f0440a66740 1 -- 172.21.15.16:0/1659063751 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04341b94f0 con 0x55cefa3331b0 2024-09-21T03:51:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.010+0000 7f0440a66740 10 client.0 _ll_register_callbacks cb 0x7f04200f8070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.011+0000 7f0432000640 1 -- 172.21.15.16:0/1659063751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa3331b0 msgr2=0x55cefa6e9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.011+0000 7f0432000640 1 --2- 172.21.15.16:0/1659063751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa3331b0 0x55cefa6e9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.011+0000 7f0432000640 1 -- 172.21.15.16:0/1659063751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa30a2f0 msgr2=0x55cefa6e8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.011+0000 7f0432000640 1 --2- 172.21.15.16:0/1659063751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa30a2f0 0x55cefa6e8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.011+0000 7f0432000640 1 -- 172.21.15.16:0/1659063751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cefa6ea2d0 con 0x55cefa3124d0 2024-09-21T03:51:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.012+0000 7f0432a00640 1 --2- 172.21.15.16:0/1659063751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa30a2f0 0x55cefa6e8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.012+0000 7f041cc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:51:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.012+0000 7f0432000640 1 --2- 172.21.15.16:0/1659063751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa3124d0 0x55cefa6e9000 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f04200aa0d0 tx=0x7f0420094380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce5c6c44f2564f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.012+0000 7f0433400640 1 --2- 172.21.15.16:0/1659063751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa3331b0 0x55cefa6e9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:51:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.013+0000 7f041cc00640 1 -- 172.21.15.16:0/1659063751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04200ac7b0 con 0x55cefa3124d0 2024-09-21T03:51:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.013+0000 7f0440a66740 1 -- 172.21.15.16:0/1659063751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0424003ca0 con 0x55cefa3124d0 2024-09-21T03:51:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.013+0000 7f041cc00640 1 -- 172.21.15.16:0/1659063751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f04200ea140 con 0x55cefa3124d0 2024-09-21T03:51:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.014+0000 7f041cc00640 1 -- 172.21.15.16:0/1659063751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f04200ac7b0 con 0x55cefa3124d0 2024-09-21T03:51:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.014+0000 7f0440a66740 10 client.16008 fetch_fsmap learned FSMap version 30 2024-09-21T03:51:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.014+0000 7f0440a66740 1 -- 172.21.15.16:0/1659063751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cefa6eb5c0 con 0x55cefa3124d0 2024-09-21T03:51:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.014+0000 7f041cc00640 1 -- 172.21.15.16:0/1659063751 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04200c7020 con 0x55cefa3124d0 2024-09-21T03:51:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.015+0000 7f041cc00640 1 -- 172.21.15.16:0/1659063751 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f04200b63f0 con 0x55cefa3124d0 2024-09-21T03:51:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.015+0000 7f0440a66740 10 client.16008 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:51:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.015+0000 7f0440a66740 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.015+0000 7f0440a66740 20 client.16008 populate_metadata read hostname 'smithi016' 2024-09-21T03:51:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.015+0000 7f0440a66740 1 --2- 172.21.15.16:0/1659063751 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cefa2c71c0 0x55cefa75d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.015+0000 7f0440a66740 4 client.16008 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:51:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.015+0000 7f0440a66740 1 -- 172.21.15.16:0/1659063751 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cefa64b0b0 con 0x55cefa2c71c0 2024-09-21T03:51:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.016+0000 7f0432a00640 1 --2- 172.21.15.16:0/1659063751 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cefa2c71c0 0x55cefa75d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.017+0000 7f0432a00640 1 --2- 172.21.15.16:0/1659063751 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cefa2c71c0 0x55cefa75d3e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4469c4e5665565f7 server_cookie=7b3dea124eaee782 in_seq=0 out_seq=0 2024-09-21T03:51:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.017+0000 7f041cc00640 10 client.16008 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.033+0000 7f041cc00640 1 -- 172.21.15.16:0/1659063751 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cefa64b0b0 con 0x55cefa2c71c0 2024-09-21T03:51:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.033+0000 7f041cc00640 10 client.16008 handle_command_reply: tid=0 2024-09-21T03:51:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.033+0000 7f0440a66740 1 client.16008 shutdown 2024-09-21T03:51:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.034+0000 7f0440a66740 1 -- 172.21.15.16:0/1659063751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa3124d0 msgr2=0x55cefa6e9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.034+0000 7f0440a66740 1 --2- 172.21.15.16:0/1659063751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa3124d0 0x55cefa6e9000 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f04200aa0d0 tx=0x7f0420094380 comp rx=0 tx=0).stop 2024-09-21T03:51:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.034+0000 7f0440a66740 1 -- 172.21.15.16:0/1659063751 shutdown_connections 2024-09-21T03:51:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.034+0000 7f0440a66740 1 --2- 172.21.15.16:0/1659063751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cefa3331b0 0x55cefa6e9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.034+0000 7f0440a66740 1 --2- 172.21.15.16:0/1659063751 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cefa2c71c0 0x55cefa75d3e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.034+0000 7f0440a66740 1 --2- 172.21.15.16:0/1659063751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cefa3124d0 0x55cefa6e9000 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.034+0000 7f0440a66740 1 --2- 172.21.15.16:0/1659063751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cefa30a2f0 0x55cefa6e8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.034+0000 7f0440a66740 1 -- 172.21.15.16:0/1659063751 >> 172.21.15.16:0/1659063751 conn(0x55cefa327300 msgr2=0x55cefa6d9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.034+0000 7f041cc00640 0 client.16008 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.034+0000 7f0440a66740 1 -- 172.21.15.16:0/1659063751 shutdown_connections 2024-09-21T03:51:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.034+0000 7f0440a66740 1 -- 172.21.15.16:0/1659063751 wait complete. 2024-09-21T03:51:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.035+0000 7f0440a66740 20 client.16008 trim_cache size 0 max 16384 2024-09-21T03:51:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:51:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:51:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:51:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:51:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.035+0000 7f0439a00640 1 -- 172.21.15.16:0/420079565 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04180b6610 msgr2=0x7f04180b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.035+0000 7f0439a00640 1 --2- 172.21.15.16:0/420079565 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04180b6610 0x7f04180b8b20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f042c005fd0 tx=0x7f042c009500 comp rx=0 tx=0).stop 2024-09-21T03:51:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.035+0000 7f0439a00640 1 -- 172.21.15.16:0/420079565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0434078ca0 msgr2=0x7f0434111020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.035+0000 7f0439a00640 1 --2- 172.21.15.16:0/420079565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0434078ca0 0x7f0434111020 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f042000d950 tx=0x7f042000de10 comp rx=0 tx=0).stop 2024-09-21T03:51:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.036+0000 7f0439a00640 1 -- 172.21.15.16:0/420079565 shutdown_connections 2024-09-21T03:51:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.036+0000 7f0439a00640 1 --2- 172.21.15.16:0/420079565 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04180b6610 0x7f04180b8b20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.036+0000 7f0439a00640 1 --2- 172.21.15.16:0/420079565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0434079640 0x7f0434111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.036+0000 7f0439a00640 1 --2- 172.21.15.16:0/420079565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0434078ca0 0x7f0434111020 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.036+0000 7f0439a00640 1 --2- 172.21.15.16:0/420079565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0434077c20 0x7f04341109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.036+0000 7f0439a00640 1 -- 172.21.15.16:0/420079565 >> 172.21.15.16:0/420079565 conn(0x7f0434101230 msgr2=0x7f04341074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.036+0000 7f0439a00640 1 -- 172.21.15.16:0/420079565 shutdown_connections 2024-09-21T03:51:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:25.036+0000 7f0439a00640 1 -- 172.21.15.16:0/420079565 wait complete. 2024-09-21T03:51:25.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:24 smithi134 ceph-mon[35031]: pgmap v571: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 244 MiB/s wr, 1.13k op/s 2024-09-21T03:51:25.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:24 smithi123 ceph-mon[34673]: pgmap v571: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 244 MiB/s wr, 1.13k op/s 2024-09-21T03:51:25.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:24 smithi016 ceph-mon[30375]: pgmap v571: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 244 MiB/s wr, 1.13k op/s 2024-09-21T03:51:27.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:26 smithi134 ceph-mon[35031]: pgmap v572: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 240 MiB/s wr, 1.10k op/s 2024-09-21T03:51:27.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:26 smithi123 ceph-mon[34673]: pgmap v572: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 240 MiB/s wr, 1.10k op/s 2024-09-21T03:51:27.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:26 smithi016 ceph-mon[30375]: pgmap v572: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 240 MiB/s wr, 1.10k op/s 2024-09-21T03:51:28.475 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-21T03:51:28.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:51:29.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:28 smithi134 ceph-mon[35031]: pgmap v573: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 230 MiB/s wr, 1.04k op/s 2024-09-21T03:51:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:29.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:28 smithi123 ceph-mon[34673]: pgmap v573: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 230 MiB/s wr, 1.04k op/s 2024-09-21T03:51:29.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:28 smithi016 ceph-mon[30375]: pgmap v573: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 230 MiB/s wr, 1.04k op/s 2024-09-21T03:51:31.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.036+0000 7f6464c00640 1 -- 172.21.15.16:0/1810787623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f646010dbf0 msgr2=0x7f646010e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:31.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.036+0000 7f6464c00640 1 --2- 172.21.15.16:0/1810787623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f646010dbf0 0x7f646010e000 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6448009900 tx=0x7f644802a440 comp rx=0 tx=0).stop 2024-09-21T03:51:31.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.038+0000 7f6464c00640 1 -- 172.21.15.16:0/1810787623 shutdown_connections 2024-09-21T03:51:31.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.038+0000 7f6464c00640 1 --2- 172.21.15.16:0/1810787623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f646011cba0 0x7f646011efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:31.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.038+0000 7f6464c00640 1 --2- 172.21.15.16:0/1810787623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f646010dbf0 0x7f646010e000 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:31.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.038+0000 7f6464c00640 1 --2- 172.21.15.16:0/1810787623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6460111cd0 0x7f646010d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:31.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.038+0000 7f6464c00640 1 -- 172.21.15.16:0/1810787623 >> 172.21.15.16:0/1810787623 conn(0x7f646006e9f0 msgr2=0x7f646006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:31.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.038+0000 7f6464c00640 1 -- 172.21.15.16:0/1810787623 shutdown_connections 2024-09-21T03:51:31.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.038+0000 7f6464c00640 1 -- 172.21.15.16:0/1810787623 wait complete. 2024-09-21T03:51:31.042 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:30 smithi016 ceph-mon[30375]: pgmap v574: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 228 MiB/s wr, 1.02k op/s 2024-09-21T03:51:31.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.040+0000 7f6464c00640 1 Processor -- start 2024-09-21T03:51:31.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.040+0000 7f6464c00640 1 -- start start 2024-09-21T03:51:31.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.041+0000 7f6464c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f646010dbf0 0x7f646011c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:31.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.041+0000 7f6464c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6460111cd0 0x7f6460115700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:31.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.041+0000 7f6464c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f646011cba0 0x7f6460115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:31.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.042+0000 7f645f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f646010dbf0 0x7f646011c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:31.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.042+0000 7f645f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f646010dbf0 0x7f646011c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:53108/0 (socket says 172.21.15.16:53108) 2024-09-21T03:51:31.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.042+0000 7f645fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f646011cba0 0x7f6460115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:31.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.042+0000 7f645fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f646011cba0 0x7f6460115ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38354/0 (socket says 172.21.15.16:38354) 2024-09-21T03:51:31.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.042+0000 7f6457e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6460111cd0 0x7f6460115700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:31.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.042+0000 7f6457e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6460111cd0 0x7f6460115700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:47900/0 (socket says 172.21.15.16:47900) 2024-09-21T03:51:31.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.042+0000 7f645f400640 1 -- 172.21.15.16:0/3753028621 learned_addr learned my addr 172.21.15.16:0/3753028621 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:31.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.042+0000 7f6464c00640 1 -- 172.21.15.16:0/3753028621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6460122750 con 0x7f646011cba0 2024-09-21T03:51:31.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.043+0000 7f6464c00640 1 -- 172.21.15.16:0/3753028621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64601225d0 con 0x7f646010dbf0 2024-09-21T03:51:31.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.043+0000 7f6464c00640 1 -- 172.21.15.16:0/3753028621 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64601228d0 con 0x7f6460111cd0 2024-09-21T03:51:31.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.044+0000 7f645f400640 1 -- 172.21.15.16:0/3753028621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6460111cd0 msgr2=0x7f6460115700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:31.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.044+0000 7f645f400640 1 --2- 172.21.15.16:0/3753028621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6460111cd0 0x7f6460115700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:31.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.045+0000 7f645f400640 1 -- 172.21.15.16:0/3753028621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f646011cba0 msgr2=0x7f6460115ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:31.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.045+0000 7f645f400640 1 --2- 172.21.15.16:0/3753028621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f646011cba0 0x7f6460115ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:31.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.045+0000 7f645f400640 1 -- 172.21.15.16:0/3753028621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64601164c0 con 0x7f646010dbf0 2024-09-21T03:51:31.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.046+0000 7f6457e00640 1 --2- 172.21.15.16:0/3753028621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6460111cd0 0x7f6460115700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:31.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.046+0000 7f645f400640 1 --2- 172.21.15.16:0/3753028621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f646010dbf0 0x7f646011c430 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f645000c970 tx=0x7f645000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:31.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.047+0000 7f645cc00640 1 -- 172.21.15.16:0/3753028621 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6450004220 con 0x7f646010dbf0 2024-09-21T03:51:31.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.048+0000 7f645cc00640 1 -- 172.21.15.16:0/3753028621 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64500043c0 con 0x7f646010dbf0 2024-09-21T03:51:31.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.048+0000 7f6464c00640 1 -- 172.21.15.16:0/3753028621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f646011b160 con 0x7f646010dbf0 2024-09-21T03:51:31.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.048+0000 7f6464c00640 1 -- 172.21.15.16:0/3753028621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f646011b670 con 0x7f646010dbf0 2024-09-21T03:51:31.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.048+0000 7f645cc00640 1 -- 172.21.15.16:0/3753028621 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645000f450 con 0x7f646010dbf0 2024-09-21T03:51:31.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.051+0000 7f6464c00640 1 -- 172.21.15.16:0/3753028621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f643c0052c0 con 0x7f646010dbf0 2024-09-21T03:51:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.053+0000 7f645cc00640 1 -- 172.21.15.16:0/3753028621 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f645000f5f0 con 0x7f646010dbf0 2024-09-21T03:51:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.059+0000 7f645cc00640 1 --2- 172.21.15.16:0/3753028621 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64440b6660 0x7f64440b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.060+0000 7f6457e00640 1 --2- 172.21.15.16:0/3753028621 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64440b6660 0x7f64440b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.060+0000 7f645cc00640 1 -- 172.21.15.16:0/3753028621 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f64500e71e0 con 0x7f646010dbf0 2024-09-21T03:51:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.061+0000 7f645cc00640 1 -- 172.21.15.16:0/3753028621 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6450014030 con 0x7f646010dbf0 2024-09-21T03:51:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.062+0000 7f6457e00640 1 --2- 172.21.15.16:0/3753028621 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64440b6660 0x7f64440b8b70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f644802a960 tx=0x7f6448005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:31.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:30 smithi134 ceph-mon[35031]: pgmap v574: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 228 MiB/s wr, 1.02k op/s 2024-09-21T03:51:31.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:30 smithi123 ceph-mon[34673]: pgmap v574: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 228 MiB/s wr, 1.02k op/s 2024-09-21T03:51:31.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.589+0000 7f6464c00640 1 -- 172.21.15.16:0/3753028621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f643c0055b0 con 0x7f646010dbf0 2024-09-21T03:51:31.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.590+0000 7f645cc00640 1 -- 172.21.15.16:0/3753028621 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f64500b4e20 con 0x7f646010dbf0 2024-09-21T03:51:31.596 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:51:31.596 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:51:31.596 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:51:31.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.594+0000 7f6464c00640 1 -- 172.21.15.16:0/3753028621 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64440b6660 msgr2=0x7f64440b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:31.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.594+0000 7f6464c00640 1 --2- 172.21.15.16:0/3753028621 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64440b6660 0x7f64440b8b70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f644802a960 tx=0x7f6448005fd0 comp rx=0 tx=0).stop 2024-09-21T03:51:31.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.594+0000 7f6464c00640 1 -- 172.21.15.16:0/3753028621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f646010dbf0 msgr2=0x7f646011c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.594+0000 7f6464c00640 1 --2- 172.21.15.16:0/3753028621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f646010dbf0 0x7f646011c430 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f645000c970 tx=0x7f645000ce30 comp rx=0 tx=0).stop 2024-09-21T03:51:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.594+0000 7f6464c00640 1 -- 172.21.15.16:0/3753028621 shutdown_connections 2024-09-21T03:51:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.594+0000 7f6464c00640 1 --2- 172.21.15.16:0/3753028621 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64440b6660 0x7f64440b8b70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.594+0000 7f6464c00640 1 --2- 172.21.15.16:0/3753028621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f646011cba0 0x7f6460115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.594+0000 7f6464c00640 1 --2- 172.21.15.16:0/3753028621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6460111cd0 0x7f6460115700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.594+0000 7f6464c00640 1 --2- 172.21.15.16:0/3753028621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f646010dbf0 0x7f646011c430 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.594+0000 7f6464c00640 1 -- 172.21.15.16:0/3753028621 >> 172.21.15.16:0/3753028621 conn(0x7f646006e9f0 msgr2=0x7f6460110980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.595+0000 7f6464c00640 1 -- 172.21.15.16:0/3753028621 shutdown_connections 2024-09-21T03:51:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:31.595+0000 7f6464c00640 1 -- 172.21.15.16:0/3753028621 wait complete. 2024-09-21T03:51:32.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:31 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3753028621' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:32.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:31 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3753028621' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:32.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:31 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3753028621' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:32.992 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:32 smithi016 ceph-mon[30375]: pgmap v575: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 237 MiB/s wr, 1.06k op/s 2024-09-21T03:51:33.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:32 smithi134 ceph-mon[35031]: pgmap v575: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 237 MiB/s wr, 1.06k op/s 2024-09-21T03:51:33.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:32 smithi123 ceph-mon[34673]: pgmap v575: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 237 MiB/s wr, 1.06k op/s 2024-09-21T03:51:33.987 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-21T03:51:33.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-21T03:51:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:35.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:35 smithi134 ceph-mon[35031]: pgmap v576: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 228 MiB/s wr, 1.04k op/s 2024-09-21T03:51:35.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:35 smithi123 ceph-mon[34673]: pgmap v576: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 228 MiB/s wr, 1.04k op/s 2024-09-21T03:51:35.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:35 smithi016 ceph-mon[30375]: pgmap v576: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 228 MiB/s wr, 1.04k op/s 2024-09-21T03:51:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.348+0000 7f17e1200640 1 -- 172.21.15.16:0/418014145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17dc103b90 msgr2=0x7f17dc103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.348+0000 7f17e1200640 1 --2- 172.21.15.16:0/418014145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17dc103b90 0x7f17dc103fa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009860 tx=0x7f17cc02a340 comp rx=0 tx=0).stop 2024-09-21T03:51:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.349+0000 7f17e1200640 1 -- 172.21.15.16:0/418014145 shutdown_connections 2024-09-21T03:51:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.349+0000 7f17e1200640 1 --2- 172.21.15.16:0/418014145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17dc109e20 0x7f17dc110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.349+0000 7f17e1200640 1 --2- 172.21.15.16:0/418014145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17dc1094d0 0x7f17dc1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.349+0000 7f17e1200640 1 --2- 172.21.15.16:0/418014145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17dc103b90 0x7f17dc103fa0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.349+0000 7f17e1200640 1 -- 172.21.15.16:0/418014145 >> 172.21.15.16:0/418014145 conn(0x7f17dc0ff5e0 msgr2=0x7f17dc101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.349+0000 7f17e1200640 1 -- 172.21.15.16:0/418014145 shutdown_connections 2024-09-21T03:51:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.349+0000 7f17e1200640 1 -- 172.21.15.16:0/418014145 wait complete. 2024-09-21T03:51:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.351+0000 7f17e1200640 1 Processor -- start 2024-09-21T03:51:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.351+0000 7f17e1200640 1 -- start start 2024-09-21T03:51:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.352+0000 7f17e1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17dc103b90 0x7f17dc19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.352+0000 7f17e1200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17dc1094d0 0x7f17dc19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.352+0000 7f17e1200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17dc109e20 0x7f17dc1a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.352+0000 7f17e1200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc115d70 con 0x7f17dc103b90 2024-09-21T03:51:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.352+0000 7f17e1200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc115bf0 con 0x7f17dc1094d0 2024-09-21T03:51:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.352+0000 7f17e1200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc115ef0 con 0x7f17dc109e20 2024-09-21T03:51:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.352+0000 7f17d9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17dc1094d0 0x7f17dc19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.352+0000 7f17d9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17dc1094d0 0x7f17dc19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:35654/0 (socket says 172.21.15.16:35654) 2024-09-21T03:51:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.352+0000 7f17d9400640 1 -- 172.21.15.16:0/1915420444 learned_addr learned my addr 172.21.15.16:0/1915420444 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.353+0000 7f17da800640 1 --2- 172.21.15.16:0/1915420444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17dc109e20 0x7f17dc1a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.353+0000 7f17d9e00640 1 --2- 172.21.15.16:0/1915420444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17dc103b90 0x7f17dc19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.354+0000 7f17d9e00640 1 -- 172.21.15.16:0/1915420444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17dc109e20 msgr2=0x7f17dc1a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.354+0000 7f17d9e00640 1 --2- 172.21.15.16:0/1915420444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17dc109e20 0x7f17dc1a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.354+0000 7f17d9e00640 1 -- 172.21.15.16:0/1915420444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17dc1094d0 msgr2=0x7f17dc19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.354+0000 7f17d9e00640 1 --2- 172.21.15.16:0/1915420444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17dc1094d0 0x7f17dc19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.354+0000 7f17d9e00640 1 -- 172.21.15.16:0/1915420444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc1a2590 con 0x7f17dc103b90 2024-09-21T03:51:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.354+0000 7f17d9e00640 1 --2- 172.21.15.16:0/1915420444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17dc103b90 0x7f17dc19c6e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f17cc02a8b0 tx=0x7f17cc030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.355+0000 7f17d2a00640 1 -- 172.21.15.16:0/1915420444 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc042070 con 0x7f17dc103b90 2024-09-21T03:51:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.355+0000 7f17d2a00640 1 -- 172.21.15.16:0/1915420444 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17cc030900 con 0x7f17dc103b90 2024-09-21T03:51:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.355+0000 7f17d2a00640 1 -- 172.21.15.16:0/1915420444 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc037040 con 0x7f17dc103b90 2024-09-21T03:51:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.355+0000 7f17e1200640 1 -- 172.21.15.16:0/1915420444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17dc1a2820 con 0x7f17dc103b90 2024-09-21T03:51:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.356+0000 7f17e1200640 1 -- 172.21.15.16:0/1915420444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17dc1a2cb0 con 0x7f17dc103b90 2024-09-21T03:51:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.361+0000 7f17e7e89740 1 Processor -- start 2024-09-21T03:51:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.361+0000 7f17e7e89740 1 -- start start 2024-09-21T03:51:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.362+0000 7f17e7e89740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 0x55aeba543b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.362+0000 7f17e7e89740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba16a980 0x55aeba5440a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.362+0000 7f17e7e89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba172b60 0x55aeba5445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.362+0000 7f17e7e89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55aeba4b0ca0 con 0x55aeba172b60 2024-09-21T03:51:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.362+0000 7f17e7e89740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aeba4a00d0 con 0x55aeba16a980 2024-09-21T03:51:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.362+0000 7f17e7e89740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55aeba4a3ee0 con 0x55aeba193840 2024-09-21T03:51:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.362+0000 7f17da800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 0x55aeba543b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.362+0000 7f17da800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 0x55aeba543b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:38896/0 (socket says 172.21.15.16:38896) 2024-09-21T03:51:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.362+0000 7f17da800640 1 -- 172.21.15.16:0/3859434339 learned_addr learned my addr 172.21.15.16:0/3859434339 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.362+0000 7f17d9400640 1 --2- 172.21.15.16:0/3859434339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba172b60 0x55aeba5445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.363+0000 7f17da800640 1 -- 172.21.15.16:0/3859434339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba16a980 msgr2=0x55aeba5440a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:51:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.363+0000 7f17da800640 1 --2- 172.21.15.16:0/3859434339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba16a980 0x55aeba5440a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.363+0000 7f17da800640 1 -- 172.21.15.16:0/3859434339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba172b60 msgr2=0x55aeba5445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.363+0000 7f17da800640 1 --2- 172.21.15.16:0/3859434339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba172b60 0x55aeba5445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.363+0000 7f17da800640 1 -- 172.21.15.16:0/3859434339 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeba452400 con 0x55aeba193840 2024-09-21T03:51:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.363+0000 7f17da800640 1 --2- 172.21.15.16:0/3859434339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 0x55aeba543b60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f17d4007f40 tx=0x7f17d400f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=107e722cf8fa5a94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.363+0000 7f17c7400640 1 -- 172.21.15.16:0/3859434339 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4010900 con 0x55aeba193840 2024-09-21T03:51:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.364+0000 7f17d2a00640 1 -- 172.21.15.16:0/1915420444 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f17cc030450 con 0x7f17dc103b90 2024-09-21T03:51:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.365+0000 7f17c7400640 1 -- 172.21.15.16:0/3859434339 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d40183f0 con 0x55aeba193840 2024-09-21T03:51:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.365+0000 7f17c7400640 1 -- 172.21.15.16:0/3859434339 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d40175e0 con 0x55aeba193840 2024-09-21T03:51:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.366+0000 7f17e7e89740 1 -- 172.21.15.16:0/3859434339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 msgr2=0x55aeba543b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.366+0000 7f17e7e89740 1 --2- 172.21.15.16:0/3859434339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 0x55aeba543b60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f17d4007f40 tx=0x7f17d400f850 comp rx=0 tx=0).stop 2024-09-21T03:51:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.366+0000 7f17e7e89740 1 -- 172.21.15.16:0/3859434339 shutdown_connections 2024-09-21T03:51:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.366+0000 7f17e7e89740 1 --2- 172.21.15.16:0/3859434339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba172b60 0x55aeba5445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.366+0000 7f17e7e89740 1 --2- 172.21.15.16:0/3859434339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba16a980 0x55aeba5440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.366+0000 7f17e7e89740 1 --2- 172.21.15.16:0/3859434339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 0x55aeba543b60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.366+0000 7f17e7e89740 1 -- 172.21.15.16:0/3859434339 >> 172.21.15.16:0/3859434339 conn(0x55aeba187990 msgr2=0x55aeba541a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.367+0000 7f17e7e89740 1 -- 172.21.15.16:0/3859434339 shutdown_connections 2024-09-21T03:51:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.367+0000 7f17e7e89740 1 -- 172.21.15.16:0/3859434339 wait complete. 2024-09-21T03:51:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.367+0000 7f17d2a00640 1 --2- 172.21.15.16:0/1915420444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f17c00b6510 0x7f17c00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.367+0000 7f17d2a00640 1 -- 172.21.15.16:0/1915420444 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f17cc106cc0 con 0x7f17dc103b90 2024-09-21T03:51:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.367+0000 7f17e7e89740 1 -- start start 2024-09-21T03:51:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.367+0000 7f17d9400640 1 --2- 172.21.15.16:0/1915420444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f17c00b6510 0x7f17c00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.367+0000 7f17e7e89740 1 Processor -- start 2024-09-21T03:51:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.368+0000 7f17e7e89740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba16a980 0x55aeba54f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.368+0000 7f17e7e89740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba172b60 0x55aeba54f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.368+0000 7f17e7e89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba193840 0x55aeba55b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.368+0000 7f17e7e89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55aeba3017a0 con 0x55aeba193840 2024-09-21T03:51:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.368+0000 7f17e7e89740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aeba445a10 con 0x55aeba172b60 2024-09-21T03:51:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.368+0000 7f17e7e89740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55aeba552340 con 0x55aeba16a980 2024-09-21T03:51:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.368+0000 7f17e7e89740 10 client.0 _ll_register_callbacks cb 0x55aeba455be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.368+0000 7f17d9e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba16a980 0x55aeba54f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.368+0000 7f17d9e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba16a980 0x55aeba54f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:38898/0 (socket says 172.21.15.16:38898) 2024-09-21T03:51:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.368+0000 7f17d9e00640 1 -- 172.21.15.16:0/1357629679 learned_addr learned my addr 172.21.15.16:0/1357629679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.369+0000 7f17da800640 1 --2- 172.21.15.16:0/1357629679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba193840 0x55aeba55b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.369+0000 7f17d9e00640 1 -- 172.21.15.16:0/1357629679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba172b60 msgr2=0x55aeba54f740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:51:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.369+0000 7f17d9e00640 1 --2- 172.21.15.16:0/1357629679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba172b60 0x55aeba54f740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.369+0000 7f17d9e00640 1 -- 172.21.15.16:0/1357629679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba193840 msgr2=0x55aeba55b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.369+0000 7f17d9e00640 1 --2- 172.21.15.16:0/1357629679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba193840 0x55aeba55b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.369+0000 7f17d9e00640 1 -- 172.21.15.16:0/1357629679 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeba20db90 con 0x55aeba16a980 2024-09-21T03:51:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.369+0000 7f17d9400640 1 --2- 172.21.15.16:0/1915420444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f17c00b6510 0x7f17c00b8a20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f17dc108f60 tx=0x7f17c8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.369+0000 7f17d9e00640 1 --2- 172.21.15.16:0/1357629679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba16a980 0x55aeba54f200 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0c9010 tx=0x7f17cc0cc8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c34371a074a752ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.369+0000 7f17bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:51:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.369+0000 7f17bbe00640 1 -- 172.21.15.16:0/1357629679 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc030740 con 0x55aeba16a980 2024-09-21T03:51:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.370+0000 7f17bbe00640 1 -- 172.21.15.16:0/1357629679 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f17cc0d1170 con 0x55aeba16a980 2024-09-21T03:51:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.370+0000 7f17e7e89740 1 -- 172.21.15.16:0/1357629679 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17d40183f0 con 0x55aeba16a980 2024-09-21T03:51:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.370+0000 7f17bbe00640 1 -- 172.21.15.16:0/1357629679 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17cc02ada0 con 0x55aeba16a980 2024-09-21T03:51:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.370+0000 7f17e7e89740 10 client.26053 fetch_fsmap learned FSMap version 30 2024-09-21T03:51:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.370+0000 7f17e7e89740 1 -- 172.21.15.16:0/1357629679 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aeba55e060 con 0x55aeba16a980 2024-09-21T03:51:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.370+0000 7f17bbe00640 1 -- 172.21.15.16:0/1357629679 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d40183f0 con 0x55aeba16a980 2024-09-21T03:51:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.370+0000 7f17bbe00640 1 -- 172.21.15.16:0/1357629679 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f17cc0c8390 con 0x55aeba16a980 2024-09-21T03:51:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.370+0000 7f17e7e89740 10 client.26053 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:51:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.370+0000 7f17e7e89740 10 client.26053 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:51:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.370+0000 7f17e7e89740 20 client.26053 populate_metadata read hostname 'smithi016' 2024-09-21T03:51:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.371+0000 7f17e7e89740 1 --2- 172.21.15.16:0/1357629679 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55aeba1521c0 0x55aeba561510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.371+0000 7f17e7e89740 4 client.26053 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.371+0000 7f17e7e89740 1 -- 172.21.15.16:0/1357629679 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aeba4b2bf0 con 0x55aeba1521c0 2024-09-21T03:51:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.371+0000 7f17d9400640 1 --2- 172.21.15.16:0/1357629679 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55aeba1521c0 0x55aeba561510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.372+0000 7f17bbe00640 10 client.26053 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.372+0000 7f17d9400640 1 --2- 172.21.15.16:0/1357629679 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55aeba1521c0 0x55aeba561510 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e03f3802abf33061 server_cookie=a2e3d04929724460 in_seq=0 out_seq=0 2024-09-21T03:51:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.373+0000 7f17bbe00640 1 -- 172.21.15.16:0/1357629679 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aeba4b2bf0 con 0x55aeba1521c0 2024-09-21T03:51:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.373+0000 7f17bbe00640 10 client.26053 handle_command_reply: tid=0 2024-09-21T03:51:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.373+0000 7f17e7e89740 1 client.26053 shutdown 2024-09-21T03:51:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.374+0000 7f17e7e89740 1 -- 172.21.15.16:0/1357629679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba16a980 msgr2=0x55aeba54f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.374+0000 7f17e7e89740 1 --2- 172.21.15.16:0/1357629679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba16a980 0x55aeba54f200 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0c9010 tx=0x7f17cc0cc8d0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.374+0000 7f17e7e89740 1 -- 172.21.15.16:0/1357629679 shutdown_connections 2024-09-21T03:51:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.374+0000 7f17e7e89740 1 --2- 172.21.15.16:0/1357629679 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55aeba1521c0 0x55aeba561510 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.374+0000 7f17e7e89740 1 --2- 172.21.15.16:0/1357629679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba193840 0x55aeba55b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.374+0000 7f17e7e89740 1 --2- 172.21.15.16:0/1357629679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba172b60 0x55aeba54f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.374+0000 7f17e7e89740 1 --2- 172.21.15.16:0/1357629679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba16a980 0x55aeba54f200 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.374+0000 7f17e7e89740 1 -- 172.21.15.16:0/1357629679 >> 172.21.15.16:0/1357629679 conn(0x55aeba187990 msgr2=0x55aeba5379a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.374+0000 7f17bbe00640 0 client.26053 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.374+0000 7f17e7e89740 1 -- 172.21.15.16:0/1357629679 shutdown_connections 2024-09-21T03:51:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.374+0000 7f17e7e89740 1 -- 172.21.15.16:0/1357629679 wait complete. 2024-09-21T03:51:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.374+0000 7f17e7e89740 20 client.26053 trim_cache size 0 max 16384 2024-09-21T03:51:36.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.436+0000 7f17e7e89740 1 Processor -- start 2024-09-21T03:51:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.442+0000 7f17e7e89740 1 -- start start 2024-09-21T03:51:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.442+0000 7f17e7e89740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba16a980 0x55aeba5452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.442+0000 7f17e7e89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba172b60 0x55aeba55b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.442+0000 7f17e7e89740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 0x55aeba54c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.443+0000 7f17e7e89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1ae940 con 0x55aeba172b60 2024-09-21T03:51:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.443+0000 7f17e7e89740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1b0480 con 0x55aeba16a980 2024-09-21T03:51:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.443+0000 7f17e7e89740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1af010 con 0x55aeba193840 2024-09-21T03:51:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.443+0000 7f17d9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba16a980 0x55aeba5452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.443+0000 7f17d9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba16a980 0x55aeba5452a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:35672/0 (socket says 172.21.15.16:35672) 2024-09-21T03:51:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.443+0000 7f17d9e00640 1 -- 172.21.15.16:0/1049480861 learned_addr learned my addr 172.21.15.16:0/1049480861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.443+0000 7f17da800640 1 --2- 172.21.15.16:0/1049480861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 0x55aeba54c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.443+0000 7f17d9e00640 1 -- 172.21.15.16:0/1049480861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 msgr2=0x55aeba54c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.443+0000 7f17d9e00640 1 --2- 172.21.15.16:0/1049480861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 0x55aeba54c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.443+0000 7f17d9e00640 1 -- 172.21.15.16:0/1049480861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba172b60 msgr2=0x55aeba55b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.443+0000 7f17d9e00640 1 --2- 172.21.15.16:0/1049480861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba172b60 0x55aeba55b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.443+0000 7f17d9e00640 1 -- 172.21.15.16:0/1049480861 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeba1523f0 con 0x55aeba16a980 2024-09-21T03:51:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.444+0000 7f17d9e00640 1 --2- 172.21.15.16:0/1049480861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba16a980 0x55aeba5452a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f17cc035040 tx=0x7f17cc0cc940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2283c9f7ea72aebe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.444+0000 7f17c6a00640 1 -- 172.21.15.16:0/1049480861 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc0c9090 con 0x55aeba16a980 2024-09-21T03:51:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.444+0000 7f17c6a00640 1 -- 172.21.15.16:0/1049480861 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17cc0f00f0 con 0x55aeba16a980 2024-09-21T03:51:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.444+0000 7f17c6a00640 1 -- 172.21.15.16:0/1049480861 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc0cd140 con 0x55aeba16a980 2024-09-21T03:51:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.447+0000 7f17e7e89740 1 -- 172.21.15.16:0/1049480861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba16a980 msgr2=0x55aeba5452a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.447+0000 7f17e7e89740 1 --2- 172.21.15.16:0/1049480861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba16a980 0x55aeba5452a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f17cc035040 tx=0x7f17cc0cc940 comp rx=0 tx=0).stop 2024-09-21T03:51:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.448+0000 7f17e7e89740 1 -- 172.21.15.16:0/1049480861 shutdown_connections 2024-09-21T03:51:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.448+0000 7f17e7e89740 1 --2- 172.21.15.16:0/1049480861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 0x55aeba54c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.448+0000 7f17e7e89740 1 --2- 172.21.15.16:0/1049480861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba172b60 0x55aeba55b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.448+0000 7f17e7e89740 1 --2- 172.21.15.16:0/1049480861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba16a980 0x55aeba5452a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.448+0000 7f17e7e89740 1 -- 172.21.15.16:0/1049480861 >> 172.21.15.16:0/1049480861 conn(0x55aeba187990 msgr2=0x55aeba544b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.448+0000 7f17e7e89740 1 -- 172.21.15.16:0/1049480861 shutdown_connections 2024-09-21T03:51:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.448+0000 7f17e7e89740 1 -- 172.21.15.16:0/1049480861 wait complete. 2024-09-21T03:51:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.449+0000 7f17e7e89740 1 -- start start 2024-09-21T03:51:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.449+0000 7f17e7e89740 1 Processor -- start 2024-09-21T03:51:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.455+0000 7f17e7e89740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba16a980 0x55aeba5538c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.455+0000 7f17e7e89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba172b60 0x55aeba553e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.455+0000 7f17e7e89740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 0x55aeba5caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.455+0000 7f17e7e89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1afdb0 con 0x55aeba172b60 2024-09-21T03:51:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.455+0000 7f17e7e89740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1af6e0 con 0x55aeba16a980 2024-09-21T03:51:36.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.455+0000 7f17e7e89740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1ac920 con 0x55aeba193840 2024-09-21T03:51:36.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.455+0000 7f17da800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 0x55aeba5caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:36.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.455+0000 7f17da800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 0x55aeba5caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:38918/0 (socket says 172.21.15.16:38918) 2024-09-21T03:51:36.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.455+0000 7f17da800640 1 -- 172.21.15.16:0/1148799322 learned_addr learned my addr 172.21.15.16:0/1148799322 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:36.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.462+0000 7f17d9e00640 1 --2- 172.21.15.16:0/1148799322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba16a980 0x55aeba5538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:36.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.467+0000 7f17e7e89740 10 client.0 _ll_register_callbacks cb 0x7f17cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.468+0000 7f17da800640 1 -- 172.21.15.16:0/1148799322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba16a980 msgr2=0x55aeba5538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.468+0000 7f17da800640 1 --2- 172.21.15.16:0/1148799322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba16a980 0x55aeba5538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.468+0000 7f17da800640 1 -- 172.21.15.16:0/1148799322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba172b60 msgr2=0x55aeba553e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.468+0000 7f17da800640 1 --2- 172.21.15.16:0/1148799322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba172b60 0x55aeba553e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.469+0000 7f17da800640 1 -- 172.21.15.16:0/1148799322 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeba5546d0 con 0x55aeba193840 2024-09-21T03:51:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.469+0000 7f17da800640 1 --2- 172.21.15.16:0/1148799322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 0x55aeba5caed0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f17d4007e60 tx=0x7f17d4003db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a9b201db2471007 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.469+0000 7f17bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:51:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.470+0000 7f17bbe00640 1 -- 172.21.15.16:0/1148799322 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4018600 con 0x55aeba193840 2024-09-21T03:51:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.470+0000 7f17bbe00640 1 -- 172.21.15.16:0/1148799322 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f17d40187a0 con 0x55aeba193840 2024-09-21T03:51:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.470+0000 7f17e7e89740 1 -- 172.21.15.16:0/1148799322 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17cc0f00f0 con 0x55aeba193840 2024-09-21T03:51:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.471+0000 7f17bbe00640 1 -- 172.21.15.16:0/1148799322 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17d4032050 con 0x55aeba193840 2024-09-21T03:51:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.471+0000 7f17e7e89740 10 client.26059 fetch_fsmap learned FSMap version 30 2024-09-21T03:51:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.471+0000 7f17e7e89740 1 -- 172.21.15.16:0/1148799322 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aeba539ab0 con 0x55aeba193840 2024-09-21T03:51:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.472+0000 7f17bbe00640 1 -- 172.21.15.16:0/1148799322 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17cc0f00f0 con 0x55aeba193840 2024-09-21T03:51:36.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.476+0000 7f17bbe00640 1 -- 172.21.15.16:0/1148799322 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f17d40050b0 con 0x55aeba193840 2024-09-21T03:51:36.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.476+0000 7f17e7e89740 10 client.26059 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:51:36.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.476+0000 7f17e7e89740 10 client.26059 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:51:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.476+0000 7f17e7e89740 20 client.26059 populate_metadata read hostname 'smithi016' 2024-09-21T03:51:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.476+0000 7f17e7e89740 1 --2- 172.21.15.16:0/1148799322 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55aeba127570 0x55aeba53bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.476+0000 7f17e7e89740 4 client.26059 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-21T03:51:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.476+0000 7f17e7e89740 1 -- 172.21.15.16:0/1148799322 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x55aeba4ae6a0 con 0x55aeba127570 2024-09-21T03:51:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.478+0000 7f17d9e00640 1 --2- 172.21.15.16:0/1148799322 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55aeba127570 0x55aeba53bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.479+0000 7f17bbe00640 10 client.26059 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.478+0000 7f17d9e00640 1 --2- 172.21.15.16:0/1148799322 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55aeba127570 0x55aeba53bf20 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37c061860f847f38 server_cookie=fb1c1d78f93943b0 in_seq=0 out_seq=0 2024-09-21T03:51:36.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.533+0000 7f17bbe00640 1 -- 172.21.15.16:0/1148799322 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55aeba4ae6a0 con 0x55aeba127570 2024-09-21T03:51:36.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.533+0000 7f17bbe00640 10 client.26059 handle_command_reply: tid=0 2024-09-21T03:51:36.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.533+0000 7f17e7e89740 1 client.26059 shutdown 2024-09-21T03:51:36.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.534+0000 7f17e7e89740 1 -- 172.21.15.16:0/1148799322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 msgr2=0x55aeba5caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:36.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.534+0000 7f17e7e89740 1 --2- 172.21.15.16:0/1148799322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 0x55aeba5caed0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f17d4007e60 tx=0x7f17d4003db0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.535+0000 7f17e7e89740 1 -- 172.21.15.16:0/1148799322 shutdown_connections 2024-09-21T03:51:36.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.535+0000 7f17e7e89740 1 --2- 172.21.15.16:0/1148799322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aeba193840 0x55aeba5caed0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.536+0000 7f17e7e89740 1 --2- 172.21.15.16:0/1148799322 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55aeba127570 0x55aeba53bf20 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.536+0000 7f17e7e89740 1 --2- 172.21.15.16:0/1148799322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aeba172b60 0x55aeba553e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.536+0000 7f17bbe00640 0 client.26059 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:36.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.536+0000 7f17e7e89740 1 --2- 172.21.15.16:0/1148799322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aeba16a980 0x55aeba5538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.536+0000 7f17e7e89740 1 -- 172.21.15.16:0/1148799322 >> 172.21.15.16:0/1148799322 conn(0x55aeba187990 msgr2=0x55aeba5379a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:36.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.538+0000 7f17e7e89740 1 -- 172.21.15.16:0/1148799322 shutdown_connections 2024-09-21T03:51:36.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.538+0000 7f17e7e89740 1 -- 172.21.15.16:0/1148799322 wait complete. 2024-09-21T03:51:36.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.538+0000 7f17e7e89740 20 client.26059 trim_cache size 0 max 16384 2024-09-21T03:51:36.543 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:51:36.543 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:51:36.543 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:51:36.544 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 18, 2024-09-21T03:51:36.544 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:51:36.544 INFO:teuthology.orchestra.run.smithi016.stdout: "58330903": { 2024-09-21T03:51:36.544 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 18, 2024-09-21T03:51:36.544 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:51:36.544 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:51:36.544 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:51:36.544 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:51:36.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:51:36.544 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:51:36.544 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 130.0, 2024-09-21T03:51:36.544 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:51:36.544 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:51:36.544 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:51:36.544 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:51:36.545 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:51:36.545 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:51:36.545 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:51:36.545 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:51:36.545 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:51:36.545 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:51:36.545 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:51:36.545 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:51:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.540+0000 7f17e1200640 1 -- 172.21.15.16:0/1915420444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f17c00b6510 msgr2=0x7f17c00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.540+0000 7f17e1200640 1 --2- 172.21.15.16:0/1915420444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f17c00b6510 0x7f17c00b8a20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f17dc108f60 tx=0x7f17c8009260 comp rx=0 tx=0).stop 2024-09-21T03:51:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.540+0000 7f17e1200640 1 -- 172.21.15.16:0/1915420444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17dc103b90 msgr2=0x7f17dc19c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.540+0000 7f17e1200640 1 --2- 172.21.15.16:0/1915420444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17dc103b90 0x7f17dc19c6e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f17cc02a8b0 tx=0x7f17cc030d80 comp rx=0 tx=0).stop 2024-09-21T03:51:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.540+0000 7f17e1200640 1 -- 172.21.15.16:0/1915420444 shutdown_connections 2024-09-21T03:51:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.540+0000 7f17e1200640 1 --2- 172.21.15.16:0/1915420444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f17c00b6510 0x7f17c00b8a20 secure :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f17dc108f60 tx=0x7f17c8009260 comp rx=0 tx=0).stop 2024-09-21T03:51:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.540+0000 7f17e1200640 1 --2- 172.21.15.16:0/1915420444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17dc109e20 0x7f17dc1a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.540+0000 7f17e1200640 1 --2- 172.21.15.16:0/1915420444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17dc1094d0 0x7f17dc19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.540+0000 7f17e1200640 1 --2- 172.21.15.16:0/1915420444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17dc103b90 0x7f17dc19c6e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.540+0000 7f17e1200640 1 -- 172.21.15.16:0/1915420444 >> 172.21.15.16:0/1915420444 conn(0x7f17dc0ff5e0 msgr2=0x7f17dc101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.541+0000 7f17e1200640 1 -- 172.21.15.16:0/1915420444 shutdown_connections 2024-09-21T03:51:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:36.542+0000 7f17e1200640 1 -- 172.21.15.16:0/1915420444 wait complete. 2024-09-21T03:51:37.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:37 smithi134 ceph-mon[35031]: pgmap v577: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 238 MiB/s wr, 1.07k op/s 2024-09-21T03:51:37.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:37 smithi123 ceph-mon[34673]: pgmap v577: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 238 MiB/s wr, 1.07k op/s 2024-09-21T03:51:37.428 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:37 smithi016 ceph-mon[30375]: pgmap v577: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 238 MiB/s wr, 1.07k op/s 2024-09-21T03:51:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.513+0000 7f37c9000640 1 -- 172.21.15.16:0/3481363224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c410cd20 msgr2=0x7f37c410f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.513+0000 7f37c9000640 1 --2- 172.21.15.16:0/3481363224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c410cd20 0x7f37c410f140 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009ca0 tx=0x7f37ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.513+0000 7f37c9000640 1 -- 172.21.15.16:0/3481363224 shutdown_connections 2024-09-21T03:51:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.514+0000 7f37c9000640 1 --2- 172.21.15.16:0/3481363224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37c410f830 0x7f37c410fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.514+0000 7f37c9000640 1 --2- 172.21.15.16:0/3481363224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c410cd20 0x7f37c410f140 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.514+0000 7f37c9000640 1 --2- 172.21.15.16:0/3481363224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37c410a3c0 0x7f37c410c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.514+0000 7f37c9000640 1 -- 172.21.15.16:0/3481363224 >> 172.21.15.16:0/3481363224 conn(0x7f37c4070640 msgr2=0x7f37c4072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.514+0000 7f37c9000640 1 -- 172.21.15.16:0/3481363224 shutdown_connections 2024-09-21T03:51:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.514+0000 7f37c9000640 1 -- 172.21.15.16:0/3481363224 wait complete. 2024-09-21T03:51:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.516+0000 7f37c9000640 1 Processor -- start 2024-09-21T03:51:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.516+0000 7f37c9000640 1 -- start start 2024-09-21T03:51:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.516+0000 7f37c9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c410a3c0 0x7f37c4119020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.517+0000 7f37c9000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37c410cd20 0x7f37c4119690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.517+0000 7f37c9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37c410f830 0x7f37c411a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.517+0000 7f37c9000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37c411dfb0 con 0x7f37c410f830 2024-09-21T03:51:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.517+0000 7f37c9000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37c411de30 con 0x7f37c410a3c0 2024-09-21T03:51:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.517+0000 7f37c9000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37c411e130 con 0x7f37c410cd20 2024-09-21T03:51:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.517+0000 7f37c3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c410a3c0 0x7f37c4119020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.517+0000 7f37c3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c410a3c0 0x7f37c4119020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:35712/0 (socket says 172.21.15.16:35712) 2024-09-21T03:51:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.517+0000 7f37c3400640 1 -- 172.21.15.16:0/508526478 learned_addr learned my addr 172.21.15.16:0/508526478 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.517+0000 7f37c2a00640 1 --2- 172.21.15.16:0/508526478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37c410cd20 0x7f37c4119690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.517+0000 7f37c3400640 1 -- 172.21.15.16:0/508526478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37c410cd20 msgr2=0x7f37c4119690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.518+0000 7f37c3400640 1 --2- 172.21.15.16:0/508526478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37c410cd20 0x7f37c4119690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.518+0000 7f37c3400640 1 -- 172.21.15.16:0/508526478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37c410f830 msgr2=0x7f37c411a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.518+0000 7f37c3400640 1 --2- 172.21.15.16:0/508526478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37c410f830 0x7f37c411a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.518+0000 7f37c3400640 1 -- 172.21.15.16:0/508526478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c411a9d0 con 0x7f37c410a3c0 2024-09-21T03:51:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.518+0000 7f37c3400640 1 --2- 172.21.15.16:0/508526478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c410a3c0 0x7f37c4119020 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00c920 tx=0x7f37bc00cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.518+0000 7f37bbe00640 1 -- 172.21.15.16:0/508526478 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc007c70 con 0x7f37c410a3c0 2024-09-21T03:51:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.518+0000 7f37bbe00640 1 -- 172.21.15.16:0/508526478 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37bc007e10 con 0x7f37c410a3c0 2024-09-21T03:51:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.518+0000 7f37bbe00640 1 -- 172.21.15.16:0/508526478 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc018820 con 0x7f37c410a3c0 2024-09-21T03:51:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.519+0000 7f37c9000640 1 -- 172.21.15.16:0/508526478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c411acc0 con 0x7f37c410a3c0 2024-09-21T03:51:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.519+0000 7f37c9000640 1 -- 172.21.15.16:0/508526478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c411b230 con 0x7f37c410a3c0 2024-09-21T03:51:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.524+0000 7f37d17f0740 1 Processor -- start 2024-09-21T03:51:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.526+0000 7f37bbe00640 1 -- 172.21.15.16:0/508526478 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f37bc00f450 con 0x7f37c410a3c0 2024-09-21T03:51:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.529+0000 7f37bbe00640 1 --2- 172.21.15.16:0/508526478 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f37a80b6650 0x7f37a80b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.529+0000 7f37d17f0740 1 -- start start 2024-09-21T03:51:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.530+0000 7f37d17f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f6801b0 0x55c17fa272b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.530+0000 7f37d17f0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 0x55c17fa27920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.530+0000 7f37d17f0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17f65f4d0 0x55c17fa28470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.531+0000 7f37d17f0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c17f99a6b0 con 0x55c17f6801b0 2024-09-21T03:51:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.531+0000 7f37d17f0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c17f989ae0 con 0x55c17f6572f0 2024-09-21T03:51:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.531+0000 7f37d17f0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c17f98d8f0 con 0x55c17f65f4d0 2024-09-21T03:51:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.532+0000 7f37bbe00640 1 -- 172.21.15.16:0/508526478 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f37bc0e7880 con 0x7f37c410a3c0 2024-09-21T03:51:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.533+0000 7f37c2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 0x55c17fa27920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.533+0000 7f37c3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17f65f4d0 0x55c17fa28470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.533+0000 7f37c2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 0x55c17fa27920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:35720/0 (socket says 172.21.15.16:35720) 2024-09-21T03:51:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.533+0000 7f37c2a00640 1 -- 172.21.15.16:0/1158881987 learned_addr learned my addr 172.21.15.16:0/1158881987 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.533+0000 7f37c3e00640 1 --2- 172.21.15.16:0/508526478 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f37a80b6650 0x7f37a80b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.534+0000 7f37c2a00640 1 -- 172.21.15.16:0/1158881987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17f65f4d0 msgr2=0x55c17fa28470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.534+0000 7f37c2a00640 1 --2- 172.21.15.16:0/1158881987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17f65f4d0 0x55c17fa28470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.534+0000 7f37c2a00640 1 -- 172.21.15.16:0/1158881987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f6801b0 msgr2=0x55c17fa272b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.534+0000 7f37c2a00640 1 --2- 172.21.15.16:0/1158881987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f6801b0 0x55c17fa272b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.534+0000 7f37c2a00640 1 -- 172.21.15.16:0/1158881987 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c17f956780 con 0x55c17f6572f0 2024-09-21T03:51:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.534+0000 7f37c3e00640 1 --2- 172.21.15.16:0/1158881987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17f65f4d0 0x55c17fa28470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.534+0000 7f37c2a00640 1 --2- 172.21.15.16:0/1158881987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 0x55c17fa27920 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f37ac006010 tx=0x7f37ac009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86d78787c4a9d986 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.535+0000 7f37b8c00640 1 -- 172.21.15.16:0/1158881987 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac031650 con 0x55c17f6572f0 2024-09-21T03:51:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.535+0000 7f37b8c00640 1 -- 172.21.15.16:0/1158881987 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37ac036070 con 0x55c17f6572f0 2024-09-21T03:51:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.535+0000 7f37b8c00640 1 -- 172.21.15.16:0/1158881987 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac004700 con 0x55c17f6572f0 2024-09-21T03:51:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.536+0000 7f37d17f0740 1 -- 172.21.15.16:0/1158881987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 msgr2=0x55c17fa27920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.536+0000 7f37d17f0740 1 --2- 172.21.15.16:0/1158881987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 0x55c17fa27920 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f37ac006010 tx=0x7f37ac009490 comp rx=0 tx=0).stop 2024-09-21T03:51:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.537+0000 7f37d17f0740 1 -- 172.21.15.16:0/1158881987 shutdown_connections 2024-09-21T03:51:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.537+0000 7f37d17f0740 1 --2- 172.21.15.16:0/1158881987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17f65f4d0 0x55c17fa28470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.537+0000 7f37d17f0740 1 --2- 172.21.15.16:0/1158881987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 0x55c17fa27920 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.537+0000 7f37d17f0740 1 --2- 172.21.15.16:0/1158881987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f6801b0 0x55c17fa272b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.537+0000 7f37d17f0740 1 -- 172.21.15.16:0/1158881987 >> 172.21.15.16:0/1158881987 conn(0x55c17f674300 msgr2=0x55c17fa22d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.537+0000 7f37c3e00640 1 --2- 172.21.15.16:0/508526478 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f37a80b6650 0x7f37a80b8b60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f37b40058c0 tx=0x7f37b4005850 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.537+0000 7f37d17f0740 1 -- 172.21.15.16:0/1158881987 shutdown_connections 2024-09-21T03:51:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.537+0000 7f37d17f0740 1 -- 172.21.15.16:0/1158881987 wait complete. 2024-09-21T03:51:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.538+0000 7f37d17f0740 1 -- start start 2024-09-21T03:51:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.539+0000 7f37d17f0740 1 Processor -- start 2024-09-21T03:51:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.539+0000 7f37d17f0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 0x55c17fa30960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.540+0000 7f37d17f0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17f65f4d0 0x55c17fa31020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.540+0000 7f37d17f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f6801b0 0x55c17fa31b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.540+0000 7f37c3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 0x55c17fa30960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.540+0000 7f37c2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17f65f4d0 0x55c17fa31020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.540+0000 7f37c2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17f65f4d0 0x55c17fa31020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:38970/0 (socket says 172.21.15.16:38970) 2024-09-21T03:51:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.541+0000 7f37c3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 0x55c17fa30960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:35724/0 (socket says 172.21.15.16:35724) 2024-09-21T03:51:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.541+0000 7f37d17f0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c17f8c1ab0 con 0x55c17f6801b0 2024-09-21T03:51:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.541+0000 7f37d17f0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c17f930570 con 0x55c17f6572f0 2024-09-21T03:51:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.541+0000 7f37d17f0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c17fa28f30 con 0x55c17f65f4d0 2024-09-21T03:51:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.541+0000 7f37d17f0740 10 client.0 _ll_register_callbacks cb 0x55c17f93f100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.542+0000 7f37c2a00640 1 -- 172.21.15.16:0/3568999002 learned_addr learned my addr 172.21.15.16:0/3568999002 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.542+0000 7f37c3400640 1 -- 172.21.15.16:0/3568999002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17f65f4d0 msgr2=0x55c17fa31020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.542+0000 7f37c3400640 1 --2- 172.21.15.16:0/3568999002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17f65f4d0 0x55c17fa31020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.542+0000 7f37c3400640 1 -- 172.21.15.16:0/3568999002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f6801b0 msgr2=0x55c17fa31b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.542+0000 7f37c3400640 1 --2- 172.21.15.16:0/3568999002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f6801b0 0x55c17fa31b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.543+0000 7f37c3400640 1 -- 172.21.15.16:0/3568999002 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c17f6eb790 con 0x55c17f6572f0 2024-09-21T03:51:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.544+0000 7f37c3400640 1 --2- 172.21.15.16:0/3568999002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 0x55c17fa30960 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0aa870 tx=0x7f37bc0ad010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdf942b4f732b760 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.544+0000 7f37b1600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:51:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.544+0000 7f37b1600640 1 -- 172.21.15.16:0/3568999002 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0b6d90 con 0x55c17f6572f0 2024-09-21T03:51:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.544+0000 7f37b1600640 1 -- 172.21.15.16:0/3568999002 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f37bc0b6040 con 0x55c17f6572f0 2024-09-21T03:51:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.547+0000 7f37d17f0740 1 -- 172.21.15.16:0/3568999002 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37ac036070 con 0x55c17f6572f0 2024-09-21T03:51:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.548+0000 7f37b1600640 1 -- 172.21.15.16:0/3568999002 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f37bc058e60 con 0x55c17f6572f0 2024-09-21T03:51:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.548+0000 7f37d17f0740 10 client.26129 fetch_fsmap learned FSMap version 30 2024-09-21T03:51:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.549+0000 7f37d17f0740 1 -- 172.21.15.16:0/3568999002 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c17f764620 con 0x55c17f6572f0 2024-09-21T03:51:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.550+0000 7f37b1600640 1 -- 172.21.15.16:0/3568999002 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37ac036070 con 0x55c17f6572f0 2024-09-21T03:51:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.550+0000 7f37b1600640 1 -- 172.21.15.16:0/3568999002 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f37bc0ac2d0 con 0x55c17f6572f0 2024-09-21T03:51:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.550+0000 7f37d17f0740 10 client.26129 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:51:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.550+0000 7f37d17f0740 10 client.26129 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.550+0000 7f37d17f0740 20 client.26129 populate_metadata read hostname 'smithi016' 2024-09-21T03:51:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.550+0000 7f37d17f0740 1 --2- 172.21.15.16:0/3568999002 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c17f63ed80 0x55c17fa3e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.550+0000 7f37d17f0740 4 client.26129 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.550+0000 7f37d17f0740 1 -- 172.21.15.16:0/3568999002 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c17f99c600 con 0x55c17f63ed80 2024-09-21T03:51:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.551+0000 7f37c2a00640 1 --2- 172.21.15.16:0/3568999002 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c17f63ed80 0x55c17fa3e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.552+0000 7f37c2a00640 1 --2- 172.21.15.16:0/3568999002 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c17f63ed80 0x55c17fa3e0e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e096a0cb7fd655d5 server_cookie=8e5dc66da3a7c1ba in_seq=0 out_seq=0 2024-09-21T03:51:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.552+0000 7f37b1600640 10 client.26129 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.555+0000 7f37b1600640 1 -- 172.21.15.16:0/3568999002 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c17f99c600 con 0x55c17f63ed80 2024-09-21T03:51:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.555+0000 7f37b1600640 10 client.26129 handle_command_reply: tid=0 2024-09-21T03:51:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.555+0000 7f37d17f0740 1 client.26129 shutdown 2024-09-21T03:51:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.557+0000 7f37d17f0740 1 -- 172.21.15.16:0/3568999002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 msgr2=0x55c17fa30960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.557+0000 7f37d17f0740 1 --2- 172.21.15.16:0/3568999002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 0x55c17fa30960 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0aa870 tx=0x7f37bc0ad010 comp rx=0 tx=0).stop 2024-09-21T03:51:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.558+0000 7f37d17f0740 1 -- 172.21.15.16:0/3568999002 shutdown_connections 2024-09-21T03:51:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.558+0000 7f37d17f0740 1 --2- 172.21.15.16:0/3568999002 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c17f63ed80 0x55c17fa3e0e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.560+0000 7f37d17f0740 1 --2- 172.21.15.16:0/3568999002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f6801b0 0x55c17fa31b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.560+0000 7f37d17f0740 1 --2- 172.21.15.16:0/3568999002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17f65f4d0 0x55c17fa31020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.560+0000 7f37d17f0740 1 --2- 172.21.15.16:0/3568999002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 0x55c17fa30960 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.560+0000 7f37d17f0740 1 -- 172.21.15.16:0/3568999002 >> 172.21.15.16:0/3568999002 conn(0x55c17f674300 msgr2=0x55c17fa21360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.560+0000 7f37b1600640 0 client.26129 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.561+0000 7f37d17f0740 1 -- 172.21.15.16:0/3568999002 shutdown_connections 2024-09-21T03:51:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.561+0000 7f37d17f0740 1 -- 172.21.15.16:0/3568999002 wait complete. 2024-09-21T03:51:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.561+0000 7f37d17f0740 20 client.26129 trim_cache size 0 max 16384 2024-09-21T03:51:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.625+0000 7f37d17f0740 1 Processor -- start 2024-09-21T03:51:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.625+0000 7f37d17f0740 1 -- start start 2024-09-21T03:51:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.625+0000 7f37d17f0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17fa3e410 0x55c17fa22370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.625+0000 7f37d17f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f674300 0x55c17fa228b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.625+0000 7f37d17f0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17f6572f0 0x55c17fa2b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.625+0000 7f37d17f0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41c3ba0 con 0x55c17f674300 2024-09-21T03:51:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.626+0000 7f37d17f0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41c56e0 con 0x55c17fa3e410 2024-09-21T03:51:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.626+0000 7f37d17f0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41c4270 con 0x55c17f6572f0 2024-09-21T03:51:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.626+0000 7f37c3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17fa3e410 0x55c17fa22370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.626+0000 7f37c3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17fa3e410 0x55c17fa22370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:35734/0 (socket says 172.21.15.16:35734) 2024-09-21T03:51:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.626+0000 7f37c3400640 1 -- 172.21.15.16:0/3190907751 learned_addr learned my addr 172.21.15.16:0/3190907751 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.626+0000 7f37c3e00640 1 --2- 172.21.15.16:0/3190907751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17f6572f0 0x55c17fa2b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.627+0000 7f37c3400640 1 -- 172.21.15.16:0/3190907751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17f6572f0 msgr2=0x55c17fa2b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.627+0000 7f37c3400640 1 --2- 172.21.15.16:0/3190907751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17f6572f0 0x55c17fa2b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.627+0000 7f37c3400640 1 -- 172.21.15.16:0/3190907751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f674300 msgr2=0x55c17fa228b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.627+0000 7f37c3400640 1 --2- 172.21.15.16:0/3190907751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f674300 0x55c17fa228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.627+0000 7f37c3400640 1 -- 172.21.15.16:0/3190907751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c17fa29070 con 0x55c17fa3e410 2024-09-21T03:51:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.628+0000 7f37c3400640 1 --2- 172.21.15.16:0/3190907751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17fa3e410 0x55c17fa22370 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f37bc007590 tx=0x7f37bc00a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82be850bc929a97a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.628+0000 7f37b3e00640 1 -- 172.21.15.16:0/3190907751 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0c7070 con 0x55c17fa3e410 2024-09-21T03:51:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.628+0000 7f37b3e00640 1 -- 172.21.15.16:0/3190907751 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37bc0b20e0 con 0x55c17fa3e410 2024-09-21T03:51:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.629+0000 7f37b3e00640 1 -- 172.21.15.16:0/3190907751 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0b6df0 con 0x55c17fa3e410 2024-09-21T03:51:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.631+0000 7f37d17f0740 1 -- 172.21.15.16:0/3190907751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17fa3e410 msgr2=0x55c17fa22370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.631+0000 7f37d17f0740 1 --2- 172.21.15.16:0/3190907751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17fa3e410 0x55c17fa22370 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f37bc007590 tx=0x7f37bc00a410 comp rx=0 tx=0).stop 2024-09-21T03:51:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.632+0000 7f37d17f0740 1 -- 172.21.15.16:0/3190907751 shutdown_connections 2024-09-21T03:51:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.632+0000 7f37d17f0740 1 --2- 172.21.15.16:0/3190907751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17f6572f0 0x55c17fa2b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.632+0000 7f37d17f0740 1 --2- 172.21.15.16:0/3190907751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f674300 0x55c17fa228b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.632+0000 7f37d17f0740 1 --2- 172.21.15.16:0/3190907751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17fa3e410 0x55c17fa22370 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.632+0000 7f37d17f0740 1 -- 172.21.15.16:0/3190907751 >> 172.21.15.16:0/3190907751 conn(0x55c17fa284b0 msgr2=0x55c17fa2fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.634+0000 7f37d17f0740 1 -- 172.21.15.16:0/3190907751 shutdown_connections 2024-09-21T03:51:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.634+0000 7f37d17f0740 1 -- 172.21.15.16:0/3190907751 wait complete. 2024-09-21T03:51:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.635+0000 7f37d17f0740 1 -- start start 2024-09-21T03:51:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.636+0000 7f37d17f0740 1 Processor -- start 2024-09-21T03:51:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.638+0000 7f37d17f0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 0x55c17fa387a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.638+0000 7f37d17f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f674300 0x55c17fa37460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.639+0000 7f37c3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 0x55c17fa387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.640+0000 7f37c3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 0x55c17fa387a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:35740/0 (socket says 172.21.15.16:35740) 2024-09-21T03:51:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.640+0000 7f37d17f0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17fa3e410 0x55c17fa379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.640+0000 7f37d17f0740 1 -- 172.21.15.16:0/2575474368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41c5010 con 0x55c17f674300 2024-09-21T03:51:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.640+0000 7f37c3400640 1 -- 172.21.15.16:0/2575474368 learned_addr learned my addr 172.21.15.16:0/2575474368 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.640+0000 7f37d17f0740 1 -- 172.21.15.16:0/2575474368 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41c4940 con 0x55c17f6572f0 2024-09-21T03:51:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.640+0000 7f37d17f0740 1 -- 172.21.15.16:0/2575474368 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41c1b80 con 0x55c17fa3e410 2024-09-21T03:51:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.640+0000 7f37d17f0740 10 client.0 _ll_register_callbacks cb 0x7f37bc00cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.639+0000 7f37c2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f674300 0x55c17fa37460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.642+0000 7f37c3e00640 1 --2- 172.21.15.16:0/2575474368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17fa3e410 0x55c17fa379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.642+0000 7f37c2a00640 1 -- 172.21.15.16:0/2575474368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17fa3e410 msgr2=0x55c17fa379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.642+0000 7f37c2a00640 1 --2- 172.21.15.16:0/2575474368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17fa3e410 0x55c17fa379e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.642+0000 7f37c2a00640 1 -- 172.21.15.16:0/2575474368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 msgr2=0x55c17fa387a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.642+0000 7f37c2a00640 1 --2- 172.21.15.16:0/2575474368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 0x55c17fa387a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.642+0000 7f37c2a00640 1 -- 172.21.15.16:0/2575474368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c17fa38170 con 0x55c17f674300 2024-09-21T03:51:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.643+0000 7f37b1600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:51:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.643+0000 7f37c2a00640 1 --2- 172.21.15.16:0/2575474368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f674300 0x55c17fa37460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55c17f966d00 tx=0x7f37ac004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a86fb6b1daec791 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.644+0000 7f37b1600640 1 -- 172.21.15.16:0/2575474368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac049040 con 0x55c17f674300 2024-09-21T03:51:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.644+0000 7f37b1600640 1 -- 172.21.15.16:0/2575474368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f37ac031780 con 0x55c17f674300 2024-09-21T03:51:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.644+0000 7f37d17f0740 1 -- 172.21.15.16:0/2575474368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37bc0b20e0 con 0x55c17f674300 2024-09-21T03:51:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.645+0000 7f37b1600640 1 -- 172.21.15.16:0/2575474368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f37ac049040 con 0x55c17f674300 2024-09-21T03:51:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.645+0000 7f37d17f0740 10 client.16026 fetch_fsmap learned FSMap version 30 2024-09-21T03:51:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.645+0000 7f37d17f0740 1 -- 172.21.15.16:0/2575474368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c17faa8ae0 con 0x55c17f674300 2024-09-21T03:51:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.646+0000 7f37b1600640 1 -- 172.21.15.16:0/2575474368 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37ac036040 con 0x55c17f674300 2024-09-21T03:51:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.646+0000 7f37b1600640 1 -- 172.21.15.16:0/2575474368 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f37ac055070 con 0x55c17f674300 2024-09-21T03:51:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.648+0000 7f37d17f0740 10 client.16026 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:51:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.648+0000 7f37d17f0740 10 client.16026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.648+0000 7f37d17f0740 20 client.16026 populate_metadata read hostname 'smithi016' 2024-09-21T03:51:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.648+0000 7f37d17f0740 1 --2- 172.21.15.16:0/2575474368 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c17f65f4d0 0x55c17faaaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.648+0000 7f37d17f0740 4 client.16026 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:51:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.648+0000 7f37d17f0740 1 -- 172.21.15.16:0/2575474368 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f37bc0b20e0 con 0x55c17f65f4d0 2024-09-21T03:51:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.650+0000 7f37c3400640 1 --2- 172.21.15.16:0/2575474368 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c17f65f4d0 0x55c17faaaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.652+0000 7f37b1600640 10 client.16026 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.652+0000 7f37c3400640 1 --2- 172.21.15.16:0/2575474368 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c17f65f4d0 0x55c17faaaf90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be32e9b108503219 server_cookie=152ac0eeff166973 in_seq=0 out_seq=0 2024-09-21T03:51:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.671+0000 7f37b1600640 1 -- 172.21.15.16:0/2575474368 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f37bc0b20e0 con 0x55c17f65f4d0 2024-09-21T03:51:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.671+0000 7f37b1600640 10 client.16026 handle_command_reply: tid=0 2024-09-21T03:51:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.672+0000 7f37d17f0740 1 client.16026 shutdown 2024-09-21T03:51:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.674+0000 7f37d17f0740 1 -- 172.21.15.16:0/2575474368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f674300 msgr2=0x55c17fa37460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.674+0000 7f37d17f0740 1 --2- 172.21.15.16:0/2575474368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f674300 0x55c17fa37460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55c17f966d00 tx=0x7f37ac004980 comp rx=0 tx=0).stop 2024-09-21T03:51:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.675+0000 7f37d17f0740 1 -- 172.21.15.16:0/2575474368 shutdown_connections 2024-09-21T03:51:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.675+0000 7f37d17f0740 1 --2- 172.21.15.16:0/2575474368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c17fa3e410 0x55c17fa379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.676+0000 7f37d17f0740 1 --2- 172.21.15.16:0/2575474368 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c17f65f4d0 0x55c17faaaf90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.676+0000 7f37d17f0740 1 --2- 172.21.15.16:0/2575474368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c17f674300 0x55c17fa37460 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.676+0000 7f37b1600640 0 client.16026 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.676+0000 7f37d17f0740 1 --2- 172.21.15.16:0/2575474368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c17f6572f0 0x55c17fa387a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.676+0000 7f37d17f0740 1 -- 172.21.15.16:0/2575474368 >> 172.21.15.16:0/2575474368 conn(0x55c17fa284b0 msgr2=0x55c17fa2fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.682+0000 7f37d17f0740 1 -- 172.21.15.16:0/2575474368 shutdown_connections 2024-09-21T03:51:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.688+0000 7f37d17f0740 1 -- 172.21.15.16:0/2575474368 wait complete. 2024-09-21T03:51:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.688+0000 7f37d17f0740 20 client.16026 trim_cache size 0 max 16384 2024-09-21T03:51:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:51:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:51:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:51:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:51:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.689+0000 7f37c9000640 1 -- 172.21.15.16:0/508526478 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f37a80b6650 msgr2=0x7f37a80b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.689+0000 7f37c9000640 1 --2- 172.21.15.16:0/508526478 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f37a80b6650 0x7f37a80b8b60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f37b40058c0 tx=0x7f37b4005850 comp rx=0 tx=0).stop 2024-09-21T03:51:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.689+0000 7f37c9000640 1 -- 172.21.15.16:0/508526478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c410a3c0 msgr2=0x7f37c4119020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.689+0000 7f37c9000640 1 --2- 172.21.15.16:0/508526478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c410a3c0 0x7f37c4119020 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00c920 tx=0x7f37bc00cde0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.689+0000 7f37c9000640 1 -- 172.21.15.16:0/508526478 shutdown_connections 2024-09-21T03:51:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.689+0000 7f37c9000640 1 --2- 172.21.15.16:0/508526478 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f37a80b6650 0x7f37a80b8b60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.689+0000 7f37c9000640 1 --2- 172.21.15.16:0/508526478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37c410f830 0x7f37c411a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.689+0000 7f37c9000640 1 --2- 172.21.15.16:0/508526478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37c410cd20 0x7f37c4119690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.689+0000 7f37c9000640 1 --2- 172.21.15.16:0/508526478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c410a3c0 0x7f37c4119020 secure :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00c920 tx=0x7f37bc00cde0 comp rx=0 tx=0).stop 2024-09-21T03:51:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.689+0000 7f37c9000640 1 -- 172.21.15.16:0/508526478 >> 172.21.15.16:0/508526478 conn(0x7f37c4070640 msgr2=0x7f37c4070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.693+0000 7f37c9000640 1 -- 172.21.15.16:0/508526478 shutdown_connections 2024-09-21T03:51:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:38.693+0000 7f37c9000640 1 -- 172.21.15.16:0/508526478 wait complete. 2024-09-21T03:51:38.792 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '58330903', quiesce took 10.3 seconds. Will release after: -0.3000000000000007 2024-09-21T03:51:38.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:51:39.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:39 smithi134 ceph-mon[35031]: pgmap v578: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 MiB/s wr, 1.06k op/s 2024-09-21T03:51:39.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:39 smithi123 ceph-mon[34673]: pgmap v578: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 MiB/s wr, 1.06k op/s 2024-09-21T03:51:39.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:39 smithi016 ceph-mon[30375]: pgmap v578: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 MiB/s wr, 1.06k op/s 2024-09-21T03:51:39.525 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:41.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:41 smithi134 ceph-mon[35031]: pgmap v579: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 234 MiB/s wr, 1.08k op/s 2024-09-21T03:51:41.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:41 smithi123 ceph-mon[34673]: pgmap v579: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 234 MiB/s wr, 1.08k op/s 2024-09-21T03:51:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:41 smithi016 ceph-mon[30375]: pgmap v579: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 234 MiB/s wr, 1.08k op/s 2024-09-21T03:51:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.315+0000 7efbf7e00640 1 -- 172.21.15.16:0/3314445675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf81047c0 msgr2=0x7efbf8104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.315+0000 7efbf7e00640 1 --2- 172.21.15.16:0/3314445675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf81047c0 0x7efbf8104bd0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009900 tx=0x7efbe802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:51:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.316+0000 7efbf7e00640 1 -- 172.21.15.16:0/3314445675 shutdown_connections 2024-09-21T03:51:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.316+0000 7efbf7e00640 1 --2- 172.21.15.16:0/3314445675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efbf810dc20 0x7efbf8110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.316+0000 7efbf7e00640 1 --2- 172.21.15.16:0/3314445675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efbf8105110 0x7efbf810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.316+0000 7efbf7e00640 1 --2- 172.21.15.16:0/3314445675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf81047c0 0x7efbf8104bd0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.316+0000 7efbf7e00640 1 -- 172.21.15.16:0/3314445675 >> 172.21.15.16:0/3314445675 conn(0x7efbf80fe2c0 msgr2=0x7efbf8100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.316+0000 7efbf7e00640 1 -- 172.21.15.16:0/3314445675 shutdown_connections 2024-09-21T03:51:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.316+0000 7efbf7e00640 1 -- 172.21.15.16:0/3314445675 wait complete. 2024-09-21T03:51:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.317+0000 7efbf7e00640 1 Processor -- start 2024-09-21T03:51:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.318+0000 7efbf7e00640 1 -- start start 2024-09-21T03:51:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.318+0000 7efbf7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efbf81047c0 0x7efbf819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.318+0000 7efbf7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf8105110 0x7efbf819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.318+0000 7efbf7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efbf810dc20 0x7efbf81a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.318+0000 7efbf7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efbf81138a0 con 0x7efbf8105110 2024-09-21T03:51:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.318+0000 7efbf7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efbf8113720 con 0x7efbf81047c0 2024-09-21T03:51:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.318+0000 7efbf7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efbf8113a20 con 0x7efbf810dc20 2024-09-21T03:51:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.319+0000 7efbf6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efbf81047c0 0x7efbf819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.319+0000 7efbf7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efbf810dc20 0x7efbf81a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.319+0000 7efbf6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efbf81047c0 0x7efbf819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:43494/0 (socket says 172.21.15.16:43494) 2024-09-21T03:51:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.319+0000 7efbf7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efbf810dc20 0x7efbf81a1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:59270/0 (socket says 172.21.15.16:59270) 2024-09-21T03:51:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.319+0000 7efbf6a00640 1 -- 172.21.15.16:0/3961357524 learned_addr learned my addr 172.21.15.16:0/3961357524 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.319+0000 7efbf6000640 1 --2- 172.21.15.16:0/3961357524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf8105110 0x7efbf819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.320+0000 7efbf6a00640 1 -- 172.21.15.16:0/3961357524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efbf810dc20 msgr2=0x7efbf81a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.320+0000 7efbf6a00640 1 --2- 172.21.15.16:0/3961357524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efbf810dc20 0x7efbf81a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.320+0000 7efbf6a00640 1 -- 172.21.15.16:0/3961357524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf8105110 msgr2=0x7efbf819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.320+0000 7efbf6a00640 1 --2- 172.21.15.16:0/3961357524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf8105110 0x7efbf819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.320+0000 7efbf6a00640 1 -- 172.21.15.16:0/3961357524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf81a24a0 con 0x7efbf81047c0 2024-09-21T03:51:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.320+0000 7efbf6a00640 1 --2- 172.21.15.16:0/3961357524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efbf81047c0 0x7efbf819c530 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efbe8005940 tx=0x7efbe80040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.321+0000 7efbef400640 1 -- 172.21.15.16:0/3961357524 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe803f030 con 0x7efbf81047c0 2024-09-21T03:51:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.321+0000 7efbf7e00640 1 -- 172.21.15.16:0/3961357524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbf81a2730 con 0x7efbf81047c0 2024-09-21T03:51:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.321+0000 7efbef400640 1 -- 172.21.15.16:0/3961357524 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbe80044c0 con 0x7efbf81047c0 2024-09-21T03:51:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.321+0000 7efbef400640 1 -- 172.21.15.16:0/3961357524 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe803f030 con 0x7efbf81047c0 2024-09-21T03:51:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.321+0000 7efbf7e00640 1 -- 172.21.15.16:0/3961357524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbf81a8ef0 con 0x7efbf81047c0 2024-09-21T03:51:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.323+0000 7efbf7e00640 1 -- 172.21.15.16:0/3961357524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbf806b560 con 0x7efbf81047c0 2024-09-21T03:51:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.325+0000 7efbef400640 1 -- 172.21.15.16:0/3961357524 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7efbe8042030 con 0x7efbf81047c0 2024-09-21T03:51:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.326+0000 7efbef400640 1 --2- 172.21.15.16:0/3961357524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efbdc0b6810 0x7efbdc0b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.326+0000 7efbef400640 1 -- 172.21.15.16:0/3961357524 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7efbe8107590 con 0x7efbf81047c0 2024-09-21T03:51:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.327+0000 7efbf6000640 1 --2- 172.21.15.16:0/3961357524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efbdc0b6810 0x7efbdc0b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.328+0000 7efbf6000640 1 --2- 172.21.15.16:0/3961357524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efbdc0b6810 0x7efbdc0b8d20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efbf810fca0 tx=0x7efbe4005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.330+0000 7efbef400640 1 -- 172.21.15.16:0/3961357524 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efbe80cda10 con 0x7efbf81047c0 2024-09-21T03:51:42.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.746+0000 7efbf7e00640 1 -- 172.21.15.16:0/3961357524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efbf81a90c0 con 0x7efbf81047c0 2024-09-21T03:51:42.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.750+0000 7efbef400640 1 -- 172.21.15.16:0/3961357524 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7efbe80d1df0 con 0x7efbf81047c0 2024-09-21T03:51:42.756 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:51:42.756 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:51:42.756 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:51:42.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.757+0000 7efbf7e00640 1 -- 172.21.15.16:0/3961357524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efbdc0b6810 msgr2=0x7efbdc0b8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:42.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.757+0000 7efbf7e00640 1 --2- 172.21.15.16:0/3961357524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efbdc0b6810 0x7efbdc0b8d20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efbf810fca0 tx=0x7efbe4005f90 comp rx=0 tx=0).stop 2024-09-21T03:51:42.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.757+0000 7efbf7e00640 1 -- 172.21.15.16:0/3961357524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efbf81047c0 msgr2=0x7efbf819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:42.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.757+0000 7efbf7e00640 1 --2- 172.21.15.16:0/3961357524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efbf81047c0 0x7efbf819c530 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efbe8005940 tx=0x7efbe80040c0 comp rx=0 tx=0).stop 2024-09-21T03:51:42.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.757+0000 7efbf7e00640 1 -- 172.21.15.16:0/3961357524 shutdown_connections 2024-09-21T03:51:42.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.757+0000 7efbf7e00640 1 --2- 172.21.15.16:0/3961357524 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efbdc0b6810 0x7efbdc0b8d20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:42.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.757+0000 7efbf7e00640 1 --2- 172.21.15.16:0/3961357524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efbf810dc20 0x7efbf81a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:42.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.757+0000 7efbf7e00640 1 --2- 172.21.15.16:0/3961357524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf8105110 0x7efbf819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:42.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.757+0000 7efbf7e00640 1 --2- 172.21.15.16:0/3961357524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efbf81047c0 0x7efbf819c530 secure :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efbe8005940 tx=0x7efbe80040c0 comp rx=0 tx=0).stop 2024-09-21T03:51:42.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.757+0000 7efbf7e00640 1 -- 172.21.15.16:0/3961357524 >> 172.21.15.16:0/3961357524 conn(0x7efbf80fe2c0 msgr2=0x7efbf80fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:42.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.758+0000 7efbf7e00640 1 -- 172.21.15.16:0/3961357524 shutdown_connections 2024-09-21T03:51:42.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:42.758+0000 7efbf7e00640 1 -- 172.21.15.16:0/3961357524 wait complete. 2024-09-21T03:51:43.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:43 smithi016 ceph-mon[30375]: pgmap v580: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 240 MiB/s wr, 1.11k op/s 2024-09-21T03:51:43.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:43 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3961357524' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:43 smithi134 ceph-mon[35031]: pgmap v580: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 240 MiB/s wr, 1.11k op/s 2024-09-21T03:51:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:43 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3961357524' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:43.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:43 smithi123 ceph-mon[34673]: pgmap v580: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 240 MiB/s wr, 1.11k op/s 2024-09-21T03:51:43.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:43 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3961357524' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:51:44.388 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db --set-id 58330903 --release --await' 2024-09-21T03:51:44.388 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db --set-id 58330903 --release --await 2024-09-21T03:51:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:51:45.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:45 smithi123 ceph-mon[34673]: pgmap v581: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 238 MiB/s wr, 1.14k op/s 2024-09-21T03:51:45.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:45 smithi016 ceph-mon[30375]: pgmap v581: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 238 MiB/s wr, 1.14k op/s 2024-09-21T03:51:45.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:45 smithi134 ceph-mon[35031]: pgmap v581: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 238 MiB/s wr, 1.14k op/s 2024-09-21T03:51:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.918+0000 7fa3b5e00640 1 -- 172.21.15.16:0/1084271988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3b01049b0 msgr2=0x7fa3b0104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.918+0000 7fa3b5e00640 1 --2- 172.21.15.16:0/1084271988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3b01049b0 0x7fa3b0104dc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009900 tx=0x7fa3a002a400 comp rx=0 tx=0).stop 2024-09-21T03:51:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.918+0000 7fa3b5e00640 1 -- 172.21.15.16:0/1084271988 shutdown_connections 2024-09-21T03:51:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.918+0000 7fa3b5e00640 1 --2- 172.21.15.16:0/1084271988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3b010de10 0x7fa3b0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.918+0000 7fa3b5e00640 1 --2- 172.21.15.16:0/1084271988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3b0105300 0x7fa3b010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.918+0000 7fa3b5e00640 1 --2- 172.21.15.16:0/1084271988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3b01049b0 0x7fa3b0104dc0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.918+0000 7fa3b5e00640 1 -- 172.21.15.16:0/1084271988 >> 172.21.15.16:0/1084271988 conn(0x7fa3b0079720 msgr2=0x7fa3b0079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.918+0000 7fa3b5e00640 1 -- 172.21.15.16:0/1084271988 shutdown_connections 2024-09-21T03:51:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.918+0000 7fa3b5e00640 1 -- 172.21.15.16:0/1084271988 wait complete. 2024-09-21T03:51:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.919+0000 7fa3b5e00640 1 Processor -- start 2024-09-21T03:51:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.919+0000 7fa3b5e00640 1 -- start start 2024-09-21T03:51:45.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.920+0000 7fa3b5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3b01049b0 0x7fa3b019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.920+0000 7fa3b5e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3b0105300 0x7fa3b019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.920+0000 7fa3b5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3b010de10 0x7fa3b01a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.920+0000 7fa3b5e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b0113a90 con 0x7fa3b01049b0 2024-09-21T03:51:45.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.920+0000 7fa3b5e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b0113910 con 0x7fa3b010de10 2024-09-21T03:51:45.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.920+0000 7fa3b5e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b0113c10 con 0x7fa3b0105300 2024-09-21T03:51:45.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.920+0000 7fa3aea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3b01049b0 0x7fa3b019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:45.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.920+0000 7fa3a6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3b0105300 0x7fa3b019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:45.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.920+0000 7fa3aea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3b01049b0 0x7fa3b019c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44672/0 (socket says 172.21.15.16:44672) 2024-09-21T03:51:45.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.920+0000 7fa3a6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3b0105300 0x7fa3b019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:59300/0 (socket says 172.21.15.16:59300) 2024-09-21T03:51:45.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.920+0000 7fa3aea00640 1 -- 172.21.15.16:0/3233486247 learned_addr learned my addr 172.21.15.16:0/3233486247 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:45.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.920+0000 7fa3af400640 1 --2- 172.21.15.16:0/3233486247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3b010de10 0x7fa3b01a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.921+0000 7fa3a6000640 1 -- 172.21.15.16:0/3233486247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3b010de10 msgr2=0x7fa3b01a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.921+0000 7fa3a6000640 1 --2- 172.21.15.16:0/3233486247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3b010de10 0x7fa3b01a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.921+0000 7fa3a6000640 1 -- 172.21.15.16:0/3233486247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3b01049b0 msgr2=0x7fa3b019c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.921+0000 7fa3a6000640 1 --2- 172.21.15.16:0/3233486247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3b01049b0 0x7fa3b019c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.921+0000 7fa3a6000640 1 -- 172.21.15.16:0/3233486247 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b01a2670 con 0x7fa3b0105300 2024-09-21T03:51:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.921+0000 7fa3a6000640 1 --2- 172.21.15.16:0/3233486247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3b0105300 0x7fa3b019cca0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa3b006b970 tx=0x7fa39800ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.921+0000 7fa3b4a00640 1 -- 172.21.15.16:0/3233486247 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39800ccc0 con 0x7fa3b0105300 2024-09-21T03:51:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.921+0000 7fa3b4a00640 1 -- 172.21.15.16:0/3233486247 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa398009e40 con 0x7fa3b0105300 2024-09-21T03:51:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.921+0000 7fa3b4a00640 1 -- 172.21.15.16:0/3233486247 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398004a00 con 0x7fa3b0105300 2024-09-21T03:51:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.921+0000 7fa3b5e00640 1 -- 172.21.15.16:0/3233486247 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b01a2960 con 0x7fa3b0105300 2024-09-21T03:51:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.921+0000 7fa3b5e00640 1 -- 172.21.15.16:0/3233486247 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b01a90c0 con 0x7fa3b0105300 2024-09-21T03:51:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.924+0000 7fa3bca6a740 1 Processor -- start 2024-09-21T03:51:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.924+0000 7fa3bca6a740 1 -- start start 2024-09-21T03:51:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.925+0000 7fa3bca6a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e5a520 0x55fb5a2011c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.925+0000 7fa3bca6a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e31660 0x55fb5a201700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.925+0000 7fa3bca6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e39840 0x55fb5a201c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.925+0000 7fa3bca6a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fb5a178290 con 0x55fb59e5a520 2024-09-21T03:51:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.925+0000 7fa3bca6a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fb5a1676c0 con 0x55fb59e31660 2024-09-21T03:51:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.925+0000 7fa3bca6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fb5a16b4d0 con 0x55fb59e39840 2024-09-21T03:51:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.925+0000 7fa3af400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e5a520 0x55fb5a2011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.925+0000 7fa3af400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e5a520 0x55fb5a2011c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44680/0 (socket says 172.21.15.16:44680) 2024-09-21T03:51:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.925+0000 7fa3af400640 1 -- 172.21.15.16:0/931157905 learned_addr learned my addr 172.21.15.16:0/931157905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.925+0000 7fa3aea00640 1 --2- 172.21.15.16:0/931157905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e31660 0x55fb5a201700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.926+0000 7fa3af400640 1 -- 172.21.15.16:0/931157905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e39840 msgr2=0x55fb5a201c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:51:45.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.926+0000 7fa3af400640 1 --2- 172.21.15.16:0/931157905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e39840 0x55fb5a201c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.926+0000 7fa3af400640 1 -- 172.21.15.16:0/931157905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e31660 msgr2=0x55fb5a201700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:45.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.926+0000 7fa3af400640 1 --2- 172.21.15.16:0/931157905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e31660 0x55fb5a201700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.926+0000 7fa3af400640 1 -- 172.21.15.16:0/931157905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5a0f4460 con 0x55fb59e5a520 2024-09-21T03:51:45.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.926+0000 7fa3af400640 1 --2- 172.21.15.16:0/931157905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e5a520 0x55fb5a2011c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8011440 tx=0x7fa3a8011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c0470263ed3259 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:45.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.926+0000 7fa3b4a00640 1 -- 172.21.15.16:0/3233486247 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa398005020 con 0x7fa3b0105300 2024-09-21T03:51:45.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.926+0000 7fa39fe00640 1 -- 172.21.15.16:0/931157905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8003c30 con 0x55fb59e5a520 2024-09-21T03:51:45.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.926+0000 7fa39fe00640 1 -- 172.21.15.16:0/931157905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a8003dd0 con 0x55fb59e5a520 2024-09-21T03:51:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.927+0000 7fa3bca6a740 1 -- 172.21.15.16:0/931157905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e5a520 msgr2=0x55fb5a2011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.927+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/931157905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e5a520 0x55fb5a2011c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8011440 tx=0x7fa3a8011900 comp rx=0 tx=0).stop 2024-09-21T03:51:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.928+0000 7fa3bca6a740 1 -- 172.21.15.16:0/931157905 shutdown_connections 2024-09-21T03:51:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.928+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/931157905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e39840 0x55fb5a201c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.928+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/931157905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e31660 0x55fb5a201700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.928+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/931157905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e5a520 0x55fb5a2011c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.928+0000 7fa3bca6a740 1 -- 172.21.15.16:0/931157905 >> 172.21.15.16:0/931157905 conn(0x55fb59e4e910 msgr2=0x55fb5a1ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.928+0000 7fa3bca6a740 1 -- 172.21.15.16:0/931157905 shutdown_connections 2024-09-21T03:51:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.928+0000 7fa3bca6a740 1 -- 172.21.15.16:0/931157905 wait complete. 2024-09-21T03:51:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.928+0000 7fa3b4a00640 1 --2- 172.21.15.16:0/3233486247 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa3940b6450 0x7fa3940b8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.928+0000 7fa3b4a00640 1 -- 172.21.15.16:0/3233486247 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fa3980e7620 con 0x7fa3b0105300 2024-09-21T03:51:45.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.928+0000 7fa3aea00640 1 --2- 172.21.15.16:0/3233486247 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa3940b6450 0x7fa3940b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:45.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.929+0000 7fa3bca6a740 1 -- start start 2024-09-21T03:51:45.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.929+0000 7fa3bca6a740 1 Processor -- start 2024-09-21T03:51:45.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.929+0000 7fa3aea00640 1 --2- 172.21.15.16:0/3233486247 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa3940b6450 0x7fa3940b8960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55fb5a1ffe40 tx=0x7fa3a0005890 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:45.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.929+0000 7fa3bca6a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e31660 0x55fb5a20cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3bca6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e39840 0x55fb5a20d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3bca6a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e5a520 0x55fb5a218d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3bca6a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fb59fc8bb0 con 0x55fb59e31660 2024-09-21T03:51:45.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3bca6a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fb5a1026c0 con 0x55fb59e5a520 2024-09-21T03:51:45.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3bca6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fb5a20f9f0 con 0x55fb59e39840 2024-09-21T03:51:45.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3bca6a740 10 client.0 _ll_register_callbacks cb 0x55fb5a11d490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:45.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3aea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e31660 0x55fb5a20cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:45.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3aea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e31660 0x55fb5a20cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44690/0 (socket says 172.21.15.16:44690) 2024-09-21T03:51:45.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3aea00640 1 -- 172.21.15.16:0/4246685275 learned_addr learned my addr 172.21.15.16:0/4246685275 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:45.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3af400640 1 --2- 172.21.15.16:0/4246685275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e5a520 0x55fb5a218d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:45.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3a6000640 1 --2- 172.21.15.16:0/4246685275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e39840 0x55fb5a20d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:45.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3aea00640 1 -- 172.21.15.16:0/4246685275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e39840 msgr2=0x55fb5a20d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:45.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3aea00640 1 --2- 172.21.15.16:0/4246685275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e39840 0x55fb5a20d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3aea00640 1 -- 172.21.15.16:0/4246685275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e5a520 msgr2=0x55fb5a218d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:45.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3aea00640 1 --2- 172.21.15.16:0/4246685275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e5a520 0x55fb5a218d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3aea00640 1 -- 172.21.15.16:0/4246685275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb59ec5b00 con 0x55fb59e31660 2024-09-21T03:51:45.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3af400640 1 --2- 172.21.15.16:0/4246685275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e5a520 0x55fb5a218d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:45.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.930+0000 7fa3a6000640 1 --2- 172.21.15.16:0/4246685275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e39840 0x55fb5a20d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:51:45.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.931+0000 7fa3aea00640 1 --2- 172.21.15.16:0/4246685275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e31660 0x55fb5a20cd80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0006010 tx=0x7fa3a0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265131f42008f722 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:45.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.931+0000 7fa39cc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:51:45.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.931+0000 7fa39cc00640 1 -- 172.21.15.16:0/4246685275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a004d030 con 0x55fb59e31660 2024-09-21T03:51:45.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.931+0000 7fa3bca6a740 1 -- 172.21.15.16:0/4246685275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3a8003dd0 con 0x55fb59e31660 2024-09-21T03:51:45.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.931+0000 7fa39cc00640 1 -- 172.21.15.16:0/4246685275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fa3a00035f0 con 0x55fb59e31660 2024-09-21T03:51:45.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.931+0000 7fa39cc00640 1 -- 172.21.15.16:0/4246685275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa3a005a050 con 0x55fb59e31660 2024-09-21T03:51:45.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.931+0000 7fa3bca6a740 10 client.16044 fetch_fsmap learned FSMap version 30 2024-09-21T03:51:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.931+0000 7fa3bca6a740 1 -- 172.21.15.16:0/4246685275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fb59f3c960 con 0x55fb59e31660 2024-09-21T03:51:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.932+0000 7fa39cc00640 1 -- 172.21.15.16:0/4246685275 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a8003dd0 con 0x55fb59e31660 2024-09-21T03:51:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.932+0000 7fa39cc00640 1 -- 172.21.15.16:0/4246685275 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa3a003d830 con 0x55fb59e31660 2024-09-21T03:51:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.932+0000 7fa3bca6a740 10 client.16044 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:51:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.932+0000 7fa3bca6a740 10 client.16044 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:51:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.932+0000 7fa3bca6a740 20 client.16044 populate_metadata read hostname 'smithi016' 2024-09-21T03:51:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.932+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/4246685275 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55fb59e18ea0 0x55fb5a21ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.932+0000 7fa3bca6a740 4 client.16044 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.932+0000 7fa3bca6a740 1 -- 172.21.15.16:0/4246685275 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb5a17a1e0 con 0x55fb59e18ea0 2024-09-21T03:51:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.933+0000 7fa3a6000640 1 --2- 172.21.15.16:0/4246685275 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55fb59e18ea0 0x55fb5a21ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.933+0000 7fa3a6000640 1 --2- 172.21.15.16:0/4246685275 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55fb59e18ea0 0x55fb5a21ee80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ae402fd93724baa server_cookie=b611656707edccb8 in_seq=0 out_seq=0 2024-09-21T03:51:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.933+0000 7fa39cc00640 10 client.16044 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.934+0000 7fa39cc00640 1 -- 172.21.15.16:0/4246685275 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fb5a17a1e0 con 0x55fb59e18ea0 2024-09-21T03:51:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.934+0000 7fa39cc00640 10 client.16044 handle_command_reply: tid=0 2024-09-21T03:51:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.934+0000 7fa3bca6a740 1 client.16044 shutdown 2024-09-21T03:51:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.934+0000 7fa3bca6a740 1 -- 172.21.15.16:0/4246685275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e31660 msgr2=0x55fb5a20cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.934+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/4246685275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e31660 0x55fb5a20cd80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0006010 tx=0x7fa3a0009260 comp rx=0 tx=0).stop 2024-09-21T03:51:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.934+0000 7fa3bca6a740 1 -- 172.21.15.16:0/4246685275 shutdown_connections 2024-09-21T03:51:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.934+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/4246685275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e5a520 0x55fb5a218d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.934+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/4246685275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e39840 0x55fb5a20d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.934+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/4246685275 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55fb59e18ea0 0x55fb5a21ee80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.934+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/4246685275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e31660 0x55fb5a20cd80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.934+0000 7fa3bca6a740 1 -- 172.21.15.16:0/4246685275 >> 172.21.15.16:0/4246685275 conn(0x55fb59e4e910 msgr2=0x55fb5a2043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.934+0000 7fa39cc00640 0 client.16044 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.934+0000 7fa3bca6a740 1 -- 172.21.15.16:0/4246685275 shutdown_connections 2024-09-21T03:51:45.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.934+0000 7fa3bca6a740 1 -- 172.21.15.16:0/4246685275 wait complete. 2024-09-21T03:51:45.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.934+0000 7fa3bca6a740 20 client.16044 trim_cache size 0 max 16384 2024-09-21T03:51:45.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.953+0000 7fa3bca6a740 1 Processor -- start 2024-09-21T03:51:45.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.953+0000 7fa3bca6a740 1 -- start start 2024-09-21T03:51:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.953+0000 7fa3bca6a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e31660 0x55fb5a202860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.953+0000 7fa3bca6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e39840 0x55fb5a204c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.953+0000 7fa3bca6a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e5a520 0x55fb5a20e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.953+0000 7fa3bca6a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b01b2d50 con 0x55fb59e5a520 2024-09-21T03:51:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.953+0000 7fa3bca6a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b01b4890 con 0x55fb59e31660 2024-09-21T03:51:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.953+0000 7fa3bca6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b01b3420 con 0x55fb59e39840 2024-09-21T03:51:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.953+0000 7fa3af400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e5a520 0x55fb5a20e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.953+0000 7fa3a6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e39840 0x55fb5a204c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.953+0000 7fa3aea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e31660 0x55fb5a202860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.953+0000 7fa3a6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e39840 0x55fb5a204c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:59314/0 (socket says 172.21.15.16:59314) 2024-09-21T03:51:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.953+0000 7fa3af400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e5a520 0x55fb5a20e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44692/0 (socket says 172.21.15.16:44692) 2024-09-21T03:51:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.953+0000 7fa3a6000640 1 -- 172.21.15.16:0/307516345 learned_addr learned my addr 172.21.15.16:0/307516345 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.954+0000 7fa3aea00640 1 -- 172.21.15.16:0/307516345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e39840 msgr2=0x55fb5a204c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.954+0000 7fa3aea00640 1 --2- 172.21.15.16:0/307516345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e39840 0x55fb5a204c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.954+0000 7fa3aea00640 1 -- 172.21.15.16:0/307516345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e5a520 msgr2=0x55fb5a20e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.954+0000 7fa3aea00640 1 --2- 172.21.15.16:0/307516345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e5a520 0x55fb5a20e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.954+0000 7fa3aea00640 1 -- 172.21.15.16:0/307516345 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5a2034b0 con 0x55fb59e31660 2024-09-21T03:51:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.954+0000 7fa3aea00640 1 --2- 172.21.15.16:0/307516345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e31660 0x55fb5a202860 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa3a003dca0 tx=0x7fa3a00319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=777d4be4280fa05d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.954+0000 7fa39f400640 1 -- 172.21.15.16:0/307516345 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0063070 con 0x55fb59e31660 2024-09-21T03:51:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.954+0000 7fa39f400640 1 -- 172.21.15.16:0/307516345 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a0049640 con 0x55fb59e31660 2024-09-21T03:51:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.954+0000 7fa39f400640 1 -- 172.21.15.16:0/307516345 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a004d040 con 0x55fb59e31660 2024-09-21T03:51:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.955+0000 7fa3bca6a740 1 -- 172.21.15.16:0/307516345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e31660 msgr2=0x55fb5a202860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.955+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/307516345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e31660 0x55fb5a202860 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa3a003dca0 tx=0x7fa3a00319f0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.955+0000 7fa3bca6a740 1 -- 172.21.15.16:0/307516345 shutdown_connections 2024-09-21T03:51:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.955+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/307516345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e5a520 0x55fb5a20e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.955+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/307516345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e39840 0x55fb5a204c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.955+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/307516345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e31660 0x55fb5a202860 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.955+0000 7fa3bca6a740 1 -- 172.21.15.16:0/307516345 >> 172.21.15.16:0/307516345 conn(0x55fb59e4e910 msgr2=0x55fb5a20e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.955+0000 7fa3bca6a740 1 -- 172.21.15.16:0/307516345 shutdown_connections 2024-09-21T03:51:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.956+0000 7fa3bca6a740 1 -- 172.21.15.16:0/307516345 wait complete. 2024-09-21T03:51:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.956+0000 7fa3bca6a740 1 -- start start 2024-09-21T03:51:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.956+0000 7fa3bca6a740 1 Processor -- start 2024-09-21T03:51:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.957+0000 7fa3bca6a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e31660 0x55fb5a2155c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.957+0000 7fa3bca6a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e39840 0x55fb5a211a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.957+0000 7fa3bca6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e5a520 0x55fb5a212100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.957+0000 7fa3bca6a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b01b41c0 con 0x55fb59e31660 2024-09-21T03:51:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.957+0000 7fa3bca6a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b01b3af0 con 0x55fb59e39840 2024-09-21T03:51:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.957+0000 7fa3bca6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b01b0d30 con 0x55fb59e5a520 2024-09-21T03:51:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.957+0000 7fa3bca6a740 10 client.0 _ll_register_callbacks cb 0x7fa3a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.957+0000 7fa3aea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e31660 0x55fb5a2155c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.957+0000 7fa3aea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e31660 0x55fb5a2155c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44702/0 (socket says 172.21.15.16:44702) 2024-09-21T03:51:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.957+0000 7fa3aea00640 1 -- 172.21.15.16:0/260049123 learned_addr learned my addr 172.21.15.16:0/260049123 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.957+0000 7fa3a6000640 1 --2- 172.21.15.16:0/260049123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e39840 0x55fb5a211a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.957+0000 7fa3af400640 1 --2- 172.21.15.16:0/260049123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e5a520 0x55fb5a212100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.957+0000 7fa3aea00640 1 -- 172.21.15.16:0/260049123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e5a520 msgr2=0x55fb5a212100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.957+0000 7fa3aea00640 1 --2- 172.21.15.16:0/260049123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e5a520 0x55fb5a212100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.957+0000 7fa3aea00640 1 -- 172.21.15.16:0/260049123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e39840 msgr2=0x55fb5a211a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.957+0000 7fa3aea00640 1 --2- 172.21.15.16:0/260049123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e39840 0x55fb5a211a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.957+0000 7fa3aea00640 1 -- 172.21.15.16:0/260049123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb5a212890 con 0x55fb59e31660 2024-09-21T03:51:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.958+0000 7fa3aea00640 1 --2- 172.21.15.16:0/260049123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e31660 0x55fb5a2155c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0005ed0 tx=0x7fa3a0005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b219d129318134ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.958+0000 7fa39cc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:51:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.958+0000 7fa39cc00640 1 -- 172.21.15.16:0/260049123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a006d060 con 0x55fb59e31660 2024-09-21T03:51:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.958+0000 7fa39cc00640 1 -- 172.21.15.16:0/260049123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fa3a00497f0 con 0x55fb59e31660 2024-09-21T03:51:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.958+0000 7fa3bca6a740 1 -- 172.21.15.16:0/260049123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3a0049640 con 0x55fb59e31660 2024-09-21T03:51:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.958+0000 7fa39cc00640 1 -- 172.21.15.16:0/260049123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa3a0074050 con 0x55fb59e31660 2024-09-21T03:51:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.958+0000 7fa3bca6a740 10 client.16050 fetch_fsmap learned FSMap version 30 2024-09-21T03:51:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.959+0000 7fa3bca6a740 1 -- 172.21.15.16:0/260049123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fb5a28a910 con 0x55fb59e31660 2024-09-21T03:51:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.959+0000 7fa39cc00640 1 -- 172.21.15.16:0/260049123 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a0049640 con 0x55fb59e31660 2024-09-21T03:51:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.959+0000 7fa39cc00640 1 -- 172.21.15.16:0/260049123 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa3a0003bc0 con 0x55fb59e31660 2024-09-21T03:51:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.959+0000 7fa3bca6a740 10 client.16050 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:51:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.959+0000 7fa3bca6a740 10 client.16050 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:51:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.959+0000 7fa3bca6a740 20 client.16050 populate_metadata read hostname 'smithi016' 2024-09-21T03:51:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.959+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/260049123 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55fb59dee250 0x55fb5a28cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.959+0000 7fa3bca6a740 4 client.16050 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "58330903", "release": true, "await": true}] 2024-09-21T03:51:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.959+0000 7fa3bca6a740 1 -- 172.21.15.16:0/260049123 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "set_id": "58330903", "release": true, "await": true}) -- 0x55fb5a175c90 con 0x55fb59dee250 2024-09-21T03:51:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.960+0000 7fa3a6000640 1 --2- 172.21.15.16:0/260049123 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55fb59dee250 0x55fb5a28cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.960+0000 7fa3a6000640 1 --2- 172.21.15.16:0/260049123 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55fb59dee250 0x55fb5a28cff0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37d60f582dbb75a8 server_cookie=edf22a7833bf74e5 in_seq=0 out_seq=0 2024-09-21T03:51:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.960+0000 7fa39cc00640 10 client.16050 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:45.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.963+0000 7fa39cc00640 1 -- 172.21.15.16:0/260049123 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fb5a175c90 con 0x55fb59dee250 2024-09-21T03:51:45.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.963+0000 7fa39cc00640 10 client.16050 handle_command_reply: tid=0 2024-09-21T03:51:45.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.963+0000 7fa3bca6a740 1 client.16050 shutdown 2024-09-21T03:51:45.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.964+0000 7fa3bca6a740 1 -- 172.21.15.16:0/260049123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e31660 msgr2=0x55fb5a2155c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:45.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.964+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/260049123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e31660 0x55fb5a2155c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0005ed0 tx=0x7fa3a0005f00 comp rx=0 tx=0).stop 2024-09-21T03:51:45.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.964+0000 7fa3bca6a740 1 -- 172.21.15.16:0/260049123 shutdown_connections 2024-09-21T03:51:45.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.964+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/260049123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb59e5a520 0x55fb5a212100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.964+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/260049123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb59e39840 0x55fb5a211a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.964+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/260049123 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55fb59dee250 0x55fb5a28cff0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.964+0000 7fa3bca6a740 1 --2- 172.21.15.16:0/260049123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb59e31660 0x55fb5a2155c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.964+0000 7fa3bca6a740 1 -- 172.21.15.16:0/260049123 >> 172.21.15.16:0/260049123 conn(0x55fb59e4e910 msgr2=0x55fb5a20bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:45.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.964+0000 7fa39cc00640 0 client.16050 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:45.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.965+0000 7fa3bca6a740 1 -- 172.21.15.16:0/260049123 shutdown_connections 2024-09-21T03:51:45.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.965+0000 7fa3bca6a740 1 -- 172.21.15.16:0/260049123 wait complete. 2024-09-21T03:51:45.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.965+0000 7fa3bca6a740 20 client.16050 trim_cache size 0 max 16384 2024-09-21T03:51:45.968 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:51:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:51:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:51:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 20, 2024-09-21T03:51:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:51:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "58330903": { 2024-09-21T03:51:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 20, 2024-09-21T03:51:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:51:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:51:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:51:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:51:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:51:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:51:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 130.0, 2024-09-21T03:51:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:51:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:51:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:51:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:51:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:51:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:51:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:51:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:51:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:51:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:51:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:51:45.970 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:51:45.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.966+0000 7fa3b5e00640 1 -- 172.21.15.16:0/3233486247 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa3940b6450 msgr2=0x7fa3940b8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.966+0000 7fa3b5e00640 1 --2- 172.21.15.16:0/3233486247 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa3940b6450 0x7fa3940b8960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55fb5a1ffe40 tx=0x7fa3a0005890 comp rx=0 tx=0).stop 2024-09-21T03:51:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.966+0000 7fa3b5e00640 1 -- 172.21.15.16:0/3233486247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3b0105300 msgr2=0x7fa3b019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.966+0000 7fa3b5e00640 1 --2- 172.21.15.16:0/3233486247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3b0105300 0x7fa3b019cca0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa3b006b970 tx=0x7fa39800ed90 comp rx=0 tx=0).stop 2024-09-21T03:51:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.967+0000 7fa3b5e00640 1 -- 172.21.15.16:0/3233486247 shutdown_connections 2024-09-21T03:51:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.967+0000 7fa3b5e00640 1 --2- 172.21.15.16:0/3233486247 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa3940b6450 0x7fa3940b8960 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.967+0000 7fa3b5e00640 1 --2- 172.21.15.16:0/3233486247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3b010de10 0x7fa3b01a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.967+0000 7fa3b5e00640 1 --2- 172.21.15.16:0/3233486247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3b0105300 0x7fa3b019cca0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.967+0000 7fa3b5e00640 1 --2- 172.21.15.16:0/3233486247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3b01049b0 0x7fa3b019c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.967+0000 7fa3b5e00640 1 -- 172.21.15.16:0/3233486247 >> 172.21.15.16:0/3233486247 conn(0x7fa3b0079720 msgr2=0x7fa3b00ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.968+0000 7fa3b5e00640 1 -- 172.21.15.16:0/3233486247 shutdown_connections 2024-09-21T03:51:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:51:45.968+0000 7fa3b5e00640 1 -- 172.21.15.16:0/3233486247 wait complete. 2024-09-21T03:51:47.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:47 smithi134 ceph-mon[35031]: pgmap v582: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 1.26k op/s 2024-09-21T03:51:47.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:47 smithi123 ceph-mon[34673]: pgmap v582: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 1.26k op/s 2024-09-21T03:51:47.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:47 smithi016 ceph-mon[30375]: pgmap v582: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 1.26k op/s 2024-09-21T03:51:49.102 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '58330903', total seconds elapsed: 20.6 2024-09-21T03:51:49.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:49 smithi016 ceph-mon[30375]: pgmap v583: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 236 MiB/s wr, 1.25k op/s 2024-09-21T03:51:49.458 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:49 smithi123 ceph-mon[34673]: pgmap v583: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 236 MiB/s wr, 1.25k op/s 2024-09-21T03:51:49.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:49 smithi134 ceph-mon[35031]: pgmap v583: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 236 MiB/s wr, 1.25k op/s 2024-09-21T03:51:51.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:51 smithi134 ceph-mon[35031]: pgmap v584: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 227 MiB/s wr, 1.25k op/s 2024-09-21T03:51:51.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:51 smithi123 ceph-mon[34673]: pgmap v584: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 227 MiB/s wr, 1.25k op/s 2024-09-21T03:51:51.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:51 smithi016 ceph-mon[30375]: pgmap v584: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 227 MiB/s wr, 1.25k op/s 2024-09-21T03:51:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.086+0000 7f786bc00640 1 -- 172.21.15.16:0/2182878731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7864106950 msgr2=0x7f7864106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.086+0000 7f786bc00640 1 --2- 172.21.15.16:0/2182878731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7864106950 0x7f7864106d60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7850009900 tx=0x7f785002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.086+0000 7f786bc00640 1 -- 172.21.15.16:0/2182878731 shutdown_connections 2024-09-21T03:51:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.086+0000 7f786bc00640 1 --2- 172.21.15.16:0/2182878731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78641072f0 0x7f786410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.086+0000 7f786bc00640 1 --2- 172.21.15.16:0/2182878731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7864106950 0x7f7864106d60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.086+0000 7f786bc00640 1 --2- 172.21.15.16:0/2182878731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78641058d0 0x7f7864105d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.086+0000 7f786bc00640 1 -- 172.21.15.16:0/2182878731 >> 172.21.15.16:0/2182878731 conn(0x7f78641011f0 msgr2=0x7f78641036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.086+0000 7f786bc00640 1 -- 172.21.15.16:0/2182878731 shutdown_connections 2024-09-21T03:51:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.086+0000 7f786bc00640 1 -- 172.21.15.16:0/2182878731 wait complete. 2024-09-21T03:51:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.088+0000 7f786bc00640 1 Processor -- start 2024-09-21T03:51:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.088+0000 7f786bc00640 1 -- start start 2024-09-21T03:51:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.089+0000 7f786bc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78641058d0 0x7f786419c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.089+0000 7f786bc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7864106950 0x7f786419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.089+0000 7f786bc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78641072f0 0x7f786419d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.089+0000 7f786bc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7864111550 con 0x7f7864106950 2024-09-21T03:51:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.089+0000 7f786bc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f78641113d0 con 0x7f78641072f0 2024-09-21T03:51:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.089+0000 7f786bc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78641116d0 con 0x7f78641058d0 2024-09-21T03:51:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.089+0000 7f7869200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78641058d0 0x7f786419c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.089+0000 7f7869200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78641058d0 0x7f786419c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:49656/0 (socket says 172.21.15.16:49656) 2024-09-21T03:51:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.089+0000 7f7869200640 1 -- 172.21.15.16:0/1204100559 learned_addr learned my addr 172.21.15.16:0/1204100559 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.090+0000 7f7863e00640 1 --2- 172.21.15.16:0/1204100559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7864106950 0x7f786419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.090+0000 7f7869200640 1 -- 172.21.15.16:0/1204100559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78641072f0 msgr2=0x7f786419d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.090+0000 7f7869200640 1 --2- 172.21.15.16:0/1204100559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78641072f0 0x7f786419d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.090+0000 7f7869200640 1 -- 172.21.15.16:0/1204100559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7864106950 msgr2=0x7f786419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.090+0000 7f7869200640 1 --2- 172.21.15.16:0/1204100559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7864106950 0x7f786419c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.090+0000 7f7869200640 1 -- 172.21.15.16:0/1204100559 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f786419db30 con 0x7f78641058d0 2024-09-21T03:51:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.090+0000 7f7869200640 1 --2- 172.21.15.16:0/1204100559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78641058d0 0x7f786419c180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f785800c920 tx=0x7f785800cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.091+0000 7f7861600640 1 -- 172.21.15.16:0/1204100559 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7858007c70 con 0x7f78641058d0 2024-09-21T03:51:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.091+0000 7f7861600640 1 -- 172.21.15.16:0/1204100559 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7858007e10 con 0x7f78641058d0 2024-09-21T03:51:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.091+0000 7f7861600640 1 -- 172.21.15.16:0/1204100559 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7858018820 con 0x7f78641058d0 2024-09-21T03:51:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.092+0000 7f786bc00640 1 -- 172.21.15.16:0/1204100559 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f786419de20 con 0x7f78641058d0 2024-09-21T03:51:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.092+0000 7f786bc00640 1 -- 172.21.15.16:0/1204100559 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78641a45a0 con 0x7f78641058d0 2024-09-21T03:51:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.104+0000 7f7872c6c740 1 Processor -- start 2024-09-21T03:51:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.104+0000 7f7872c6c740 1 -- start start 2024-09-21T03:51:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.105+0000 7f7872c6c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f2102d1b0 0x559f213cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.105+0000 7f7872c6c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f210042f0 0x559f213cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.105+0000 7f7872c6c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f2100c4d0 0x559f213d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.105+0000 7f7872c6c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559f213476b0 con 0x559f2102d1b0 2024-09-21T03:51:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.105+0000 7f7872c6c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559f21336ae0 con 0x559f210042f0 2024-09-21T03:51:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.105+0000 7f7872c6c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f2133a8f0 con 0x559f2100c4d0 2024-09-21T03:51:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.106+0000 7f7869c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f2102d1b0 0x559f213cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.106+0000 7f7869c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f2102d1b0 0x559f213cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33622/0 (socket says 172.21.15.16:33622) 2024-09-21T03:51:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.106+0000 7f7869c00640 1 -- 172.21.15.16:0/3353229073 learned_addr learned my addr 172.21.15.16:0/3353229073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.109+0000 7f7863e00640 1 --2- 172.21.15.16:0/3353229073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f2100c4d0 0x559f213d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.109+0000 7f7863e00640 1 -- 172.21.15.16:0/3353229073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f210042f0 msgr2=0x559f213cfdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:51:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.109+0000 7f7863e00640 1 --2- 172.21.15.16:0/3353229073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f210042f0 0x559f213cfdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.109+0000 7f7863e00640 1 -- 172.21.15.16:0/3353229073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f2102d1b0 msgr2=0x559f213cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.109+0000 7f7863e00640 1 --2- 172.21.15.16:0/3353229073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f2102d1b0 0x559f213cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.109+0000 7f7863e00640 1 -- 172.21.15.16:0/3353229073 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f21303780 con 0x559f2100c4d0 2024-09-21T03:51:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.110+0000 7f7863e00640 1 --2- 172.21.15.16:0/3353229073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f2100c4d0 0x559f213d0940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7850006010 tx=0x7f785002f510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af5ee1273ea6cb59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.110+0000 7f7856000640 1 -- 172.21.15.16:0/3353229073 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785003f070 con 0x559f2100c4d0 2024-09-21T03:51:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.110+0000 7f7861600640 1 -- 172.21.15.16:0/1204100559 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f785800f450 con 0x7f78641058d0 2024-09-21T03:51:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.110+0000 7f7856000640 1 -- 172.21.15.16:0/3353229073 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7850004440 con 0x559f2100c4d0 2024-09-21T03:51:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.111+0000 7f7872c6c740 1 -- 172.21.15.16:0/3353229073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f2100c4d0 msgr2=0x559f213d0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.112+0000 7f7856000640 1 -- 172.21.15.16:0/3353229073 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785003f070 con 0x559f2100c4d0 2024-09-21T03:51:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.112+0000 7f7872c6c740 1 --2- 172.21.15.16:0/3353229073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f2100c4d0 0x559f213d0940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7850006010 tx=0x7f785002f510 comp rx=0 tx=0).stop 2024-09-21T03:51:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.113+0000 7f7872c6c740 1 -- 172.21.15.16:0/3353229073 shutdown_connections 2024-09-21T03:51:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.113+0000 7f7872c6c740 1 --2- 172.21.15.16:0/3353229073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f2100c4d0 0x559f213d0940 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.113+0000 7f7872c6c740 1 --2- 172.21.15.16:0/3353229073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f210042f0 0x559f213cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.113+0000 7f7872c6c740 1 --2- 172.21.15.16:0/3353229073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f2102d1b0 0x559f213cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.113+0000 7f7872c6c740 1 -- 172.21.15.16:0/3353229073 >> 172.21.15.16:0/3353229073 conn(0x559f21021300 msgr2=0x559f213d8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.113+0000 7f7872c6c740 1 -- 172.21.15.16:0/3353229073 shutdown_connections 2024-09-21T03:51:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.113+0000 7f7872c6c740 1 -- 172.21.15.16:0/3353229073 wait complete. 2024-09-21T03:51:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.114+0000 7f7872c6c740 1 -- start start 2024-09-21T03:51:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.114+0000 7f7872c6c740 1 Processor -- start 2024-09-21T03:51:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.116+0000 7f7872c6c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f210042f0 0x559f213ee7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.116+0000 7f7872c6c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f2100c4d0 0x559f213eee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.117+0000 7f7872c6c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f2102d1b0 0x559f213ef9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.117+0000 7f7872c6c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559f2126eab0 con 0x559f2100c4d0 2024-09-21T03:51:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.117+0000 7f7872c6c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559f212dd570 con 0x559f2102d1b0 2024-09-21T03:51:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.117+0000 7f7872c6c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f213d11c0 con 0x559f210042f0 2024-09-21T03:51:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.117+0000 7f7869c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f210042f0 0x559f213ee7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.117+0000 7f7869c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f210042f0 0x559f213ee7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:49672/0 (socket says 172.21.15.16:49672) 2024-09-21T03:51:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.117+0000 7f7869c00640 1 -- 172.21.15.16:0/1657727465 learned_addr learned my addr 172.21.15.16:0/1657727465 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.117+0000 7f7872c6c740 10 client.0 _ll_register_callbacks cb 0x559f212ec100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.118+0000 7f7869c00640 1 -- 172.21.15.16:0/1657727465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f2102d1b0 msgr2=0x559f213ef9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:51:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.118+0000 7f7869c00640 1 --2- 172.21.15.16:0/1657727465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f2102d1b0 0x559f213ef9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.118+0000 7f7869c00640 1 -- 172.21.15.16:0/1657727465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f2100c4d0 msgr2=0x559f213eee60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:51:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.118+0000 7f7869c00640 1 --2- 172.21.15.16:0/1657727465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f2100c4d0 0x559f213eee60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.118+0000 7f7869c00640 1 -- 172.21.15.16:0/1657727465 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f21098790 con 0x559f210042f0 2024-09-21T03:51:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.118+0000 7f7869c00640 1 --2- 172.21.15.16:0/1657727465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f210042f0 0x559f213ee7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f785c010680 tx=0x7f785c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=505340a95228c67d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.118+0000 7f784aa00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:51:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.119+0000 7f784aa00640 1 -- 172.21.15.16:0/1657727465 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c004d20 con 0x559f210042f0 2024-09-21T03:51:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.119+0000 7f784aa00640 1 -- 172.21.15.16:0/1657727465 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f785c00e650 con 0x559f210042f0 2024-09-21T03:51:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.123+0000 7f7872c6c740 1 -- 172.21.15.16:0/1657727465 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7850004440 con 0x559f210042f0 2024-09-21T03:51:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.123+0000 7f784aa00640 1 -- 172.21.15.16:0/1657727465 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f785c00e850 con 0x559f210042f0 2024-09-21T03:51:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.124+0000 7f7872c6c740 10 client.26104 fetch_fsmap learned FSMap version 30 2024-09-21T03:51:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.124+0000 7f7872c6c740 1 -- 172.21.15.16:0/1657727465 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f21111620 con 0x559f210042f0 2024-09-21T03:51:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.125+0000 7f784aa00640 1 -- 172.21.15.16:0/1657727465 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7850004440 con 0x559f210042f0 2024-09-21T03:51:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.133+0000 7f7861600640 1 --2- 172.21.15.16:0/1204100559 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f784c0b64d0 0x7f784c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.133+0000 7f7861600640 1 -- 172.21.15.16:0/1204100559 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f78580e7880 con 0x7f78641058d0 2024-09-21T03:51:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.136+0000 7f784aa00640 1 -- 172.21.15.16:0/1657727465 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f785c005760 con 0x559f210042f0 2024-09-21T03:51:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.136+0000 7f7872c6c740 10 client.26104 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:51:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.138+0000 7f7872c6c740 10 client.26104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.139+0000 7f7872c6c740 20 client.26104 populate_metadata read hostname 'smithi016' 2024-09-21T03:51:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.139+0000 7f7872c6c740 1 --2- 172.21.15.16:0/1657727465 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x559f20febd80 0x559f213fc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.139+0000 7f7872c6c740 4 client.26104 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:51:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.139+0000 7f7872c6c740 1 -- 172.21.15.16:0/1657727465 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f21349600 con 0x559f20febd80 2024-09-21T03:51:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.140+0000 7f7869200640 1 --2- 172.21.15.16:0/1204100559 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f784c0b64d0 0x7f784c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.141+0000 7f7869200640 1 --2- 172.21.15.16:0/1204100559 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f784c0b64d0 0x7f784c0b89e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f78580ad590 tx=0x7f785800fe60 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.142+0000 7f7863e00640 1 --2- 172.21.15.16:0/1657727465 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x559f20febd80 0x559f213fc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.144+0000 7f7863e00640 1 --2- 172.21.15.16:0/1657727465 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x559f20febd80 0x559f213fc020 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f590c8f013bcaf server_cookie=8a53e7c5ebddaaab in_seq=0 out_seq=0 2024-09-21T03:51:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.146+0000 7f784aa00640 10 client.26104 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.146+0000 7f784aa00640 1 -- 172.21.15.16:0/1657727465 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f21349600 con 0x559f20febd80 2024-09-21T03:51:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.146+0000 7f784aa00640 10 client.26104 handle_command_reply: tid=0 2024-09-21T03:51:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.149+0000 7f7872c6c740 1 client.26104 shutdown 2024-09-21T03:51:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.149+0000 7f7872c6c740 1 -- 172.21.15.16:0/1657727465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f210042f0 msgr2=0x559f213ee7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.149+0000 7f7872c6c740 1 --2- 172.21.15.16:0/1657727465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f210042f0 0x559f213ee7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f785c010680 tx=0x7f785c009f90 comp rx=0 tx=0).stop 2024-09-21T03:51:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.150+0000 7f7872c6c740 1 -- 172.21.15.16:0/1657727465 shutdown_connections 2024-09-21T03:51:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.150+0000 7f7872c6c740 1 --2- 172.21.15.16:0/1657727465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f2102d1b0 0x559f213ef9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.150+0000 7f7872c6c740 1 --2- 172.21.15.16:0/1657727465 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x559f20febd80 0x559f213fc020 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.150+0000 7f7872c6c740 1 --2- 172.21.15.16:0/1657727465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f2100c4d0 0x559f213eee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.150+0000 7f7872c6c740 1 --2- 172.21.15.16:0/1657727465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f210042f0 0x559f213ee7a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.150+0000 7f7872c6c740 1 -- 172.21.15.16:0/1657727465 >> 172.21.15.16:0/1657727465 conn(0x559f21021300 msgr2=0x559f213ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.150+0000 7f784aa00640 0 client.26104 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.150+0000 7f7872c6c740 1 -- 172.21.15.16:0/1657727465 shutdown_connections 2024-09-21T03:51:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.150+0000 7f7872c6c740 1 -- 172.21.15.16:0/1657727465 wait complete. 2024-09-21T03:51:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.154+0000 7f7872c6c740 20 client.26104 trim_cache size 0 max 16384 2024-09-21T03:51:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.214+0000 7f7872c6c740 1 Processor -- start 2024-09-21T03:51:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.215+0000 7f7872c6c740 1 -- start start 2024-09-21T03:51:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.220+0000 7f7872c6c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f210042f0 0x559f213d3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.220+0000 7f7872c6c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f2100c4d0 0x559f213d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.220+0000 7f7872c6c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f2102d1b0 0x559f213ed9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.220+0000 7f7872c6c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78641aa4f0 con 0x559f210042f0 2024-09-21T03:51:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.220+0000 7f7872c6c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f78641ac030 con 0x559f2102d1b0 2024-09-21T03:51:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.220+0000 7f7872c6c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78641aabc0 con 0x559f2100c4d0 2024-09-21T03:51:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.220+0000 7f7863e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f2100c4d0 0x559f213d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.220+0000 7f7863e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f2100c4d0 0x559f213d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:49674/0 (socket says 172.21.15.16:49674) 2024-09-21T03:51:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.223+0000 7f7863e00640 1 -- 172.21.15.16:0/3769606627 learned_addr learned my addr 172.21.15.16:0/3769606627 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.220+0000 7f7869c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f2102d1b0 0x559f213ed9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.223+0000 7f7869c00640 1 --2- 172.21.15.16:0/3769606627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f210042f0 0x559f213d3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.223+0000 7f7863e00640 1 -- 172.21.15.16:0/3769606627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f2102d1b0 msgr2=0x559f213ed9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.223+0000 7f7863e00640 1 --2- 172.21.15.16:0/3769606627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f2102d1b0 0x559f213ed9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.223+0000 7f7863e00640 1 -- 172.21.15.16:0/3769606627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f210042f0 msgr2=0x559f213d3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.224+0000 7f7863e00640 1 --2- 172.21.15.16:0/3769606627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f210042f0 0x559f213d3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.224+0000 7f7863e00640 1 -- 172.21.15.16:0/3769606627 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f213ee530 con 0x559f2100c4d0 2024-09-21T03:51:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.224+0000 7f7869c00640 1 --2- 172.21.15.16:0/3769606627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f210042f0 0x559f213d3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:51:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.224+0000 7f7863e00640 1 --2- 172.21.15.16:0/3769606627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f2100c4d0 0x559f213d05c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7850004650 tx=0x7f7850000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f61dc1635e939fda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.225+0000 7f7855600640 1 -- 172.21.15.16:0/3769606627 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785003f040 con 0x559f2100c4d0 2024-09-21T03:51:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.225+0000 7f7855600640 1 -- 172.21.15.16:0/3769606627 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7850004240 con 0x559f2100c4d0 2024-09-21T03:51:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.225+0000 7f7855600640 1 -- 172.21.15.16:0/3769606627 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785003f040 con 0x559f2100c4d0 2024-09-21T03:51:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.228+0000 7f7872c6c740 1 -- 172.21.15.16:0/3769606627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f2100c4d0 msgr2=0x559f213d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.228+0000 7f7872c6c740 1 --2- 172.21.15.16:0/3769606627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f2100c4d0 0x559f213d05c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7850004650 tx=0x7f7850000bc0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.229+0000 7f7872c6c740 1 -- 172.21.15.16:0/3769606627 shutdown_connections 2024-09-21T03:51:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.229+0000 7f7872c6c740 1 --2- 172.21.15.16:0/3769606627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f2102d1b0 0x559f213ed9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.229+0000 7f7872c6c740 1 --2- 172.21.15.16:0/3769606627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f2100c4d0 0x559f213d05c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.229+0000 7f7872c6c740 1 --2- 172.21.15.16:0/3769606627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f210042f0 0x559f213d3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.229+0000 7f7872c6c740 1 -- 172.21.15.16:0/3769606627 >> 172.21.15.16:0/3769606627 conn(0x559f21021300 msgr2=0x559f213d7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.229+0000 7f7872c6c740 1 -- 172.21.15.16:0/3769606627 shutdown_connections 2024-09-21T03:51:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.229+0000 7f7872c6c740 1 -- 172.21.15.16:0/3769606627 wait complete. 2024-09-21T03:51:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.230+0000 7f7872c6c740 1 -- start start 2024-09-21T03:51:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.230+0000 7f7872c6c740 1 Processor -- start 2024-09-21T03:51:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.232+0000 7f7872c6c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f210042f0 0x559f213f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.232+0000 7f7872c6c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f2100c4d0 0x559f213f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.232+0000 7f7872c6c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f2102d1b0 0x559f2145eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.232+0000 7f7872c6c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78641ab960 con 0x559f2102d1b0 2024-09-21T03:51:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.232+0000 7f7872c6c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f78641ab290 con 0x559f210042f0 2024-09-21T03:51:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.232+0000 7f7872c6c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78641a8590 con 0x559f2100c4d0 2024-09-21T03:51:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.232+0000 7f7872c6c740 10 client.0 _ll_register_callbacks cb 0x7f785c009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:51:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.233+0000 7f7869c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f210042f0 0x559f213f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.233+0000 7f7869c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f210042f0 0x559f213f5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:43736/0 (socket says 172.21.15.16:43736) 2024-09-21T03:51:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.233+0000 7f7869c00640 1 -- 172.21.15.16:0/3284163442 learned_addr learned my addr 172.21.15.16:0/3284163442 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:51:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.233+0000 7f7869c00640 1 --2- 172.21.15.16:0/3284163442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f2102d1b0 0x559f2145eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.234+0000 7f7869c00640 1 -- 172.21.15.16:0/3284163442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f2100c4d0 msgr2=0x559f213f66b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:51:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.234+0000 7f7869c00640 1 --2- 172.21.15.16:0/3284163442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f2100c4d0 0x559f213f66b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.234+0000 7f7869c00640 1 -- 172.21.15.16:0/3284163442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f2102d1b0 msgr2=0x559f2145eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:51:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.234+0000 7f7869c00640 1 --2- 172.21.15.16:0/3284163442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f2102d1b0 0x559f2145eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.234+0000 7f7869c00640 1 -- 172.21.15.16:0/3284163442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f2145f530 con 0x559f210042f0 2024-09-21T03:51:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.234+0000 7f784aa00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:51:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.235+0000 7f7869c00640 1 --2- 172.21.15.16:0/3284163442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f210042f0 0x559f213f5ff0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f785c0099a0 tx=0x7f785c009ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdb8144c099cda01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:51:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.235+0000 7f784aa00640 1 -- 172.21.15.16:0/3284163442 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c00e9f0 con 0x559f210042f0 2024-09-21T03:51:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.235+0000 7f7872c6c740 1 -- 172.21.15.16:0/3284163442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7850004240 con 0x559f210042f0 2024-09-21T03:51:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.236+0000 7f784aa00640 1 -- 172.21.15.16:0/3284163442 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f785c002030 con 0x559f210042f0 2024-09-21T03:51:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.236+0000 7f784aa00640 1 -- 172.21.15.16:0/3284163442 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f785c00e9f0 con 0x559f210042f0 2024-09-21T03:51:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.236+0000 7f7872c6c740 10 client.26165 fetch_fsmap learned FSMap version 30 2024-09-21T03:51:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.236+0000 7f7872c6c740 1 -- 172.21.15.16:0/3284163442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f21466830 con 0x559f210042f0 2024-09-21T03:51:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.237+0000 7f784aa00640 1 -- 172.21.15.16:0/3284163442 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7850004240 con 0x559f210042f0 2024-09-21T03:51:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.237+0000 7f784aa00640 1 -- 172.21.15.16:0/3284163442 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f785c0051b0 con 0x559f210042f0 2024-09-21T03:51:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.237+0000 7f7872c6c740 10 client.26165 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:51:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.237+0000 7f7872c6c740 10 client.26165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:51:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.237+0000 7f7872c6c740 20 client.26165 populate_metadata read hostname 'smithi016' 2024-09-21T03:51:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.237+0000 7f7872c6c740 1 --2- 172.21.15.16:0/3284163442 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x559f20fc11c0 0x559f21469200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:51:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.237+0000 7f7872c6c740 4 client.26165 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:51:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.237+0000 7f7872c6c740 1 -- 172.21.15.16:0/3284163442 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f213450b0 con 0x559f20fc11c0 2024-09-21T03:51:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.238+0000 7f7863e00640 1 --2- 172.21.15.16:0/3284163442 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x559f20fc11c0 0x559f21469200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:51:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.240+0000 7f7863e00640 1 --2- 172.21.15.16:0/3284163442 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x559f20fc11c0 0x559f21469200 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7931f9f11b5a98df server_cookie=ca6ac85fa76a2a39 in_seq=0 out_seq=0 2024-09-21T03:51:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.240+0000 7f784aa00640 10 client.26165 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.262+0000 7f784aa00640 1 -- 172.21.15.16:0/3284163442 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f213450b0 con 0x559f20fc11c0 2024-09-21T03:51:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.262+0000 7f784aa00640 10 client.26165 handle_command_reply: tid=0 2024-09-21T03:51:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.262+0000 7f7872c6c740 1 client.26165 shutdown 2024-09-21T03:51:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.263+0000 7f7872c6c740 1 -- 172.21.15.16:0/3284163442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f210042f0 msgr2=0x559f213f5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.263+0000 7f7872c6c740 1 --2- 172.21.15.16:0/3284163442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f210042f0 0x559f213f5ff0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f785c0099a0 tx=0x7f785c009ab0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.263+0000 7f7872c6c740 1 -- 172.21.15.16:0/3284163442 shutdown_connections 2024-09-21T03:51:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.263+0000 7f7872c6c740 1 --2- 172.21.15.16:0/3284163442 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x559f20fc11c0 0x559f21469200 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.263+0000 7f7872c6c740 1 --2- 172.21.15.16:0/3284163442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f2102d1b0 0x559f2145eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.263+0000 7f7872c6c740 1 --2- 172.21.15.16:0/3284163442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f2100c4d0 0x559f213f66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.263+0000 7f7872c6c740 1 --2- 172.21.15.16:0/3284163442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559f210042f0 0x559f213f5ff0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.263+0000 7f7872c6c740 1 -- 172.21.15.16:0/3284163442 >> 172.21.15.16:0/3284163442 conn(0x559f21021300 msgr2=0x559f213d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.263+0000 7f784aa00640 0 client.26165 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:51:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.263+0000 7f7872c6c740 1 -- 172.21.15.16:0/3284163442 shutdown_connections 2024-09-21T03:51:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.263+0000 7f7872c6c740 1 -- 172.21.15.16:0/3284163442 wait complete. 2024-09-21T03:51:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.264+0000 7f7872c6c740 20 client.26165 trim_cache size 0 max 16384 2024-09-21T03:51:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:51:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:51:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:51:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:51:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.265+0000 7f786bc00640 1 -- 172.21.15.16:0/1204100559 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f784c0b64d0 msgr2=0x7f784c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.265+0000 7f786bc00640 1 --2- 172.21.15.16:0/1204100559 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f784c0b64d0 0x7f784c0b89e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f78580ad590 tx=0x7f785800fe60 comp rx=0 tx=0).stop 2024-09-21T03:51:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.265+0000 7f786bc00640 1 -- 172.21.15.16:0/1204100559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78641058d0 msgr2=0x7f786419c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:51:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.265+0000 7f786bc00640 1 --2- 172.21.15.16:0/1204100559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78641058d0 0x7f786419c180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f785800c920 tx=0x7f785800cde0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.265+0000 7f786bc00640 1 -- 172.21.15.16:0/1204100559 shutdown_connections 2024-09-21T03:51:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.265+0000 7f786bc00640 1 --2- 172.21.15.16:0/1204100559 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f784c0b64d0 0x7f784c0b89e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.265+0000 7f786bc00640 1 --2- 172.21.15.16:0/1204100559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78641072f0 0x7f786419d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.265+0000 7f786bc00640 1 --2- 172.21.15.16:0/1204100559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7864106950 0x7f786419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.265+0000 7f786bc00640 1 --2- 172.21.15.16:0/1204100559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78641058d0 0x7f786419c180 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:51:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.265+0000 7f786bc00640 1 -- 172.21.15.16:0/1204100559 >> 172.21.15.16:0/1204100559 conn(0x7f78641011f0 msgr2=0x7f7864109c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:51:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.265+0000 7f786bc00640 1 -- 172.21.15.16:0/1204100559 shutdown_connections 2024-09-21T03:51:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:51:52.265+0000 7f786bc00640 1 -- 172.21.15.16:0/1204100559 wait complete. 2024-09-21T03:51:53.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:53 smithi134 ceph-mon[35031]: pgmap v585: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 213 MiB/s wr, 1.21k op/s 2024-09-21T03:51:53.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:53 smithi123 ceph-mon[34673]: pgmap v585: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 213 MiB/s wr, 1.21k op/s 2024-09-21T03:51:53.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:53 smithi016 ceph-mon[30375]: pgmap v585: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 213 MiB/s wr, 1.21k op/s 2024-09-21T03:51:55.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:55 smithi134 ceph-mon[35031]: pgmap v586: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 222 MiB/s wr, 1.28k op/s 2024-09-21T03:51:55.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:55 smithi123 ceph-mon[34673]: pgmap v586: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 222 MiB/s wr, 1.28k op/s 2024-09-21T03:51:55.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:55 smithi016 ceph-mon[30375]: pgmap v586: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 222 MiB/s wr, 1.28k op/s 2024-09-21T03:51:57.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:57 smithi134 ceph-mon[35031]: pgmap v587: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 225 MiB/s wr, 1.29k op/s 2024-09-21T03:51:57.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:57 smithi123 ceph-mon[34673]: pgmap v587: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 225 MiB/s wr, 1.29k op/s 2024-09-21T03:51:57.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:57 smithi016 ceph-mon[30375]: pgmap v587: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 225 MiB/s wr, 1.29k op/s 2024-09-21T03:51:59.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:51:59 smithi134 ceph-mon[35031]: pgmap v588: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 219 MiB/s wr, 1.23k op/s 2024-09-21T03:51:59.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:51:59 smithi123 ceph-mon[34673]: pgmap v588: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 219 MiB/s wr, 1.23k op/s 2024-09-21T03:51:59.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:51:59 smithi016 ceph-mon[30375]: pgmap v588: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 219 MiB/s wr, 1.23k op/s 2024-09-21T03:52:01.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:01 smithi134 ceph-mon[35031]: pgmap v589: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 221 MiB/s wr, 1.21k op/s 2024-09-21T03:52:01.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:01 smithi123 ceph-mon[34673]: pgmap v589: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 221 MiB/s wr, 1.21k op/s 2024-09-21T03:52:01.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:01 smithi016 ceph-mon[30375]: pgmap v589: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 221 MiB/s wr, 1.21k op/s 2024-09-21T03:52:03.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:03 smithi016 ceph-mon[30375]: pgmap v590: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 220 MiB/s wr, 1.18k op/s 2024-09-21T03:52:03.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:03 smithi134 ceph-mon[35031]: pgmap v590: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 220 MiB/s wr, 1.18k op/s 2024-09-21T03:52:03.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:03 smithi123 ceph-mon[34673]: pgmap v590: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 220 MiB/s wr, 1.18k op/s 2024-09-21T03:52:05.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:05 smithi134 ceph-mon[35031]: pgmap v591: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 246 MiB/s wr, 1.32k op/s 2024-09-21T03:52:05.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:05 smithi123 ceph-mon[34673]: pgmap v591: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 246 MiB/s wr, 1.32k op/s 2024-09-21T03:52:05.668 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:05 smithi016 ceph-mon[30375]: pgmap v591: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 246 MiB/s wr, 1.32k op/s 2024-09-21T03:52:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.761+0000 7fe967c00640 1 -- 172.21.15.16:0/231061815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe960111f90 msgr2=0x7fe9601143b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.761+0000 7fe967c00640 1 --2- 172.21.15.16:0/231061815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe960111f90 0x7fe9601143b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe958009ca0 tx=0x7fe95802a8b0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.763+0000 7fe967c00640 1 -- 172.21.15.16:0/231061815 shutdown_connections 2024-09-21T03:52:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.763+0000 7fe967c00640 1 --2- 172.21.15.16:0/231061815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe960111f90 0x7fe9601143b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.763+0000 7fe967c00640 1 --2- 172.21.15.16:0/231061815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe96010adf0 0x7fe96010b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.763+0000 7fe967c00640 1 --2- 172.21.15.16:0/231061815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe96010a3c0 0x7fe96010a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.763+0000 7fe967c00640 1 -- 172.21.15.16:0/231061815 >> 172.21.15.16:0/231061815 conn(0x7fe960070c70 msgr2=0x7fe960073120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.764+0000 7fe967c00640 1 -- 172.21.15.16:0/231061815 shutdown_connections 2024-09-21T03:52:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.764+0000 7fe967c00640 1 -- 172.21.15.16:0/231061815 wait complete. 2024-09-21T03:52:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.765+0000 7fe967c00640 1 Processor -- start 2024-09-21T03:52:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.766+0000 7fe967c00640 1 -- start start 2024-09-21T03:52:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.766+0000 7fe967c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe96010a3c0 0x7fe9601a8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.766+0000 7fe967c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe96010adf0 0x7fe9601a9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.767+0000 7fe95fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe96010adf0 0x7fe9601a9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.767+0000 7fe95fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe96010adf0 0x7fe9601a9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59262/0 (socket says 172.21.15.16:59262) 2024-09-21T03:52:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.768+0000 7fe965200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe96010a3c0 0x7fe9601a8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.768+0000 7fe965200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe96010a3c0 0x7fe9601a8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34950/0 (socket says 172.21.15.16:34950) 2024-09-21T03:52:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.770+0000 7fe967c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe960111f90 0x7fe9601a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.770+0000 7fe967c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe96010daa0 con 0x7fe96010adf0 2024-09-21T03:52:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.770+0000 7fe967c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe96010d920 con 0x7fe96010a3c0 2024-09-21T03:52:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.770+0000 7fe967c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe96010dc20 con 0x7fe960111f90 2024-09-21T03:52:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.770+0000 7fe95fe00640 1 -- 172.21.15.16:0/3829945279 learned_addr learned my addr 172.21.15.16:0/3829945279 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.771+0000 7fe965c00640 1 --2- 172.21.15.16:0/3829945279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe960111f90 0x7fe9601a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.771+0000 7fe95fe00640 1 -- 172.21.15.16:0/3829945279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe960111f90 msgr2=0x7fe9601a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.771+0000 7fe95fe00640 1 --2- 172.21.15.16:0/3829945279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe960111f90 0x7fe9601a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.771+0000 7fe95fe00640 1 -- 172.21.15.16:0/3829945279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe96010a3c0 msgr2=0x7fe9601a8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.771+0000 7fe95fe00640 1 --2- 172.21.15.16:0/3829945279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe96010a3c0 0x7fe9601a8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.771+0000 7fe95fe00640 1 -- 172.21.15.16:0/3829945279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9601aa710 con 0x7fe96010adf0 2024-09-21T03:52:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.771+0000 7fe965c00640 1 --2- 172.21.15.16:0/3829945279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe960111f90 0x7fe9601a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.772+0000 7fe965200640 1 --2- 172.21.15.16:0/3829945279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe96010a3c0 0x7fe9601a8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.772+0000 7fe95fe00640 1 --2- 172.21.15.16:0/3829945279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe96010adf0 0x7fe9601a9340 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00e9f0 tx=0x7fe94c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.773+0000 7fe95d600640 1 -- 172.21.15.16:0/3829945279 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c00cd80 con 0x7fe96010adf0 2024-09-21T03:52:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.773+0000 7fe95d600640 1 -- 172.21.15.16:0/3829945279 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe94c0108e0 con 0x7fe96010adf0 2024-09-21T03:52:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.773+0000 7fe967c00640 1 -- 172.21.15.16:0/3829945279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9601aaa00 con 0x7fe96010adf0 2024-09-21T03:52:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.774+0000 7fe967c00640 1 -- 172.21.15.16:0/3829945279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9601b10f0 con 0x7fe96010adf0 2024-09-21T03:52:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.774+0000 7fe95d600640 1 -- 172.21.15.16:0/3829945279 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c018980 con 0x7fe96010adf0 2024-09-21T03:52:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.779+0000 7fe96ec3f740 1 Processor -- start 2024-09-21T03:52:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.780+0000 7fe95d600640 1 -- 172.21.15.16:0/3829945279 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe94c018b20 con 0x7fe96010adf0 2024-09-21T03:52:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.780+0000 7fe96ec3f740 1 -- start start 2024-09-21T03:52:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.781+0000 7fe96ec3f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028c61b0 0x558102c6d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.781+0000 7fe96ec3f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55810289d2f0 0x558102c6d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.781+0000 7fe96ec3f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581028a54d0 0x558102c6e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.781+0000 7fe96ec3f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558102be06b0 con 0x5581028a54d0 2024-09-21T03:52:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.781+0000 7fe96ec3f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558102bcfae0 con 0x5581028c61b0 2024-09-21T03:52:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.781+0000 7fe96ec3f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558102bd38f0 con 0x55810289d2f0 2024-09-21T03:52:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.781+0000 7fe965c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028c61b0 0x558102c6d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.782+0000 7fe965200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55810289d2f0 0x558102c6d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.782+0000 7fe965c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028c61b0 0x558102c6d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34952/0 (socket says 172.21.15.16:34952) 2024-09-21T03:52:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.782+0000 7fe965200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55810289d2f0 0x558102c6d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34456/0 (socket says 172.21.15.16:34456) 2024-09-21T03:52:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.782+0000 7fe965c00640 1 -- 172.21.15.16:0/2855255213 learned_addr learned my addr 172.21.15.16:0/2855255213 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.782+0000 7fe965200640 1 -- 172.21.15.16:0/2855255213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028c61b0 msgr2=0x558102c6d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.782+0000 7fe965200640 1 --2- 172.21.15.16:0/2855255213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028c61b0 0x558102c6d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.782+0000 7fe965200640 1 -- 172.21.15.16:0/2855255213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581028a54d0 msgr2=0x558102c6e290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:52:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.783+0000 7fe965200640 1 --2- 172.21.15.16:0/2855255213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581028a54d0 0x558102c6e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.783+0000 7fe965200640 1 -- 172.21.15.16:0/2855255213 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558102b9c780 con 0x55810289d2f0 2024-09-21T03:52:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.783+0000 7fe95d600640 1 --2- 172.21.15.16:0/3829945279 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe9440b66a0 0x7fe9440b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.783+0000 7fe95d600640 1 -- 172.21.15.16:0/3829945279 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fe94c0e79a0 con 0x7fe96010adf0 2024-09-21T03:52:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.783+0000 7fe965200640 1 --2- 172.21.15.16:0/2855255213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55810289d2f0 0x558102c6d740 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe95000bc80 tx=0x7fe95000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2947f33c4044b0fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.783+0000 7fe965c00640 1 --2- 172.21.15.16:0/3829945279 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe9440b66a0 0x7fe9440b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.785+0000 7fe956000640 1 -- 172.21.15.16:0/2855255213 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950015070 con 0x55810289d2f0 2024-09-21T03:52:05.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.785+0000 7fe956000640 1 -- 172.21.15.16:0/2855255213 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95001a910 con 0x55810289d2f0 2024-09-21T03:52:05.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.786+0000 7fe96ec3f740 1 -- 172.21.15.16:0/2855255213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55810289d2f0 msgr2=0x558102c6d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.786+0000 7fe965c00640 1 --2- 172.21.15.16:0/3829945279 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe9440b66a0 0x7fe9440b8bb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe958009bf0 tx=0x7fe958030f80 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.787+0000 7fe956000640 1 -- 172.21.15.16:0/2855255213 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950011860 con 0x55810289d2f0 2024-09-21T03:52:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.787+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/2855255213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55810289d2f0 0x558102c6d740 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe95000bc80 tx=0x7fe95000f730 comp rx=0 tx=0).stop 2024-09-21T03:52:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.788+0000 7fe96ec3f740 1 -- 172.21.15.16:0/2855255213 shutdown_connections 2024-09-21T03:52:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.788+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/2855255213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581028a54d0 0x558102c6e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.788+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/2855255213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55810289d2f0 0x558102c6d740 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.788+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/2855255213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028c61b0 0x558102c6d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.788+0000 7fe96ec3f740 1 -- 172.21.15.16:0/2855255213 >> 172.21.15.16:0/2855255213 conn(0x5581028ba300 msgr2=0x558102c68b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.789+0000 7fe96ec3f740 1 -- 172.21.15.16:0/2855255213 shutdown_connections 2024-09-21T03:52:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.793+0000 7fe96ec3f740 1 -- 172.21.15.16:0/2855255213 wait complete. 2024-09-21T03:52:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.793+0000 7fe96ec3f740 1 -- start start 2024-09-21T03:52:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.794+0000 7fe96ec3f740 1 Processor -- start 2024-09-21T03:52:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.795+0000 7fe96ec3f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810289d2f0 0x558102c76830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.796+0000 7fe96ec3f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028a54d0 0x558102c76ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.796+0000 7fe96ec3f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581028c61b0 0x558102c77a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.796+0000 7fe96ec3f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558102b07ab0 con 0x55810289d2f0 2024-09-21T03:52:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.796+0000 7fe96ec3f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558102b76570 con 0x5581028a54d0 2024-09-21T03:52:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.796+0000 7fe96ec3f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558102c6ee00 con 0x5581028c61b0 2024-09-21T03:52:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.796+0000 7fe96ec3f740 10 client.0 _ll_register_callbacks cb 0x558102b85100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.796+0000 7fe95fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028a54d0 0x558102c76ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.796+0000 7fe95fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028a54d0 0x558102c76ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34968/0 (socket says 172.21.15.16:34968) 2024-09-21T03:52:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.796+0000 7fe95fe00640 1 -- 172.21.15.16:0/4177277109 learned_addr learned my addr 172.21.15.16:0/4177277109 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.797+0000 7fe95fe00640 1 -- 172.21.15.16:0/4177277109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581028c61b0 msgr2=0x558102c77a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:52:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.797+0000 7fe95fe00640 1 --2- 172.21.15.16:0/4177277109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581028c61b0 0x558102c77a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.797+0000 7fe95fe00640 1 -- 172.21.15.16:0/4177277109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810289d2f0 msgr2=0x558102c76830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.797+0000 7fe95fe00640 1 --2- 172.21.15.16:0/4177277109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810289d2f0 0x558102c76830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.797+0000 7fe95fe00640 1 -- 172.21.15.16:0/4177277109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558102931790 con 0x5581028a54d0 2024-09-21T03:52:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.797+0000 7fe95fe00640 1 --2- 172.21.15.16:0/4177277109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028a54d0 0x558102c76ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0a9cf0 tx=0x7fe94c010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=870452503d611129 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.803+0000 7fe94aa00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:52:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.803+0000 7fe94aa00640 1 -- 172.21.15.16:0/4177277109 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c0ad230 con 0x5581028a54d0 2024-09-21T03:52:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.803+0000 7fe94aa00640 1 -- 172.21.15.16:0/4177277109 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fe94c0ac3f0 con 0x5581028a54d0 2024-09-21T03:52:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.804+0000 7fe96ec3f740 1 -- 172.21.15.16:0/4177277109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe95001a910 con 0x5581028a54d0 2024-09-21T03:52:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.804+0000 7fe94aa00640 1 -- 172.21.15.16:0/4177277109 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe94c0ade20 con 0x5581028a54d0 2024-09-21T03:52:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.805+0000 7fe96ec3f740 10 client.26177 fetch_fsmap learned FSMap version 30 2024-09-21T03:52:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.805+0000 7fe96ec3f740 1 -- 172.21.15.16:0/4177277109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5581029aa620 con 0x5581028a54d0 2024-09-21T03:52:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.808+0000 7fe94aa00640 1 -- 172.21.15.16:0/4177277109 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95001a910 con 0x5581028a54d0 2024-09-21T03:52:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.817+0000 7fe94aa00640 1 -- 172.21.15.16:0/4177277109 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe94c010a90 con 0x5581028a54d0 2024-09-21T03:52:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.818+0000 7fe96ec3f740 10 client.26177 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:52:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.818+0000 7fe96ec3f740 10 client.26177 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.818+0000 7fe96ec3f740 20 client.26177 populate_metadata read hostname 'smithi016' 2024-09-21T03:52:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.818+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/4177277109 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x558102884d80 0x558102c83f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.818+0000 7fe96ec3f740 4 client.26177 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.818+0000 7fe96ec3f740 1 -- 172.21.15.16:0/4177277109 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558102be2600 con 0x558102884d80 2024-09-21T03:52:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.818+0000 7fe965200640 1 --2- 172.21.15.16:0/4177277109 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x558102884d80 0x558102c83f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.819+0000 7fe965200640 1 --2- 172.21.15.16:0/4177277109 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x558102884d80 0x558102c83f90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75365dbbb3b00cd2 server_cookie=dbf16f771fd7d124 in_seq=0 out_seq=0 2024-09-21T03:52:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.819+0000 7fe94aa00640 10 client.26177 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.820+0000 7fe94aa00640 1 -- 172.21.15.16:0/4177277109 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558102be2600 con 0x558102884d80 2024-09-21T03:52:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.820+0000 7fe94aa00640 10 client.26177 handle_command_reply: tid=0 2024-09-21T03:52:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.820+0000 7fe96ec3f740 1 client.26177 shutdown 2024-09-21T03:52:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.822+0000 7fe96ec3f740 1 -- 172.21.15.16:0/4177277109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028a54d0 msgr2=0x558102c76ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.822+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/4177277109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028a54d0 0x558102c76ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0a9cf0 tx=0x7fe94c010800 comp rx=0 tx=0).stop 2024-09-21T03:52:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.822+0000 7fe96ec3f740 1 -- 172.21.15.16:0/4177277109 shutdown_connections 2024-09-21T03:52:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.822+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/4177277109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581028c61b0 0x558102c77a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.822+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/4177277109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028a54d0 0x558102c76ef0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.822+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/4177277109 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x558102884d80 0x558102c83f90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.822+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/4177277109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810289d2f0 0x558102c76830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.822+0000 7fe96ec3f740 1 -- 172.21.15.16:0/4177277109 >> 172.21.15.16:0/4177277109 conn(0x5581028ba300 msgr2=0x558102c670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.825+0000 7fe94aa00640 0 client.26177 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.832+0000 7fe96ec3f740 1 -- 172.21.15.16:0/4177277109 shutdown_connections 2024-09-21T03:52:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.832+0000 7fe96ec3f740 1 -- 172.21.15.16:0/4177277109 wait complete. 2024-09-21T03:52:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.832+0000 7fe96ec3f740 20 client.26177 trim_cache size 0 max 16384 2024-09-21T03:52:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.874+0000 7fe96ec3f740 1 Processor -- start 2024-09-21T03:52:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.874+0000 7fe96ec3f740 1 -- start start 2024-09-21T03:52:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.875+0000 7fe96ec3f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55810289d2f0 0x558102c6d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.875+0000 7fe96ec3f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581028a54d0 0x558102c7e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.875+0000 7fe96ec3f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581028c61b0 0x558102c69fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.877+0000 7fe96ec3f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe94000fda0 con 0x5581028c61b0 2024-09-21T03:52:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.877+0000 7fe96ec3f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9400118e0 con 0x55810289d2f0 2024-09-21T03:52:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.877+0000 7fe96ec3f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe940010470 con 0x5581028a54d0 2024-09-21T03:52:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.876+0000 7fe95fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581028a54d0 0x558102c7e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.877+0000 7fe95fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581028a54d0 0x558102c7e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34464/0 (socket says 172.21.15.16:34464) 2024-09-21T03:52:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.877+0000 7fe95fe00640 1 -- 172.21.15.16:0/1436067782 learned_addr learned my addr 172.21.15.16:0/1436067782 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.877+0000 7fe965200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55810289d2f0 0x558102c6d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.878+0000 7fe965c00640 1 --2- 172.21.15.16:0/1436067782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581028c61b0 0x558102c69fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.878+0000 7fe965200640 1 -- 172.21.15.16:0/1436067782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581028a54d0 msgr2=0x558102c7e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.878+0000 7fe965200640 1 --2- 172.21.15.16:0/1436067782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581028a54d0 0x558102c7e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.878+0000 7fe965200640 1 -- 172.21.15.16:0/1436067782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581028c61b0 msgr2=0x558102c69fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.879+0000 7fe965200640 1 --2- 172.21.15.16:0/1436067782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581028c61b0 0x558102c69fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.879+0000 7fe965200640 1 -- 172.21.15.16:0/1436067782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558102c77dd0 con 0x55810289d2f0 2024-09-21T03:52:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.879+0000 7fe95fe00640 1 --2- 172.21.15.16:0/1436067782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581028a54d0 0x558102c7e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.880+0000 7fe965c00640 1 --2- 172.21.15.16:0/1436067782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581028c61b0 0x558102c69fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.880+0000 7fe965200640 1 --2- 172.21.15.16:0/1436067782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55810289d2f0 0x558102c6d550 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x558102b07c00 tx=0x7fe950000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f3778e4e8e8900e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.886+0000 7fe955600640 1 -- 172.21.15.16:0/1436067782 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950015040 con 0x55810289d2f0 2024-09-21T03:52:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.886+0000 7fe955600640 1 -- 172.21.15.16:0/1436067782 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe950005050 con 0x55810289d2f0 2024-09-21T03:52:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.886+0000 7fe955600640 1 -- 172.21.15.16:0/1436067782 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9500048c0 con 0x55810289d2f0 2024-09-21T03:52:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.887+0000 7fe96ec3f740 1 -- 172.21.15.16:0/1436067782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55810289d2f0 msgr2=0x558102c6d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.887+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/1436067782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55810289d2f0 0x558102c6d550 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x558102b07c00 tx=0x7fe950000bc0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.891+0000 7fe96ec3f740 1 -- 172.21.15.16:0/1436067782 shutdown_connections 2024-09-21T03:52:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.891+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/1436067782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581028c61b0 0x558102c69fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.891+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/1436067782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581028a54d0 0x558102c7e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.891+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/1436067782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55810289d2f0 0x558102c6d550 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.891+0000 7fe96ec3f740 1 -- 172.21.15.16:0/1436067782 >> 172.21.15.16:0/1436067782 conn(0x5581028ba300 msgr2=0x558102c638e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.891+0000 7fe96ec3f740 1 -- 172.21.15.16:0/1436067782 shutdown_connections 2024-09-21T03:52:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.891+0000 7fe96ec3f740 1 -- 172.21.15.16:0/1436067782 wait complete. 2024-09-21T03:52:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.892+0000 7fe96ec3f740 1 -- start start 2024-09-21T03:52:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.892+0000 7fe96ec3f740 1 Processor -- start 2024-09-21T03:52:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.895+0000 7fe96ec3f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55810289d2f0 0x558102c7d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.896+0000 7fe96ec3f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028a54d0 0x558102c7db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.896+0000 7fe965200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55810289d2f0 0x558102c7d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.896+0000 7fe965200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55810289d2f0 0x558102c7d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34474/0 (socket says 172.21.15.16:34474) 2024-09-21T03:52:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.896+0000 7fe965200640 1 -- 172.21.15.16:0/1815054837 learned_addr learned my addr 172.21.15.16:0/1815054837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.896+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/1815054837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581028c61b0 0x558102c7e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.897+0000 7fe95fe00640 1 --2- 172.21.15.16:0/1815054837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028a54d0 0x558102c7db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.897+0000 7fe96ec3f740 1 -- 172.21.15.16:0/1815054837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe940011210 con 0x5581028c61b0 2024-09-21T03:52:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.897+0000 7fe96ec3f740 1 -- 172.21.15.16:0/1815054837 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe940010b40 con 0x5581028a54d0 2024-09-21T03:52:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.897+0000 7fe96ec3f740 1 -- 172.21.15.16:0/1815054837 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe94000dd40 con 0x55810289d2f0 2024-09-21T03:52:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.897+0000 7fe96ec3f740 10 client.0 _ll_register_callbacks cb 0x7fe94c0097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.898+0000 7fe965200640 1 -- 172.21.15.16:0/1815054837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028a54d0 msgr2=0x558102c7db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.898+0000 7fe965200640 1 --2- 172.21.15.16:0/1815054837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028a54d0 0x558102c7db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.898+0000 7fe965200640 1 -- 172.21.15.16:0/1815054837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581028c61b0 msgr2=0x558102c7e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.898+0000 7fe965200640 1 --2- 172.21.15.16:0/1815054837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581028c61b0 0x558102c7e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.898+0000 7fe965200640 1 -- 172.21.15.16:0/1815054837 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558102c7c250 con 0x55810289d2f0 2024-09-21T03:52:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.899+0000 7fe965200640 1 --2- 172.21.15.16:0/1815054837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55810289d2f0 0x558102c7d640 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe95000ba40 tx=0x7fe95001acd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db5ace115fb5f70d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.899+0000 7fe94aa00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:52:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.900+0000 7fe94aa00640 1 -- 172.21.15.16:0/1815054837 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950015070 con 0x55810289d2f0 2024-09-21T03:52:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.900+0000 7fe94aa00640 1 -- 172.21.15.16:0/1815054837 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fe95001a710 con 0x55810289d2f0 2024-09-21T03:52:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.901+0000 7fe96ec3f740 1 -- 172.21.15.16:0/1815054837 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe950005050 con 0x55810289d2f0 2024-09-21T03:52:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.902+0000 7fe94aa00640 1 -- 172.21.15.16:0/1815054837 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe95001adc0 con 0x55810289d2f0 2024-09-21T03:52:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.902+0000 7fe96ec3f740 10 client.26119 fetch_fsmap learned FSMap version 30 2024-09-21T03:52:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.902+0000 7fe96ec3f740 1 -- 172.21.15.16:0/1815054837 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558102ceeb90 con 0x55810289d2f0 2024-09-21T03:52:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.902+0000 7fe94aa00640 1 -- 172.21.15.16:0/1815054837 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe950005410 con 0x55810289d2f0 2024-09-21T03:52:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.905+0000 7fe94aa00640 1 -- 172.21.15.16:0/1815054837 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe950004be0 con 0x55810289d2f0 2024-09-21T03:52:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.905+0000 7fe96ec3f740 10 client.26119 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:52:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.905+0000 7fe96ec3f740 10 client.26119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.905+0000 7fe96ec3f740 20 client.26119 populate_metadata read hostname 'smithi016' 2024-09-21T03:52:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.905+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/1815054837 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55810285a1c0 0x558102cf1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.905+0000 7fe96ec3f740 4 client.26119 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:52:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.905+0000 7fe96ec3f740 1 -- 172.21.15.16:0/1815054837 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe950005050 con 0x55810285a1c0 2024-09-21T03:52:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.906+0000 7fe95fe00640 1 --2- 172.21.15.16:0/1815054837 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55810285a1c0 0x558102cf1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.906+0000 7fe95fe00640 1 --2- 172.21.15.16:0/1815054837 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55810285a1c0 0x558102cf1340 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97009933f4944b77 server_cookie=6ebf56526c7ce531 in_seq=0 out_seq=0 2024-09-21T03:52:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.906+0000 7fe94aa00640 10 client.26119 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.926+0000 7fe94aa00640 1 -- 172.21.15.16:0/1815054837 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe950005050 con 0x55810285a1c0 2024-09-21T03:52:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.926+0000 7fe94aa00640 10 client.26119 handle_command_reply: tid=0 2024-09-21T03:52:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.926+0000 7fe96ec3f740 1 client.26119 shutdown 2024-09-21T03:52:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.928+0000 7fe96ec3f740 1 -- 172.21.15.16:0/1815054837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55810289d2f0 msgr2=0x558102c7d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.928+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/1815054837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55810289d2f0 0x558102c7d640 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe95000ba40 tx=0x7fe95001acd0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.928+0000 7fe96ec3f740 1 -- 172.21.15.16:0/1815054837 shutdown_connections 2024-09-21T03:52:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.928+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/1815054837 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55810285a1c0 0x558102cf1340 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.928+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/1815054837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581028c61b0 0x558102c7e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.928+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/1815054837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581028a54d0 0x558102c7db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.928+0000 7fe96ec3f740 1 --2- 172.21.15.16:0/1815054837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55810289d2f0 0x558102c7d640 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.928+0000 7fe96ec3f740 1 -- 172.21.15.16:0/1815054837 >> 172.21.15.16:0/1815054837 conn(0x5581028ba300 msgr2=0x558102c711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.928+0000 7fe94aa00640 0 client.26119 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.929+0000 7fe96ec3f740 1 -- 172.21.15.16:0/1815054837 shutdown_connections 2024-09-21T03:52:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.929+0000 7fe96ec3f740 1 -- 172.21.15.16:0/1815054837 wait complete. 2024-09-21T03:52:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.929+0000 7fe96ec3f740 20 client.26119 trim_cache size 0 max 16384 2024-09-21T03:52:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:52:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:52:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:52:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:52:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.931+0000 7fe967c00640 1 -- 172.21.15.16:0/3829945279 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe9440b66a0 msgr2=0x7fe9440b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.931+0000 7fe967c00640 1 --2- 172.21.15.16:0/3829945279 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe9440b66a0 0x7fe9440b8bb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe958009bf0 tx=0x7fe958030f80 comp rx=0 tx=0).stop 2024-09-21T03:52:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.931+0000 7fe967c00640 1 -- 172.21.15.16:0/3829945279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe96010adf0 msgr2=0x7fe9601a9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.931+0000 7fe967c00640 1 --2- 172.21.15.16:0/3829945279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe96010adf0 0x7fe9601a9340 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00e9f0 tx=0x7fe94c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.932+0000 7fe967c00640 1 -- 172.21.15.16:0/3829945279 shutdown_connections 2024-09-21T03:52:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.932+0000 7fe967c00640 1 --2- 172.21.15.16:0/3829945279 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe9440b66a0 0x7fe9440b8bb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.932+0000 7fe967c00640 1 --2- 172.21.15.16:0/3829945279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe960111f90 0x7fe9601a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.932+0000 7fe967c00640 1 --2- 172.21.15.16:0/3829945279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe96010adf0 0x7fe9601a9340 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.932+0000 7fe967c00640 1 --2- 172.21.15.16:0/3829945279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe96010a3c0 0x7fe9601a8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.932+0000 7fe967c00640 1 -- 172.21.15.16:0/3829945279 >> 172.21.15.16:0/3829945279 conn(0x7fe960070c70 msgr2=0x7fe960116290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.933+0000 7fe967c00640 1 -- 172.21.15.16:0/3829945279 shutdown_connections 2024-09-21T03:52:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:05.933+0000 7fe967c00640 1 -- 172.21.15.16:0/3829945279 wait complete. 2024-09-21T03:52:07.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:07 smithi134 ceph-mon[35031]: pgmap v592: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 244 MiB/s wr, 1.31k op/s 2024-09-21T03:52:07.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:07 smithi123 ceph-mon[34673]: pgmap v592: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 244 MiB/s wr, 1.31k op/s 2024-09-21T03:52:07.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:07 smithi016 ceph-mon[30375]: pgmap v592: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 244 MiB/s wr, 1.31k op/s 2024-09-21T03:52:07.675 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T03:52:07.675 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.25 2024-09-21T03:52:07.675 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:52:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:09.373 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:09 smithi016 ceph-mon[30375]: pgmap v593: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 236 MiB/s wr, 1.27k op/s 2024-09-21T03:52:09.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:09 smithi134 ceph-mon[35031]: pgmap v593: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 236 MiB/s wr, 1.27k op/s 2024-09-21T03:52:09.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:09 smithi123 ceph-mon[34673]: pgmap v593: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 236 MiB/s wr, 1.27k op/s 2024-09-21T03:52:10.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.573+0000 7f0aed600640 1 -- 172.21.15.16:0/2561303465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ae81094a0 msgr2=0x7f0ae81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:10.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.573+0000 7f0aed600640 1 --2- 172.21.15.16:0/2561303465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ae81094a0 0x7f0ae81098b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009900 tx=0x7f0ad402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:52:10.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.573+0000 7f0aed600640 1 -- 172.21.15.16:0/2561303465 shutdown_connections 2024-09-21T03:52:10.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.573+0000 7f0aed600640 1 --2- 172.21.15.16:0/2561303465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae8109df0 0x7f0ae8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:10.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.573+0000 7f0aed600640 1 --2- 172.21.15.16:0/2561303465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ae81094a0 0x7f0ae81098b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:10.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.573+0000 7f0aed600640 1 --2- 172.21.15.16:0/2561303465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ae8103b60 0x7f0ae8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:10.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.573+0000 7f0aed600640 1 -- 172.21.15.16:0/2561303465 >> 172.21.15.16:0/2561303465 conn(0x7f0ae807b850 msgr2=0x7f0ae807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:10.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.573+0000 7f0aed600640 1 -- 172.21.15.16:0/2561303465 shutdown_connections 2024-09-21T03:52:10.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.574+0000 7f0aed600640 1 -- 172.21.15.16:0/2561303465 wait complete. 2024-09-21T03:52:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.576+0000 7f0aed600640 1 Processor -- start 2024-09-21T03:52:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.577+0000 7f0aed600640 1 -- start start 2024-09-21T03:52:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.577+0000 7f0aed600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ae8103b60 0x7f0ae81a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.577+0000 7f0aed600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae81094a0 0x7f0ae81a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.578+0000 7f0aed600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ae8109df0 0x7f0ae81ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.578+0000 7f0ae6a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ae8103b60 0x7f0ae81a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.578+0000 7f0ae7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ae8109df0 0x7f0ae81ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.578+0000 7f0ae6a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ae8103b60 0x7f0ae81a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34482/0 (socket says 172.21.15.16:34482) 2024-09-21T03:52:10.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.578+0000 7f0ae6a00640 1 -- 172.21.15.16:0/777017679 learned_addr learned my addr 172.21.15.16:0/777017679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:10.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.578+0000 7f0aed600640 1 -- 172.21.15.16:0/777017679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae8113ab0 con 0x7f0ae81094a0 2024-09-21T03:52:10.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.578+0000 7f0aed600640 1 -- 172.21.15.16:0/777017679 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae8113930 con 0x7f0ae8109df0 2024-09-21T03:52:10.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.578+0000 7f0aed600640 1 -- 172.21.15.16:0/777017679 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae8113c30 con 0x7f0ae8103b60 2024-09-21T03:52:10.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.579+0000 7f0ae6000640 1 --2- 172.21.15.16:0/777017679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae81094a0 0x7f0ae81a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:10.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.581+0000 7f0ae6a00640 1 -- 172.21.15.16:0/777017679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ae8109df0 msgr2=0x7f0ae81ac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:10.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.581+0000 7f0ae6a00640 1 --2- 172.21.15.16:0/777017679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ae8109df0 0x7f0ae81ac7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:10.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.581+0000 7f0ae6a00640 1 -- 172.21.15.16:0/777017679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae81094a0 msgr2=0x7f0ae81a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:10.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.581+0000 7f0ae6a00640 1 --2- 172.21.15.16:0/777017679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae81094a0 0x7f0ae81a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:10.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.581+0000 7f0ae6a00640 1 -- 172.21.15.16:0/777017679 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae81acd10 con 0x7f0ae8103b60 2024-09-21T03:52:10.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.581+0000 7f0ae6a00640 1 --2- 172.21.15.16:0/777017679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ae8103b60 0x7f0ae81a1750 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800e9f0 tx=0x7f0ad800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.582+0000 7f0adf400640 1 -- 172.21.15.16:0/777017679 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad800ccc0 con 0x7f0ae8103b60 2024-09-21T03:52:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.582+0000 7f0adf400640 1 -- 172.21.15.16:0/777017679 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad800ce60 con 0x7f0ae8103b60 2024-09-21T03:52:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.582+0000 7f0adf400640 1 -- 172.21.15.16:0/777017679 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8010430 con 0x7f0ae8103b60 2024-09-21T03:52:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.582+0000 7f0aed600640 1 -- 172.21.15.16:0/777017679 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae81acee0 con 0x7f0ae8103b60 2024-09-21T03:52:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.582+0000 7f0aed600640 1 -- 172.21.15.16:0/777017679 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae81ad2e0 con 0x7f0ae8103b60 2024-09-21T03:52:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.584+0000 7f0aed600640 1 -- 172.21.15.16:0/777017679 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae8103f70 con 0x7f0ae8103b60 2024-09-21T03:52:10.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.587+0000 7f0adf400640 1 -- 172.21.15.16:0/777017679 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0ad80046c0 con 0x7f0ae8103b60 2024-09-21T03:52:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.589+0000 7f0adf400640 1 --2- 172.21.15.16:0/777017679 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0acc0b6660 0x7f0acc0b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.589+0000 7f0adf400640 1 -- 172.21.15.16:0/777017679 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0ad80e7a50 con 0x7f0ae8103b60 2024-09-21T03:52:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.589+0000 7f0ae6000640 1 --2- 172.21.15.16:0/777017679 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0acc0b6660 0x7f0acc0b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:10.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.593+0000 7f0ae6000640 1 --2- 172.21.15.16:0/777017679 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0acc0b6660 0x7f0acc0b8b70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009230 tx=0x7f0ad4031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:10.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:10.593+0000 7f0adf400640 1 -- 172.21.15.16:0/777017679 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0ad80adf30 con 0x7f0ae8103b60 2024-09-21T03:52:11.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:11.002+0000 7f0aed600640 1 -- 172.21.15.16:0/777017679 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ae81ad5a0 con 0x7f0ae8103b60 2024-09-21T03:52:11.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:11.004+0000 7f0adf400640 1 -- 172.21.15.16:0/777017679 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f0ad80b2310 con 0x7f0ae8103b60 2024-09-21T03:52:11.007 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:52:11.007 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:52:11.007 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:52:11.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:11.012+0000 7f0aed600640 1 -- 172.21.15.16:0/777017679 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0acc0b6660 msgr2=0x7f0acc0b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:11.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:11.012+0000 7f0aed600640 1 --2- 172.21.15.16:0/777017679 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0acc0b6660 0x7f0acc0b8b70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009230 tx=0x7f0ad4031400 comp rx=0 tx=0).stop 2024-09-21T03:52:11.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:11.012+0000 7f0aed600640 1 -- 172.21.15.16:0/777017679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ae8103b60 msgr2=0x7f0ae81a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:11.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:11.012+0000 7f0aed600640 1 --2- 172.21.15.16:0/777017679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ae8103b60 0x7f0ae81a1750 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800e9f0 tx=0x7f0ad800edf0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:11.012+0000 7f0aed600640 1 -- 172.21.15.16:0/777017679 shutdown_connections 2024-09-21T03:52:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:11.012+0000 7f0aed600640 1 --2- 172.21.15.16:0/777017679 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0acc0b6660 0x7f0acc0b8b70 secure :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009230 tx=0x7f0ad4031400 comp rx=0 tx=0).stop 2024-09-21T03:52:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:11.012+0000 7f0aed600640 1 --2- 172.21.15.16:0/777017679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ae8109df0 0x7f0ae81ac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:11.012+0000 7f0aed600640 1 --2- 172.21.15.16:0/777017679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae81094a0 0x7f0ae81a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:11.012+0000 7f0aed600640 1 --2- 172.21.15.16:0/777017679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ae8103b60 0x7f0ae81a1750 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:11.012+0000 7f0aed600640 1 -- 172.21.15.16:0/777017679 >> 172.21.15.16:0/777017679 conn(0x7f0ae807b850 msgr2=0x7f0ae8105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:11.013+0000 7f0aed600640 1 -- 172.21.15.16:0/777017679 shutdown_connections 2024-09-21T03:52:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:11.014+0000 7f0aed600640 1 -- 172.21.15.16:0/777017679 wait complete. 2024-09-21T03:52:11.291 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:11 smithi016 ceph-mon[30375]: pgmap v594: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 225 MiB/s wr, 1.22k op/s 2024-09-21T03:52:11.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:11 smithi134 ceph-mon[35031]: pgmap v594: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 225 MiB/s wr, 1.22k op/s 2024-09-21T03:52:11.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:11 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/777017679' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:11.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:11 smithi123 ceph-mon[34673]: pgmap v594: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 225 MiB/s wr, 1.22k op/s 2024-09-21T03:52:11.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:11 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/777017679' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:11.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:11 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/777017679' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:13.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:13 smithi134 ceph-mon[35031]: pgmap v595: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 215 MiB/s wr, 1.17k op/s 2024-09-21T03:52:13.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:13 smithi123 ceph-mon[34673]: pgmap v595: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 215 MiB/s wr, 1.17k op/s 2024-09-21T03:52:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:13 smithi016 ceph-mon[30375]: pgmap v595: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 215 MiB/s wr, 1.17k op/s 2024-09-21T03:52:14.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:14 smithi134 ceph-mon[35031]: pgmap v596: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 242 MiB/s wr, 1.28k op/s 2024-09-21T03:52:14.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:14 smithi123 ceph-mon[34673]: pgmap v596: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 242 MiB/s wr, 1.28k op/s 2024-09-21T03:52:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:14 smithi016 ceph-mon[30375]: pgmap v596: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 242 MiB/s wr, 1.28k op/s 2024-09-21T03:52:15.029 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0' 2024-09-21T03:52:15.029 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0 2024-09-21T03:52:16.619 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:17.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:16 smithi134 ceph-mon[35031]: pgmap v597: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 242 MiB/s wr, 1.22k op/s 2024-09-21T03:52:17.146 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:16 smithi016 ceph-mon[30375]: pgmap v597: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 242 MiB/s wr, 1.22k op/s 2024-09-21T03:52:17.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:16 smithi123 ceph-mon[34673]: pgmap v597: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 242 MiB/s wr, 1.22k op/s 2024-09-21T03:52:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.821+0000 7f746b000640 1 -- 172.21.15.16:0/2818683232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7464105bf0 msgr2=0x7f7464106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.821+0000 7f746b000640 1 --2- 172.21.15.16:0/2818683232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7464105bf0 0x7f7464106000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7458009900 tx=0x7f745802a440 comp rx=0 tx=0).stop 2024-09-21T03:52:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.821+0000 7f746b000640 1 -- 172.21.15.16:0/2818683232 shutdown_connections 2024-09-21T03:52:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.821+0000 7f746b000640 1 --2- 172.21.15.16:0/2818683232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f746410be80 0x7f74641126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.821+0000 7f746b000640 1 --2- 172.21.15.16:0/2818683232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f746410b530 0x7f746410b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.821+0000 7f746b000640 1 --2- 172.21.15.16:0/2818683232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7464105bf0 0x7f7464106000 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.821+0000 7f746b000640 1 -- 172.21.15.16:0/2818683232 >> 172.21.15.16:0/2818683232 conn(0x7f7464101680 msgr2=0x7f7464103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.822+0000 7f746b000640 1 -- 172.21.15.16:0/2818683232 shutdown_connections 2024-09-21T03:52:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.822+0000 7f746b000640 1 -- 172.21.15.16:0/2818683232 wait complete. 2024-09-21T03:52:18.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.826+0000 7f746b000640 1 Processor -- start 2024-09-21T03:52:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.826+0000 7f746b000640 1 -- start start 2024-09-21T03:52:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.827+0000 7f746b000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7464105bf0 0x7f746407b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.827+0000 7f746b000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f746410b530 0x7f7464077920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.827+0000 7f746b000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f746410be80 0x7f7464078040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.827+0000 7f746b000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7464115b90 con 0x7f746410be80 2024-09-21T03:52:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.827+0000 7f746b000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7464115a10 con 0x7f7464105bf0 2024-09-21T03:52:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.827+0000 7f746b000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7464115d10 con 0x7f746410b530 2024-09-21T03:52:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.828+0000 7f7469200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f746410b530 0x7f7464077920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.828+0000 7f7469200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f746410b530 0x7f7464077920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:58434/0 (socket says 172.21.15.16:58434) 2024-09-21T03:52:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.828+0000 7f7469200640 1 -- 172.21.15.16:0/889663029 learned_addr learned my addr 172.21.15.16:0/889663029 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:18.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.829+0000 7f7469c00640 1 --2- 172.21.15.16:0/889663029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7464105bf0 0x7f746407b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.829+0000 7f7469200640 1 -- 172.21.15.16:0/889663029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7464105bf0 msgr2=0x7f746407b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:18.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.829+0000 7f746a600640 1 --2- 172.21.15.16:0/889663029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f746410be80 0x7f7464078040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.829+0000 7f7469200640 1 --2- 172.21.15.16:0/889663029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7464105bf0 0x7f746407b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.829+0000 7f7469200640 1 -- 172.21.15.16:0/889663029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f746410be80 msgr2=0x7f7464078040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.829+0000 7f7469200640 1 --2- 172.21.15.16:0/889663029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f746410be80 0x7f7464078040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.829+0000 7f7469200640 1 -- 172.21.15.16:0/889663029 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746419c270 con 0x7f746410b530 2024-09-21T03:52:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.829+0000 7f7469c00640 1 --2- 172.21.15.16:0/889663029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7464105bf0 0x7f746407b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.829+0000 7f746a600640 1 --2- 172.21.15.16:0/889663029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f746410be80 0x7f7464078040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.830+0000 7f7469200640 1 --2- 172.21.15.16:0/889663029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f746410b530 0x7f7464077920 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f745000c8c0 tx=0x7f745000cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.830+0000 7f7462000640 1 -- 172.21.15.16:0/889663029 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450007bb0 con 0x7f746410b530 2024-09-21T03:52:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.830+0000 7f7462000640 1 -- 172.21.15.16:0/889663029 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7450007d50 con 0x7f746410b530 2024-09-21T03:52:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.830+0000 7f7462000640 1 -- 172.21.15.16:0/889663029 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450007bb0 con 0x7f746410b530 2024-09-21T03:52:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.831+0000 7f746b000640 1 -- 172.21.15.16:0/889663029 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746419c560 con 0x7f746410b530 2024-09-21T03:52:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.831+0000 7f746b000640 1 -- 172.21.15.16:0/889663029 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f746419c9f0 con 0x7f746410b530 2024-09-21T03:52:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.844+0000 7f7472f22740 1 Processor -- start 2024-09-21T03:52:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.844+0000 7f7462000640 1 -- 172.21.15.16:0/889663029 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f745000fa20 con 0x7f746410b530 2024-09-21T03:52:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.844+0000 7f7472f22740 1 -- start start 2024-09-21T03:52:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.851+0000 7f7472f22740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9fa850 0x556f6fda1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.851+0000 7f7472f22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 0x556f6fda20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.851+0000 7f7472f22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d9b70 0x556f6fda2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.851+0000 7f7472f22740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556f6fd17a90 con 0x556f6f9fa850 2024-09-21T03:52:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.851+0000 7f7472f22740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556f6fd06ec0 con 0x556f6f9d1990 2024-09-21T03:52:18.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.851+0000 7f7472f22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556f6fd0acd0 con 0x556f6f9d9b70 2024-09-21T03:52:18.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.851+0000 7f7469200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d9b70 0x556f6fda2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.852+0000 7f7469200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d9b70 0x556f6fda2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:58438/0 (socket says 172.21.15.16:58438) 2024-09-21T03:52:18.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.852+0000 7f7469200640 1 -- 172.21.15.16:0/3956359378 learned_addr learned my addr 172.21.15.16:0/3956359378 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:18.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.854+0000 7f7469200640 1 -- 172.21.15.16:0/3956359378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 msgr2=0x556f6fda20d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:52:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.854+0000 7f7469c00640 1 --2- 172.21.15.16:0/3956359378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 0x556f6fda20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.857+0000 7f7469200640 1 --2- 172.21.15.16:0/3956359378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 0x556f6fda20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.857+0000 7f7469200640 1 -- 172.21.15.16:0/3956359378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9fa850 msgr2=0x556f6fda1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.857+0000 7f7469200640 1 --2- 172.21.15.16:0/3956359378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9fa850 0x556f6fda1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.857+0000 7f7469200640 1 -- 172.21.15.16:0/3956359378 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f6fcac9d0 con 0x556f6f9d9b70 2024-09-21T03:52:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.858+0000 7f7469c00640 1 --2- 172.21.15.16:0/3956359378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 0x556f6fda20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.859+0000 7f7469200640 1 --2- 172.21.15.16:0/3956359378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d9b70 0x556f6fda2610 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f74500edb60 tx=0x7f74500edc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4b107aad2450b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.861+0000 7f7456a00640 1 -- 172.21.15.16:0/3956359378 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74500efd90 con 0x556f6f9d9b70 2024-09-21T03:52:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.861+0000 7f7456a00640 1 -- 172.21.15.16:0/3956359378 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74500f4070 con 0x556f6f9d9b70 2024-09-21T03:52:18.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.861+0000 7f7456a00640 1 -- 172.21.15.16:0/3956359378 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74500e92b0 con 0x556f6f9d9b70 2024-09-21T03:52:18.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.862+0000 7f7462000640 1 --2- 172.21.15.16:0/889663029 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f744c0b67c0 0x7f744c0b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.863+0000 7f7472f22740 1 -- 172.21.15.16:0/3956359378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d9b70 msgr2=0x556f6fda2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.863+0000 7f7472f22740 1 --2- 172.21.15.16:0/3956359378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d9b70 0x556f6fda2610 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f74500edb60 tx=0x7f74500edc50 comp rx=0 tx=0).stop 2024-09-21T03:52:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.863+0000 7f7472f22740 1 -- 172.21.15.16:0/3956359378 shutdown_connections 2024-09-21T03:52:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.863+0000 7f7472f22740 1 --2- 172.21.15.16:0/3956359378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d9b70 0x556f6fda2610 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.863+0000 7f7472f22740 1 --2- 172.21.15.16:0/3956359378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 0x556f6fda20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.863+0000 7f7472f22740 1 --2- 172.21.15.16:0/3956359378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9fa850 0x556f6fda1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.863+0000 7f7472f22740 1 -- 172.21.15.16:0/3956359378 >> 172.21.15.16:0/3956359378 conn(0x556f6f9eec40 msgr2=0x556f6fd9fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:18.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.863+0000 7f7472f22740 1 -- 172.21.15.16:0/3956359378 shutdown_connections 2024-09-21T03:52:18.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.864+0000 7f7472f22740 1 -- 172.21.15.16:0/3956359378 wait complete. 2024-09-21T03:52:18.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.865+0000 7f7462000640 1 -- 172.21.15.16:0/889663029 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f74500e7770 con 0x7f746410b530 2024-09-21T03:52:18.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.865+0000 7f7472f22740 1 -- start start 2024-09-21T03:52:18.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.866+0000 7f7472f22740 1 Processor -- start 2024-09-21T03:52:18.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.866+0000 7f746a600640 1 --2- 172.21.15.16:0/889663029 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f744c0b67c0 0x7f744c0b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.866+0000 7f7472f22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 0x556f6fdad740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.866+0000 7f7472f22740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9d9b70 0x556f6fdadc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.866+0000 7f7472f22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9fa850 0x556f6fdb9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.866+0000 7f7472f22740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556f6fb687b0 con 0x556f6f9d9b70 2024-09-21T03:52:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.867+0000 7f7472f22740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556f6fcab650 con 0x556f6f9d1990 2024-09-21T03:52:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.867+0000 7f7472f22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556f6fdb0370 con 0x556f6f9fa850 2024-09-21T03:52:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.867+0000 7f7472f22740 10 client.0 _ll_register_callbacks cb 0x556f6fa120a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.867+0000 7f7469c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 0x556f6fdad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.867+0000 7f7469c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 0x556f6fdad740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:60090/0 (socket says 172.21.15.16:60090) 2024-09-21T03:52:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.867+0000 7f7469c00640 1 -- 172.21.15.16:0/523543634 learned_addr learned my addr 172.21.15.16:0/523543634 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.867+0000 7f746a600640 1 --2- 172.21.15.16:0/523543634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9fa850 0x556f6fdb9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.868+0000 7f7469200640 1 --2- 172.21.15.16:0/523543634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9d9b70 0x556f6fdadc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.868+0000 7f746a600640 1 --2- 172.21.15.16:0/889663029 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f744c0b67c0 0x7f744c0b8cd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f745c005fd0 tx=0x7f745c0117b0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.869+0000 7f746a600640 1 -- 172.21.15.16:0/523543634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 msgr2=0x556f6fdad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.869+0000 7f746a600640 1 --2- 172.21.15.16:0/523543634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 0x556f6fdad740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.869+0000 7f746a600640 1 -- 172.21.15.16:0/523543634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9d9b70 msgr2=0x556f6fdadc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.870+0000 7f746a600640 1 --2- 172.21.15.16:0/523543634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9d9b70 0x556f6fdadc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.870+0000 7f746a600640 1 -- 172.21.15.16:0/523543634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f6fa74ba0 con 0x556f6f9fa850 2024-09-21T03:52:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.870+0000 7f7469200640 1 --2- 172.21.15.16:0/523543634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9d9b70 0x556f6fdadc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.870+0000 7f746a600640 1 --2- 172.21.15.16:0/523543634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9fa850 0x556f6fdb9690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f745c012dd0 tx=0x7f745c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc21477189311d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.870+0000 7f744b400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:52:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.870+0000 7f744b400640 1 -- 172.21.15.16:0/523543634 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c01c420 con 0x556f6f9fa850 2024-09-21T03:52:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.871+0000 7f744b400640 1 -- 172.21.15.16:0/523543634 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f745c01c5c0 con 0x556f6f9fa850 2024-09-21T03:52:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.871+0000 7f7472f22740 1 -- 172.21.15.16:0/523543634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74500f4070 con 0x556f6f9fa850 2024-09-21T03:52:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.871+0000 7f744b400640 1 -- 172.21.15.16:0/523543634 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f745c02b050 con 0x556f6f9fa850 2024-09-21T03:52:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.871+0000 7f7472f22740 10 client.26134 fetch_fsmap learned FSMap version 30 2024-09-21T03:52:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.871+0000 7f7472f22740 1 -- 172.21.15.16:0/523543634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556f6fdcd3a0 con 0x556f6f9fa850 2024-09-21T03:52:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.872+0000 7f744b400640 1 -- 172.21.15.16:0/523543634 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74500f4070 con 0x556f6f9fa850 2024-09-21T03:52:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.872+0000 7f744b400640 1 -- 172.21.15.16:0/523543634 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f745c00f4a0 con 0x556f6f9fa850 2024-09-21T03:52:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.872+0000 7f7472f22740 10 client.26134 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:52:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.872+0000 7f7472f22740 10 client.26134 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:52:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.872+0000 7f7472f22740 20 client.26134 populate_metadata read hostname 'smithi016' 2024-09-21T03:52:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.872+0000 7f7472f22740 1 --2- 172.21.15.16:0/523543634 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556f6f9b91d0 0x556f6fdd0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.872+0000 7f7472f22740 4 client.26134 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.872+0000 7f7472f22740 1 -- 172.21.15.16:0/523543634 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f6fd199e0 con 0x556f6f9b91d0 2024-09-21T03:52:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.873+0000 7f7469c00640 1 --2- 172.21.15.16:0/523543634 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556f6f9b91d0 0x556f6fdd0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.873+0000 7f744b400640 10 client.26134 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.874+0000 7f7469c00640 1 --2- 172.21.15.16:0/523543634 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556f6f9b91d0 0x556f6fdd0940 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9af714fd3b52ab9 server_cookie=98a2b5e2e21e24ba in_seq=0 out_seq=0 2024-09-21T03:52:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.880+0000 7f744b400640 1 -- 172.21.15.16:0/523543634 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f6fd199e0 con 0x556f6f9b91d0 2024-09-21T03:52:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.880+0000 7f744b400640 10 client.26134 handle_command_reply: tid=0 2024-09-21T03:52:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.880+0000 7f7472f22740 1 client.26134 shutdown 2024-09-21T03:52:18.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.883+0000 7f7472f22740 1 -- 172.21.15.16:0/523543634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9fa850 msgr2=0x556f6fdb9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:18.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.883+0000 7f7472f22740 1 --2- 172.21.15.16:0/523543634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9fa850 0x556f6fdb9690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f745c012dd0 tx=0x7f745c014000 comp rx=0 tx=0).stop 2024-09-21T03:52:18.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.883+0000 7f7472f22740 1 -- 172.21.15.16:0/523543634 shutdown_connections 2024-09-21T03:52:18.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.883+0000 7f7472f22740 1 --2- 172.21.15.16:0/523543634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9fa850 0x556f6fdb9690 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.883+0000 7f7472f22740 1 --2- 172.21.15.16:0/523543634 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556f6f9b91d0 0x556f6fdd0940 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.883+0000 7f7472f22740 1 --2- 172.21.15.16:0/523543634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9d9b70 0x556f6fdadc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.883+0000 7f7472f22740 1 --2- 172.21.15.16:0/523543634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 0x556f6fdad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.883+0000 7f7472f22740 1 -- 172.21.15.16:0/523543634 >> 172.21.15.16:0/523543634 conn(0x556f6f9eec40 msgr2=0x556f6fda4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:18.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.883+0000 7f744b400640 0 client.26134 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:18.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.884+0000 7f7472f22740 1 -- 172.21.15.16:0/523543634 shutdown_connections 2024-09-21T03:52:18.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.884+0000 7f7472f22740 1 -- 172.21.15.16:0/523543634 wait complete. 2024-09-21T03:52:18.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.884+0000 7f7472f22740 20 client.26134 trim_cache size 0 max 16384 2024-09-21T03:52:18.972 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:18 smithi016 ceph-mon[30375]: pgmap v598: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 231 MiB/s wr, 1.12k op/s 2024-09-21T03:52:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.969+0000 7f7472f22740 1 Processor -- start 2024-09-21T03:52:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.969+0000 7f7472f22740 1 -- start start 2024-09-21T03:52:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.970+0000 7f7472f22740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9eec40 0x556f6fdb9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.970+0000 7f7472f22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 0x556f6fda32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.970+0000 7f7472f22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d9b70 0x556f6fdaa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.970+0000 7f7472f22740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f746406eb90 con 0x556f6f9eec40 2024-09-21T03:52:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.970+0000 7f7472f22740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74640706d0 con 0x556f6f9d1990 2024-09-21T03:52:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.970+0000 7f7472f22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f746406f260 con 0x556f6f9d9b70 2024-09-21T03:52:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.972+0000 7f746a600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d9b70 0x556f6fdaa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.972+0000 7f746a600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d9b70 0x556f6fdaa730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:58462/0 (socket says 172.21.15.16:58462) 2024-09-21T03:52:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.972+0000 7f746a600640 1 -- 172.21.15.16:0/1493951405 learned_addr learned my addr 172.21.15.16:0/1493951405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.973+0000 7f7469200640 1 --2- 172.21.15.16:0/1493951405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 0x556f6fda32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.973+0000 7f7469200640 1 -- 172.21.15.16:0/1493951405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d9b70 msgr2=0x556f6fdaa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.974+0000 7f7469200640 1 --2- 172.21.15.16:0/1493951405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d9b70 0x556f6fdaa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.974+0000 7f7469200640 1 -- 172.21.15.16:0/1493951405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9eec40 msgr2=0x556f6fdb9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.974+0000 7f7469200640 1 --2- 172.21.15.16:0/1493951405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9eec40 0x556f6fdb9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.974+0000 7f7469200640 1 -- 172.21.15.16:0/1493951405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f6fdae800 con 0x556f6f9d1990 2024-09-21T03:52:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.974+0000 7f7469200640 1 --2- 172.21.15.16:0/1493951405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 0x556f6fda32d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f74500aa7c0 tx=0x7f7450000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3242aea5a6a5e331 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.974+0000 7f7456000640 1 -- 172.21.15.16:0/1493951405 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74500a93f0 con 0x556f6f9d1990 2024-09-21T03:52:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.975+0000 7f7456000640 1 -- 172.21.15.16:0/1493951405 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74500f4040 con 0x556f6f9d1990 2024-09-21T03:52:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.975+0000 7f7456000640 1 -- 172.21.15.16:0/1493951405 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74500accc0 con 0x556f6f9d1990 2024-09-21T03:52:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.978+0000 7f7472f22740 1 -- 172.21.15.16:0/1493951405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 msgr2=0x556f6fda32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.978+0000 7f7472f22740 1 --2- 172.21.15.16:0/1493951405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 0x556f6fda32d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f74500aa7c0 tx=0x7f7450000bc0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.979+0000 7f7472f22740 1 -- 172.21.15.16:0/1493951405 shutdown_connections 2024-09-21T03:52:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.979+0000 7f7472f22740 1 --2- 172.21.15.16:0/1493951405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d9b70 0x556f6fdaa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.979+0000 7f7472f22740 1 --2- 172.21.15.16:0/1493951405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9d1990 0x556f6fda32d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.979+0000 7f7472f22740 1 --2- 172.21.15.16:0/1493951405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9eec40 0x556f6fdb9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.979+0000 7f7472f22740 1 -- 172.21.15.16:0/1493951405 >> 172.21.15.16:0/1493951405 conn(0x556f6fdade00 msgr2=0x556f6fda2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.980+0000 7f7472f22740 1 -- 172.21.15.16:0/1493951405 shutdown_connections 2024-09-21T03:52:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.981+0000 7f7472f22740 1 -- 172.21.15.16:0/1493951405 wait complete. 2024-09-21T03:52:18.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.982+0000 7f7472f22740 1 -- start start 2024-09-21T03:52:18.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.983+0000 7f7472f22740 1 Processor -- start 2024-09-21T03:52:18.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.986+0000 7f7472f22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d1990 0x556f6fdb5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.987+0000 7f7472f22740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9d9b70 0x556f6fdb20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.987+0000 7f7472f22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9eec40 0x556f6fdb2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.987+0000 7f7472f22740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7464070000 con 0x556f6f9d9b70 2024-09-21T03:52:18.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.987+0000 7f7472f22740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f746406f930 con 0x556f6f9eec40 2024-09-21T03:52:18.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.987+0000 7f7472f22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f74641a4810 con 0x556f6f9d1990 2024-09-21T03:52:18.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.988+0000 7f7472f22740 10 client.0 _ll_register_callbacks cb 0x7f745c008420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:18.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.988+0000 7f7469c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d1990 0x556f6fdb5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.988+0000 7f7469c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d1990 0x556f6fdb5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:58472/0 (socket says 172.21.15.16:58472) 2024-09-21T03:52:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.988+0000 7f7469c00640 1 -- 172.21.15.16:0/1972033535 learned_addr learned my addr 172.21.15.16:0/1972033535 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.988+0000 7f7469200640 1 --2- 172.21.15.16:0/1972033535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9d9b70 0x556f6fdb20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.989+0000 7f7469c00640 1 -- 172.21.15.16:0/1972033535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9eec40 msgr2=0x556f6fdb2610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:52:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.989+0000 7f7469c00640 1 --2- 172.21.15.16:0/1972033535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9eec40 0x556f6fdb2610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.989+0000 7f7469c00640 1 -- 172.21.15.16:0/1972033535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9d9b70 msgr2=0x556f6fdb20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.989+0000 7f7469c00640 1 --2- 172.21.15.16:0/1972033535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9d9b70 0x556f6fdb20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.989+0000 7f7469c00640 1 -- 172.21.15.16:0/1972033535 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f6fdb2bd0 con 0x556f6f9d1990 2024-09-21T03:52:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.989+0000 7f7469c00640 1 --2- 172.21.15.16:0/1972033535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d1990 0x556f6fdb5c10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7458033040 tx=0x7f745802abe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66ae1b44c649a0de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.990+0000 7f744b400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:52:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.990+0000 7f744b400640 1 -- 172.21.15.16:0/1972033535 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458031710 con 0x556f6f9d1990 2024-09-21T03:52:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.990+0000 7f744b400640 1 -- 172.21.15.16:0/1972033535 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f745804a040 con 0x556f6f9d1990 2024-09-21T03:52:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.991+0000 7f7472f22740 1 -- 172.21.15.16:0/1972033535 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74500f4040 con 0x556f6f9d1990 2024-09-21T03:52:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.992+0000 7f744b400640 1 -- 172.21.15.16:0/1972033535 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7458031710 con 0x556f6f9d1990 2024-09-21T03:52:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.992+0000 7f7472f22740 10 client.26140 fetch_fsmap learned FSMap version 30 2024-09-21T03:52:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.992+0000 7f7472f22740 1 -- 172.21.15.16:0/1972033535 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556f6fe3c050 con 0x556f6f9d1990 2024-09-21T03:52:18.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.993+0000 7f744b400640 1 -- 172.21.15.16:0/1972033535 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7458031d60 con 0x556f6f9d1990 2024-09-21T03:52:18.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.993+0000 7f744b400640 1 -- 172.21.15.16:0/1972033535 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7458058c10 con 0x556f6f9d1990 2024-09-21T03:52:18.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.993+0000 7f7472f22740 10 client.26140 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:52:18.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.993+0000 7f7472f22740 10 client.26140 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:52:18.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.993+0000 7f7472f22740 20 client.26140 populate_metadata read hostname 'smithi016' 2024-09-21T03:52:18.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.993+0000 7f7472f22740 1 --2- 172.21.15.16:0/1972033535 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556f6f9fa850 0x556f6fe3e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:18.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.993+0000 7f7472f22740 4 client.26140 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}] 2024-09-21T03:52:18.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.993+0000 7f7472f22740 1 -- 172.21.15.16:0/1972033535 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}) -- 0x7f74500f4040 con 0x556f6f9fa850 2024-09-21T03:52:18.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.994+0000 7f7469200640 1 --2- 172.21.15.16:0/1972033535 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556f6f9fa850 0x556f6fe3e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:18.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.994+0000 7f744b400640 10 client.26140 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:18.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:18.994+0000 7f7469200640 1 --2- 172.21.15.16:0/1972033535 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556f6f9fa850 0x556f6fe3e8b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d42d9b30ae4260 server_cookie=5f3d9d8a8f6f1379 in_seq=0 out_seq=0 2024-09-21T03:52:19.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.038+0000 7f744b400640 1 -- 172.21.15.16:0/1972033535 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f74500f4040 con 0x556f6f9fa850 2024-09-21T03:52:19.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.038+0000 7f744b400640 10 client.26140 handle_command_reply: tid=0 2024-09-21T03:52:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.038+0000 7f7472f22740 1 client.26140 shutdown 2024-09-21T03:52:19.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.043+0000 7f7472f22740 1 -- 172.21.15.16:0/1972033535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d1990 msgr2=0x556f6fdb5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:19.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.043+0000 7f7472f22740 1 --2- 172.21.15.16:0/1972033535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d1990 0x556f6fdb5c10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7458033040 tx=0x7f745802abe0 comp rx=0 tx=0).stop 2024-09-21T03:52:19.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.043+0000 7f7472f22740 1 -- 172.21.15.16:0/1972033535 shutdown_connections 2024-09-21T03:52:19.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.043+0000 7f7472f22740 1 --2- 172.21.15.16:0/1972033535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6f9eec40 0x556f6fdb2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:19.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.043+0000 7f7472f22740 1 --2- 172.21.15.16:0/1972033535 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556f6f9fa850 0x556f6fe3e8b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:19.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.043+0000 7f7472f22740 1 --2- 172.21.15.16:0/1972033535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f6f9d9b70 0x556f6fdb20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:19.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.043+0000 7f7472f22740 1 --2- 172.21.15.16:0/1972033535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f6f9d1990 0x556f6fdb5c10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:19.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.043+0000 7f744b400640 0 client.26140 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:19.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.043+0000 7f7472f22740 1 -- 172.21.15.16:0/1972033535 >> 172.21.15.16:0/1972033535 conn(0x556f6fdade00 msgr2=0x556f6fdb8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:19.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.044+0000 7f7472f22740 1 -- 172.21.15.16:0/1972033535 shutdown_connections 2024-09-21T03:52:19.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.044+0000 7f7472f22740 1 -- 172.21.15.16:0/1972033535 wait complete. 2024-09-21T03:52:19.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.044+0000 7f7472f22740 20 client.26140 trim_cache size 0 max 16384 2024-09-21T03:52:19.047 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:52:19.048 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:52:19.048 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:52:19.048 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 22, 2024-09-21T03:52:19.048 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:52:19.048 INFO:teuthology.orchestra.run.smithi016.stdout: "2e29e1f3": { 2024-09-21T03:52:19.048 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 22, 2024-09-21T03:52:19.048 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:52:19.048 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:52:19.048 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:52:19.048 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:52:19.048 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:52:19.048 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:52:19.048 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 139.2, 2024-09-21T03:52:19.048 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:52:19.049 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:52:19.049 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:52:19.049 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:52:19.049 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:52:19.049 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:52:19.049 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:52:19.049 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:52:19.049 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:52:19.049 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:52:19.049 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:52:19.049 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:52:19.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.048+0000 7f746b000640 1 -- 172.21.15.16:0/889663029 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f744c0b67c0 msgr2=0x7f744c0b8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.048+0000 7f746b000640 1 --2- 172.21.15.16:0/889663029 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f744c0b67c0 0x7f744c0b8cd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f745c005fd0 tx=0x7f745c0117b0 comp rx=0 tx=0).stop 2024-09-21T03:52:19.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.048+0000 7f746b000640 1 -- 172.21.15.16:0/889663029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f746410b530 msgr2=0x7f7464077920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:19.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.048+0000 7f746b000640 1 --2- 172.21.15.16:0/889663029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f746410b530 0x7f7464077920 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f745000c8c0 tx=0x7f745000cd80 comp rx=0 tx=0).stop 2024-09-21T03:52:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.051+0000 7f746b000640 1 -- 172.21.15.16:0/889663029 shutdown_connections 2024-09-21T03:52:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.051+0000 7f746b000640 1 --2- 172.21.15.16:0/889663029 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f744c0b67c0 0x7f744c0b8cd0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.051+0000 7f746b000640 1 --2- 172.21.15.16:0/889663029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f746410be80 0x7f7464078040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.051+0000 7f746b000640 1 --2- 172.21.15.16:0/889663029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f746410b530 0x7f7464077920 secure :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f745000c8c0 tx=0x7f745000cd80 comp rx=0 tx=0).stop 2024-09-21T03:52:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.051+0000 7f746b000640 1 --2- 172.21.15.16:0/889663029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7464105bf0 0x7f746407b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.051+0000 7f746b000640 1 -- 172.21.15.16:0/889663029 >> 172.21.15.16:0/889663029 conn(0x7f7464101680 msgr2=0x7f746410ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.054+0000 7f746b000640 1 -- 172.21.15.16:0/889663029 shutdown_connections 2024-09-21T03:52:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:19.056+0000 7f746b000640 1 -- 172.21.15.16:0/889663029 wait complete. 2024-09-21T03:52:19.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:18 smithi134 ceph-mon[35031]: pgmap v598: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 231 MiB/s wr, 1.12k op/s 2024-09-21T03:52:19.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:18 smithi123 ceph-mon[34673]: pgmap v598: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 231 MiB/s wr, 1.12k op/s 2024-09-21T03:52:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.340+0000 7f6c24c00640 1 -- 172.21.15.16:0/3701942802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c201066c0 msgr2=0x7f6c20108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.340+0000 7f6c24c00640 1 --2- 172.21.15.16:0/3701942802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c201066c0 0x7f6c20108b40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009900 tx=0x7f6c0c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.340+0000 7f6c24c00640 1 -- 172.21.15.16:0/3701942802 shutdown_connections 2024-09-21T03:52:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.340+0000 7f6c24c00640 1 --2- 172.21.15.16:0/3701942802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c20109280 0x7f6c2010b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.340+0000 7f6c24c00640 1 --2- 172.21.15.16:0/3701942802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c201066c0 0x7f6c20108b40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.340+0000 7f6c24c00640 1 --2- 172.21.15.16:0/3701942802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c2006b710 0x7f6c20106180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.340+0000 7f6c24c00640 1 -- 172.21.15.16:0/3701942802 >> 172.21.15.16:0/3701942802 conn(0x7f6c200fdd80 msgr2=0x7f6c20100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.342+0000 7f6c24c00640 1 -- 172.21.15.16:0/3701942802 shutdown_connections 2024-09-21T03:52:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.342+0000 7f6c24c00640 1 -- 172.21.15.16:0/3701942802 wait complete. 2024-09-21T03:52:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.344+0000 7f6c24c00640 1 Processor -- start 2024-09-21T03:52:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.344+0000 7f6c24c00640 1 -- start start 2024-09-21T03:52:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.345+0000 7f6c24c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c2006b710 0x7f6c201037c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.345+0000 7f6c24c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c201066c0 0x7f6c20102080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.345+0000 7f6c24c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c20109280 0x7f6c201025c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.345+0000 7f6c24c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2010f170 con 0x7f6c201066c0 2024-09-21T03:52:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.345+0000 7f6c24c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2010eff0 con 0x7f6c20109280 2024-09-21T03:52:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.345+0000 7f6c24c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2010f2f0 con 0x7f6c2006b710 2024-09-21T03:52:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.345+0000 7f6c1f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c2006b710 0x7f6c201037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.345+0000 7f6c1f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c2006b710 0x7f6c201037c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:58490/0 (socket says 172.21.15.16:58490) 2024-09-21T03:52:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.345+0000 7f6c1f400640 1 -- 172.21.15.16:0/1422493514 learned_addr learned my addr 172.21.15.16:0/1422493514 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.345+0000 7f6c1fe00640 1 --2- 172.21.15.16:0/1422493514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c20109280 0x7f6c201025c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.345+0000 7f6c1ea00640 1 --2- 172.21.15.16:0/1422493514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c201066c0 0x7f6c20102080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.346+0000 7f6c1f400640 1 -- 172.21.15.16:0/1422493514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c20109280 msgr2=0x7f6c201025c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.346+0000 7f6c1f400640 1 --2- 172.21.15.16:0/1422493514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c20109280 0x7f6c201025c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.346+0000 7f6c1f400640 1 -- 172.21.15.16:0/1422493514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c201066c0 msgr2=0x7f6c20102080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.346+0000 7f6c1f400640 1 --2- 172.21.15.16:0/1422493514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c201066c0 0x7f6c20102080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.346+0000 7f6c1f400640 1 -- 172.21.15.16:0/1422493514 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c20102cf0 con 0x7f6c2006b710 2024-09-21T03:52:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.346+0000 7f6c1ea00640 1 --2- 172.21.15.16:0/1422493514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c201066c0 0x7f6c20102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.346+0000 7f6c1fe00640 1 --2- 172.21.15.16:0/1422493514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c20109280 0x7f6c201025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:52:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.346+0000 7f6c1f400640 1 --2- 172.21.15.16:0/1422493514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c2006b710 0x7f6c201037c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000e9f0 tx=0x7f6c1000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.347+0000 7f6c17e00640 1 -- 172.21.15.16:0/1422493514 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1000cd80 con 0x7f6c2006b710 2024-09-21T03:52:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.347+0000 7f6c17e00640 1 -- 172.21.15.16:0/1422493514 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c100054d0 con 0x7f6c2006b710 2024-09-21T03:52:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.347+0000 7f6c17e00640 1 -- 172.21.15.16:0/1422493514 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10010690 con 0x7f6c2006b710 2024-09-21T03:52:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.347+0000 7f6c24c00640 1 -- 172.21.15.16:0/1422493514 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c201a3e70 con 0x7f6c2006b710 2024-09-21T03:52:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.347+0000 7f6c24c00640 1 -- 172.21.15.16:0/1422493514 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c201a4220 con 0x7f6c2006b710 2024-09-21T03:52:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.351+0000 7f6c17e00640 1 -- 172.21.15.16:0/1422493514 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6c10004e90 con 0x7f6c2006b710 2024-09-21T03:52:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.361+0000 7f6c17e00640 1 --2- 172.21.15.16:0/1422493514 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6c040b6590 0x7f6c040b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.361+0000 7f6c17e00640 1 -- 172.21.15.16:0/1422493514 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f6c100e7a80 con 0x7f6c2006b710 2024-09-21T03:52:20.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.363+0000 7f6c1ea00640 1 --2- 172.21.15.16:0/1422493514 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6c040b6590 0x7f6c040b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.366+0000 7f6c1ea00640 1 --2- 172.21.15.16:0/1422493514 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6c040b6590 0x7f6c040b8aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6c2006bcc0 tx=0x7f6c0c005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.398+0000 7f6c2d1c4740 1 Processor -- start 2024-09-21T03:52:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.400+0000 7f6c2d1c4740 1 -- start start 2024-09-21T03:52:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.400+0000 7f6c2d1c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a012a01b0 0x557a0164fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.400+0000 7f6c2d1c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a012772f0 0x557a01650130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.400+0000 7f6c2d1c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0127f4d0 0x557a01650c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.400+0000 7f6c2d1c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557a015ba6b0 con 0x557a012772f0 2024-09-21T03:52:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.400+0000 7f6c2d1c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557a015a9ae0 con 0x557a012a01b0 2024-09-21T03:52:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.400+0000 7f6c2d1c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557a015ad8f0 con 0x557a0127f4d0 2024-09-21T03:52:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.403+0000 7f6c1f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a012a01b0 0x557a0164fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.403+0000 7f6c1f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a012a01b0 0x557a0164fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:60134/0 (socket says 172.21.15.16:60134) 2024-09-21T03:52:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.403+0000 7f6c1f400640 1 -- 172.21.15.16:0/1600435126 learned_addr learned my addr 172.21.15.16:0/1600435126 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.403+0000 7f6c1f400640 1 -- 172.21.15.16:0/1600435126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0127f4d0 msgr2=0x557a01650c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:52:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.403+0000 7f6c1f400640 1 --2- 172.21.15.16:0/1600435126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0127f4d0 0x557a01650c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.403+0000 7f6c1f400640 1 -- 172.21.15.16:0/1600435126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a012772f0 msgr2=0x557a01650130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:52:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.403+0000 7f6c1f400640 1 --2- 172.21.15.16:0/1600435126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a012772f0 0x557a01650130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.403+0000 7f6c1f400640 1 -- 172.21.15.16:0/1600435126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a01576780 con 0x557a012a01b0 2024-09-21T03:52:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.404+0000 7f6c1f400640 1 --2- 172.21.15.16:0/1600435126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a012a01b0 0x557a0164fac0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6c100a9e30 tx=0x7f6c100ad6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fe5e4dd5c92e97e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.405+0000 7f6c0be00640 1 -- 172.21.15.16:0/1600435126 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c100b2020 con 0x557a012a01b0 2024-09-21T03:52:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.405+0000 7f6c0be00640 1 -- 172.21.15.16:0/1600435126 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c100ad370 con 0x557a012a01b0 2024-09-21T03:52:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.405+0000 7f6c0be00640 1 -- 172.21.15.16:0/1600435126 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c100aeb90 con 0x557a012a01b0 2024-09-21T03:52:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.407+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/1600435126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a012a01b0 msgr2=0x557a0164fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.407+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/1600435126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a012a01b0 0x557a0164fac0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6c100a9e30 tx=0x7f6c100ad6f0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.409+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/1600435126 shutdown_connections 2024-09-21T03:52:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.409+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/1600435126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0127f4d0 0x557a01650c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.409+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/1600435126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a012772f0 0x557a01650130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.409+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/1600435126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a012a01b0 0x557a0164fac0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.410+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/1600435126 >> 172.21.15.16:0/1600435126 conn(0x557a01294300 msgr2=0x557a01642d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.410+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/1600435126 shutdown_connections 2024-09-21T03:52:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.411+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/1600435126 wait complete. 2024-09-21T03:52:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.412+0000 7f6c2d1c4740 1 -- start start 2024-09-21T03:52:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.412+0000 7f6c2d1c4740 1 Processor -- start 2024-09-21T03:52:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.414+0000 7f6c2d1c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a012772f0 0x557a01659210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.414+0000 7f6c2d1c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0127f4d0 0x557a016598d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.414+0000 7f6c2d1c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a012a01b0 0x557a0165a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.414+0000 7f6c2d1c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557a014e1ab0 con 0x557a0127f4d0 2024-09-21T03:52:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.414+0000 7f6c2d1c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557a01550570 con 0x557a012a01b0 2024-09-21T03:52:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.414+0000 7f6c2d1c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557a016517a0 con 0x557a012772f0 2024-09-21T03:52:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.414+0000 7f6c2d1c4740 10 client.0 _ll_register_callbacks cb 0x557a0155f100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.415+0000 7f6c1f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a012772f0 0x557a01659210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.415+0000 7f6c1f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a012772f0 0x557a01659210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:58502/0 (socket says 172.21.15.16:58502) 2024-09-21T03:52:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.415+0000 7f6c1f400640 1 -- 172.21.15.16:0/1599678617 learned_addr learned my addr 172.21.15.16:0/1599678617 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.415+0000 7f6c1f400640 1 -- 172.21.15.16:0/1599678617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a012a01b0 msgr2=0x557a0165a440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:52:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.415+0000 7f6c1f400640 1 --2- 172.21.15.16:0/1599678617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a012a01b0 0x557a0165a440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.415+0000 7f6c1f400640 1 -- 172.21.15.16:0/1599678617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0127f4d0 msgr2=0x557a016598d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:52:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.416+0000 7f6c1f400640 1 --2- 172.21.15.16:0/1599678617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0127f4d0 0x557a016598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.416+0000 7f6c1f400640 1 -- 172.21.15.16:0/1599678617 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a0130b790 con 0x557a012772f0 2024-09-21T03:52:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.416+0000 7f6c1f400640 1 --2- 172.21.15.16:0/1599678617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a012772f0 0x557a01659210 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6c100aaad0 tx=0x7f6c100ad270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd2c7fc914442e0c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.416+0000 7f6c08c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:52:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.417+0000 7f6c08c00640 1 -- 172.21.15.16:0/1599678617 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c100b2040 con 0x557a012772f0 2024-09-21T03:52:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.417+0000 7f6c08c00640 1 -- 172.21.15.16:0/1599678617 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f6c100aeb90 con 0x557a012772f0 2024-09-21T03:52:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.417+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/1599678617 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c100ad370 con 0x557a012772f0 2024-09-21T03:52:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.418+0000 7f6c08c00640 1 -- 172.21.15.16:0/1599678617 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6c10005180 con 0x557a012772f0 2024-09-21T03:52:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.418+0000 7f6c2d1c4740 10 client.26149 fetch_fsmap learned FSMap version 30 2024-09-21T03:52:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.418+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/1599678617 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557a01384620 con 0x557a012772f0 2024-09-21T03:52:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.418+0000 7f6c08c00640 1 -- 172.21.15.16:0/1599678617 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c100ad370 con 0x557a012772f0 2024-09-21T03:52:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.419+0000 7f6c08c00640 1 -- 172.21.15.16:0/1599678617 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6c100ac530 con 0x557a012772f0 2024-09-21T03:52:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.419+0000 7f6c2d1c4740 10 client.26149 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:52:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.419+0000 7f6c2d1c4740 10 client.26149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.419+0000 7f6c2d1c4740 20 client.26149 populate_metadata read hostname 'smithi016' 2024-09-21T03:52:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.419+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/1599678617 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557a0125ed80 0x557a0164a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.419+0000 7f6c2d1c4740 4 client.26149 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.419+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/1599678617 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a015bc600 con 0x557a0125ed80 2024-09-21T03:52:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.420+0000 7f6c1ea00640 1 --2- 172.21.15.16:0/1599678617 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557a0125ed80 0x557a0164a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.420+0000 7f6c1ea00640 1 --2- 172.21.15.16:0/1599678617 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557a0125ed80 0x557a0164a070 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a5b77032c58f744 server_cookie=e8d7295b685f87ec in_seq=0 out_seq=0 2024-09-21T03:52:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.420+0000 7f6c08c00640 10 client.26149 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.421+0000 7f6c08c00640 1 -- 172.21.15.16:0/1599678617 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557a015bc600 con 0x557a0125ed80 2024-09-21T03:52:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.422+0000 7f6c08c00640 10 client.26149 handle_command_reply: tid=0 2024-09-21T03:52:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.422+0000 7f6c2d1c4740 1 client.26149 shutdown 2024-09-21T03:52:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.423+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/1599678617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a012772f0 msgr2=0x557a01659210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.423+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/1599678617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a012772f0 0x557a01659210 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6c100aaad0 tx=0x7f6c100ad270 comp rx=0 tx=0).stop 2024-09-21T03:52:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.423+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/1599678617 shutdown_connections 2024-09-21T03:52:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.423+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/1599678617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a012a01b0 0x557a0165a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.423+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/1599678617 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557a0125ed80 0x557a0164a070 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.423+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/1599678617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a0127f4d0 0x557a016598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.423+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/1599678617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a012772f0 0x557a01659210 secure :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6c100aaad0 tx=0x7f6c100ad270 comp rx=0 tx=0).stop 2024-09-21T03:52:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.423+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/1599678617 >> 172.21.15.16:0/1599678617 conn(0x557a01294300 msgr2=0x557a01641360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.424+0000 7f6c08c00640 0 client.26149 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.424+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/1599678617 shutdown_connections 2024-09-21T03:52:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.424+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/1599678617 wait complete. 2024-09-21T03:52:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.424+0000 7f6c2d1c4740 20 client.26149 trim_cache size 0 max 16384 2024-09-21T03:52:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.451+0000 7f6c2d1c4740 1 Processor -- start 2024-09-21T03:52:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.451+0000 7f6c2d1c4740 1 -- start start 2024-09-21T03:52:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.451+0000 7f6c2d1c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0127f4d0 0x557a01657760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.451+0000 7f6c2d1c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a012a01b0 0x557a01657ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.451+0000 7f6c2d1c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a012341c0 0x557a01643760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.451+0000 7f6c2d1c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201aa420 con 0x557a012341c0 2024-09-21T03:52:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.451+0000 7f6c2d1c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201abf60 con 0x557a0127f4d0 2024-09-21T03:52:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.451+0000 7f6c2d1c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201aaaf0 con 0x557a012a01b0 2024-09-21T03:52:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.452+0000 7f6c1f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0127f4d0 0x557a01657760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.452+0000 7f6c1f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0127f4d0 0x557a01657760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:60140/0 (socket says 172.21.15.16:60140) 2024-09-21T03:52:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.452+0000 7f6c1f400640 1 -- 172.21.15.16:0/3210770705 learned_addr learned my addr 172.21.15.16:0/3210770705 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.452+0000 7f6c1ea00640 1 --2- 172.21.15.16:0/3210770705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a012a01b0 0x557a01657ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.452+0000 7f6c1fe00640 1 --2- 172.21.15.16:0/3210770705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a012341c0 0x557a01643760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.452+0000 7f6c1f400640 1 -- 172.21.15.16:0/3210770705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a012a01b0 msgr2=0x557a01657ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.452+0000 7f6c1f400640 1 --2- 172.21.15.16:0/3210770705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a012a01b0 0x557a01657ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.452+0000 7f6c1f400640 1 -- 172.21.15.16:0/3210770705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a012341c0 msgr2=0x557a01643760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.452+0000 7f6c1f400640 1 --2- 172.21.15.16:0/3210770705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a012341c0 0x557a01643760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.452+0000 7f6c1f400640 1 -- 172.21.15.16:0/3210770705 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a016573e0 con 0x557a0127f4d0 2024-09-21T03:52:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.452+0000 7f6c1fe00640 1 --2- 172.21.15.16:0/3210770705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a012341c0 0x557a01643760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.453+0000 7f6c1f400640 1 --2- 172.21.15.16:0/3210770705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0127f4d0 0x557a01657760 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6c100aca50 tx=0x7f6c100b3700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a359fda6c4d23a60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.453+0000 7f6c0b400640 1 -- 172.21.15.16:0/3210770705 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c100b2050 con 0x557a0127f4d0 2024-09-21T03:52:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.453+0000 7f6c0b400640 1 -- 172.21.15.16:0/3210770705 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c10071030 con 0x557a0127f4d0 2024-09-21T03:52:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.453+0000 7f6c0b400640 1 -- 172.21.15.16:0/3210770705 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c100b7280 con 0x557a0127f4d0 2024-09-21T03:52:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.454+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/3210770705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0127f4d0 msgr2=0x557a01657760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.454+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/3210770705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0127f4d0 0x557a01657760 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6c100aca50 tx=0x7f6c100b3700 comp rx=0 tx=0).stop 2024-09-21T03:52:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.455+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/3210770705 shutdown_connections 2024-09-21T03:52:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.455+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/3210770705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a012341c0 0x557a01643760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.455+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/3210770705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a012a01b0 0x557a01657ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.455+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/3210770705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0127f4d0 0x557a01657760 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.455+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/3210770705 >> 172.21.15.16:0/3210770705 conn(0x557a01294300 msgr2=0x557a01642680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.455+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/3210770705 shutdown_connections 2024-09-21T03:52:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.455+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/3210770705 wait complete. 2024-09-21T03:52:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.456+0000 7f6c2d1c4740 1 -- start start 2024-09-21T03:52:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.456+0000 7f6c2d1c4740 1 Processor -- start 2024-09-21T03:52:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.457+0000 7f6c2d1c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a012341c0 0x557a01668c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.457+0000 7f6c2d1c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0127f4d0 0x557a01667590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.457+0000 7f6c2d1c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a012a01b0 0x557a01667ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.457+0000 7f6c2d1c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201ab890 con 0x557a012341c0 2024-09-21T03:52:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.457+0000 7f6c2d1c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201ab1c0 con 0x557a012a01b0 2024-09-21T03:52:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.457+0000 7f6c2d1c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201a8400 con 0x557a0127f4d0 2024-09-21T03:52:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.457+0000 7f6c2d1c4740 10 client.0 _ll_register_callbacks cb 0x7f6c100a9fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.457+0000 7f6c1ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0127f4d0 0x557a01667590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.457+0000 7f6c1ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0127f4d0 0x557a01667590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:58512/0 (socket says 172.21.15.16:58512) 2024-09-21T03:52:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.457+0000 7f6c1ea00640 1 -- 172.21.15.16:0/4200937216 learned_addr learned my addr 172.21.15.16:0/4200937216 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.457+0000 7f6c1fe00640 1 --2- 172.21.15.16:0/4200937216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a012a01b0 0x557a01667ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.457+0000 7f6c1f400640 1 --2- 172.21.15.16:0/4200937216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a012341c0 0x557a01668c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.458+0000 7f6c1ea00640 1 -- 172.21.15.16:0/4200937216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a012a01b0 msgr2=0x557a01667ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.458+0000 7f6c1ea00640 1 --2- 172.21.15.16:0/4200937216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a012a01b0 0x557a01667ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.458+0000 7f6c1ea00640 1 -- 172.21.15.16:0/4200937216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a012341c0 msgr2=0x557a01668c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.458+0000 7f6c1ea00640 1 --2- 172.21.15.16:0/4200937216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a012341c0 0x557a01668c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.458+0000 7f6c1ea00640 1 -- 172.21.15.16:0/4200937216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a016d85a0 con 0x557a0127f4d0 2024-09-21T03:52:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.458+0000 7f6c1ea00640 1 --2- 172.21.15.16:0/4200937216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0127f4d0 0x557a01667590 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x557a0164a7b0 tx=0x7f6c0c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87bea821cd2814f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.458+0000 7f6c08c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:52:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.460+0000 7f6c08c00640 1 -- 172.21.15.16:0/4200937216 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c035050 con 0x557a0127f4d0 2024-09-21T03:52:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.460+0000 7f6c08c00640 1 -- 172.21.15.16:0/4200937216 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f6c0c004ad0 con 0x557a0127f4d0 2024-09-21T03:52:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.460+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/4200937216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c10071030 con 0x557a0127f4d0 2024-09-21T03:52:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.461+0000 7f6c08c00640 1 -- 172.21.15.16:0/4200937216 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6c0c005590 con 0x557a0127f4d0 2024-09-21T03:52:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.461+0000 7f6c2d1c4740 10 client.26155 fetch_fsmap learned FSMap version 30 2024-09-21T03:52:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.461+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/4200937216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557a016d9750 con 0x557a0127f4d0 2024-09-21T03:52:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.462+0000 7f6c08c00640 1 -- 172.21.15.16:0/4200937216 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c10071030 con 0x557a0127f4d0 2024-09-21T03:52:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.463+0000 7f6c08c00640 1 -- 172.21.15.16:0/4200937216 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6c0c0035f0 con 0x557a0127f4d0 2024-09-21T03:52:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.463+0000 7f6c2d1c4740 10 client.26155 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:52:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.463+0000 7f6c2d1c4740 10 client.26155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.463+0000 7f6c2d1c4740 20 client.26155 populate_metadata read hostname 'smithi016' 2024-09-21T03:52:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.463+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/4200937216 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557a016da060 0x557a016dc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.464+0000 7f6c2d1c4740 4 client.26155 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:52:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.464+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/4200937216 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a015b80b0 con 0x557a016da060 2024-09-21T03:52:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.464+0000 7f6c1f400640 1 --2- 172.21.15.16:0/4200937216 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557a016da060 0x557a016dc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.465+0000 7f6c08c00640 10 client.26155 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.465+0000 7f6c1f400640 1 --2- 172.21.15.16:0/4200937216 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557a016da060 0x557a016dc480 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2570f0e71dec817a server_cookie=f7751864f3588c29 in_seq=0 out_seq=0 2024-09-21T03:52:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.481+0000 7f6c08c00640 1 -- 172.21.15.16:0/4200937216 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a015b80b0 con 0x557a016da060 2024-09-21T03:52:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.482+0000 7f6c08c00640 10 client.26155 handle_command_reply: tid=0 2024-09-21T03:52:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.482+0000 7f6c2d1c4740 1 client.26155 shutdown 2024-09-21T03:52:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.482+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/4200937216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0127f4d0 msgr2=0x557a01667590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.482+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/4200937216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0127f4d0 0x557a01667590 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x557a0164a7b0 tx=0x7f6c0c000bc0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.483+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/4200937216 shutdown_connections 2024-09-21T03:52:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.483+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/4200937216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a012a01b0 0x557a01667ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.483+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/4200937216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0127f4d0 0x557a01667590 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.483+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/4200937216 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x557a016da060 0x557a016dc480 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.483+0000 7f6c2d1c4740 1 --2- 172.21.15.16:0/4200937216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a012341c0 0x557a01668c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.484+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/4200937216 >> 172.21.15.16:0/4200937216 conn(0x557a01294300 msgr2=0x557a01657700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.484+0000 7f6c08c00640 0 client.26155 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.484+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/4200937216 shutdown_connections 2024-09-21T03:52:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.484+0000 7f6c2d1c4740 1 -- 172.21.15.16:0/4200937216 wait complete. 2024-09-21T03:52:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.484+0000 7f6c2d1c4740 20 client.26155 trim_cache size 0 max 16384 2024-09-21T03:52:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:52:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:52:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:52:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:52:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.486+0000 7f6c24c00640 1 -- 172.21.15.16:0/1422493514 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6c040b6590 msgr2=0x7f6c040b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.486+0000 7f6c24c00640 1 --2- 172.21.15.16:0/1422493514 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6c040b6590 0x7f6c040b8aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6c2006bcc0 tx=0x7f6c0c005fd0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.486+0000 7f6c24c00640 1 -- 172.21.15.16:0/1422493514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c2006b710 msgr2=0x7f6c201037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.486+0000 7f6c24c00640 1 --2- 172.21.15.16:0/1422493514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c2006b710 0x7f6c201037c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000e9f0 tx=0x7f6c1000edf0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.486+0000 7f6c24c00640 1 -- 172.21.15.16:0/1422493514 shutdown_connections 2024-09-21T03:52:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.486+0000 7f6c24c00640 1 --2- 172.21.15.16:0/1422493514 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6c040b6590 0x7f6c040b8aa0 secure :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6c2006bcc0 tx=0x7f6c0c005fd0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.486+0000 7f6c24c00640 1 --2- 172.21.15.16:0/1422493514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c20109280 0x7f6c201025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.486+0000 7f6c24c00640 1 --2- 172.21.15.16:0/1422493514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c201066c0 0x7f6c20102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.486+0000 7f6c24c00640 1 --2- 172.21.15.16:0/1422493514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c2006b710 0x7f6c201037c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.486+0000 7f6c24c00640 1 -- 172.21.15.16:0/1422493514 >> 172.21.15.16:0/1422493514 conn(0x7f6c200fdd80 msgr2=0x7f6c20104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.486+0000 7f6c24c00640 1 -- 172.21.15.16:0/1422493514 shutdown_connections 2024-09-21T03:52:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:20.487+0000 7f6c24c00640 1 -- 172.21.15.16:0/1422493514 wait complete. 2024-09-21T03:52:20.958 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:20 smithi123 ceph-mon[34673]: pgmap v599: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 229 MiB/s wr, 1.06k op/s 2024-09-21T03:52:21.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:20 smithi134 ceph-mon[35031]: pgmap v599: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 229 MiB/s wr, 1.06k op/s 2024-09-21T03:52:21.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:20 smithi016 ceph-mon[30375]: pgmap v599: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 229 MiB/s wr, 1.06k op/s 2024-09-21T03:52:23.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:22 smithi134 ceph-mon[35031]: pgmap v600: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 236 MiB/s wr, 1.05k op/s 2024-09-21T03:52:23.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:22 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:52:23.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:22 smithi123 ceph-mon[34673]: pgmap v600: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 236 MiB/s wr, 1.05k op/s 2024-09-21T03:52:23.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:22 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:52:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:22 smithi016 ceph-mon[30375]: pgmap v600: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 236 MiB/s wr, 1.05k op/s 2024-09-21T03:52:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:22 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:52:23.973 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:23 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:23.973 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:23 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:23.973 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:23 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:23.973 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:23 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:23.973 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:23 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:23.973 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:23 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:23.974 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:23 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:23.974 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:23 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:23.974 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:23 smithi016 ceph-mon[30375]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-21T03:52:23.974 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:23 smithi016 ceph-mon[30375]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-09-21T03:52:23.974 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:23 smithi016 ceph-mon[30375]: Unable to set osd_memory_target on smithi134 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-21T03:52:23.974 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:23 smithi016 ceph-mon[30375]: Unable to set osd_memory_target on smithi123 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-21T03:52:23.986 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2e29e1f3', quiesce took 16.3 seconds. Will release after: 2.9499999999999993 2024-09-21T03:52:24.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:23 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:24.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:23 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:24.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:23 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:24.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:23 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:24.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:23 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:24.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:23 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:24.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:23 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:24.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:23 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:24.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:23 smithi134 ceph-mon[35031]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-21T03:52:24.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:23 smithi134 ceph-mon[35031]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-09-21T03:52:24.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:23 smithi134 ceph-mon[35031]: Unable to set osd_memory_target on smithi134 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-21T03:52:24.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:23 smithi134 ceph-mon[35031]: Unable to set osd_memory_target on smithi123 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-21T03:52:24.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:23 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:24.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:23 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:24.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:23 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:24.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:23 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:24.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:23 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:24.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:23 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:24.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:23 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:24.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:23 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:52:24.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:23 smithi123 ceph-mon[34673]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-21T03:52:24.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:23 smithi123 ceph-mon[34673]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-09-21T03:52:24.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:23 smithi123 ceph-mon[34673]: Unable to set osd_memory_target on smithi134 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-21T03:52:24.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:23 smithi123 ceph-mon[34673]: Unable to set osd_memory_target on smithi123 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-21T03:52:25.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:24 smithi134 ceph-mon[35031]: pgmap v601: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 254 MiB/s wr, 1.14k op/s 2024-09-21T03:52:25.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:24 smithi123 ceph-mon[34673]: pgmap v601: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 254 MiB/s wr, 1.14k op/s 2024-09-21T03:52:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:24 smithi016 ceph-mon[30375]: pgmap v601: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 254 MiB/s wr, 1.14k op/s 2024-09-21T03:52:26.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:25 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:52:26.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:25 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:52:26.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:25 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:52:26.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:25 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:52:26.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:25 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:52:26.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:25 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:52:26.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:25 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:52:26.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:25 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:52:26.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:25 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:52:26.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:52:27.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:26 smithi123 ceph-mon[34673]: pgmap v602: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 244 MiB/s wr, 1.12k op/s 2024-09-21T03:52:27.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:26 smithi016 ceph-mon[30375]: pgmap v602: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 244 MiB/s wr, 1.12k op/s 2024-09-21T03:52:27.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:26 smithi134 ceph-mon[35031]: pgmap v602: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 244 MiB/s wr, 1.12k op/s 2024-09-21T03:52:27.688 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:28.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.663+0000 7f8d54e00640 1 -- 172.21.15.16:0/1105797580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d480ab190 msgr2=0x7f8d480ab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.663+0000 7f8d54e00640 1 --2- 172.21.15.16:0/1105797580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d480ab190 0x7f8d480ab9b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009900 tx=0x7f8d3c02a440 comp rx=0 tx=0).stop 2024-09-21T03:52:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.663+0000 7f8d54e00640 1 -- 172.21.15.16:0/1105797580 shutdown_connections 2024-09-21T03:52:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.663+0000 7f8d54e00640 1 --2- 172.21.15.16:0/1105797580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d480ab190 0x7f8d480ab9b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.663+0000 7f8d54e00640 1 --2- 172.21.15.16:0/1105797580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d480aa840 0x7f8d480aac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.663+0000 7f8d54e00640 1 --2- 172.21.15.16:0/1105797580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d480a4f10 0x7f8d480a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.663+0000 7f8d54e00640 1 -- 172.21.15.16:0/1105797580 >> 172.21.15.16:0/1105797580 conn(0x7f8d4801cc60 msgr2=0x7f8d4801d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.663+0000 7f8d54e00640 1 -- 172.21.15.16:0/1105797580 shutdown_connections 2024-09-21T03:52:28.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.663+0000 7f8d54e00640 1 -- 172.21.15.16:0/1105797580 wait complete. 2024-09-21T03:52:28.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.668+0000 7f8d54e00640 1 Processor -- start 2024-09-21T03:52:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.668+0000 7f8d54e00640 1 -- start start 2024-09-21T03:52:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.669+0000 7f8d54e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d480a4f10 0x7f8d480baf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.669+0000 7f8d54e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d480aa840 0x7f8d480b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.669+0000 7f8d4f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d480a4f10 0x7f8d480baf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.670+0000 7f8d4f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d480a4f10 0x7f8d480baf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:41716/0 (socket says 172.21.15.16:41716) 2024-09-21T03:52:28.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.670+0000 7f8d4f400640 1 -- 172.21.15.16:0/300462006 learned_addr learned my addr 172.21.15.16:0/300462006 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:28.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.670+0000 7f8d54e00640 1 --2- 172.21.15.16:0/300462006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d480ab190 0x7f8d480b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.670+0000 7f8d4ea00640 1 --2- 172.21.15.16:0/300462006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d480aa840 0x7f8d480b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.670+0000 7f8d54e00640 1 -- 172.21.15.16:0/300462006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d480bd6e0 con 0x7f8d480ab190 2024-09-21T03:52:28.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.670+0000 7f8d54e00640 1 -- 172.21.15.16:0/300462006 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d480bd560 con 0x7f8d480aa840 2024-09-21T03:52:28.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.671+0000 7f8d54e00640 1 -- 172.21.15.16:0/300462006 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d480bd860 con 0x7f8d480a4f10 2024-09-21T03:52:28.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.672+0000 7f8d4f400640 1 -- 172.21.15.16:0/300462006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d480aa840 msgr2=0x7f8d480b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.672+0000 7f8d56400640 1 --2- 172.21.15.16:0/300462006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d480ab190 0x7f8d480b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.675+0000 7f8d4f400640 1 --2- 172.21.15.16:0/300462006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d480aa840 0x7f8d480b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.675+0000 7f8d4f400640 1 -- 172.21.15.16:0/300462006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d480ab190 msgr2=0x7f8d480b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.675+0000 7f8d4f400640 1 --2- 172.21.15.16:0/300462006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d480ab190 0x7f8d480b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.675+0000 7f8d4f400640 1 -- 172.21.15.16:0/300462006 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d480b4f00 con 0x7f8d480a4f10 2024-09-21T03:52:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.675+0000 7f8d4ea00640 1 --2- 172.21.15.16:0/300462006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d480aa840 0x7f8d480b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:52:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.676+0000 7f8d56400640 1 --2- 172.21.15.16:0/300462006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d480ab190 0x7f8d480b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.676+0000 7f8d4f400640 1 --2- 172.21.15.16:0/300462006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d480a4f10 0x7f8d480baf70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8d500524b0 tx=0x7f8d5006a4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.677+0000 7f8d43e00640 1 -- 172.21.15.16:0/300462006 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d50073b00 con 0x7f8d480a4f10 2024-09-21T03:52:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.677+0000 7f8d54e00640 1 -- 172.21.15.16:0/300462006 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d480b9be0 con 0x7f8d480a4f10 2024-09-21T03:52:28.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.677+0000 7f8d54e00640 1 -- 172.21.15.16:0/300462006 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d480ba040 con 0x7f8d480a4f10 2024-09-21T03:52:28.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.679+0000 7f8d54e00640 1 -- 172.21.15.16:0/300462006 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d480aac50 con 0x7f8d480a4f10 2024-09-21T03:52:28.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.682+0000 7f8d43e00640 1 -- 172.21.15.16:0/300462006 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d50073ca0 con 0x7f8d480a4f10 2024-09-21T03:52:28.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.682+0000 7f8d43e00640 1 -- 172.21.15.16:0/300462006 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d50077380 con 0x7f8d480a4f10 2024-09-21T03:52:28.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.700+0000 7f8d43e00640 1 -- 172.21.15.16:0/300462006 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8d5007f030 con 0x7f8d480a4f10 2024-09-21T03:52:28.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.704+0000 7f8d43e00640 1 --2- 172.21.15.16:0/300462006 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8d380b6610 0x7f8d380b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:28.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.704+0000 7f8d43e00640 1 -- 172.21.15.16:0/300462006 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f8d5014c6a0 con 0x7f8d480a4f10 2024-09-21T03:52:28.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.705+0000 7f8d43e00640 1 -- 172.21.15.16:0/300462006 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8d50086020 con 0x7f8d480a4f10 2024-09-21T03:52:28.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.706+0000 7f8d4ea00640 1 --2- 172.21.15.16:0/300462006 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8d380b6610 0x7f8d380b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:28.722+0000 7f8d4ea00640 1 --2- 172.21.15.16:0/300462006 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8d380b6610 0x7f8d380b8b20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8d480057b0 tx=0x7f8d44005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:28 smithi016 ceph-mon[30375]: pgmap v603: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 238 MiB/s wr, 1.16k op/s 2024-09-21T03:52:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:29.181+0000 7f8d54e00640 1 -- 172.21.15.16:0/300462006 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d480b5540 con 0x7f8d480a4f10 2024-09-21T03:52:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:29.183+0000 7f8d43e00640 1 -- 172.21.15.16:0/300462006 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f8d480b5540 con 0x7f8d480a4f10 2024-09-21T03:52:29.187 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:52:29.187 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:52:29.187 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:52:29.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:29.187+0000 7f8d54e00640 1 -- 172.21.15.16:0/300462006 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8d380b6610 msgr2=0x7f8d380b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:29.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:29.187+0000 7f8d54e00640 1 --2- 172.21.15.16:0/300462006 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8d380b6610 0x7f8d380b8b20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8d480057b0 tx=0x7f8d44005f90 comp rx=0 tx=0).stop 2024-09-21T03:52:29.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:29.187+0000 7f8d54e00640 1 -- 172.21.15.16:0/300462006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d480a4f10 msgr2=0x7f8d480baf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:29.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:29.187+0000 7f8d54e00640 1 --2- 172.21.15.16:0/300462006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d480a4f10 0x7f8d480baf70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8d500524b0 tx=0x7f8d5006a4f0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:29.188+0000 7f8d54e00640 1 -- 172.21.15.16:0/300462006 shutdown_connections 2024-09-21T03:52:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:29.188+0000 7f8d54e00640 1 --2- 172.21.15.16:0/300462006 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8d380b6610 0x7f8d380b8b20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:29.188+0000 7f8d54e00640 1 --2- 172.21.15.16:0/300462006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d480ab190 0x7f8d480b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:29.188+0000 7f8d54e00640 1 --2- 172.21.15.16:0/300462006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d480aa840 0x7f8d480b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:29.188+0000 7f8d54e00640 1 --2- 172.21.15.16:0/300462006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d480a4f10 0x7f8d480baf70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:29.188+0000 7f8d54e00640 1 -- 172.21.15.16:0/300462006 >> 172.21.15.16:0/300462006 conn(0x7f8d4801cc60 msgr2=0x7f8d480a3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:29.188+0000 7f8d54e00640 1 -- 172.21.15.16:0/300462006 shutdown_connections 2024-09-21T03:52:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:29.188+0000 7f8d54e00640 1 -- 172.21.15.16:0/300462006 wait complete. 2024-09-21T03:52:29.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:28 smithi134 ceph-mon[35031]: pgmap v603: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 238 MiB/s wr, 1.16k op/s 2024-09-21T03:52:29.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:28 smithi123 ceph-mon[34673]: pgmap v603: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 238 MiB/s wr, 1.16k op/s 2024-09-21T03:52:30.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:29 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/300462006' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:30.387 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:29 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/300462006' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:30.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:29 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/300462006' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:30.836 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db --set-id 2e29e1f3 --release --await' 2024-09-21T03:52:30.836 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db --set-id 2e29e1f3 --release --await 2024-09-21T03:52:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:30 smithi016 ceph-mon[30375]: pgmap v604: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 235 MiB/s wr, 1.23k op/s 2024-09-21T03:52:31.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:30 smithi134 ceph-mon[35031]: pgmap v604: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 235 MiB/s wr, 1.23k op/s 2024-09-21T03:52:31.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:30 smithi123 ceph-mon[34673]: pgmap v604: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 235 MiB/s wr, 1.23k op/s 2024-09-21T03:52:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:33.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:32 smithi134 ceph-mon[35031]: pgmap v605: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 230 MiB/s wr, 1.28k op/s 2024-09-21T03:52:33.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:32 smithi123 ceph-mon[34673]: pgmap v605: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 230 MiB/s wr, 1.28k op/s 2024-09-21T03:52:33.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:32 smithi016 ceph-mon[30375]: pgmap v605: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 230 MiB/s wr, 1.28k op/s 2024-09-21T03:52:35.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:34 smithi134 ceph-mon[35031]: pgmap v606: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 236 MiB/s wr, 1.47k op/s 2024-09-21T03:52:35.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:34 smithi123 ceph-mon[34673]: pgmap v606: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 236 MiB/s wr, 1.47k op/s 2024-09-21T03:52:35.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:34 smithi016 ceph-mon[30375]: pgmap v606: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 236 MiB/s wr, 1.47k op/s 2024-09-21T03:52:35.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.544+0000 7fb61cc00640 1 -- 172.21.15.16:0/3544136019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6100956b0 msgr2=0x7fb610099be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:35.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.545+0000 7fb61cc00640 1 --2- 172.21.15.16:0/3544136019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6100956b0 0x7fb610099be0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb604009930 tx=0x7fb60402a460 comp rx=0 tx=0).stop 2024-09-21T03:52:35.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.546+0000 7fb61cc00640 1 -- 172.21.15.16:0/3544136019 shutdown_connections 2024-09-21T03:52:35.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.546+0000 7fb61cc00640 1 --2- 172.21.15.16:0/3544136019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb61009a1a0 0x7fb61009a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.546+0000 7fb61cc00640 1 --2- 172.21.15.16:0/3544136019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6100956b0 0x7fb610099be0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.546+0000 7fb61cc00640 1 --2- 172.21.15.16:0/3544136019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb610094d60 0x7fb610095170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.546+0000 7fb61cc00640 1 -- 172.21.15.16:0/3544136019 >> 172.21.15.16:0/3544136019 conn(0x7fb61000cd70 msgr2=0x7fb61000d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:35.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.547+0000 7fb61cc00640 1 -- 172.21.15.16:0/3544136019 shutdown_connections 2024-09-21T03:52:35.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.547+0000 7fb61cc00640 1 -- 172.21.15.16:0/3544136019 wait complete. 2024-09-21T03:52:35.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.550+0000 7fb61cc00640 1 Processor -- start 2024-09-21T03:52:35.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.550+0000 7fb61cc00640 1 -- start start 2024-09-21T03:52:35.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.552+0000 7fb61cc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb610094d60 0x7fb6101330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.552+0000 7fb61cc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6100956b0 0x7fb610133650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.552+0000 7fb61cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb61009a1a0 0x7fb6101388f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.552+0000 7fb61cc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6100ac6c0 con 0x7fb61009a1a0 2024-09-21T03:52:35.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.552+0000 7fb61cc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb6100ac540 con 0x7fb6100956b0 2024-09-21T03:52:35.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.552+0000 7fb61cc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb6100ac840 con 0x7fb610094d60 2024-09-21T03:52:35.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.554+0000 7fb617400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb610094d60 0x7fb6101330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:35.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.555+0000 7fb616a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6100956b0 0x7fb610133650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:35.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.558+0000 7fb617400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb610094d60 0x7fb6101330b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39018/0 (socket says 172.21.15.16:39018) 2024-09-21T03:52:35.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.558+0000 7fb617400640 1 -- 172.21.15.16:0/3161640403 learned_addr learned my addr 172.21.15.16:0/3161640403 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.562+0000 7fb617e00640 1 --2- 172.21.15.16:0/3161640403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb61009a1a0 0x7fb6101388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.564+0000 7fb617400640 1 -- 172.21.15.16:0/3161640403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6100956b0 msgr2=0x7fb610133650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.564+0000 7fb617400640 1 --2- 172.21.15.16:0/3161640403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6100956b0 0x7fb610133650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.564+0000 7fb617400640 1 -- 172.21.15.16:0/3161640403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb61009a1a0 msgr2=0x7fb6101388f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.564+0000 7fb617400640 1 --2- 172.21.15.16:0/3161640403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb61009a1a0 0x7fb6101388f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.564+0000 7fb617400640 1 -- 172.21.15.16:0/3161640403 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb610139020 con 0x7fb610094d60 2024-09-21T03:52:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.569+0000 7fb617400640 1 --2- 172.21.15.16:0/3161640403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb610094d60 0x7fb6101330b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00e9f0 tx=0x7fb60c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.569+0000 7fb60be00640 1 -- 172.21.15.16:0/3161640403 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c00cd80 con 0x7fb610094d60 2024-09-21T03:52:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.570+0000 7fb60be00640 1 -- 172.21.15.16:0/3161640403 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb60c0108e0 con 0x7fb610094d60 2024-09-21T03:52:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.570+0000 7fb60be00640 1 -- 172.21.15.16:0/3161640403 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c018980 con 0x7fb610094d60 2024-09-21T03:52:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.570+0000 7fb61cc00640 1 -- 172.21.15.16:0/3161640403 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb610139310 con 0x7fb610094d60 2024-09-21T03:52:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.570+0000 7fb61cc00640 1 -- 172.21.15.16:0/3161640403 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb61009b1b0 con 0x7fb610094d60 2024-09-21T03:52:35.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.583+0000 7fb624fd3740 1 Processor -- start 2024-09-21T03:52:35.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.585+0000 7fb624fd3740 1 -- start start 2024-09-21T03:52:35.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.585+0000 7fb624fd3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f516d520 0x55b3f5514340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.585+0000 7fb624fd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 0x55b3f5514880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.585+0000 7fb624fd3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f514c840 0x55b3f5514dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.585+0000 7fb624fd3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b3f548b290 con 0x55b3f516d520 2024-09-21T03:52:35.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.585+0000 7fb624fd3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b3f547a6c0 con 0x55b3f514c840 2024-09-21T03:52:35.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.585+0000 7fb624fd3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b3f547e4d0 con 0x55b3f5144660 2024-09-21T03:52:35.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.586+0000 7fb616a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f514c840 0x55b3f5514dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:35.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.587+0000 7fb60be00640 1 -- 172.21.15.16:0/3161640403 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb60c018b20 con 0x7fb610094d60 2024-09-21T03:52:35.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.587+0000 7fb616a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f514c840 0x55b3f5514dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42196/0 (socket says 172.21.15.16:42196) 2024-09-21T03:52:35.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.587+0000 7fb617e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f516d520 0x55b3f5514340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:35.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.587+0000 7fb616a00640 1 -- 172.21.15.16:0/1579247856 learned_addr learned my addr 172.21.15.16:0/1579247856 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:35.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.588+0000 7fb617400640 1 --2- 172.21.15.16:0/1579247856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 0x55b3f5514880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:35.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.588+0000 7fb616a00640 1 -- 172.21.15.16:0/1579247856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 msgr2=0x55b3f5514880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:35.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.588+0000 7fb616a00640 1 --2- 172.21.15.16:0/1579247856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 0x55b3f5514880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.588+0000 7fb616a00640 1 -- 172.21.15.16:0/1579247856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f516d520 msgr2=0x55b3f5514340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:35.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.588+0000 7fb616a00640 1 --2- 172.21.15.16:0/1579247856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f516d520 0x55b3f5514340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.588+0000 7fb616a00640 1 -- 172.21.15.16:0/1579247856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3f5407460 con 0x55b3f514c840 2024-09-21T03:52:35.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.590+0000 7fb616a00640 1 --2- 172.21.15.16:0/1579247856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f514c840 0x55b3f5514dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb60402aa50 tx=0x7fb604004310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40d8b2b4a2174eca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:35.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.591+0000 7fb5ffe00640 1 -- 172.21.15.16:0/1579247856 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb604035040 con 0x55b3f514c840 2024-09-21T03:52:35.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.591+0000 7fb5ffe00640 1 -- 172.21.15.16:0/1579247856 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6040046c0 con 0x55b3f514c840 2024-09-21T03:52:35.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.591+0000 7fb5ffe00640 1 -- 172.21.15.16:0/1579247856 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb604003850 con 0x55b3f514c840 2024-09-21T03:52:35.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.592+0000 7fb60be00640 1 --2- 172.21.15.16:0/3161640403 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6000b6720 0x7fb6000b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.592+0000 7fb60be00640 1 -- 172.21.15.16:0/3161640403 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb60c0e77e0 con 0x7fb610094d60 2024-09-21T03:52:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.593+0000 7fb617e00640 1 --2- 172.21.15.16:0/3161640403 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6000b6720 0x7fb6000b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.594+0000 7fb624fd3740 1 -- 172.21.15.16:0/1579247856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f514c840 msgr2=0x55b3f5514dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.595+0000 7fb624fd3740 1 --2- 172.21.15.16:0/1579247856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f514c840 0x55b3f5514dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb60402aa50 tx=0x7fb604004310 comp rx=0 tx=0).stop 2024-09-21T03:52:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.595+0000 7fb624fd3740 1 -- 172.21.15.16:0/1579247856 shutdown_connections 2024-09-21T03:52:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.595+0000 7fb624fd3740 1 --2- 172.21.15.16:0/1579247856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f514c840 0x55b3f5514dc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.595+0000 7fb624fd3740 1 --2- 172.21.15.16:0/1579247856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 0x55b3f5514880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.595+0000 7fb624fd3740 1 --2- 172.21.15.16:0/1579247856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f516d520 0x55b3f5514340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.595+0000 7fb624fd3740 1 -- 172.21.15.16:0/1579247856 >> 172.21.15.16:0/1579247856 conn(0x55b3f5161910 msgr2=0x55b3f5512230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.596+0000 7fb624fd3740 1 -- 172.21.15.16:0/1579247856 shutdown_connections 2024-09-21T03:52:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.596+0000 7fb624fd3740 1 -- 172.21.15.16:0/1579247856 wait complete. 2024-09-21T03:52:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.597+0000 7fb617e00640 1 --2- 172.21.15.16:0/3161640403 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6000b6720 0x7fb6000b8c30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb618066c70 tx=0x7fb618051ed0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.597+0000 7fb624fd3740 1 -- start start 2024-09-21T03:52:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.598+0000 7fb624fd3740 1 Processor -- start 2024-09-21T03:52:35.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.601+0000 7fb624fd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 0x55b3f551fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.601+0000 7fb624fd3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f514c840 0x55b3f5520400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.601+0000 7fb624fd3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f516d520 0x55b3f552be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.601+0000 7fb624fd3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b3f52dbbb0 con 0x55b3f514c840 2024-09-21T03:52:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.601+0000 7fb624fd3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b3f54156c0 con 0x55b3f516d520 2024-09-21T03:52:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.601+0000 7fb624fd3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b3f5522ac0 con 0x55b3f5144660 2024-09-21T03:52:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.601+0000 7fb624fd3740 10 client.0 _ll_register_callbacks cb 0x55b3f5430490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.601+0000 7fb617400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 0x55b3f551fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.601+0000 7fb617400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 0x55b3f551fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39036/0 (socket says 172.21.15.16:39036) 2024-09-21T03:52:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.601+0000 7fb617400640 1 -- 172.21.15.16:0/1031266597 learned_addr learned my addr 172.21.15.16:0/1031266597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.602+0000 7fb617e00640 1 --2- 172.21.15.16:0/1031266597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f516d520 0x55b3f552be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.602+0000 7fb617400640 1 -- 172.21.15.16:0/1031266597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f516d520 msgr2=0x55b3f552be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.602+0000 7fb617400640 1 --2- 172.21.15.16:0/1031266597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f516d520 0x55b3f552be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.602+0000 7fb617400640 1 -- 172.21.15.16:0/1031266597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f514c840 msgr2=0x55b3f5520400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.602+0000 7fb617400640 1 --2- 172.21.15.16:0/1031266597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f514c840 0x55b3f5520400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.602+0000 7fb617400640 1 -- 172.21.15.16:0/1031266597 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3f51d8b00 con 0x55b3f5144660 2024-09-21T03:52:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.603+0000 7fb617400640 1 --2- 172.21.15.16:0/1031266597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 0x55b3f551fec0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00c6a0 tx=0x7fb60c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4db522d2276226c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.603+0000 7fb5fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:52:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.603+0000 7fb5fcc00640 1 -- 172.21.15.16:0/1031266597 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c0a9400 con 0x55b3f5144660 2024-09-21T03:52:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.603+0000 7fb5fcc00640 1 -- 172.21.15.16:0/1031266597 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb60c0ac230 con 0x55b3f5144660 2024-09-21T03:52:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.603+0000 7fb624fd3740 1 -- 172.21.15.16:0/1031266597 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6040046c0 con 0x55b3f5144660 2024-09-21T03:52:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.604+0000 7fb5fcc00640 1 -- 172.21.15.16:0/1031266597 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb60c0accd0 con 0x55b3f5144660 2024-09-21T03:52:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.604+0000 7fb624fd3740 10 client.26170 fetch_fsmap learned FSMap version 30 2024-09-21T03:52:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.604+0000 7fb624fd3740 1 -- 172.21.15.16:0/1031266597 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3f524f960 con 0x55b3f5144660 2024-09-21T03:52:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.605+0000 7fb5fcc00640 1 -- 172.21.15.16:0/1031266597 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6040046c0 con 0x55b3f5144660 2024-09-21T03:52:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.605+0000 7fb5fcc00640 1 -- 172.21.15.16:0/1031266597 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb60c0eace0 con 0x55b3f5144660 2024-09-21T03:52:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.605+0000 7fb624fd3740 10 client.26170 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:52:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.605+0000 7fb624fd3740 10 client.26170 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:52:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.606+0000 7fb624fd3740 20 client.26170 populate_metadata read hostname 'smithi016' 2024-09-21T03:52:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.606+0000 7fb624fd3740 1 --2- 172.21.15.16:0/1031266597 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55b3f512bea0 0x55b3f5531e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.606+0000 7fb624fd3740 4 client.26170 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.606+0000 7fb624fd3740 1 -- 172.21.15.16:0/1031266597 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3f548d1e0 con 0x55b3f512bea0 2024-09-21T03:52:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.607+0000 7fb616a00640 1 --2- 172.21.15.16:0/1031266597 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55b3f512bea0 0x55b3f5531e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.608+0000 7fb616a00640 1 --2- 172.21.15.16:0/1031266597 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55b3f512bea0 0x55b3f5531e80 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a74605052ed1bb37 server_cookie=cdc5aa2321367aad in_seq=0 out_seq=0 2024-09-21T03:52:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.608+0000 7fb5fcc00640 10 client.26170 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.609+0000 7fb5fcc00640 1 -- 172.21.15.16:0/1031266597 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b3f548d1e0 con 0x55b3f512bea0 2024-09-21T03:52:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.609+0000 7fb5fcc00640 10 client.26170 handle_command_reply: tid=0 2024-09-21T03:52:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.609+0000 7fb624fd3740 1 client.26170 shutdown 2024-09-21T03:52:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.610+0000 7fb624fd3740 1 -- 172.21.15.16:0/1031266597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 msgr2=0x55b3f551fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.610+0000 7fb624fd3740 1 --2- 172.21.15.16:0/1031266597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 0x55b3f551fec0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00c6a0 tx=0x7fb60c010800 comp rx=0 tx=0).stop 2024-09-21T03:52:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.610+0000 7fb624fd3740 1 -- 172.21.15.16:0/1031266597 shutdown_connections 2024-09-21T03:52:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.610+0000 7fb624fd3740 1 --2- 172.21.15.16:0/1031266597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f516d520 0x55b3f552be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.610+0000 7fb624fd3740 1 --2- 172.21.15.16:0/1031266597 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55b3f512bea0 0x55b3f5531e80 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.610+0000 7fb624fd3740 1 --2- 172.21.15.16:0/1031266597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f514c840 0x55b3f5520400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.611+0000 7fb5fcc00640 0 client.26170 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.611+0000 7fb624fd3740 1 --2- 172.21.15.16:0/1031266597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 0x55b3f551fec0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.611+0000 7fb624fd3740 1 -- 172.21.15.16:0/1031266597 >> 172.21.15.16:0/1031266597 conn(0x55b3f5161910 msgr2=0x55b3f55174d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.612+0000 7fb624fd3740 1 -- 172.21.15.16:0/1031266597 shutdown_connections 2024-09-21T03:52:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.612+0000 7fb624fd3740 1 -- 172.21.15.16:0/1031266597 wait complete. 2024-09-21T03:52:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.612+0000 7fb624fd3740 20 client.26170 trim_cache size 0 max 16384 2024-09-21T03:52:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.652+0000 7fb624fd3740 1 Processor -- start 2024-09-21T03:52:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.652+0000 7fb624fd3740 1 -- start start 2024-09-21T03:52:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.652+0000 7fb624fd3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f512bea0 0x55b3f552bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.652+0000 7fb624fd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 0x55b3f552c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.652+0000 7fb624fd3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f514c840 0x55b3f5517440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.652+0000 7fb624fd3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f800fd10 con 0x55b3f514c840 2024-09-21T03:52:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.652+0000 7fb624fd3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f8011850 con 0x55b3f512bea0 2024-09-21T03:52:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.652+0000 7fb624fd3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f80103e0 con 0x55b3f5144660 2024-09-21T03:52:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.653+0000 7fb616a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 0x55b3f552c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.653+0000 7fb616a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 0x55b3f552c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39052/0 (socket says 172.21.15.16:39052) 2024-09-21T03:52:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.653+0000 7fb616a00640 1 -- 172.21.15.16:0/1062387196 learned_addr learned my addr 172.21.15.16:0/1062387196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.653+0000 7fb617400640 1 --2- 172.21.15.16:0/1062387196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f512bea0 0x55b3f552bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.653+0000 7fb617e00640 1 --2- 172.21.15.16:0/1062387196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f514c840 0x55b3f5517440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.653+0000 7fb616a00640 1 -- 172.21.15.16:0/1062387196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f512bea0 msgr2=0x55b3f552bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.653+0000 7fb616a00640 1 --2- 172.21.15.16:0/1062387196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f512bea0 0x55b3f552bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.653+0000 7fb616a00640 1 -- 172.21.15.16:0/1062387196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f514c840 msgr2=0x55b3f5517440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.653+0000 7fb616a00640 1 --2- 172.21.15.16:0/1062387196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f514c840 0x55b3f5517440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.653+0000 7fb616a00640 1 -- 172.21.15.16:0/1062387196 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3f551b550 con 0x55b3f5144660 2024-09-21T03:52:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.653+0000 7fb616a00640 1 --2- 172.21.15.16:0/1062387196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 0x55b3f552c250 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb604005a70 tx=0x7fb604005310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdd3df4c6a86153b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.654+0000 7fb5ff400640 1 -- 172.21.15.16:0/1062387196 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb604051040 con 0x55b3f5144660 2024-09-21T03:52:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.654+0000 7fb5ff400640 1 -- 172.21.15.16:0/1062387196 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb604004b70 con 0x55b3f5144660 2024-09-21T03:52:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.654+0000 7fb5ff400640 1 -- 172.21.15.16:0/1062387196 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb604035040 con 0x55b3f5144660 2024-09-21T03:52:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.655+0000 7fb624fd3740 1 -- 172.21.15.16:0/1062387196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 msgr2=0x55b3f552c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.655+0000 7fb624fd3740 1 --2- 172.21.15.16:0/1062387196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 0x55b3f552c250 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb604005a70 tx=0x7fb604005310 comp rx=0 tx=0).stop 2024-09-21T03:52:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.655+0000 7fb624fd3740 1 -- 172.21.15.16:0/1062387196 shutdown_connections 2024-09-21T03:52:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.655+0000 7fb624fd3740 1 --2- 172.21.15.16:0/1062387196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f514c840 0x55b3f5517440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.655+0000 7fb624fd3740 1 --2- 172.21.15.16:0/1062387196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f5144660 0x55b3f552c250 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.655+0000 7fb624fd3740 1 --2- 172.21.15.16:0/1062387196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f512bea0 0x55b3f552bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.655+0000 7fb624fd3740 1 -- 172.21.15.16:0/1062387196 >> 172.21.15.16:0/1062387196 conn(0x55b3f5161910 msgr2=0x55b3f551fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.655+0000 7fb624fd3740 1 -- 172.21.15.16:0/1062387196 shutdown_connections 2024-09-21T03:52:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.656+0000 7fb624fd3740 1 -- 172.21.15.16:0/1062387196 wait complete. 2024-09-21T03:52:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.656+0000 7fb624fd3740 1 -- start start 2024-09-21T03:52:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.656+0000 7fb624fd3740 1 Processor -- start 2024-09-21T03:52:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.657+0000 7fb624fd3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f512bea0 0x55b3f5524820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.657+0000 7fb624fd3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f5144660 0x55b3f5524e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.657+0000 7fb624fd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f514c840 0x55b3f55a6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.657+0000 7fb624fd3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f8011180 con 0x55b3f512bea0 2024-09-21T03:52:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.657+0000 7fb624fd3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f8010ab0 con 0x55b3f5144660 2024-09-21T03:52:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.657+0000 7fb624fd3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f800dcf0 con 0x55b3f514c840 2024-09-21T03:52:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.657+0000 7fb624fd3740 10 client.0 _ll_register_callbacks cb 0x7fb60c0056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.657+0000 7fb617e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f514c840 0x55b3f55a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.657+0000 7fb617e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f514c840 0x55b3f55a6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39060/0 (socket says 172.21.15.16:39060) 2024-09-21T03:52:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.657+0000 7fb616a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f5144660 0x55b3f5524e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.657+0000 7fb617e00640 1 -- 172.21.15.16:0/4286003662 learned_addr learned my addr 172.21.15.16:0/4286003662 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.657+0000 7fb617e00640 1 -- 172.21.15.16:0/4286003662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f5144660 msgr2=0x55b3f5524e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.657+0000 7fb617e00640 1 --2- 172.21.15.16:0/4286003662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f5144660 0x55b3f5524e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.657+0000 7fb617e00640 1 -- 172.21.15.16:0/4286003662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f512bea0 msgr2=0x55b3f5524820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.657+0000 7fb617e00640 1 --2- 172.21.15.16:0/4286003662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f512bea0 0x55b3f5524820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.657+0000 7fb617e00640 1 -- 172.21.15.16:0/4286003662 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3f55a6fb0 con 0x55b3f514c840 2024-09-21T03:52:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.657+0000 7fb616a00640 1 --2- 172.21.15.16:0/4286003662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f5144660 0x55b3f5524e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:52:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.658+0000 7fb617e00640 1 --2- 172.21.15.16:0/4286003662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f514c840 0x55b3f55a6940 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb618069050 tx=0x7fb618064940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=984a5c13929effe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.658+0000 7fb5fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:52:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.659+0000 7fb5fcc00640 1 -- 172.21.15.16:0/4286003662 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618091410 con 0x55b3f514c840 2024-09-21T03:52:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.659+0000 7fb5fcc00640 1 -- 172.21.15.16:0/4286003662 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb61807f040 con 0x55b3f514c840 2024-09-21T03:52:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.659+0000 7fb624fd3740 1 -- 172.21.15.16:0/4286003662 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb604004b70 con 0x55b3f514c840 2024-09-21T03:52:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.659+0000 7fb5fcc00640 1 -- 172.21.15.16:0/4286003662 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb618099050 con 0x55b3f514c840 2024-09-21T03:52:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.659+0000 7fb624fd3740 10 client.26176 fetch_fsmap learned FSMap version 30 2024-09-21T03:52:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.659+0000 7fb624fd3740 1 -- 172.21.15.16:0/4286003662 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3f55ae470 con 0x55b3f514c840 2024-09-21T03:52:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.659+0000 7fb5fcc00640 1 -- 172.21.15.16:0/4286003662 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb604004b70 con 0x55b3f514c840 2024-09-21T03:52:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.660+0000 7fb5fcc00640 1 -- 172.21.15.16:0/4286003662 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb618090550 con 0x55b3f514c840 2024-09-21T03:52:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.660+0000 7fb624fd3740 10 client.26176 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:52:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.660+0000 7fb624fd3740 10 client.26176 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:52:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.660+0000 7fb624fd3740 20 client.26176 populate_metadata read hostname 'smithi016' 2024-09-21T03:52:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.660+0000 7fb624fd3740 1 --2- 172.21.15.16:0/4286003662 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55b3f516d520 0x55b3f55b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.660+0000 7fb624fd3740 4 client.26176 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2e29e1f3", "release": true, "await": true}] 2024-09-21T03:52:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.660+0000 7fb624fd3740 1 -- 172.21.15.16:0/4286003662 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2e29e1f3", "release": true, "await": true}) -- 0x55b3f5488c90 con 0x55b3f516d520 2024-09-21T03:52:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.661+0000 7fb617400640 1 --2- 172.21.15.16:0/4286003662 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55b3f516d520 0x55b3f55b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.661+0000 7fb617400640 1 --2- 172.21.15.16:0/4286003662 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55b3f516d520 0x55b3f55b0c80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df9d4fc5234e9574 server_cookie=78f3a1fb22524983 in_seq=0 out_seq=0 2024-09-21T03:52:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.661+0000 7fb5fcc00640 10 client.26176 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.665+0000 7fb5fcc00640 1 -- 172.21.15.16:0/4286003662 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b3f5488c90 con 0x55b3f516d520 2024-09-21T03:52:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.665+0000 7fb5fcc00640 10 client.26176 handle_command_reply: tid=0 2024-09-21T03:52:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.665+0000 7fb624fd3740 1 client.26176 shutdown 2024-09-21T03:52:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.665+0000 7fb624fd3740 1 -- 172.21.15.16:0/4286003662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f514c840 msgr2=0x55b3f55a6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.665+0000 7fb624fd3740 1 --2- 172.21.15.16:0/4286003662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f514c840 0x55b3f55a6940 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb618069050 tx=0x7fb618064940 comp rx=0 tx=0).stop 2024-09-21T03:52:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.665+0000 7fb624fd3740 1 -- 172.21.15.16:0/4286003662 shutdown_connections 2024-09-21T03:52:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.665+0000 7fb624fd3740 1 --2- 172.21.15.16:0/4286003662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3f514c840 0x55b3f55a6940 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.665+0000 7fb624fd3740 1 --2- 172.21.15.16:0/4286003662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f5144660 0x55b3f5524e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.665+0000 7fb624fd3740 1 --2- 172.21.15.16:0/4286003662 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55b3f516d520 0x55b3f55b0c80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.665+0000 7fb624fd3740 1 --2- 172.21.15.16:0/4286003662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3f512bea0 0x55b3f5524820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.665+0000 7fb624fd3740 1 -- 172.21.15.16:0/4286003662 >> 172.21.15.16:0/4286003662 conn(0x55b3f5161910 msgr2=0x55b3f5517440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.665+0000 7fb5fcc00640 0 client.26176 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.666+0000 7fb624fd3740 1 -- 172.21.15.16:0/4286003662 shutdown_connections 2024-09-21T03:52:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.667+0000 7fb624fd3740 1 -- 172.21.15.16:0/4286003662 wait complete. 2024-09-21T03:52:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.667+0000 7fb624fd3740 20 client.26176 trim_cache size 0 max 16384 2024-09-21T03:52:35.671 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:52:35.671 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:52:35.671 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:52:35.671 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 24, 2024-09-21T03:52:35.671 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:52:35.671 INFO:teuthology.orchestra.run.smithi016.stdout: "2e29e1f3": { 2024-09-21T03:52:35.672 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 24, 2024-09-21T03:52:35.672 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:52:35.672 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:52:35.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:52:35.672 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:52:35.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:52:35.672 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:52:35.672 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 139.2, 2024-09-21T03:52:35.672 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:52:35.673 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:52:35.673 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:52:35.673 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:52:35.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:52:35.673 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:52:35.673 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:52:35.673 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:52:35.673 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:52:35.673 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:52:35.673 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:52:35.673 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:52:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.673+0000 7fb61cc00640 1 -- 172.21.15.16:0/3161640403 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6000b6720 msgr2=0x7fb6000b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.673+0000 7fb61cc00640 1 --2- 172.21.15.16:0/3161640403 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6000b6720 0x7fb6000b8c30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb618066c70 tx=0x7fb618051ed0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.673+0000 7fb61cc00640 1 -- 172.21.15.16:0/3161640403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb610094d60 msgr2=0x7fb6101330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.673+0000 7fb61cc00640 1 --2- 172.21.15.16:0/3161640403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb610094d60 0x7fb6101330b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00e9f0 tx=0x7fb60c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.673+0000 7fb61cc00640 1 -- 172.21.15.16:0/3161640403 shutdown_connections 2024-09-21T03:52:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.673+0000 7fb61cc00640 1 --2- 172.21.15.16:0/3161640403 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6000b6720 0x7fb6000b8c30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.673+0000 7fb61cc00640 1 --2- 172.21.15.16:0/3161640403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb61009a1a0 0x7fb6101388f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.673+0000 7fb61cc00640 1 --2- 172.21.15.16:0/3161640403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6100956b0 0x7fb610133650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.673+0000 7fb61cc00640 1 --2- 172.21.15.16:0/3161640403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb610094d60 0x7fb6101330b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.673+0000 7fb61cc00640 1 -- 172.21.15.16:0/3161640403 >> 172.21.15.16:0/3161640403 conn(0x7fb61000cd70 msgr2=0x7fb610097bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.674+0000 7fb61cc00640 1 -- 172.21.15.16:0/3161640403 shutdown_connections 2024-09-21T03:52:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:35.674+0000 7fb61cc00640 1 -- 172.21.15.16:0/3161640403 wait complete. 2024-09-21T03:52:37.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:36 smithi134 ceph-mon[35031]: pgmap v607: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 240 MiB/s wr, 1.66k op/s 2024-09-21T03:52:37.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:36 smithi123 ceph-mon[34673]: pgmap v607: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 240 MiB/s wr, 1.66k op/s 2024-09-21T03:52:37.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:36 smithi016 ceph-mon[30375]: pgmap v607: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 240 MiB/s wr, 1.66k op/s 2024-09-21T03:52:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.544+0000 7f959be00640 1 -- 172.21.15.16:0/1204301647 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959402f310 con 0x7f959c104740 2024-09-21T03:52:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.547+0000 7f95a4200640 1 -- 172.21.15.16:0/1204301647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f959c104740 msgr2=0x7f959c104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.547+0000 7f95a4200640 1 --2- 172.21.15.16:0/1204301647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f959c104740 0x7f959c104b50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9594009ca0 tx=0x7f959402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.549+0000 7f95a4200640 1 -- 172.21.15.16:0/1204301647 shutdown_connections 2024-09-21T03:52:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.549+0000 7f95a4200640 1 --2- 172.21.15.16:0/1204301647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f959c1050e0 0x7f959c10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.549+0000 7f95a4200640 1 --2- 172.21.15.16:0/1204301647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f959c104740 0x7f959c104b50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.549+0000 7f95a4200640 1 --2- 172.21.15.16:0/1204301647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f959c1036c0 0x7f959c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.549+0000 7f95a4200640 1 -- 172.21.15.16:0/1204301647 >> 172.21.15.16:0/1204301647 conn(0x7f959c0fefe0 msgr2=0x7f959c1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.551+0000 7f95a4200640 1 -- 172.21.15.16:0/1204301647 shutdown_connections 2024-09-21T03:52:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.551+0000 7f95a4200640 1 -- 172.21.15.16:0/1204301647 wait complete. 2024-09-21T03:52:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.552+0000 7f95a4200640 1 Processor -- start 2024-09-21T03:52:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.556+0000 7f95a4200640 1 -- start start 2024-09-21T03:52:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.557+0000 7f95a4200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f959c1036c0 0x7f959c1131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.557+0000 7f95a4200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f959c104740 0x7f959c113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.557+0000 7f95a4200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f959c1050e0 0x7f959c114340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.557+0000 7f95a4200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f959c10d220 con 0x7f959c1050e0 2024-09-21T03:52:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.557+0000 7f95a4200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f959c10d0a0 con 0x7f959c104740 2024-09-21T03:52:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.557+0000 7f95a4200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f959c10d3a0 con 0x7f959c1036c0 2024-09-21T03:52:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.558+0000 7f95a0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f959c104740 0x7f959c113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.558+0000 7f95a0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f959c104740 0x7f959c113810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42228/0 (socket says 172.21.15.16:42228) 2024-09-21T03:52:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.558+0000 7f95a0e00640 1 -- 172.21.15.16:0/3200252144 learned_addr learned my addr 172.21.15.16:0/3200252144 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.560+0000 7f95a1800640 1 --2- 172.21.15.16:0/3200252144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f959c1036c0 0x7f959c1131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.560+0000 7f95a0e00640 1 -- 172.21.15.16:0/3200252144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f959c1036c0 msgr2=0x7f959c1131a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.560+0000 7f95a0e00640 1 --2- 172.21.15.16:0/3200252144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f959c1036c0 0x7f959c1131a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.560+0000 7f95a0e00640 1 -- 172.21.15.16:0/3200252144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f959c1050e0 msgr2=0x7f959c114340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.560+0000 7f95a0e00640 1 --2- 172.21.15.16:0/3200252144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f959c1050e0 0x7f959c114340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.560+0000 7f95a0e00640 1 -- 172.21.15.16:0/3200252144 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c114b40 con 0x7f959c104740 2024-09-21T03:52:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.560+0000 7f95a1800640 1 --2- 172.21.15.16:0/3200252144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f959c1036c0 0x7f959c1131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.564+0000 7f95a0e00640 1 --2- 172.21.15.16:0/3200252144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f959c104740 0x7f959c113810 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f959402ade0 tx=0x7f9594003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.566+0000 7f959a000640 1 -- 172.21.15.16:0/3200252144 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594004040 con 0x7f959c104740 2024-09-21T03:52:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.566+0000 7f959a000640 1 -- 172.21.15.16:0/3200252144 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9594004690 con 0x7f959c104740 2024-09-21T03:52:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.566+0000 7f959a000640 1 -- 172.21.15.16:0/3200252144 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959402fed0 con 0x7f959c104740 2024-09-21T03:52:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.568+0000 7f95a4200640 1 -- 172.21.15.16:0/3200252144 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959c114dd0 con 0x7f959c104740 2024-09-21T03:52:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.568+0000 7f95a4200640 1 -- 172.21.15.16:0/3200252144 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959c1b11b0 con 0x7f959c104740 2024-09-21T03:52:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.580+0000 7f95ab29d740 1 Processor -- start 2024-09-21T03:52:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.582+0000 7f959a000640 1 -- 172.21.15.16:0/3200252144 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f959402f620 con 0x7f959c104740 2024-09-21T03:52:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.583+0000 7f95ab29d740 1 -- start start 2024-09-21T03:52:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.583+0000 7f95ab29d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2adb1b0 0x55fce2e820d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.583+0000 7f95ab29d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2ab22f0 0x55fce2e82740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.583+0000 7f95ab29d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2e83290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.583+0000 7f95ab29d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fce2df56b0 con 0x55fce2ab22f0 2024-09-21T03:52:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.583+0000 7f95ab29d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fce2de4ae0 con 0x55fce2adb1b0 2024-09-21T03:52:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.584+0000 7f95ab29d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fce2de88f0 con 0x55fce2aba4d0 2024-09-21T03:52:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.584+0000 7f95a0e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2e83290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.584+0000 7f95a0e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2e83290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39090/0 (socket says 172.21.15.16:39090) 2024-09-21T03:52:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.584+0000 7f95a0e00640 1 -- 172.21.15.16:0/175924424 learned_addr learned my addr 172.21.15.16:0/175924424 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.584+0000 7f95a2200640 1 --2- 172.21.15.16:0/175924424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2adb1b0 0x55fce2e820d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.584+0000 7f95a0e00640 1 -- 172.21.15.16:0/175924424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2adb1b0 msgr2=0x55fce2e820d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.584+0000 7f95a0e00640 1 --2- 172.21.15.16:0/175924424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2adb1b0 0x55fce2e820d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.584+0000 7f95a0e00640 1 -- 172.21.15.16:0/175924424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2ab22f0 msgr2=0x55fce2e82740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.584+0000 7f95a0e00640 1 --2- 172.21.15.16:0/175924424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2ab22f0 0x55fce2e82740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.584+0000 7f95a0e00640 1 -- 172.21.15.16:0/175924424 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce2db1780 con 0x55fce2aba4d0 2024-09-21T03:52:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.585+0000 7f95a2200640 1 --2- 172.21.15.16:0/175924424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2adb1b0 0x55fce2e820d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.590+0000 7f959a000640 1 --2- 172.21.15.16:0/3200252144 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f95840b6660 0x7f95840b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.590+0000 7f959a000640 1 -- 172.21.15.16:0/3200252144 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f9594108f50 con 0x7f959c104740 2024-09-21T03:52:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.592+0000 7f95a2200640 1 --2- 172.21.15.16:0/3200252144 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f95840b6660 0x7f95840b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.593+0000 7f95a0e00640 1 --2- 172.21.15.16:0/175924424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2e83290 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f959410ced0 tx=0x7f95941110e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d1cc028a0a5ecf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.593+0000 7f958ea00640 1 -- 172.21.15.16:0/175924424 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95940cb380 con 0x55fce2aba4d0 2024-09-21T03:52:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.594+0000 7f958ea00640 1 -- 172.21.15.16:0/175924424 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9594036030 con 0x55fce2aba4d0 2024-09-21T03:52:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.594+0000 7f958ea00640 1 -- 172.21.15.16:0/175924424 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95940ce4a0 con 0x55fce2aba4d0 2024-09-21T03:52:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.595+0000 7f95ab29d740 1 -- 172.21.15.16:0/175924424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 msgr2=0x55fce2e83290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.595+0000 7f95ab29d740 1 --2- 172.21.15.16:0/175924424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2e83290 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f959410ced0 tx=0x7f95941110e0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.597+0000 7f95a2200640 1 --2- 172.21.15.16:0/3200252144 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f95840b6660 0x7f95840b8b70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f959000a8d0 tx=0x7f9590008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.601+0000 7f95ab29d740 1 -- 172.21.15.16:0/175924424 shutdown_connections 2024-09-21T03:52:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.601+0000 7f95ab29d740 1 --2- 172.21.15.16:0/175924424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2e83290 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.601+0000 7f95ab29d740 1 --2- 172.21.15.16:0/175924424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2ab22f0 0x55fce2e82740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.601+0000 7f95ab29d740 1 --2- 172.21.15.16:0/175924424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2adb1b0 0x55fce2e820d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.601+0000 7f95ab29d740 1 -- 172.21.15.16:0/175924424 >> 172.21.15.16:0/175924424 conn(0x55fce2acf300 msgr2=0x55fce2e7db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.601+0000 7f95ab29d740 1 -- 172.21.15.16:0/175924424 shutdown_connections 2024-09-21T03:52:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.603+0000 7f95ab29d740 1 -- 172.21.15.16:0/175924424 wait complete. 2024-09-21T03:52:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.603+0000 7f95ab29d740 1 -- start start 2024-09-21T03:52:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.604+0000 7f95ab29d740 1 Processor -- start 2024-09-21T03:52:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.605+0000 7f95ab29d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2ab22f0 0x55fce2e9c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.606+0000 7f95ab29d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2e9ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.606+0000 7f95ab29d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2adb1b0 0x55fce2e9d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.606+0000 7f95ab29d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fce2d1cab0 con 0x55fce2ab22f0 2024-09-21T03:52:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.607+0000 7f95a1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2ab22f0 0x55fce2e9c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.607+0000 7f95a0e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2e9ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.607+0000 7f95a0e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2e9ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39106/0 (socket says 172.21.15.16:39106) 2024-09-21T03:52:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.607+0000 7f95a0e00640 1 -- 172.21.15.16:0/2955578660 learned_addr learned my addr 172.21.15.16:0/2955578660 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.607+0000 7f95a2200640 1 --2- 172.21.15.16:0/2955578660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2adb1b0 0x55fce2e9d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.607+0000 7f95ab29d740 1 -- 172.21.15.16:0/2955578660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fce2d8b570 con 0x55fce2adb1b0 2024-09-21T03:52:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.607+0000 7f95ab29d740 1 -- 172.21.15.16:0/2955578660 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fce2de88f0 con 0x55fce2aba4d0 2024-09-21T03:52:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.608+0000 7f95a0e00640 1 -- 172.21.15.16:0/2955578660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2adb1b0 msgr2=0x55fce2e9d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.608+0000 7f95a0e00640 1 --2- 172.21.15.16:0/2955578660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2adb1b0 0x55fce2e9d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.608+0000 7f95a0e00640 1 -- 172.21.15.16:0/2955578660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2ab22f0 msgr2=0x55fce2e9c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.608+0000 7f95a0e00640 1 --2- 172.21.15.16:0/2955578660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2ab22f0 0x55fce2e9c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.608+0000 7f95a0e00640 1 -- 172.21.15.16:0/2955578660 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce2b46790 con 0x55fce2aba4d0 2024-09-21T03:52:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.608+0000 7f95a1800640 1 --2- 172.21.15.16:0/2955578660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2ab22f0 0x55fce2e9c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.608+0000 7f95a0e00640 1 --2- 172.21.15.16:0/2955578660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2e9ca20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f959410cc90 tx=0x7f95940d84f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cf51b7a8bea62bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.609+0000 7f9583400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:52:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.609+0000 7f9583400640 1 -- 172.21.15.16:0/2955578660 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95940ce680 con 0x55fce2aba4d0 2024-09-21T03:52:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.609+0000 7f9583400640 1 -- 172.21.15.16:0/2955578660 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f95940d6da0 con 0x55fce2aba4d0 2024-09-21T03:52:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.609+0000 7f9583400640 10 client.0 inactive, discarding osd_map(86..86 src has 1..86) 2024-09-21T03:52:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.609+0000 7f95ab29d740 10 client.0 _ll_register_callbacks cb 0x55fce2d9a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.609+0000 7f95ab29d740 1 -- 172.21.15.16:0/2955578660 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9594036030 con 0x55fce2aba4d0 2024-09-21T03:52:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.611+0000 7f9583400640 1 -- 172.21.15.16:0/2955578660 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f95940ceb00 con 0x55fce2aba4d0 2024-09-21T03:52:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.611+0000 7f95ab29d740 10 client.26185 fetch_fsmap learned FSMap version 30 2024-09-21T03:52:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.611+0000 7f95ab29d740 1 -- 172.21.15.16:0/2955578660 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55fce2bbf620 con 0x55fce2aba4d0 2024-09-21T03:52:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.613+0000 7f9583400640 1 -- 172.21.15.16:0/2955578660 <== mon.1 v2:172.21.15.134:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f959410c170 con 0x55fce2aba4d0 2024-09-21T03:52:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.614+0000 7f95ab29d740 10 client.26185 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:52:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.614+0000 7f95ab29d740 10 client.26185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.614+0000 7f95ab29d740 20 client.26185 populate_metadata read hostname 'smithi016' 2024-09-21T03:52:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.614+0000 7f95ab29d740 1 --2- 172.21.15.16:0/2955578660 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55fce2a99d80 0x55fce2ea9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.614+0000 7f95ab29d740 4 client.26185 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.615+0000 7f95ab29d740 1 -- 172.21.15.16:0/2955578660 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9594036030 con 0x55fce2a99d80 2024-09-21T03:52:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.615+0000 7f95a1800640 1 --2- 172.21.15.16:0/2955578660 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55fce2a99d80 0x55fce2ea9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.622+0000 7f95a1800640 1 --2- 172.21.15.16:0/2955578660 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55fce2a99d80 0x55fce2ea9400 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98189fe910f4dbe8 server_cookie=df51416c21fae46c in_seq=0 out_seq=0 2024-09-21T03:52:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.622+0000 7f9583400640 10 client.26185 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.626+0000 7f9583400640 1 -- 172.21.15.16:0/2955578660 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f9594036030 con 0x55fce2a99d80 2024-09-21T03:52:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.626+0000 7f9583400640 10 client.26185 handle_command_reply: tid=0 2024-09-21T03:52:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.626+0000 7f95ab29d740 1 client.26185 shutdown 2024-09-21T03:52:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.627+0000 7f95ab29d740 1 -- 172.21.15.16:0/2955578660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 msgr2=0x55fce2e9ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.627+0000 7f95ab29d740 1 --2- 172.21.15.16:0/2955578660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2e9ca20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f959410cc90 tx=0x7f95940d84f0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.627+0000 7f95ab29d740 1 -- 172.21.15.16:0/2955578660 shutdown_connections 2024-09-21T03:52:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.627+0000 7f95ab29d740 1 --2- 172.21.15.16:0/2955578660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2adb1b0 0x55fce2e9d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.627+0000 7f95ab29d740 1 --2- 172.21.15.16:0/2955578660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2e9ca20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.627+0000 7f95ab29d740 1 --2- 172.21.15.16:0/2955578660 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55fce2a99d80 0x55fce2ea9400 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.627+0000 7f95ab29d740 1 --2- 172.21.15.16:0/2955578660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2ab22f0 0x55fce2e9c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.627+0000 7f95ab29d740 1 -- 172.21.15.16:0/2955578660 >> 172.21.15.16:0/2955578660 conn(0x55fce2acf300 msgr2=0x55fce2e7c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.628+0000 7f9583400640 0 client.26185 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.629+0000 7f95ab29d740 1 -- 172.21.15.16:0/2955578660 shutdown_connections 2024-09-21T03:52:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.629+0000 7f95ab29d740 1 -- 172.21.15.16:0/2955578660 wait complete. 2024-09-21T03:52:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.632+0000 7f95ab29d740 20 client.26185 trim_cache size 0 max 16384 2024-09-21T03:52:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.672+0000 7f95ab29d740 1 Processor -- start 2024-09-21T03:52:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.673+0000 7f95ab29d740 1 -- start start 2024-09-21T03:52:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.673+0000 7f95ab29d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2ab22f0 0x55fce2e87240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.674+0000 7f95ab29d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2e87780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.675+0000 7f95ab29d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2adb1b0 0x55fce2e87cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.676+0000 7f95ab29d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1b2f20 con 0x55fce2adb1b0 2024-09-21T03:52:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.676+0000 7f95ab29d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1b4a60 con 0x55fce2ab22f0 2024-09-21T03:52:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.676+0000 7f95ab29d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1b35f0 con 0x55fce2aba4d0 2024-09-21T03:52:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.676+0000 7f95a1800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2ab22f0 0x55fce2e87240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.676+0000 7f95a0e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2e87780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.677+0000 7f95a0e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2e87780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39118/0 (socket says 172.21.15.16:39118) 2024-09-21T03:52:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.677+0000 7f95a0e00640 1 -- 172.21.15.16:0/227657633 learned_addr learned my addr 172.21.15.16:0/227657633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.677+0000 7f95a1800640 1 -- 172.21.15.16:0/227657633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 msgr2=0x55fce2e87780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.677+0000 7f95a1800640 1 --2- 172.21.15.16:0/227657633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2e87780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.677+0000 7f95a1800640 1 -- 172.21.15.16:0/227657633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2adb1b0 msgr2=0x55fce2e87cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.678+0000 7f95a2200640 1 --2- 172.21.15.16:0/227657633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2adb1b0 0x55fce2e87cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.678+0000 7f95a1800640 1 --2- 172.21.15.16:0/227657633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2adb1b0 0x55fce2e87cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.678+0000 7f95a1800640 1 -- 172.21.15.16:0/227657633 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce2e9d900 con 0x55fce2ab22f0 2024-09-21T03:52:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.679+0000 7f95a1800640 1 --2- 172.21.15.16:0/227657633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2ab22f0 0x55fce2e87240 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55fce2ea9960 tx=0x7f958800a4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a44dddcb5714a04 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.679+0000 7f95a2200640 1 --2- 172.21.15.16:0/227657633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2adb1b0 0x55fce2e87cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.679+0000 7f958e000640 1 -- 172.21.15.16:0/227657633 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588020020 con 0x55fce2ab22f0 2024-09-21T03:52:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.679+0000 7f958e000640 1 -- 172.21.15.16:0/227657633 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f958800ae90 con 0x55fce2ab22f0 2024-09-21T03:52:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.679+0000 7f958e000640 1 -- 172.21.15.16:0/227657633 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958801cce0 con 0x55fce2ab22f0 2024-09-21T03:52:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.685+0000 7f95ab29d740 1 -- 172.21.15.16:0/227657633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2ab22f0 msgr2=0x55fce2e87240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.685+0000 7f95ab29d740 1 --2- 172.21.15.16:0/227657633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2ab22f0 0x55fce2e87240 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55fce2ea9960 tx=0x7f958800a4b0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.686+0000 7f95ab29d740 1 -- 172.21.15.16:0/227657633 shutdown_connections 2024-09-21T03:52:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.686+0000 7f95ab29d740 1 --2- 172.21.15.16:0/227657633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2adb1b0 0x55fce2e87cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.686+0000 7f95ab29d740 1 --2- 172.21.15.16:0/227657633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2e87780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.686+0000 7f95ab29d740 1 --2- 172.21.15.16:0/227657633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2ab22f0 0x55fce2e87240 secure :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55fce2ea9960 tx=0x7f958800a4b0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.686+0000 7f95ab29d740 1 -- 172.21.15.16:0/227657633 >> 172.21.15.16:0/227657633 conn(0x55fce2acf300 msgr2=0x55fce2e82680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.691+0000 7f95ab29d740 1 -- 172.21.15.16:0/227657633 shutdown_connections 2024-09-21T03:52:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.694+0000 7f95ab29d740 1 -- 172.21.15.16:0/227657633 wait complete. 2024-09-21T03:52:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.695+0000 7f95ab29d740 1 -- start start 2024-09-21T03:52:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.695+0000 7f95ab29d740 1 Processor -- start 2024-09-21T03:52:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.698+0000 7f95ab29d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2f0ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.698+0000 7f95ab29d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2adb1b0 0x55fce2f104c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.699+0000 7f95ab29d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2a6f1c0 0x55fce2f12be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.700+0000 7f95ab29d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1b4390 con 0x55fce2adb1b0 2024-09-21T03:52:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.700+0000 7f95a2200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2a6f1c0 0x55fce2f12be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.700+0000 7f95a2200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2a6f1c0 0x55fce2f12be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42252/0 (socket says 172.21.15.16:42252) 2024-09-21T03:52:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.700+0000 7f95a2200640 1 -- 172.21.15.16:0/714415697 learned_addr learned my addr 172.21.15.16:0/714415697 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.701+0000 7f95ab29d740 1 -- 172.21.15.16:0/714415697 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1b3cc0 con 0x55fce2a6f1c0 2024-09-21T03:52:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.701+0000 7f95ab29d740 1 -- 172.21.15.16:0/714415697 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f959c075040 con 0x55fce2aba4d0 2024-09-21T03:52:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.702+0000 7f95a2200640 1 -- 172.21.15.16:0/714415697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 msgr2=0x55fce2f0ddf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:52:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.702+0000 7f95a2200640 1 --2- 172.21.15.16:0/714415697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2f0ddf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.702+0000 7f95a2200640 1 -- 172.21.15.16:0/714415697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2adb1b0 msgr2=0x55fce2f104c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:52:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.702+0000 7f95a2200640 1 --2- 172.21.15.16:0/714415697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2adb1b0 0x55fce2f104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.702+0000 7f95a2200640 1 -- 172.21.15.16:0/714415697 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce2f13370 con 0x55fce2a6f1c0 2024-09-21T03:52:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.704+0000 7f95ab29d740 10 client.0 _ll_register_callbacks cb 0x7f959410cd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.704+0000 7f95ab29d740 1 -- 172.21.15.16:0/714415697 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f958800ae90 con 0x55fce2a6f1c0 2024-09-21T03:52:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.711+0000 7f95a2200640 1 --2- 172.21.15.16:0/714415697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2a6f1c0 0x55fce2f12be0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f959000cf90 tx=0x7f95900047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b2ec19788953c4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.711+0000 7f9583400640 10 client.26246 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:52:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.712+0000 7f9583400640 1 -- 172.21.15.16:0/714415697 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959001e7f0 con 0x55fce2a6f1c0 2024-09-21T03:52:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.716+0000 7f9583400640 1 -- 172.21.15.16:0/714415697 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f959001e990 con 0x55fce2a6f1c0 2024-09-21T03:52:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.717+0000 7f9583400640 1 -- 172.21.15.16:0/714415697 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f959001eb90 con 0x55fce2a6f1c0 2024-09-21T03:52:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.718+0000 7f95ab29d740 10 client.26246 fetch_fsmap learned FSMap version 30 2024-09-21T03:52:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.718+0000 7f95ab29d740 1 -- 172.21.15.16:0/714415697 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55fce2f14510 con 0x55fce2a6f1c0 2024-09-21T03:52:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.721+0000 7f9583400640 1 -- 172.21.15.16:0/714415697 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f959001f480 con 0x55fce2a6f1c0 2024-09-21T03:52:37.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.721+0000 7f95ab29d740 10 client.26246 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:52:37.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.721+0000 7f95ab29d740 10 client.26246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:37.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.721+0000 7f95ab29d740 20 client.26246 populate_metadata read hostname 'smithi016' 2024-09-21T03:52:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.721+0000 7f95ab29d740 1 --2- 172.21.15.16:0/714415697 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55fce2f14e30 0x55fce2f17340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.721+0000 7f95ab29d740 4 client.26246 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:52:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.721+0000 7f95ab29d740 1 -- 172.21.15.16:0/714415697 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fce2df7600 con 0x55fce2f14e30 2024-09-21T03:52:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.723+0000 7f95a1800640 1 --2- 172.21.15.16:0/714415697 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55fce2f14e30 0x55fce2f17340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.724+0000 7f95a1800640 1 --2- 172.21.15.16:0/714415697 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55fce2f14e30 0x55fce2f17340 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb9d48b9b7b207ff server_cookie=64b8a0041741fcb7 in_seq=0 out_seq=0 2024-09-21T03:52:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:37.724+0000 7f9583400640 10 client.26246 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.080+0000 7f9583400640 1 -- 172.21.15.16:0/714415697 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fce2df7600 con 0x55fce2f14e30 2024-09-21T03:52:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.080+0000 7f9583400640 10 client.26246 handle_command_reply: tid=0 2024-09-21T03:52:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.092+0000 7f95ab29d740 1 client.26246 shutdown 2024-09-21T03:52:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.093+0000 7f95ab29d740 1 -- 172.21.15.16:0/714415697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2a6f1c0 msgr2=0x55fce2f12be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.093+0000 7f95ab29d740 1 --2- 172.21.15.16:0/714415697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2a6f1c0 0x55fce2f12be0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f959000cf90 tx=0x7f95900047d0 comp rx=0 tx=0).stop 2024-09-21T03:52:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.095+0000 7f95ab29d740 1 -- 172.21.15.16:0/714415697 shutdown_connections 2024-09-21T03:52:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.095+0000 7f95ab29d740 1 --2- 172.21.15.16:0/714415697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fce2a6f1c0 0x55fce2f12be0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.095+0000 7f95ab29d740 1 --2- 172.21.15.16:0/714415697 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55fce2f14e30 0x55fce2f17340 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.095+0000 7f95ab29d740 1 --2- 172.21.15.16:0/714415697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fce2adb1b0 0x55fce2f104c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.095+0000 7f95ab29d740 1 --2- 172.21.15.16:0/714415697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fce2aba4d0 0x55fce2f0ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.095+0000 7f95ab29d740 1 -- 172.21.15.16:0/714415697 >> 172.21.15.16:0/714415697 conn(0x55fce2acf300 msgr2=0x55fce2e9b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.095+0000 7f9583400640 0 client.26246 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.096+0000 7f95ab29d740 1 -- 172.21.15.16:0/714415697 shutdown_connections 2024-09-21T03:52:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.096+0000 7f95ab29d740 1 -- 172.21.15.16:0/714415697 wait complete. 2024-09-21T03:52:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.097+0000 7f95ab29d740 20 client.26246 trim_cache size 0 max 16384 2024-09-21T03:52:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:52:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:52:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:52:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:52:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.098+0000 7f95a4200640 1 -- 172.21.15.16:0/3200252144 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f95840b6660 msgr2=0x7f95840b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.098+0000 7f95a4200640 1 --2- 172.21.15.16:0/3200252144 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f95840b6660 0x7f95840b8b70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f959000a8d0 tx=0x7f9590008040 comp rx=0 tx=0).stop 2024-09-21T03:52:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.098+0000 7f95a4200640 1 -- 172.21.15.16:0/3200252144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f959c104740 msgr2=0x7f959c113810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.098+0000 7f95a4200640 1 --2- 172.21.15.16:0/3200252144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f959c104740 0x7f959c113810 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f959402ade0 tx=0x7f9594003ba0 comp rx=0 tx=0).stop 2024-09-21T03:52:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.099+0000 7f95a4200640 1 -- 172.21.15.16:0/3200252144 shutdown_connections 2024-09-21T03:52:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.099+0000 7f95a4200640 1 --2- 172.21.15.16:0/3200252144 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f95840b6660 0x7f95840b8b70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.099+0000 7f95a4200640 1 --2- 172.21.15.16:0/3200252144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f959c1050e0 0x7f959c114340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.099+0000 7f95a4200640 1 --2- 172.21.15.16:0/3200252144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f959c104740 0x7f959c113810 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.099+0000 7f95a4200640 1 --2- 172.21.15.16:0/3200252144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f959c1036c0 0x7f959c1131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.099+0000 7f95a4200640 1 -- 172.21.15.16:0/3200252144 >> 172.21.15.16:0/3200252144 conn(0x7f959c0fefe0 msgr2=0x7f959c107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.099+0000 7f95a4200640 1 -- 172.21.15.16:0/3200252144 shutdown_connections 2024-09-21T03:52:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:38.099+0000 7f95a4200640 1 -- 172.21.15.16:0/3200252144 wait complete. 2024-09-21T03:52:39.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:39 smithi134 ceph-mon[35031]: pgmap v608: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 228 MiB/s wr, 1.72k op/s 2024-09-21T03:52:39.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:39 smithi123 ceph-mon[34673]: pgmap v608: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 228 MiB/s wr, 1.72k op/s 2024-09-21T03:52:39.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:38 smithi016 ceph-mon[30375]: pgmap v608: 129 pgs: 129 active+clean; 10 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 228 MiB/s wr, 1.72k op/s 2024-09-21T03:52:39.604 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2e29e1f3', total seconds elapsed: 31.9 2024-09-21T03:52:39.604 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.25 2024-09-21T03:52:39.605 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:52:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:41.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:41 smithi134 ceph-mon[35031]: pgmap v609: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 214 MiB/s wr, 1.75k op/s 2024-09-21T03:52:41.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:41 smithi123 ceph-mon[34673]: pgmap v609: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 214 MiB/s wr, 1.75k op/s 2024-09-21T03:52:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:41 smithi016 ceph-mon[30375]: pgmap v609: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 214 MiB/s wr, 1.75k op/s 2024-09-21T03:52:43.331 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:43 smithi016 ceph-mon[30375]: pgmap v610: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 207 MiB/s wr, 1.78k op/s 2024-09-21T03:52:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:43 smithi134 ceph-mon[35031]: pgmap v610: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 207 MiB/s wr, 1.78k op/s 2024-09-21T03:52:43.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:43 smithi123 ceph-mon[34673]: pgmap v610: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 207 MiB/s wr, 1.78k op/s 2024-09-21T03:52:45.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.247+0000 7fb3aac00640 1 -- 172.21.15.16:0/2529492130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb39c0956b0 msgr2=0x7fb39c099be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.247+0000 7fb3aac00640 1 --2- 172.21.15.16:0/2529492130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb39c0956b0 0x7fb39c099be0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb390009930 tx=0x7fb39002a420 comp rx=0 tx=0).stop 2024-09-21T03:52:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.248+0000 7fb3aac00640 1 -- 172.21.15.16:0/2529492130 shutdown_connections 2024-09-21T03:52:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.249+0000 7fb3aac00640 1 --2- 172.21.15.16:0/2529492130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb39c09a1a0 0x7fb39c09a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.249+0000 7fb3aac00640 1 --2- 172.21.15.16:0/2529492130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb39c0956b0 0x7fb39c099be0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.249+0000 7fb3aac00640 1 --2- 172.21.15.16:0/2529492130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb39c094d60 0x7fb39c095170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.249+0000 7fb3aac00640 1 -- 172.21.15.16:0/2529492130 >> 172.21.15.16:0/2529492130 conn(0x7fb39c00cd70 msgr2=0x7fb39c00d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.251+0000 7fb3aac00640 1 -- 172.21.15.16:0/2529492130 shutdown_connections 2024-09-21T03:52:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.251+0000 7fb3aac00640 1 -- 172.21.15.16:0/2529492130 wait complete. 2024-09-21T03:52:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.253+0000 7fb3aac00640 1 Processor -- start 2024-09-21T03:52:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.254+0000 7fb3aac00640 1 -- start start 2024-09-21T03:52:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.258+0000 7fb3aac00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb39c094d60 0x7fb39c0a92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.258+0000 7fb3aac00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb39c0956b0 0x7fb39c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.258+0000 7fb3aac00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb39c09a1a0 0x7fb39c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.258+0000 7fb3aac00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c0ac6d0 con 0x7fb39c094d60 2024-09-21T03:52:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.258+0000 7fb3aac00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c0ac550 con 0x7fb39c0956b0 2024-09-21T03:52:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.258+0000 7fb3aac00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c0ac850 con 0x7fb39c09a1a0 2024-09-21T03:52:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.259+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb39c094d60 0x7fb39c0a92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.259+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb39c094d60 0x7fb39c0a92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45410/0 (socket says 172.21.15.16:45410) 2024-09-21T03:52:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.259+0000 7fb3a3e00640 1 -- 172.21.15.16:0/644194342 learned_addr learned my addr 172.21.15.16:0/644194342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.259+0000 7fb3a3400640 1 --2- 172.21.15.16:0/644194342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb39c0956b0 0x7fb39c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.260+0000 7fb3a3e00640 1 -- 172.21.15.16:0/644194342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb39c09a1a0 msgr2=0x7fb39c0a2b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:52:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.260+0000 7fb3a3e00640 1 --2- 172.21.15.16:0/644194342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb39c09a1a0 0x7fb39c0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.260+0000 7fb3a3e00640 1 -- 172.21.15.16:0/644194342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb39c0956b0 msgr2=0x7fb39c0a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.260+0000 7fb3a3e00640 1 --2- 172.21.15.16:0/644194342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb39c0956b0 0x7fb39c0a2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.260+0000 7fb3a3e00640 1 -- 172.21.15.16:0/644194342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39c0a3370 con 0x7fb39c094d60 2024-09-21T03:52:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.261+0000 7fb3a3e00640 1 --2- 172.21.15.16:0/644194342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb39c094d60 0x7fb39c0a92b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb39800b4a0 tx=0x7fb39800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.264+0000 7fb3a0c00640 1 -- 172.21.15.16:0/644194342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398014070 con 0x7fb39c094d60 2024-09-21T03:52:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.264+0000 7fb3a0c00640 1 -- 172.21.15.16:0/644194342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb398010ac0 con 0x7fb39c094d60 2024-09-21T03:52:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.264+0000 7fb3a0c00640 1 -- 172.21.15.16:0/644194342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398017be0 con 0x7fb39c094d60 2024-09-21T03:52:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.264+0000 7fb3aac00640 1 -- 172.21.15.16:0/644194342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39c0a8040 con 0x7fb39c094d60 2024-09-21T03:52:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.265+0000 7fb3aac00640 1 -- 172.21.15.16:0/644194342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb39c0a8580 con 0x7fb39c094d60 2024-09-21T03:52:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.269+0000 7fb3aac00640 1 -- 172.21.15.16:0/644194342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb39c004e80 con 0x7fb39c094d60 2024-09-21T03:52:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.272+0000 7fb3a0c00640 1 -- 172.21.15.16:0/644194342 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb398017070 con 0x7fb39c094d60 2024-09-21T03:52:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.275+0000 7fb3a0c00640 1 --2- 172.21.15.16:0/644194342 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb38c0b6720 0x7fb38c0b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.275+0000 7fb3a0c00640 1 -- 172.21.15.16:0/644194342 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb3980e6910 con 0x7fb39c094d60 2024-09-21T03:52:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.276+0000 7fb3a3400640 1 --2- 172.21.15.16:0/644194342 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb38c0b6720 0x7fb38c0b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.278+0000 7fb3a3400640 1 --2- 172.21.15.16:0/644194342 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb38c0b6720 0x7fb38c0b8c30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb390009850 tx=0x7fb390031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.282+0000 7fb3a0c00640 1 -- 172.21.15.16:0/644194342 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb3980acd90 con 0x7fb39c094d60 2024-09-21T03:52:45.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:45 smithi016 ceph-mon[30375]: pgmap v611: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 211 MiB/s wr, 1.85k op/s 2024-09-21T03:52:45.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:45 smithi134 ceph-mon[35031]: pgmap v611: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 211 MiB/s wr, 1.85k op/s 2024-09-21T03:52:45.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:45 smithi123 ceph-mon[34673]: pgmap v611: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 211 MiB/s wr, 1.85k op/s 2024-09-21T03:52:45.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.733+0000 7fb3aac00640 1 -- 172.21.15.16:0/644194342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb39c099be0 con 0x7fb39c094d60 2024-09-21T03:52:45.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.737+0000 7fb3a0c00640 1 -- 172.21.15.16:0/644194342 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7fb3980b1170 con 0x7fb39c094d60 2024-09-21T03:52:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:52:45.742 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:52:45.742 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:52:45.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.744+0000 7fb3aac00640 1 -- 172.21.15.16:0/644194342 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb38c0b6720 msgr2=0x7fb38c0b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:45.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.744+0000 7fb3aac00640 1 --2- 172.21.15.16:0/644194342 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb38c0b6720 0x7fb38c0b8c30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb390009850 tx=0x7fb390031400 comp rx=0 tx=0).stop 2024-09-21T03:52:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.744+0000 7fb3aac00640 1 -- 172.21.15.16:0/644194342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb39c094d60 msgr2=0x7fb39c0a92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.744+0000 7fb3aac00640 1 --2- 172.21.15.16:0/644194342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb39c094d60 0x7fb39c0a92b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb39800b4a0 tx=0x7fb39800b960 comp rx=0 tx=0).stop 2024-09-21T03:52:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.744+0000 7fb3aac00640 1 -- 172.21.15.16:0/644194342 shutdown_connections 2024-09-21T03:52:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.744+0000 7fb3aac00640 1 --2- 172.21.15.16:0/644194342 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb38c0b6720 0x7fb38c0b8c30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.744+0000 7fb3aac00640 1 --2- 172.21.15.16:0/644194342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb39c09a1a0 0x7fb39c0a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.744+0000 7fb3aac00640 1 --2- 172.21.15.16:0/644194342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb39c0956b0 0x7fb39c0a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.744+0000 7fb3aac00640 1 --2- 172.21.15.16:0/644194342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb39c094d60 0x7fb39c0a92b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.744+0000 7fb3aac00640 1 -- 172.21.15.16:0/644194342 >> 172.21.15.16:0/644194342 conn(0x7fb39c00cd70 msgr2=0x7fb39c097bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.745+0000 7fb3aac00640 1 -- 172.21.15.16:0/644194342 shutdown_connections 2024-09-21T03:52:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:45.745+0000 7fb3aac00640 1 -- 172.21.15.16:0/644194342 wait complete. 2024-09-21T03:52:46.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:46 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/644194342' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:46.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:46 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/644194342' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:46.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:46 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/644194342' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:52:47.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:47 smithi016 ceph-mon[30375]: pgmap v612: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 202 MiB/s wr, 1.78k op/s 2024-09-21T03:52:47.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:47 smithi134 ceph-mon[35031]: pgmap v612: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 202 MiB/s wr, 1.78k op/s 2024-09-21T03:52:47.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:47 smithi123 ceph-mon[34673]: pgmap v612: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 202 MiB/s wr, 1.78k op/s 2024-09-21T03:52:48.428 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0' 2024-09-21T03:52:48.428 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0 2024-09-21T03:52:49.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:49 smithi134 ceph-mon[35031]: pgmap v613: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 190 MiB/s wr, 1.64k op/s 2024-09-21T03:52:49.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:49 smithi123 ceph-mon[34673]: pgmap v613: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 190 MiB/s wr, 1.64k op/s 2024-09-21T03:52:49.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:49 smithi016 ceph-mon[30375]: pgmap v613: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 190 MiB/s wr, 1.64k op/s 2024-09-21T03:52:49.454 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:52:51.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:51 smithi134 ceph-mon[35031]: pgmap v614: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 187 MiB/s wr, 1.55k op/s 2024-09-21T03:52:51.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:51 smithi123 ceph-mon[34673]: pgmap v614: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 187 MiB/s wr, 1.55k op/s 2024-09-21T03:52:51.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:51 smithi016 ceph-mon[30375]: pgmap v614: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 187 MiB/s wr, 1.55k op/s 2024-09-21T03:52:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.528+0000 7fdeaf200640 1 -- 172.21.15.16:0/3996050595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb0106760 msgr2=0x7fdeb0106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.528+0000 7fdeaf200640 1 --2- 172.21.15.16:0/3996050595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb0106760 0x7fdeb0106b70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009900 tx=0x7fdea002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.528+0000 7fdeaf200640 1 -- 172.21.15.16:0/3996050595 shutdown_connections 2024-09-21T03:52:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.528+0000 7fdeaf200640 1 --2- 172.21.15.16:0/3996050595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeb0107100 0x7fdeb010daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.528+0000 7fdeaf200640 1 --2- 172.21.15.16:0/3996050595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb0106760 0x7fdeb0106b70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.528+0000 7fdeaf200640 1 --2- 172.21.15.16:0/3996050595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeb01056e0 0x7fdeb0105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.528+0000 7fdeaf200640 1 -- 172.21.15.16:0/3996050595 >> 172.21.15.16:0/3996050595 conn(0x7fdeb0101000 msgr2=0x7fdeb01034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.528+0000 7fdeaf200640 1 -- 172.21.15.16:0/3996050595 shutdown_connections 2024-09-21T03:52:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.528+0000 7fdeaf200640 1 -- 172.21.15.16:0/3996050595 wait complete. 2024-09-21T03:52:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.529+0000 7fdeaf200640 1 Processor -- start 2024-09-21T03:52:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.529+0000 7fdeaf200640 1 -- start start 2024-09-21T03:52:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.530+0000 7fdeaf200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeb01056e0 0x7fdeb019bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.530+0000 7fdeaf200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeb0106760 0x7fdeb019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.530+0000 7fdeaf200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb0107100 0x7fdeb019d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.530+0000 7fdeaf200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb0111360 con 0x7fdeb0106760 2024-09-21T03:52:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.530+0000 7fdeaf200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01111e0 con 0x7fdeb0107100 2024-09-21T03:52:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.530+0000 7fdeaf200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01114e0 con 0x7fdeb01056e0 2024-09-21T03:52:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.530+0000 7fdeade00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeb01056e0 0x7fdeb019bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.530+0000 7fdeade00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeb01056e0 0x7fdeb019bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:57746/0 (socket says 172.21.15.16:57746) 2024-09-21T03:52:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.530+0000 7fdeae800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb0107100 0x7fdeb019d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.530+0000 7fdeade00640 1 -- 172.21.15.16:0/2835124975 learned_addr learned my addr 172.21.15.16:0/2835124975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.530+0000 7fdead400640 1 --2- 172.21.15.16:0/2835124975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeb0106760 0x7fdeb019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.530+0000 7fdeade00640 1 -- 172.21.15.16:0/2835124975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb0107100 msgr2=0x7fdeb019d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.530+0000 7fdeade00640 1 --2- 172.21.15.16:0/2835124975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb0107100 0x7fdeb019d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.530+0000 7fdeade00640 1 -- 172.21.15.16:0/2835124975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeb0106760 msgr2=0x7fdeb019c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.530+0000 7fdeade00640 1 --2- 172.21.15.16:0/2835124975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeb0106760 0x7fdeb019c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.530+0000 7fdeade00640 1 -- 172.21.15.16:0/2835124975 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb019d9c0 con 0x7fdeb01056e0 2024-09-21T03:52:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.530+0000 7fdeae800640 1 --2- 172.21.15.16:0/2835124975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb0107100 0x7fdeb019d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:52:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.531+0000 7fdeade00640 1 --2- 172.21.15.16:0/2835124975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeb01056e0 0x7fdeb019bf80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdea800ebc0 tx=0x7fdea800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.531+0000 7fdea6a00640 1 -- 172.21.15.16:0/2835124975 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8010630 con 0x7fdeb01056e0 2024-09-21T03:52:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.531+0000 7fdeaf200640 1 -- 172.21.15.16:0/2835124975 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb019dcb0 con 0x7fdeb01056e0 2024-09-21T03:52:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.531+0000 7fdea6a00640 1 -- 172.21.15.16:0/2835124975 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdea8010c80 con 0x7fdeb01056e0 2024-09-21T03:52:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.531+0000 7fdea6a00640 1 -- 172.21.15.16:0/2835124975 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8018b60 con 0x7fdeb01056e0 2024-09-21T03:52:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.531+0000 7fdeaf200640 1 -- 172.21.15.16:0/2835124975 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb01a43a0 con 0x7fdeb01056e0 2024-09-21T03:52:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.534+0000 7fdebc200740 1 Processor -- start 2024-09-21T03:52:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.535+0000 7fdebc200740 1 -- start start 2024-09-21T03:52:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.535+0000 7fdea6a00640 1 -- 172.21.15.16:0/2835124975 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fdea8018d00 con 0x7fdeb01056e0 2024-09-21T03:52:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.535+0000 7fdebc200740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9afe1b0 0x5568b9ea52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.535+0000 7fdebc200740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9ad52f0 0x5568b9ea5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.535+0000 7fdebc200740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9add4d0 0x5568b9ea6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.535+0000 7fdebc200740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5568b9e186b0 con 0x5568b9afe1b0 2024-09-21T03:52:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.535+0000 7fdebc200740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5568b9e07ae0 con 0x5568b9add4d0 2024-09-21T03:52:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.535+0000 7fdebc200740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5568b9e0b8f0 con 0x5568b9ad52f0 2024-09-21T03:52:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.535+0000 7fdead400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9add4d0 0x5568b9ea6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.535+0000 7fdeade00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9ad52f0 0x5568b9ea5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.535+0000 7fdead400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9add4d0 0x5568b9ea6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42750/0 (socket says 172.21.15.16:42750) 2024-09-21T03:52:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.535+0000 7fdead400640 1 -- 172.21.15.16:0/3476497982 learned_addr learned my addr 172.21.15.16:0/3476497982 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.535+0000 7fdeae800640 1 --2- 172.21.15.16:0/3476497982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9afe1b0 0x5568b9ea52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.536+0000 7fdead400640 1 -- 172.21.15.16:0/3476497982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9ad52f0 msgr2=0x5568b9ea5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.536+0000 7fdead400640 1 --2- 172.21.15.16:0/3476497982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9ad52f0 0x5568b9ea5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.536+0000 7fdead400640 1 -- 172.21.15.16:0/3476497982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9afe1b0 msgr2=0x5568b9ea52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.536+0000 7fdead400640 1 --2- 172.21.15.16:0/3476497982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9afe1b0 0x5568b9ea52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.536+0000 7fdead400640 1 -- 172.21.15.16:0/3476497982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b9dd4780 con 0x5568b9add4d0 2024-09-21T03:52:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.536+0000 7fdeae800640 1 --2- 172.21.15.16:0/3476497982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9afe1b0 0x5568b9ea52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.536+0000 7fdead400640 1 --2- 172.21.15.16:0/3476497982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9add4d0 0x5568b9ea6470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009230 tx=0x7fdea00040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e558de30c566c8e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.536+0000 7fde9f400640 1 -- 172.21.15.16:0/3476497982 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0035030 con 0x5568b9add4d0 2024-09-21T03:52:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.536+0000 7fde9f400640 1 -- 172.21.15.16:0/3476497982 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdea0040070 con 0x5568b9add4d0 2024-09-21T03:52:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.536+0000 7fde9f400640 1 -- 172.21.15.16:0/3476497982 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0031920 con 0x5568b9add4d0 2024-09-21T03:52:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.537+0000 7fdea6a00640 1 --2- 172.21.15.16:0/2835124975 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fde940b6510 0x7fde940b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.537+0000 7fdea6a00640 1 -- 172.21.15.16:0/2835124975 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fdea80e7cd0 con 0x7fdeb01056e0 2024-09-21T03:52:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.537+0000 7fdeae800640 1 --2- 172.21.15.16:0/2835124975 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fde940b6510 0x7fde940b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.537+0000 7fdebc200740 1 -- 172.21.15.16:0/3476497982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9add4d0 msgr2=0x5568b9ea6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.538+0000 7fdebc200740 1 --2- 172.21.15.16:0/3476497982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9add4d0 0x5568b9ea6470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009230 tx=0x7fdea00040c0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.538+0000 7fdebc200740 1 -- 172.21.15.16:0/3476497982 shutdown_connections 2024-09-21T03:52:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.538+0000 7fdebc200740 1 --2- 172.21.15.16:0/3476497982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9add4d0 0x5568b9ea6470 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.538+0000 7fdebc200740 1 --2- 172.21.15.16:0/3476497982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9ad52f0 0x5568b9ea5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.538+0000 7fdebc200740 1 --2- 172.21.15.16:0/3476497982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9afe1b0 0x5568b9ea52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.538+0000 7fdebc200740 1 -- 172.21.15.16:0/3476497982 >> 172.21.15.16:0/3476497982 conn(0x5568b9af2300 msgr2=0x5568b9ea0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.538+0000 7fdebc200740 1 -- 172.21.15.16:0/3476497982 shutdown_connections 2024-09-21T03:52:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.538+0000 7fdebc200740 1 -- 172.21.15.16:0/3476497982 wait complete. 2024-09-21T03:52:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.538+0000 7fdeae800640 1 --2- 172.21.15.16:0/2835124975 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fde940b6510 0x7fde940b8a20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fde98005fd0 tx=0x7fde98005cc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.539+0000 7fdebc200740 1 -- start start 2024-09-21T03:52:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.539+0000 7fdebc200740 1 Processor -- start 2024-09-21T03:52:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.539+0000 7fdebc200740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9ad52f0 0x5568b9eae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.539+0000 7fdebc200740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9add4d0 0x5568b9eaefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.539+0000 7fdebc200740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9afe1b0 0x5568b9eafb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.539+0000 7fdebc200740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5568b9d3fab0 con 0x5568b9add4d0 2024-09-21T03:52:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.539+0000 7fdebc200740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5568b9dae570 con 0x5568b9ad52f0 2024-09-21T03:52:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.539+0000 7fdebc200740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5568b9ea6f30 con 0x5568b9afe1b0 2024-09-21T03:52:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.539+0000 7fdebc200740 10 client.0 _ll_register_callbacks cb 0x5568b9dbd100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.540+0000 7fdeade00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9ad52f0 0x5568b9eae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.540+0000 7fdeade00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9ad52f0 0x5568b9eae910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42752/0 (socket says 172.21.15.16:42752) 2024-09-21T03:52:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.540+0000 7fdeade00640 1 -- 172.21.15.16:0/1047616145 learned_addr learned my addr 172.21.15.16:0/1047616145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.540+0000 7fdeae800640 1 --2- 172.21.15.16:0/1047616145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9afe1b0 0x5568b9eafb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.540+0000 7fdead400640 1 --2- 172.21.15.16:0/1047616145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9add4d0 0x5568b9eaefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.540+0000 7fdeade00640 1 -- 172.21.15.16:0/1047616145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9afe1b0 msgr2=0x5568b9eafb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.540+0000 7fdeade00640 1 --2- 172.21.15.16:0/1047616145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9afe1b0 0x5568b9eafb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.540+0000 7fdeade00640 1 -- 172.21.15.16:0/1047616145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9add4d0 msgr2=0x5568b9eaefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.540+0000 7fdeade00640 1 --2- 172.21.15.16:0/1047616145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9add4d0 0x5568b9eaefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.540+0000 7fdeade00640 1 -- 172.21.15.16:0/1047616145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b9b69790 con 0x5568b9ad52f0 2024-09-21T03:52:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.540+0000 7fdead400640 1 --2- 172.21.15.16:0/1047616145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9add4d0 0x5568b9eaefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.540+0000 7fdeade00640 1 --2- 172.21.15.16:0/1047616145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9ad52f0 0x5568b9eae910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdea80ada90 tx=0x7fdea8010ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6263501242b0085e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.540+0000 7fde93e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:52:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.541+0000 7fde93e00640 1 -- 172.21.15.16:0/1047616145 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea80aa150 con 0x5568b9ad52f0 2024-09-21T03:52:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.541+0000 7fde93e00640 1 -- 172.21.15.16:0/1047616145 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fdea80ac7d0 con 0x5568b9ad52f0 2024-09-21T03:52:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.541+0000 7fdebc200740 1 -- 172.21.15.16:0/1047616145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea0040070 con 0x5568b9ad52f0 2024-09-21T03:52:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.541+0000 7fde93e00640 1 -- 172.21.15.16:0/1047616145 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdea80a9270 con 0x5568b9ad52f0 2024-09-21T03:52:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.541+0000 7fdebc200740 10 client.26264 fetch_fsmap learned FSMap version 30 2024-09-21T03:52:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.541+0000 7fdebc200740 1 -- 172.21.15.16:0/1047616145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5568b9be2620 con 0x5568b9ad52f0 2024-09-21T03:52:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.541+0000 7fde93e00640 1 -- 172.21.15.16:0/1047616145 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdea0040070 con 0x5568b9ad52f0 2024-09-21T03:52:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.541+0000 7fde93e00640 1 -- 172.21.15.16:0/1047616145 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdea80ac9d0 con 0x5568b9ad52f0 2024-09-21T03:52:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.541+0000 7fdebc200740 10 client.26264 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:52:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.541+0000 7fdebc200740 10 client.26264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.542+0000 7fdebc200740 20 client.26264 populate_metadata read hostname 'smithi016' 2024-09-21T03:52:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.542+0000 7fdebc200740 1 --2- 172.21.15.16:0/1047616145 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5568b9abcd80 0x5568b9ebc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.542+0000 7fdebc200740 4 client.26264 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.542+0000 7fdebc200740 1 -- 172.21.15.16:0/1047616145 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568b9e1a600 con 0x5568b9abcd80 2024-09-21T03:52:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.542+0000 7fdead400640 1 --2- 172.21.15.16:0/1047616145 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5568b9abcd80 0x5568b9ebc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.542+0000 7fdead400640 1 --2- 172.21.15.16:0/1047616145 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5568b9abcd80 0x5568b9ebc0b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=238665308aa1b942 server_cookie=8cc4837b40184e7d in_seq=0 out_seq=0 2024-09-21T03:52:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.543+0000 7fde93e00640 10 client.26264 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.543+0000 7fde93e00640 1 -- 172.21.15.16:0/1047616145 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5568b9e1a600 con 0x5568b9abcd80 2024-09-21T03:52:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.543+0000 7fde93e00640 10 client.26264 handle_command_reply: tid=0 2024-09-21T03:52:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.543+0000 7fdebc200740 1 client.26264 shutdown 2024-09-21T03:52:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.544+0000 7fdebc200740 1 -- 172.21.15.16:0/1047616145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9ad52f0 msgr2=0x5568b9eae910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.544+0000 7fdebc200740 1 --2- 172.21.15.16:0/1047616145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9ad52f0 0x5568b9eae910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdea80ada90 tx=0x7fdea8010ba0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.544+0000 7fdebc200740 1 -- 172.21.15.16:0/1047616145 shutdown_connections 2024-09-21T03:52:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.544+0000 7fdebc200740 1 --2- 172.21.15.16:0/1047616145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9afe1b0 0x5568b9eafb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.544+0000 7fdebc200740 1 --2- 172.21.15.16:0/1047616145 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5568b9abcd80 0x5568b9ebc0b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.544+0000 7fdebc200740 1 --2- 172.21.15.16:0/1047616145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9add4d0 0x5568b9eaefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.544+0000 7fdebc200740 1 --2- 172.21.15.16:0/1047616145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9ad52f0 0x5568b9eae910 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.544+0000 7fde93e00640 0 client.26264 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.544+0000 7fdebc200740 1 -- 172.21.15.16:0/1047616145 >> 172.21.15.16:0/1047616145 conn(0x5568b9af2300 msgr2=0x5568b9e9f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.544+0000 7fdebc200740 1 -- 172.21.15.16:0/1047616145 shutdown_connections 2024-09-21T03:52:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.544+0000 7fdebc200740 1 -- 172.21.15.16:0/1047616145 wait complete. 2024-09-21T03:52:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.544+0000 7fdebc200740 20 client.26264 trim_cache size 0 max 16384 2024-09-21T03:52:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.561+0000 7fdebc200740 1 Processor -- start 2024-09-21T03:52:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.561+0000 7fdebc200740 1 -- start start 2024-09-21T03:52:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.561+0000 7fdebc200740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9ad52f0 0x5568b9ea17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.561+0000 7fdebc200740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9add4d0 0x5568b9ea1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.561+0000 7fdebc200740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9afe1b0 0x5568b9ea2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.561+0000 7fdebc200740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01ae770 con 0x5568b9ad52f0 2024-09-21T03:52:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.561+0000 7fdebc200740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01b02b0 con 0x5568b9add4d0 2024-09-21T03:52:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.561+0000 7fdebc200740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01aee40 con 0x5568b9afe1b0 2024-09-21T03:52:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.562+0000 7fdead400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9add4d0 0x5568b9ea1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.562+0000 7fdeae800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9afe1b0 0x5568b9ea2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.562+0000 7fdead400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9add4d0 0x5568b9ea1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42756/0 (socket says 172.21.15.16:42756) 2024-09-21T03:52:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.562+0000 7fdead400640 1 -- 172.21.15.16:0/3043073 learned_addr learned my addr 172.21.15.16:0/3043073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.562+0000 7fdeade00640 1 --2- 172.21.15.16:0/3043073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9ad52f0 0x5568b9ea17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.562+0000 7fdead400640 1 -- 172.21.15.16:0/3043073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9afe1b0 msgr2=0x5568b9ea2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.562+0000 7fdead400640 1 --2- 172.21.15.16:0/3043073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9afe1b0 0x5568b9ea2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.562+0000 7fdead400640 1 -- 172.21.15.16:0/3043073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9ad52f0 msgr2=0x5568b9ea17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.562+0000 7fdead400640 1 --2- 172.21.15.16:0/3043073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9ad52f0 0x5568b9ea17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.562+0000 7fdead400640 1 -- 172.21.15.16:0/3043073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b9eafeb0 con 0x5568b9add4d0 2024-09-21T03:52:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.562+0000 7fdead400640 1 --2- 172.21.15.16:0/3043073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9add4d0 0x5568b9ea1d20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdea00045d0 tx=0x7fdea0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f74c81ca228bb231 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.563+0000 7fde9ea00640 1 -- 172.21.15.16:0/3043073 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0035040 con 0x5568b9add4d0 2024-09-21T03:52:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.563+0000 7fde9ea00640 1 -- 172.21.15.16:0/3043073 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdea0040040 con 0x5568b9add4d0 2024-09-21T03:52:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.563+0000 7fde9ea00640 1 -- 172.21.15.16:0/3043073 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0031b30 con 0x5568b9add4d0 2024-09-21T03:52:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.564+0000 7fdebc200740 1 -- 172.21.15.16:0/3043073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9add4d0 msgr2=0x5568b9ea1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.564+0000 7fdebc200740 1 --2- 172.21.15.16:0/3043073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9add4d0 0x5568b9ea1d20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdea00045d0 tx=0x7fdea0000bc0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.564+0000 7fdebc200740 1 -- 172.21.15.16:0/3043073 shutdown_connections 2024-09-21T03:52:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.564+0000 7fdebc200740 1 --2- 172.21.15.16:0/3043073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9afe1b0 0x5568b9ea2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.564+0000 7fdebc200740 1 --2- 172.21.15.16:0/3043073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9add4d0 0x5568b9ea1d20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.564+0000 7fdebc200740 1 --2- 172.21.15.16:0/3043073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9ad52f0 0x5568b9ea17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.564+0000 7fdebc200740 1 -- 172.21.15.16:0/3043073 >> 172.21.15.16:0/3043073 conn(0x5568b9af2300 msgr2=0x5568b9ea6140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.564+0000 7fdebc200740 1 -- 172.21.15.16:0/3043073 shutdown_connections 2024-09-21T03:52:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.565+0000 7fdebc200740 1 -- 172.21.15.16:0/3043073 wait complete. 2024-09-21T03:52:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.565+0000 7fdebc200740 1 -- start start 2024-09-21T03:52:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.565+0000 7fdebc200740 1 Processor -- start 2024-09-21T03:52:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.566+0000 7fdebc200740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9ad52f0 0x5568b9eb67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.566+0000 7fdebc200740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9add4d0 0x5568b9eb54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.566+0000 7fdebc200740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9afe1b0 0x5568b9eb5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.566+0000 7fdebc200740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01afbe0 con 0x5568b9add4d0 2024-09-21T03:52:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.566+0000 7fdebc200740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01af510 con 0x5568b9ad52f0 2024-09-21T03:52:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.566+0000 7fdebc200740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01ac750 con 0x5568b9afe1b0 2024-09-21T03:52:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.566+0000 7fdebc200740 10 client.0 _ll_register_callbacks cb 0x7fdea80eb400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.567+0000 7fdeae800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9afe1b0 0x5568b9eb5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.567+0000 7fdeae800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9afe1b0 0x5568b9eb5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:57776/0 (socket says 172.21.15.16:57776) 2024-09-21T03:52:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.567+0000 7fdeae800640 1 -- 172.21.15.16:0/575055723 learned_addr learned my addr 172.21.15.16:0/575055723 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.567+0000 7fdead400640 1 --2- 172.21.15.16:0/575055723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9add4d0 0x5568b9eb54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.567+0000 7fdeade00640 1 --2- 172.21.15.16:0/575055723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9ad52f0 0x5568b9eb67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.567+0000 7fdeae800640 1 -- 172.21.15.16:0/575055723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9ad52f0 msgr2=0x5568b9eb67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.567+0000 7fdeae800640 1 --2- 172.21.15.16:0/575055723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9ad52f0 0x5568b9eb67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.567+0000 7fdeae800640 1 -- 172.21.15.16:0/575055723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9add4d0 msgr2=0x5568b9eb54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.568+0000 7fdeae800640 1 --2- 172.21.15.16:0/575055723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9add4d0 0x5568b9eb54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.568+0000 7fdeae800640 1 -- 172.21.15.16:0/575055723 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b9eb6270 con 0x5568b9afe1b0 2024-09-21T03:52:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.568+0000 7fdeade00640 1 --2- 172.21.15.16:0/575055723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9ad52f0 0x5568b9eb67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.568+0000 7fdead400640 1 --2- 172.21.15.16:0/575055723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9add4d0 0x5568b9eb54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:52:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.568+0000 7fdeae800640 1 --2- 172.21.15.16:0/575055723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9afe1b0 0x5568b9eb5b00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fde98010a60 tx=0x7fde98014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b6216037ba08207 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.569+0000 7fde93e00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:52:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.569+0000 7fde93e00640 1 -- 172.21.15.16:0/575055723 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9802e060 con 0x5568b9afe1b0 2024-09-21T03:52:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.569+0000 7fde93e00640 1 -- 172.21.15.16:0/575055723 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fde9801daa0 con 0x5568b9afe1b0 2024-09-21T03:52:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.569+0000 7fdebc200740 1 -- 172.21.15.16:0/575055723 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea0040040 con 0x5568b9afe1b0 2024-09-21T03:52:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.570+0000 7fde93e00640 1 -- 172.21.15.16:0/575055723 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fde9801dca0 con 0x5568b9afe1b0 2024-09-21T03:52:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.570+0000 7fdebc200740 10 client.26209 fetch_fsmap learned FSMap version 30 2024-09-21T03:52:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.570+0000 7fdebc200740 1 -- 172.21.15.16:0/575055723 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5568b9f26be0 con 0x5568b9afe1b0 2024-09-21T03:52:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.570+0000 7fde93e00640 1 -- 172.21.15.16:0/575055723 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdea0040040 con 0x5568b9afe1b0 2024-09-21T03:52:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.570+0000 7fde93e00640 1 -- 172.21.15.16:0/575055723 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fde9801c400 con 0x5568b9afe1b0 2024-09-21T03:52:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.571+0000 7fdebc200740 10 client.26209 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:52:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.571+0000 7fdebc200740 10 client.26209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:52:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.571+0000 7fdebc200740 20 client.26209 populate_metadata read hostname 'smithi016' 2024-09-21T03:52:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.571+0000 7fdebc200740 1 --2- 172.21.15.16:0/575055723 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5568b9a921c0 0x5568b9f295e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.571+0000 7fdebc200740 4 client.26209 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:52:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.571+0000 7fdebc200740 1 -- 172.21.15.16:0/575055723 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568b9e160b0 con 0x5568b9a921c0 2024-09-21T03:52:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.571+0000 7fdeade00640 1 --2- 172.21.15.16:0/575055723 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5568b9a921c0 0x5568b9f295e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.572+0000 7fde93e00640 10 client.26209 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.572+0000 7fdeade00640 1 --2- 172.21.15.16:0/575055723 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5568b9a921c0 0x5568b9f295e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3ce03544544ae16 server_cookie=f4b755dd8df2f7ae in_seq=0 out_seq=0 2024-09-21T03:52:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.590+0000 7fde93e00640 1 -- 172.21.15.16:0/575055723 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568b9e160b0 con 0x5568b9a921c0 2024-09-21T03:52:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.590+0000 7fde93e00640 10 client.26209 handle_command_reply: tid=0 2024-09-21T03:52:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.590+0000 7fdebc200740 1 client.26209 shutdown 2024-09-21T03:52:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.591+0000 7fdebc200740 1 -- 172.21.15.16:0/575055723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9afe1b0 msgr2=0x5568b9eb5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.591+0000 7fdebc200740 1 --2- 172.21.15.16:0/575055723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9afe1b0 0x5568b9eb5b00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fde98010a60 tx=0x7fde98014040 comp rx=0 tx=0).stop 2024-09-21T03:52:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.591+0000 7fdebc200740 1 -- 172.21.15.16:0/575055723 shutdown_connections 2024-09-21T03:52:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.591+0000 7fdebc200740 1 --2- 172.21.15.16:0/575055723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568b9afe1b0 0x5568b9eb5b00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.591+0000 7fdebc200740 1 --2- 172.21.15.16:0/575055723 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5568b9a921c0 0x5568b9f295e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.591+0000 7fdebc200740 1 --2- 172.21.15.16:0/575055723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b9add4d0 0x5568b9eb54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.591+0000 7fdebc200740 1 --2- 172.21.15.16:0/575055723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568b9ad52f0 0x5568b9eb67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.591+0000 7fdebc200740 1 -- 172.21.15.16:0/575055723 >> 172.21.15.16:0/575055723 conn(0x5568b9af2300 msgr2=0x5568b9ea9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.591+0000 7fde93e00640 0 client.26209 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.592+0000 7fdebc200740 1 -- 172.21.15.16:0/575055723 shutdown_connections 2024-09-21T03:52:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.592+0000 7fdebc200740 1 -- 172.21.15.16:0/575055723 wait complete. 2024-09-21T03:52:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.592+0000 7fdebc200740 20 client.26209 trim_cache size 0 max 16384 2024-09-21T03:52:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:52:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:52:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:52:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:52:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.593+0000 7fdeaf200640 1 -- 172.21.15.16:0/2835124975 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fde940b6510 msgr2=0x7fde940b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.593+0000 7fdeaf200640 1 --2- 172.21.15.16:0/2835124975 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fde940b6510 0x7fde940b8a20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fde98005fd0 tx=0x7fde98005cc0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.593+0000 7fdeaf200640 1 -- 172.21.15.16:0/2835124975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeb01056e0 msgr2=0x7fdeb019bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.593+0000 7fdeaf200640 1 --2- 172.21.15.16:0/2835124975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeb01056e0 0x7fdeb019bf80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdea800ebc0 tx=0x7fdea800c6a0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.593+0000 7fdeaf200640 1 -- 172.21.15.16:0/2835124975 shutdown_connections 2024-09-21T03:52:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.593+0000 7fdeaf200640 1 --2- 172.21.15.16:0/2835124975 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fde940b6510 0x7fde940b8a20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.593+0000 7fdeaf200640 1 --2- 172.21.15.16:0/2835124975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeb0107100 0x7fdeb019d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.593+0000 7fdeaf200640 1 --2- 172.21.15.16:0/2835124975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeb0106760 0x7fdeb019c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.593+0000 7fdeaf200640 1 --2- 172.21.15.16:0/2835124975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeb01056e0 0x7fdeb019bf80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.593+0000 7fdeaf200640 1 -- 172.21.15.16:0/2835124975 >> 172.21.15.16:0/2835124975 conn(0x7fdeb0101000 msgr2=0x7fdeb0109a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.593+0000 7fdeaf200640 1 -- 172.21.15.16:0/2835124975 shutdown_connections 2024-09-21T03:52:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:52:51.594+0000 7fdeaf200640 1 -- 172.21.15.16:0/2835124975 wait complete. 2024-09-21T03:52:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.101+0000 7f6a0c600640 1 -- 172.21.15.16:0/1771547450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a04078290 msgr2=0x7f6a04114200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.101+0000 7f6a0c600640 1 --2- 172.21.15.16:0/1771547450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a04078290 0x7f6a04114200 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009900 tx=0x7f69f802a410 comp rx=0 tx=0).stop 2024-09-21T03:52:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.102+0000 7f6a0c600640 1 -- 172.21.15.16:0/1771547450 shutdown_connections 2024-09-21T03:52:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.102+0000 7f6a0c600640 1 --2- 172.21.15.16:0/1771547450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a04114740 0x7f6a04116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.102+0000 7f6a0c600640 1 --2- 172.21.15.16:0/1771547450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a04078290 0x7f6a04114200 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.102+0000 7f6a0c600640 1 --2- 172.21.15.16:0/1771547450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a04077940 0x7f6a04077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.102+0000 7f6a0c600640 1 -- 172.21.15.16:0/1771547450 >> 172.21.15.16:0/1771547450 conn(0x7f6a041017d0 msgr2=0x7f6a04103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.102+0000 7f6a0c600640 1 -- 172.21.15.16:0/1771547450 shutdown_connections 2024-09-21T03:52:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.102+0000 7f6a0c600640 1 -- 172.21.15.16:0/1771547450 wait complete. 2024-09-21T03:52:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.104+0000 7f6a0c600640 1 Processor -- start 2024-09-21T03:52:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.104+0000 7f6a0c600640 1 -- start start 2024-09-21T03:52:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.105+0000 7f6a0c600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a04077940 0x7f6a041a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.105+0000 7f6a0c600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a04078290 0x7f6a041a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.105+0000 7f6a0c600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a04114740 0x7f6a041a6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.105+0000 7f6a0c600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0411a1d0 con 0x7f6a04077940 2024-09-21T03:52:52.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.105+0000 7f6a0c600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0411a050 con 0x7f6a04114740 2024-09-21T03:52:52.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.105+0000 7f6a0c600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0411a350 con 0x7f6a04078290 2024-09-21T03:52:52.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.105+0000 7f6a09200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a04078290 0x7f6a041a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:52.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.105+0000 7f6a09200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a04078290 0x7f6a041a11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34500/0 (socket says 172.21.15.16:34500) 2024-09-21T03:52:52.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.105+0000 7f6a09200640 1 -- 172.21.15.16:0/3117721621 learned_addr learned my addr 172.21.15.16:0/3117721621 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:52.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.106+0000 7f6a0a600640 1 --2- 172.21.15.16:0/3117721621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a04114740 0x7f6a041a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:52.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.106+0000 7f6a09200640 1 -- 172.21.15.16:0/3117721621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a04114740 msgr2=0x7f6a041a6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:52.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.106+0000 7f6a09200640 1 --2- 172.21.15.16:0/3117721621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a04114740 0x7f6a041a6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.106+0000 7f6a09200640 1 -- 172.21.15.16:0/3117721621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a04077940 msgr2=0x7f6a041a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:52.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.106+0000 7f6a09200640 1 --2- 172.21.15.16:0/3117721621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a04077940 0x7f6a041a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.106+0000 7f6a09200640 1 -- 172.21.15.16:0/3117721621 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a041a6b90 con 0x7f6a04078290 2024-09-21T03:52:52.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.108+0000 7f6a09200640 1 --2- 172.21.15.16:0/3117721621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a04078290 0x7f6a041a11c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f69f802aa00 tx=0x7f69f8003d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:52.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.109+0000 7f6a02000640 1 -- 172.21.15.16:0/3117721621 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f803f070 con 0x7f6a04078290 2024-09-21T03:52:52.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.109+0000 7f6a02000640 1 -- 172.21.15.16:0/3117721621 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69f8031d60 con 0x7f6a04078290 2024-09-21T03:52:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.110+0000 7f6a0c600640 1 -- 172.21.15.16:0/3117721621 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a041a6e20 con 0x7f6a04078290 2024-09-21T03:52:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.110+0000 7f6a02000640 1 -- 172.21.15.16:0/3117721621 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8035030 con 0x7f6a04078290 2024-09-21T03:52:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.110+0000 7f6a0c600640 1 -- 172.21.15.16:0/3117721621 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a041ad5e0 con 0x7f6a04078290 2024-09-21T03:52:52.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.116+0000 7f6a12f22740 1 Processor -- start 2024-09-21T03:52:52.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.117+0000 7f6a12f22740 1 -- start start 2024-09-21T03:52:52.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.124+0000 7f6a12f22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f214850 0x55d92f5bb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.124+0000 7f6a12f22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1eb990 0x55d92f5bbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.124+0000 7f6a02000640 1 -- 172.21.15.16:0/3117721621 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f69f8042070 con 0x7f6a04078290 2024-09-21T03:52:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.124+0000 7f6a0a600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f214850 0x55d92f5bb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.124+0000 7f6a0a600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f214850 0x55d92f5bb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34514/0 (socket says 172.21.15.16:34514) 2024-09-21T03:52:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.124+0000 7f6a0a600640 1 -- 172.21.15.16:0/1088692791 learned_addr learned my addr 172.21.15.16:0/1088692791 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.125+0000 7f6a12f22740 1 --2- 172.21.15.16:0/1088692791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f1f3b70 0x55d92f5bc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.125+0000 7f6a12f22740 1 -- 172.21.15.16:0/1088692791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d92f531a90 con 0x55d92f1f3b70 2024-09-21T03:52:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.125+0000 7f6a12f22740 1 -- 172.21.15.16:0/1088692791 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d92f520ec0 con 0x55d92f1eb990 2024-09-21T03:52:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.125+0000 7f6a12f22740 1 -- 172.21.15.16:0/1088692791 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d92f524cd0 con 0x55d92f214850 2024-09-21T03:52:52.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.125+0000 7f6a09c00640 1 --2- 172.21.15.16:0/1088692791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1eb990 0x55d92f5bbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:52.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.125+0000 7f6a0a600640 1 -- 172.21.15.16:0/1088692791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1eb990 msgr2=0x55d92f5bbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:52.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.125+0000 7f6a0a600640 1 --2- 172.21.15.16:0/1088692791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1eb990 0x55d92f5bbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.125+0000 7f6a0a600640 1 -- 172.21.15.16:0/1088692791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f1f3b70 msgr2=0x55d92f5bc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:52.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.125+0000 7f6a0a600640 1 --2- 172.21.15.16:0/1088692791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f1f3b70 0x55d92f5bc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.125+0000 7f6a0a600640 1 -- 172.21.15.16:0/1088692791 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d92f4c69d0 con 0x55d92f214850 2024-09-21T03:52:52.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.126+0000 7f6a0a600640 1 --2- 172.21.15.16:0/1088692791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f214850 0x55d92f5bb9b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00b500 tx=0x7f69fc009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8e5a4b9d5b04b26 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:52.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.126+0000 7f69f6a00640 1 -- 172.21.15.16:0/1088692791 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc00a970 con 0x55d92f214850 2024-09-21T03:52:52.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.127+0000 7f69f6a00640 1 -- 172.21.15.16:0/1088692791 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69fc019070 con 0x55d92f214850 2024-09-21T03:52:52.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.127+0000 7f69f6a00640 1 -- 172.21.15.16:0/1088692791 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc013380 con 0x55d92f214850 2024-09-21T03:52:52.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.129+0000 7f6a09c00640 1 --2- 172.21.15.16:0/1088692791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1eb990 0x55d92f5bbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:52:52.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.129+0000 7f6a12f22740 1 -- 172.21.15.16:0/1088692791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f214850 msgr2=0x55d92f5bb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:52.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.129+0000 7f6a12f22740 1 --2- 172.21.15.16:0/1088692791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f214850 0x55d92f5bb9b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00b500 tx=0x7f69fc009630 comp rx=0 tx=0).stop 2024-09-21T03:52:52.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.130+0000 7f6a12f22740 1 -- 172.21.15.16:0/1088692791 shutdown_connections 2024-09-21T03:52:52.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.130+0000 7f6a12f22740 1 --2- 172.21.15.16:0/1088692791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f1f3b70 0x55d92f5bc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.130+0000 7f6a12f22740 1 --2- 172.21.15.16:0/1088692791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1eb990 0x55d92f5bbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.130+0000 7f6a12f22740 1 --2- 172.21.15.16:0/1088692791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f214850 0x55d92f5bb9b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.130+0000 7f6a12f22740 1 -- 172.21.15.16:0/1088692791 >> 172.21.15.16:0/1088692791 conn(0x55d92f208c40 msgr2=0x55d92f5b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.130+0000 7f6a02000640 1 --2- 172.21.15.16:0/3117721621 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f69ec0b6610 0x7f69ec0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.132+0000 7f6a12f22740 1 -- 172.21.15.16:0/1088692791 shutdown_connections 2024-09-21T03:52:52.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.133+0000 7f6a02000640 1 -- 172.21.15.16:0/3117721621 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f69f8107d80 con 0x7f6a04078290 2024-09-21T03:52:52.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.133+0000 7f6a09c00640 1 --2- 172.21.15.16:0/3117721621 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f69ec0b6610 0x7f69ec0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:52.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.134+0000 7f6a12f22740 1 -- 172.21.15.16:0/1088692791 wait complete. 2024-09-21T03:52:52.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.135+0000 7f6a12f22740 1 -- start start 2024-09-21T03:52:52.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.135+0000 7f6a12f22740 1 Processor -- start 2024-09-21T03:52:52.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.137+0000 7f6a09c00640 1 --2- 172.21.15.16:0/3117721621 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f69ec0b6610 0x7f69ec0b8b20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55d92f5ba630 tx=0x7f69f0008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:52.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.140+0000 7f6a12f22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1eb990 0x55d92f5c7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.140+0000 7f6a12f22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f1f3b70 0x55d92f5c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.140+0000 7f6a12f22740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f214850 0x55d92f5d3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.140+0000 7f6a09200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f1f3b70 0x55d92f5c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:52.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.141+0000 7f6a09200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f1f3b70 0x55d92f5c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34524/0 (socket says 172.21.15.16:34524) 2024-09-21T03:52:52.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.141+0000 7f6a12f22740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d92f3827b0 con 0x55d92f214850 2024-09-21T03:52:52.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.141+0000 7f6a12f22740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d92f4c5650 con 0x55d92f1eb990 2024-09-21T03:52:52.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.141+0000 7f6a12f22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d92f5ca190 con 0x55d92f1f3b70 2024-09-21T03:52:52.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.141+0000 7f6a09c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1eb990 0x55d92f5c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:52.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.141+0000 7f6a09c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1eb990 0x55d92f5c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:51574/0 (socket says 172.21.15.16:51574) 2024-09-21T03:52:52.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.141+0000 7f6a12f22740 10 client.0 _ll_register_callbacks cb 0x55d92f22c0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:52.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.141+0000 7f6a09c00640 1 -- 172.21.15.16:0/1296552544 learned_addr learned my addr 172.21.15.16:0/1296552544 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:52.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.141+0000 7f6a0a600640 1 --2- 172.21.15.16:0/1296552544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f214850 0x55d92f5d3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:52.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.141+0000 7f6a09c00640 1 -- 172.21.15.16:0/1296552544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f1f3b70 msgr2=0x55d92f5c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:52.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.141+0000 7f6a09c00640 1 --2- 172.21.15.16:0/1296552544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f1f3b70 0x55d92f5c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.141+0000 7f6a09c00640 1 -- 172.21.15.16:0/1296552544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f214850 msgr2=0x55d92f5d3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:52.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.142+0000 7f6a09c00640 1 --2- 172.21.15.16:0/1296552544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f214850 0x55d92f5d3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.142+0000 7f6a09c00640 1 -- 172.21.15.16:0/1296552544 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d92f28eba0 con 0x55d92f1eb990 2024-09-21T03:52:52.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.142+0000 7f6a0a600640 1 --2- 172.21.15.16:0/1296552544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f214850 0x55d92f5d3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:52:52.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.143+0000 7f69eb400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:52:52.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.144+0000 7f6a09c00640 1 --2- 172.21.15.16:0/1296552544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1eb990 0x55d92f5c7570 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f69f000cd60 tx=0x7f69f0004840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64de8e5c0b2fea4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:52.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.144+0000 7f69eb400640 1 -- 172.21.15.16:0/1296552544 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f0021020 con 0x55d92f1eb990 2024-09-21T03:52:52.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.144+0000 7f69eb400640 1 -- 172.21.15.16:0/1296552544 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f69f001d3f0 con 0x55d92f1eb990 2024-09-21T03:52:52.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.144+0000 7f6a12f22740 1 -- 172.21.15.16:0/1296552544 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69fc019070 con 0x55d92f1eb990 2024-09-21T03:52:52.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.145+0000 7f69eb400640 1 -- 172.21.15.16:0/1296552544 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69f0019040 con 0x55d92f1eb990 2024-09-21T03:52:52.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.145+0000 7f6a12f22740 10 client.26279 fetch_fsmap learned FSMap version 30 2024-09-21T03:52:52.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.145+0000 7f6a12f22740 1 -- 172.21.15.16:0/1296552544 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d92f5d6180 con 0x55d92f1eb990 2024-09-21T03:52:52.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.145+0000 7f69eb400640 1 -- 172.21.15.16:0/1296552544 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69fc019070 con 0x55d92f1eb990 2024-09-21T03:52:52.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.147+0000 7f69eb400640 1 -- 172.21.15.16:0/1296552544 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f69f002e020 con 0x55d92f1eb990 2024-09-21T03:52:52.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.147+0000 7f6a12f22740 10 client.26279 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:52:52.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.147+0000 7f6a12f22740 10 client.26279 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:52:52.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.147+0000 7f6a12f22740 20 client.26279 populate_metadata read hostname 'smithi016' 2024-09-21T03:52:52.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.147+0000 7f6a12f22740 1 --2- 172.21.15.16:0/1296552544 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d92f1d31d0 0x55d92f5d9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.147+0000 7f6a12f22740 4 client.26279 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:52:52.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.147+0000 7f6a12f22740 1 -- 172.21.15.16:0/1296552544 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d92f5339e0 con 0x55d92f1d31d0 2024-09-21T03:52:52.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.147+0000 7f6a09200640 1 --2- 172.21.15.16:0/1296552544 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d92f1d31d0 0x55d92f5d9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.149+0000 7f6a09200640 1 --2- 172.21.15.16:0/1296552544 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d92f1d31d0 0x55d92f5d9700 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e464115b39e79a9 server_cookie=1b49fa67be85f7b5 in_seq=0 out_seq=0 2024-09-21T03:52:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.149+0000 7f69eb400640 10 client.26279 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.149+0000 7f69eb400640 1 -- 172.21.15.16:0/1296552544 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d92f5339e0 con 0x55d92f1d31d0 2024-09-21T03:52:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.149+0000 7f69eb400640 10 client.26279 handle_command_reply: tid=0 2024-09-21T03:52:52.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.150+0000 7f6a12f22740 1 client.26279 shutdown 2024-09-21T03:52:52.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.150+0000 7f6a12f22740 1 -- 172.21.15.16:0/1296552544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1eb990 msgr2=0x55d92f5c7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.150+0000 7f6a12f22740 1 --2- 172.21.15.16:0/1296552544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1eb990 0x55d92f5c7570 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f69f000cd60 tx=0x7f69f0004840 comp rx=0 tx=0).stop 2024-09-21T03:52:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.150+0000 7f6a12f22740 1 -- 172.21.15.16:0/1296552544 shutdown_connections 2024-09-21T03:52:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.150+0000 7f6a12f22740 1 --2- 172.21.15.16:0/1296552544 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d92f1d31d0 0x55d92f5d9700 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.150+0000 7f6a12f22740 1 --2- 172.21.15.16:0/1296552544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f214850 0x55d92f5d3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.150+0000 7f6a12f22740 1 --2- 172.21.15.16:0/1296552544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f1f3b70 0x55d92f5c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.150+0000 7f69eb400640 0 client.26279 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.150+0000 7f6a12f22740 1 --2- 172.21.15.16:0/1296552544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1eb990 0x55d92f5c7570 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.150+0000 7f6a12f22740 1 -- 172.21.15.16:0/1296552544 >> 172.21.15.16:0/1296552544 conn(0x55d92f208c40 msgr2=0x55d92f5bc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.152+0000 7f6a12f22740 1 -- 172.21.15.16:0/1296552544 shutdown_connections 2024-09-21T03:52:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.152+0000 7f6a12f22740 1 -- 172.21.15.16:0/1296552544 wait complete. 2024-09-21T03:52:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.152+0000 7f6a12f22740 20 client.26279 trim_cache size 0 max 16384 2024-09-21T03:52:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.182+0000 7f6a12f22740 1 Processor -- start 2024-09-21T03:52:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.182+0000 7f6a12f22740 1 -- start start 2024-09-21T03:52:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.182+0000 7f6a12f22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f208c40 0x55d92f5bc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.182+0000 7f6a12f22740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f1eb990 0x55d92f5bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.182+0000 7f6a12f22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1f3b70 0x55d92f5c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.182+0000 7f6a12f22740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69e400fd20 con 0x55d92f1eb990 2024-09-21T03:52:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.182+0000 7f6a12f22740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69e4011860 con 0x55d92f1f3b70 2024-09-21T03:52:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.182+0000 7f6a12f22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f69e40103f0 con 0x55d92f208c40 2024-09-21T03:52:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.182+0000 7f6a09c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f208c40 0x55d92f5bc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.183+0000 7f6a09c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f208c40 0x55d92f5bc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34536/0 (socket says 172.21.15.16:34536) 2024-09-21T03:52:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.183+0000 7f6a09c00640 1 -- 172.21.15.16:0/2664381569 learned_addr learned my addr 172.21.15.16:0/2664381569 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.183+0000 7f6a0a600640 1 --2- 172.21.15.16:0/2664381569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1f3b70 0x55d92f5c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.183+0000 7f6a0a600640 1 -- 172.21.15.16:0/2664381569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f208c40 msgr2=0x55d92f5bc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.183+0000 7f6a0a600640 1 --2- 172.21.15.16:0/2664381569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f208c40 0x55d92f5bc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.183+0000 7f6a0a600640 1 -- 172.21.15.16:0/2664381569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f1eb990 msgr2=0x55d92f5bcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.183+0000 7f6a0a600640 1 --2- 172.21.15.16:0/2664381569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f1eb990 0x55d92f5bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.183+0000 7f6a0a600640 1 -- 172.21.15.16:0/2664381569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d92f5b8b80 con 0x55d92f1f3b70 2024-09-21T03:52:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.184+0000 7f6a0a600640 1 --2- 172.21.15.16:0/2664381569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1f3b70 0x55d92f5c23b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0093b0 tx=0x7f69fc003c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f52c10116af9672e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.184+0000 7f69f6000640 1 -- 172.21.15.16:0/2664381569 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc00a970 con 0x55d92f1f3b70 2024-09-21T03:52:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.184+0000 7f69f6000640 1 -- 172.21.15.16:0/2664381569 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69fc019070 con 0x55d92f1f3b70 2024-09-21T03:52:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.184+0000 7f69f6000640 1 -- 172.21.15.16:0/2664381569 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc004870 con 0x55d92f1f3b70 2024-09-21T03:52:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.185+0000 7f6a12f22740 1 -- 172.21.15.16:0/2664381569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1f3b70 msgr2=0x55d92f5c23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.185+0000 7f6a12f22740 1 --2- 172.21.15.16:0/2664381569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1f3b70 0x55d92f5c23b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0093b0 tx=0x7f69fc003c90 comp rx=0 tx=0).stop 2024-09-21T03:52:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.186+0000 7f6a12f22740 1 -- 172.21.15.16:0/2664381569 shutdown_connections 2024-09-21T03:52:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.186+0000 7f6a12f22740 1 --2- 172.21.15.16:0/2664381569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f1f3b70 0x55d92f5c23b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.186+0000 7f6a12f22740 1 --2- 172.21.15.16:0/2664381569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f1eb990 0x55d92f5bcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.186+0000 7f6a12f22740 1 --2- 172.21.15.16:0/2664381569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f208c40 0x55d92f5bc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.186+0000 7f6a12f22740 1 -- 172.21.15.16:0/2664381569 >> 172.21.15.16:0/2664381569 conn(0x55d92f5d3d00 msgr2=0x55d92f5c7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.186+0000 7f6a12f22740 1 -- 172.21.15.16:0/2664381569 shutdown_connections 2024-09-21T03:52:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.186+0000 7f6a12f22740 1 -- 172.21.15.16:0/2664381569 wait complete. 2024-09-21T03:52:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.187+0000 7f6a12f22740 1 -- start start 2024-09-21T03:52:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.187+0000 7f6a12f22740 1 Processor -- start 2024-09-21T03:52:52.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.188+0000 7f6a12f22740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f1eb990 0x55d92f5cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.188+0000 7f6a12f22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f1f3b70 0x55d92f5cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.188+0000 7f6a09c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f1eb990 0x55d92f5cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:52.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.188+0000 7f6a09c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f1eb990 0x55d92f5cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38426/0 (socket says 172.21.15.16:38426) 2024-09-21T03:52:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.189+0000 7f6a09c00640 1 -- 172.21.15.16:0/3495968496 learned_addr learned my addr 172.21.15.16:0/3495968496 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:52:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.189+0000 7f6a12f22740 1 --2- 172.21.15.16:0/3495968496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f208c40 0x55d92f5ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.189+0000 7f6a12f22740 1 -- 172.21.15.16:0/3495968496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69e4011190 con 0x55d92f1eb990 2024-09-21T03:52:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.189+0000 7f6a12f22740 1 -- 172.21.15.16:0/3495968496 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69e4010ac0 con 0x55d92f208c40 2024-09-21T03:52:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.189+0000 7f6a12f22740 1 -- 172.21.15.16:0/3495968496 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f69e400dd00 con 0x55d92f1f3b70 2024-09-21T03:52:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.189+0000 7f6a12f22740 10 client.0 _ll_register_callbacks cb 0x7f69f000a4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:52:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.190+0000 7f6a09c00640 1 -- 172.21.15.16:0/3495968496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f1f3b70 msgr2=0x55d92f5cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:52:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.190+0000 7f6a09c00640 1 --2- 172.21.15.16:0/3495968496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f1f3b70 0x55d92f5cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.190+0000 7f6a09c00640 1 -- 172.21.15.16:0/3495968496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f208c40 msgr2=0x55d92f5ca4f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:52:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.190+0000 7f6a0a600640 1 --2- 172.21.15.16:0/3495968496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f208c40 0x55d92f5ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.190+0000 7f6a09c00640 1 --2- 172.21.15.16:0/3495968496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f208c40 0x55d92f5ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.190+0000 7f6a09c00640 1 -- 172.21.15.16:0/3495968496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d92f5cac80 con 0x55d92f1eb990 2024-09-21T03:52:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.191+0000 7f6a09c00640 1 --2- 172.21.15.16:0/3495968496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f1eb990 0x55d92f5cf7e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f69f0026910 tx=0x7f69f0026620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9568c8cb4f0c9220 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:52:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.191+0000 7f69eb400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:52:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.192+0000 7f69eb400640 1 -- 172.21.15.16:0/3495968496 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f0021040 con 0x55d92f1eb990 2024-09-21T03:52:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.192+0000 7f69eb400640 1 -- 172.21.15.16:0/3495968496 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f69f001d600 con 0x55d92f1eb990 2024-09-21T03:52:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.192+0000 7f6a12f22740 1 -- 172.21.15.16:0/3495968496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69fc019070 con 0x55d92f1eb990 2024-09-21T03:52:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.193+0000 7f69eb400640 1 -- 172.21.15.16:0/3495968496 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69f0021040 con 0x55d92f1eb990 2024-09-21T03:52:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.193+0000 7f6a12f22740 10 client.16122 fetch_fsmap learned FSMap version 30 2024-09-21T03:52:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.193+0000 7f6a12f22740 1 -- 172.21.15.16:0/3495968496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d92f655b50 con 0x55d92f1eb990 2024-09-21T03:52:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.194+0000 7f69eb400640 1 -- 172.21.15.16:0/3495968496 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69fc019070 con 0x55d92f1eb990 2024-09-21T03:52:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.194+0000 7f69eb400640 1 -- 172.21.15.16:0/3495968496 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f69f002e070 con 0x55d92f1eb990 2024-09-21T03:52:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.194+0000 7f6a12f22740 10 client.16122 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:52:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.194+0000 7f6a12f22740 10 client.16122 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:52:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.194+0000 7f6a12f22740 20 client.16122 populate_metadata read hostname 'smithi016' 2024-09-21T03:52:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.194+0000 7f6a12f22740 1 --2- 172.21.15.16:0/3495968496 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d92f214850 0x55d92f6583d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:52:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.194+0000 7f6a12f22740 4 client.16122 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}] 2024-09-21T03:52:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.194+0000 7f6a12f22740 1 -- 172.21.15.16:0/3495968496 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}) -- 0x55d92f52f490 con 0x55d92f214850 2024-09-21T03:52:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.195+0000 7f6a09200640 1 --2- 172.21.15.16:0/3495968496 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d92f214850 0x55d92f6583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:52:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.195+0000 7f6a09200640 1 --2- 172.21.15.16:0/3495968496 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d92f214850 0x55d92f6583d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acad45e494dcdfdd server_cookie=5b459cfe0176bfa4 in_seq=0 out_seq=0 2024-09-21T03:52:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.195+0000 7f69eb400640 10 client.16122 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:52.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.235+0000 7f69eb400640 1 -- 172.21.15.16:0/3495968496 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d92f52f490 con 0x55d92f214850 2024-09-21T03:52:52.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.235+0000 7f69eb400640 10 client.16122 handle_command_reply: tid=0 2024-09-21T03:52:52.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.236+0000 7f6a12f22740 1 client.16122 shutdown 2024-09-21T03:52:52.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.236+0000 7f6a12f22740 1 -- 172.21.15.16:0/3495968496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f1eb990 msgr2=0x55d92f5cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:52.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.236+0000 7f6a12f22740 1 --2- 172.21.15.16:0/3495968496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f1eb990 0x55d92f5cf7e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f69f0026910 tx=0x7f69f0026620 comp rx=0 tx=0).stop 2024-09-21T03:52:52.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.236+0000 7f6a12f22740 1 -- 172.21.15.16:0/3495968496 shutdown_connections 2024-09-21T03:52:52.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.236+0000 7f6a12f22740 1 --2- 172.21.15.16:0/3495968496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d92f208c40 0x55d92f5ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.236+0000 7f6a12f22740 1 --2- 172.21.15.16:0/3495968496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d92f1f3b70 0x55d92f5cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.236+0000 7f6a12f22740 1 --2- 172.21.15.16:0/3495968496 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d92f214850 0x55d92f6583d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.236+0000 7f6a12f22740 1 --2- 172.21.15.16:0/3495968496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d92f1eb990 0x55d92f5cf7e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.236+0000 7f6a12f22740 1 -- 172.21.15.16:0/3495968496 >> 172.21.15.16:0/3495968496 conn(0x55d92f5d3d00 msgr2=0x55d92f5c7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:52.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.236+0000 7f69eb400640 0 client.16122 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:52:52.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.237+0000 7f6a12f22740 1 -- 172.21.15.16:0/3495968496 shutdown_connections 2024-09-21T03:52:52.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.237+0000 7f6a12f22740 1 -- 172.21.15.16:0/3495968496 wait complete. 2024-09-21T03:52:52.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.237+0000 7f6a12f22740 20 client.16122 trim_cache size 0 max 16384 2024-09-21T03:52:52.239 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:52:52.240 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:52:52.240 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:52:52.240 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 26, 2024-09-21T03:52:52.240 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:52:52.240 INFO:teuthology.orchestra.run.smithi016.stdout: "fd60747c": { 2024-09-21T03:52:52.240 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 26, 2024-09-21T03:52:52.240 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:52:52.240 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:52:52.240 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:52:52.240 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:52:52.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:52:52.240 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:52:52.240 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 139.2, 2024-09-21T03:52:52.240 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:52:52.241 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:52:52.241 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:52:52.241 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:52:52.241 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:52:52.241 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:52:52.241 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:52:52.241 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:52:52.241 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:52:52.241 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:52:52.241 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:52:52.241 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:52:52.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.238+0000 7f6a0c600640 1 -- 172.21.15.16:0/3117721621 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f69ec0b6610 msgr2=0x7f69ec0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:52.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.238+0000 7f6a0c600640 1 --2- 172.21.15.16:0/3117721621 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f69ec0b6610 0x7f69ec0b8b20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55d92f5ba630 tx=0x7f69f0008040 comp rx=0 tx=0).stop 2024-09-21T03:52:52.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.238+0000 7f6a0c600640 1 -- 172.21.15.16:0/3117721621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a04078290 msgr2=0x7f6a041a11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:52:52.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.238+0000 7f6a0c600640 1 --2- 172.21.15.16:0/3117721621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a04078290 0x7f6a041a11c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f69f802aa00 tx=0x7f69f8003d70 comp rx=0 tx=0).stop 2024-09-21T03:52:52.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.238+0000 7f6a0c600640 1 -- 172.21.15.16:0/3117721621 shutdown_connections 2024-09-21T03:52:52.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.238+0000 7f6a0c600640 1 --2- 172.21.15.16:0/3117721621 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f69ec0b6610 0x7f69ec0b8b20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.238+0000 7f6a0c600640 1 --2- 172.21.15.16:0/3117721621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a04114740 0x7f6a041a6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.238+0000 7f6a0c600640 1 --2- 172.21.15.16:0/3117721621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a04078290 0x7f6a041a11c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.238+0000 7f6a0c600640 1 --2- 172.21.15.16:0/3117721621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a04077940 0x7f6a041a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:52:52.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.238+0000 7f6a0c600640 1 -- 172.21.15.16:0/3117721621 >> 172.21.15.16:0/3117721621 conn(0x7f6a041017d0 msgr2=0x7f6a04103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:52:52.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.239+0000 7f6a0c600640 1 -- 172.21.15.16:0/3117721621 shutdown_connections 2024-09-21T03:52:52.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:52:52.239+0000 7f6a0c600640 1 -- 172.21.15.16:0/3117721621 wait complete. 2024-09-21T03:52:53.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:53 smithi134 ceph-mon[35031]: pgmap v615: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 196 MiB/s wr, 1.53k op/s 2024-09-21T03:52:53.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:53 smithi123 ceph-mon[34673]: pgmap v615: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 196 MiB/s wr, 1.53k op/s 2024-09-21T03:52:53.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:53 smithi016 ceph-mon[30375]: pgmap v615: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 196 MiB/s wr, 1.53k op/s 2024-09-21T03:52:55.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:55 smithi134 ceph-mon[35031]: pgmap v616: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 211 MiB/s wr, 1.56k op/s 2024-09-21T03:52:55.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:55 smithi123 ceph-mon[34673]: pgmap v616: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 211 MiB/s wr, 1.56k op/s 2024-09-21T03:52:55.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:55 smithi016 ceph-mon[30375]: pgmap v616: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 211 MiB/s wr, 1.56k op/s 2024-09-21T03:52:56.797 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fd60747c', quiesce took 17.2 seconds. Will release after: 2.0500000000000007 2024-09-21T03:52:57.095 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:57 smithi016 ceph-mon[30375]: pgmap v617: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 223 MiB/s wr, 1.56k op/s 2024-09-21T03:52:57.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:57 smithi134 ceph-mon[35031]: pgmap v617: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 223 MiB/s wr, 1.56k op/s 2024-09-21T03:52:57.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:57 smithi123 ceph-mon[34673]: pgmap v617: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 223 MiB/s wr, 1.56k op/s 2024-09-21T03:52:58.848 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:52:59.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:52:59 smithi123 ceph-mon[34673]: pgmap v618: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 219 MiB/s wr, 1.46k op/s 2024-09-21T03:52:59.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:52:59 smithi016 ceph-mon[30375]: pgmap v618: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 219 MiB/s wr, 1.46k op/s 2024-09-21T03:52:59.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:52:59 smithi134 ceph-mon[35031]: pgmap v618: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 219 MiB/s wr, 1.46k op/s 2024-09-21T03:52:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:01.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:01 smithi123 ceph-mon[34673]: pgmap v619: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 226 MiB/s wr, 1.44k op/s 2024-09-21T03:53:01.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:01 smithi016 ceph-mon[30375]: pgmap v619: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 226 MiB/s wr, 1.44k op/s 2024-09-21T03:53:01.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:01 smithi134 ceph-mon[35031]: pgmap v619: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 226 MiB/s wr, 1.44k op/s 2024-09-21T03:53:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.782+0000 7fb0a9c00640 1 -- 172.21.15.16:0/106000332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0a41025b0 msgr2=0x7fb0a41029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.782+0000 7fb0a9c00640 1 --2- 172.21.15.16:0/106000332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0a41025b0 0x7fb0a41029c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb09c01acb0 tx=0x7fb09c03b900 comp rx=0 tx=0).stop 2024-09-21T03:53:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.784+0000 7fb0a9c00640 1 -- 172.21.15.16:0/106000332 shutdown_connections 2024-09-21T03:53:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.784+0000 7fb0a9c00640 1 --2- 172.21.15.16:0/106000332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a41079f0 0x7fb0a4107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.784+0000 7fb0a9c00640 1 --2- 172.21.15.16:0/106000332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0a4102f00 0x7fb0a4107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.784+0000 7fb0a9c00640 1 --2- 172.21.15.16:0/106000332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0a41025b0 0x7fb0a41029c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.784+0000 7fb0a9c00640 1 -- 172.21.15.16:0/106000332 >> 172.21.15.16:0/106000332 conn(0x7fb0a40fe2c0 msgr2=0x7fb0a4100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.784+0000 7fb0a9c00640 1 -- 172.21.15.16:0/106000332 shutdown_connections 2024-09-21T03:53:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.785+0000 7fb0a9c00640 1 -- 172.21.15.16:0/106000332 wait complete. 2024-09-21T03:53:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.786+0000 7fb0a9c00640 1 Processor -- start 2024-09-21T03:53:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.789+0000 7fb0a9c00640 1 -- start start 2024-09-21T03:53:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.790+0000 7fb0a9c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0a41025b0 0x7fb0a4074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.790+0000 7fb0a9c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0a4102f00 0x7fb0a40748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.791+0000 7fb0a9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a41079f0 0x7fb0a410ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:01.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.791+0000 7fb0a3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0a41025b0 0x7fb0a4074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:01.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.794+0000 7fb0a3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0a41025b0 0x7fb0a4074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:38388/0 (socket says 172.21.15.16:38388) 2024-09-21T03:53:01.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.794+0000 7fb0a3e00640 1 -- 172.21.15.16:0/2683297186 learned_addr learned my addr 172.21.15.16:0/2683297186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:01.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.791+0000 7fb0a9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a41079f0 0x7fb0a410ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:01.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.794+0000 7fb0a9c00640 1 -- 172.21.15.16:0/2683297186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a410b2f0 con 0x7fb0a41079f0 2024-09-21T03:53:01.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.792+0000 7fb0a3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0a4102f00 0x7fb0a40748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:01.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.794+0000 7fb0a9c00640 1 -- 172.21.15.16:0/2683297186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a410b170 con 0x7fb0a4102f00 2024-09-21T03:53:01.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.794+0000 7fb0a9c00640 1 -- 172.21.15.16:0/2683297186 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a410b470 con 0x7fb0a41025b0 2024-09-21T03:53:01.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.795+0000 7fb0a3e00640 1 -- 172.21.15.16:0/2683297186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0a4102f00 msgr2=0x7fb0a40748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:01.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.795+0000 7fb0a3e00640 1 --2- 172.21.15.16:0/2683297186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0a4102f00 0x7fb0a40748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:01.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.795+0000 7fb0a3e00640 1 -- 172.21.15.16:0/2683297186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a41079f0 msgr2=0x7fb0a410ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:01.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.795+0000 7fb0a3e00640 1 --2- 172.21.15.16:0/2683297186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a41079f0 0x7fb0a410ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:01.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.795+0000 7fb0a3e00640 1 -- 172.21.15.16:0/2683297186 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a410f5c0 con 0x7fb0a41025b0 2024-09-21T03:53:01.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.796+0000 7fb0a3e00640 1 --2- 172.21.15.16:0/2683297186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0a41025b0 0x7fb0a4074340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb09c01add0 tx=0x7fb09c003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:01.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.797+0000 7fb0a0c00640 1 -- 172.21.15.16:0/2683297186 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c045070 con 0x7fb0a41025b0 2024-09-21T03:53:01.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.797+0000 7fb0a0c00640 1 -- 172.21.15.16:0/2683297186 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb09c003d10 con 0x7fb0a41025b0 2024-09-21T03:53:01.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.797+0000 7fb0a0c00640 1 -- 172.21.15.16:0/2683297186 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c0045f0 con 0x7fb0a41025b0 2024-09-21T03:53:01.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.798+0000 7fb0a9c00640 1 -- 172.21.15.16:0/2683297186 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a410f850 con 0x7fb0a41025b0 2024-09-21T03:53:01.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.798+0000 7fb0a9c00640 1 -- 172.21.15.16:0/2683297186 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a410fce0 con 0x7fb0a41025b0 2024-09-21T03:53:01.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.802+0000 7fb0a9c00640 1 -- 172.21.15.16:0/2683297186 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a4107430 con 0x7fb0a41025b0 2024-09-21T03:53:01.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.811+0000 7fb0a0c00640 1 -- 172.21.15.16:0/2683297186 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb09c048020 con 0x7fb0a41025b0 2024-09-21T03:53:01.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.813+0000 7fb0a0c00640 1 --2- 172.21.15.16:0/2683297186 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb0880b6720 0x7fb0880b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:01.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.814+0000 7fb0a0c00640 1 -- 172.21.15.16:0/2683297186 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb09c1193a0 con 0x7fb0a41025b0 2024-09-21T03:53:01.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.814+0000 7fb0a3400640 1 --2- 172.21.15.16:0/2683297186 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb0880b6720 0x7fb0880b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:01.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.819+0000 7fb0a3400640 1 --2- 172.21.15.16:0/2683297186 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb0880b6720 0x7fb0880b8c30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4108270 tx=0x7fb094009110 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:01.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:01.819+0000 7fb0a0c00640 1 -- 172.21.15.16:0/2683297186 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb09c0df820 con 0x7fb0a41025b0 2024-09-21T03:53:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:02.367+0000 7fb0a9c00640 1 -- 172.21.15.16:0/2683297186 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0a41ad430 con 0x7fb0a41025b0 2024-09-21T03:53:02.376 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:53:02.376 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:53:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:02.370+0000 7fb0a0c00640 1 -- 172.21.15.16:0/2683297186 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7fb09c0e3c00 con 0x7fb0a41025b0 2024-09-21T03:53:02.377 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:53:02.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:02.374+0000 7fb0a9c00640 1 -- 172.21.15.16:0/2683297186 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb0880b6720 msgr2=0x7fb0880b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:02.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:02.374+0000 7fb0a9c00640 1 --2- 172.21.15.16:0/2683297186 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb0880b6720 0x7fb0880b8c30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4108270 tx=0x7fb094009110 comp rx=0 tx=0).stop 2024-09-21T03:53:02.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:02.374+0000 7fb0a9c00640 1 -- 172.21.15.16:0/2683297186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0a41025b0 msgr2=0x7fb0a4074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:02.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:02.374+0000 7fb0a9c00640 1 --2- 172.21.15.16:0/2683297186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0a41025b0 0x7fb0a4074340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb09c01add0 tx=0x7fb09c003ba0 comp rx=0 tx=0).stop 2024-09-21T03:53:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:02.374+0000 7fb0a9c00640 1 -- 172.21.15.16:0/2683297186 shutdown_connections 2024-09-21T03:53:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:02.374+0000 7fb0a9c00640 1 --2- 172.21.15.16:0/2683297186 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb0880b6720 0x7fb0880b8c30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:02.374+0000 7fb0a9c00640 1 --2- 172.21.15.16:0/2683297186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a41079f0 0x7fb0a410ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:02.374+0000 7fb0a9c00640 1 --2- 172.21.15.16:0/2683297186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0a4102f00 0x7fb0a40748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:02.374+0000 7fb0a9c00640 1 --2- 172.21.15.16:0/2683297186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0a41025b0 0x7fb0a4074340 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:02.374+0000 7fb0a9c00640 1 -- 172.21.15.16:0/2683297186 >> 172.21.15.16:0/2683297186 conn(0x7fb0a40fe2c0 msgr2=0x7fb0a4105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:02.375+0000 7fb0a9c00640 1 -- 172.21.15.16:0/2683297186 shutdown_connections 2024-09-21T03:53:02.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:02.378+0000 7fb0a9c00640 1 -- 172.21.15.16:0/2683297186 wait complete. 2024-09-21T03:53:03.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:03 smithi123 ceph-mon[34673]: pgmap v620: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 236 MiB/s wr, 1.46k op/s 2024-09-21T03:53:03.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:03 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2683297186' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:53:03.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:03 smithi016 ceph-mon[30375]: pgmap v620: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 236 MiB/s wr, 1.46k op/s 2024-09-21T03:53:03.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:03 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2683297186' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:53:03.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:03 smithi134 ceph-mon[35031]: pgmap v620: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 236 MiB/s wr, 1.46k op/s 2024-09-21T03:53:03.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:03 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2683297186' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:53:03.676 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db --set-id fd60747c --release --await' 2024-09-21T03:53:03.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db --set-id fd60747c --release --await 2024-09-21T03:53:04.561 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:53:05.160 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:05 smithi016 ceph-mon[30375]: pgmap v621: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 244 MiB/s wr, 1.47k op/s 2024-09-21T03:53:05.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:05 smithi134 ceph-mon[35031]: pgmap v621: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 244 MiB/s wr, 1.47k op/s 2024-09-21T03:53:05.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:05 smithi123 ceph-mon[34673]: pgmap v621: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 244 MiB/s wr, 1.47k op/s 2024-09-21T03:53:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.044+0000 7f3ff9000640 1 -- 172.21.15.16:0/3278260929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff410ad10 msgr2=0x7f3ff4111700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.044+0000 7f3ff9000640 1 --2- 172.21.15.16:0/3278260929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff410ad10 0x7f3ff4111700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009860 tx=0x7f3fe002a380 comp rx=0 tx=0).stop 2024-09-21T03:53:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.047+0000 7f3ff9000640 1 -- 172.21.15.16:0/3278260929 shutdown_connections 2024-09-21T03:53:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.047+0000 7f3ff9000640 1 --2- 172.21.15.16:0/3278260929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4111c40 0x7f3ff4114060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.047+0000 7f3ff9000640 1 --2- 172.21.15.16:0/3278260929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff410ad10 0x7f3ff4111700 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.047+0000 7f3ff9000640 1 --2- 172.21.15.16:0/3278260929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff410a3c0 0x7f3ff410a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.047+0000 7f3ff9000640 1 -- 172.21.15.16:0/3278260929 >> 172.21.15.16:0/3278260929 conn(0x7f3ff4070640 msgr2=0x7f3ff4072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.047+0000 7f3ff9000640 1 -- 172.21.15.16:0/3278260929 shutdown_connections 2024-09-21T03:53:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.047+0000 7f3ff9000640 1 -- 172.21.15.16:0/3278260929 wait complete. 2024-09-21T03:53:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.050+0000 7f3ff9000640 1 Processor -- start 2024-09-21T03:53:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.050+0000 7f3ff9000640 1 -- start start 2024-09-21T03:53:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.051+0000 7f3ff9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff410a3c0 0x7f3ff41a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.051+0000 7f3ff9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff410ad10 0x7f3ff41a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.051+0000 7f3ff9000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff4111c40 0x7f3ff41a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.052+0000 7f3ff9000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff41179a0 con 0x7f3ff410a3c0 2024-09-21T03:53:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.052+0000 7f3ff2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff410ad10 0x7f3ff41a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.052+0000 7f3ff2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff410ad10 0x7f3ff41a2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44614/0 (socket says 172.21.15.16:44614) 2024-09-21T03:53:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.052+0000 7f3ff2a00640 1 -- 172.21.15.16:0/2544285817 learned_addr learned my addr 172.21.15.16:0/2544285817 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.052+0000 7f3ff3e00640 1 --2- 172.21.15.16:0/2544285817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff4111c40 0x7f3ff41a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.052+0000 7f3ff9000640 1 -- 172.21.15.16:0/2544285817 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff4117820 con 0x7f3ff410ad10 2024-09-21T03:53:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.053+0000 7f3ff9000640 1 -- 172.21.15.16:0/2544285817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff4117b20 con 0x7f3ff4111c40 2024-09-21T03:53:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.053+0000 7f3ff3400640 1 --2- 172.21.15.16:0/2544285817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff410a3c0 0x7f3ff41a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.053+0000 7f3ff3e00640 1 -- 172.21.15.16:0/2544285817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff410ad10 msgr2=0x7f3ff41a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.053+0000 7f3ff3e00640 1 --2- 172.21.15.16:0/2544285817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff410ad10 0x7f3ff41a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.053+0000 7f3ff3e00640 1 -- 172.21.15.16:0/2544285817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff410a3c0 msgr2=0x7f3ff41a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.053+0000 7f3ff3e00640 1 --2- 172.21.15.16:0/2544285817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff410a3c0 0x7f3ff41a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.053+0000 7f3ff3e00640 1 -- 172.21.15.16:0/2544285817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff41a3e20 con 0x7f3ff4111c40 2024-09-21T03:53:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.054+0000 7f3ff3e00640 1 --2- 172.21.15.16:0/2544285817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff4111c40 0x7f3ff41a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00ff40 tx=0x7f3fec00ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.055+0000 7f3febe00640 1 -- 172.21.15.16:0/2544285817 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec00d0d0 con 0x7f3ff4111c40 2024-09-21T03:53:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.055+0000 7f3ff9000640 1 -- 172.21.15.16:0/2544285817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff41a40b0 con 0x7f3ff4111c40 2024-09-21T03:53:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.055+0000 7f3ff9000640 1 -- 172.21.15.16:0/2544285817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff410b730 con 0x7f3ff4111c40 2024-09-21T03:53:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.056+0000 7f3febe00640 1 -- 172.21.15.16:0/2544285817 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fec00b6c0 con 0x7f3ff4111c40 2024-09-21T03:53:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.056+0000 7f3febe00640 1 -- 172.21.15.16:0/2544285817 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec00d0d0 con 0x7f3ff4111c40 2024-09-21T03:53:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.062+0000 7f40017d1740 1 Processor -- start 2024-09-21T03:53:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.064+0000 7f3febe00640 1 -- 172.21.15.16:0/2544285817 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3fec00b870 con 0x7f3ff4111c40 2024-09-21T03:53:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.065+0000 7f40017d1740 1 -- start start 2024-09-21T03:53:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.065+0000 7f40017d1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6241b0 0x55d53e9cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.065+0000 7f40017d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e5fb2f0 0x55d53e9cb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.065+0000 7f40017d1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e6034d0 0x55d53e9cc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.065+0000 7f40017d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d53e93e6b0 con 0x55d53e5fb2f0 2024-09-21T03:53:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.065+0000 7f40017d1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d53e92dae0 con 0x55d53e6034d0 2024-09-21T03:53:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.065+0000 7f40017d1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d53e9318f0 con 0x55d53e6241b0 2024-09-21T03:53:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.065+0000 7f3ff3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e5fb2f0 0x55d53e9cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.065+0000 7f3ff2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6241b0 0x55d53e9cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.065+0000 7f3ff2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6241b0 0x55d53e9cb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:38422/0 (socket says 172.21.15.16:38422) 2024-09-21T03:53:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.065+0000 7f3ff3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e5fb2f0 0x55d53e9cb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34568/0 (socket says 172.21.15.16:34568) 2024-09-21T03:53:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.065+0000 7f3ff2a00640 1 -- 172.21.15.16:0/2625095740 learned_addr learned my addr 172.21.15.16:0/2625095740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.066+0000 7f3ff2a00640 1 --2- 172.21.15.16:0/2625095740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e6034d0 0x55d53e9cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.066+0000 7f3ff3400640 1 -- 172.21.15.16:0/2625095740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6241b0 msgr2=0x55d53e9cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.066+0000 7f3ff3400640 1 --2- 172.21.15.16:0/2625095740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6241b0 0x55d53e9cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.066+0000 7f3ff3400640 1 -- 172.21.15.16:0/2625095740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e6034d0 msgr2=0x55d53e9cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.066+0000 7f3ff3400640 1 --2- 172.21.15.16:0/2625095740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e6034d0 0x55d53e9cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.066+0000 7f3ff3400640 1 -- 172.21.15.16:0/2625095740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d53e8fa780 con 0x55d53e5fb2f0 2024-09-21T03:53:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.066+0000 7f3ff3400640 1 --2- 172.21.15.16:0/2625095740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e5fb2f0 0x55d53e9cb920 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009c50 tx=0x7f3fe40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7421babb544fac11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.067+0000 7f3fe8c00640 1 -- 172.21.15.16:0/2625095740 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4012450 con 0x55d53e5fb2f0 2024-09-21T03:53:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.067+0000 7f3fe8c00640 1 -- 172.21.15.16:0/2625095740 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe4012aa0 con 0x55d53e5fb2f0 2024-09-21T03:53:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.067+0000 7f3fe8c00640 1 -- 172.21.15.16:0/2625095740 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe400fae0 con 0x55d53e5fb2f0 2024-09-21T03:53:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.067+0000 7f3febe00640 1 --2- 172.21.15.16:0/2544285817 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fd80b6790 0x7f3fd80b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.068+0000 7f3ff2a00640 1 --2- 172.21.15.16:0/2544285817 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fd80b6790 0x7f3fd80b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.068+0000 7f3febe00640 1 -- 172.21.15.16:0/2544285817 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f3fec0ea8a0 con 0x7f3ff4111c40 2024-09-21T03:53:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.068+0000 7f40017d1740 1 -- 172.21.15.16:0/2625095740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e5fb2f0 msgr2=0x55d53e9cb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.068+0000 7f40017d1740 1 --2- 172.21.15.16:0/2625095740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e5fb2f0 0x55d53e9cb920 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009c50 tx=0x7f3fe40117b0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.069+0000 7f40017d1740 1 -- 172.21.15.16:0/2625095740 shutdown_connections 2024-09-21T03:53:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.069+0000 7f40017d1740 1 --2- 172.21.15.16:0/2625095740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e6034d0 0x55d53e9cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.069+0000 7f40017d1740 1 --2- 172.21.15.16:0/2625095740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e5fb2f0 0x55d53e9cb920 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.069+0000 7f40017d1740 1 --2- 172.21.15.16:0/2625095740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6241b0 0x55d53e9cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.069+0000 7f40017d1740 1 -- 172.21.15.16:0/2625095740 >> 172.21.15.16:0/2625095740 conn(0x55d53e618300 msgr2=0x55d53e9c6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.069+0000 7f40017d1740 1 -- 172.21.15.16:0/2625095740 shutdown_connections 2024-09-21T03:53:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.069+0000 7f40017d1740 1 -- 172.21.15.16:0/2625095740 wait complete. 2024-09-21T03:53:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.069+0000 7f3ff2a00640 1 --2- 172.21.15.16:0/2544285817 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fd80b6790 0x7f3fd80b8ca0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00095f0 tx=0x7f3fe0030470 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.070+0000 7f40017d1740 1 -- start start 2024-09-21T03:53:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.070+0000 7f40017d1740 1 Processor -- start 2024-09-21T03:53:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f40017d1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e5fb2f0 0x55d53e9d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f40017d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e6034d0 0x55d53e9d5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f40017d1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6241b0 0x55d53e9d5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f40017d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d53e865ab0 con 0x55d53e6034d0 2024-09-21T03:53:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f40017d1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d53e8d4570 con 0x55d53e5fb2f0 2024-09-21T03:53:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f40017d1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d53e9ccf90 con 0x55d53e6241b0 2024-09-21T03:53:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f40017d1740 10 client.0 _ll_register_callbacks cb 0x55d53e8e3100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f3ff3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e5fb2f0 0x55d53e9d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f3ff3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e5fb2f0 0x55d53e9d49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44630/0 (socket says 172.21.15.16:44630) 2024-09-21T03:53:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f3ff3400640 1 -- 172.21.15.16:0/3164206792 learned_addr learned my addr 172.21.15.16:0/3164206792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f3ff3e00640 1 --2- 172.21.15.16:0/3164206792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6241b0 0x55d53e9d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f3ff2a00640 1 --2- 172.21.15.16:0/3164206792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e6034d0 0x55d53e9d5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f3ff3e00640 1 -- 172.21.15.16:0/3164206792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e5fb2f0 msgr2=0x55d53e9d49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f3ff3e00640 1 --2- 172.21.15.16:0/3164206792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e5fb2f0 0x55d53e9d49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f3ff3e00640 1 -- 172.21.15.16:0/3164206792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e6034d0 msgr2=0x55d53e9d5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f3ff3e00640 1 --2- 172.21.15.16:0/3164206792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e6034d0 0x55d53e9d5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f3ff3e00640 1 -- 172.21.15.16:0/3164206792 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d53e68f790 con 0x55d53e6241b0 2024-09-21T03:53:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.071+0000 7f3ff3400640 1 --2- 172.21.15.16:0/3164206792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e5fb2f0 0x55d53e9d49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.072+0000 7f3ff2a00640 1 --2- 172.21.15.16:0/3164206792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e6034d0 0x55d53e9d5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.072+0000 7f3ff3e00640 1 --2- 172.21.15.16:0/3164206792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6241b0 0x55d53e9d5b80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0e8040 tx=0x7f3fec0affd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cda93da70d954f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.072+0000 7f3fdd600640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:53:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.072+0000 7f3fdd600640 1 -- 172.21.15.16:0/3164206792 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec0b0190 con 0x55d53e6241b0 2024-09-21T03:53:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.072+0000 7f3fdd600640 1 -- 172.21.15.16:0/3164206792 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f3fec00d270 con 0x55d53e6241b0 2024-09-21T03:53:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.072+0000 7f40017d1740 1 -- 172.21.15.16:0/3164206792 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe4012aa0 con 0x55d53e6241b0 2024-09-21T03:53:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.072+0000 7f3fdd600640 1 -- 172.21.15.16:0/3164206792 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3fec0b0d80 con 0x55d53e6241b0 2024-09-21T03:53:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.072+0000 7f40017d1740 10 client.26239 fetch_fsmap learned FSMap version 30 2024-09-21T03:53:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.072+0000 7f40017d1740 1 -- 172.21.15.16:0/3164206792 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d53e708620 con 0x55d53e6241b0 2024-09-21T03:53:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.073+0000 7f3fdd600640 1 -- 172.21.15.16:0/3164206792 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe4012aa0 con 0x55d53e6241b0 2024-09-21T03:53:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.073+0000 7f3fdd600640 1 -- 172.21.15.16:0/3164206792 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3fec0b62f0 con 0x55d53e6241b0 2024-09-21T03:53:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.073+0000 7f40017d1740 10 client.26239 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:53:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.073+0000 7f40017d1740 10 client.26239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.073+0000 7f40017d1740 20 client.26239 populate_metadata read hostname 'smithi016' 2024-09-21T03:53:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.073+0000 7f40017d1740 1 --2- 172.21.15.16:0/3164206792 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d53e5e2d80 0x55d53e9e2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.073+0000 7f40017d1740 4 client.26239 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.073+0000 7f40017d1740 1 -- 172.21.15.16:0/3164206792 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d53e940600 con 0x55d53e5e2d80 2024-09-21T03:53:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.073+0000 7f3ff3400640 1 --2- 172.21.15.16:0/3164206792 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d53e5e2d80 0x55d53e9e2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.074+0000 7f3ff3400640 1 --2- 172.21.15.16:0/3164206792 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d53e5e2d80 0x55d53e9e2120 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3541495bcd6a0285 server_cookie=da3528e1c74ce7a5 in_seq=0 out_seq=0 2024-09-21T03:53:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.074+0000 7f3fdd600640 10 client.26239 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.074+0000 7f3fdd600640 1 -- 172.21.15.16:0/3164206792 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d53e940600 con 0x55d53e5e2d80 2024-09-21T03:53:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.074+0000 7f3fdd600640 10 client.26239 handle_command_reply: tid=0 2024-09-21T03:53:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.075+0000 7f40017d1740 1 client.26239 shutdown 2024-09-21T03:53:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.075+0000 7f40017d1740 1 -- 172.21.15.16:0/3164206792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6241b0 msgr2=0x55d53e9d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.075+0000 7f40017d1740 1 --2- 172.21.15.16:0/3164206792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6241b0 0x55d53e9d5b80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0e8040 tx=0x7f3fec0affd0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.075+0000 7f40017d1740 1 -- 172.21.15.16:0/3164206792 shutdown_connections 2024-09-21T03:53:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.075+0000 7f40017d1740 1 --2- 172.21.15.16:0/3164206792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6241b0 0x55d53e9d5b80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.075+0000 7f40017d1740 1 --2- 172.21.15.16:0/3164206792 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d53e5e2d80 0x55d53e9e2120 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.075+0000 7f40017d1740 1 --2- 172.21.15.16:0/3164206792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e6034d0 0x55d53e9d5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.075+0000 7f40017d1740 1 --2- 172.21.15.16:0/3164206792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e5fb2f0 0x55d53e9d49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.075+0000 7f3fdd600640 0 client.26239 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.075+0000 7f40017d1740 1 -- 172.21.15.16:0/3164206792 >> 172.21.15.16:0/3164206792 conn(0x55d53e618300 msgr2=0x55d53e9c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.075+0000 7f40017d1740 1 -- 172.21.15.16:0/3164206792 shutdown_connections 2024-09-21T03:53:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.075+0000 7f40017d1740 1 -- 172.21.15.16:0/3164206792 wait complete. 2024-09-21T03:53:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.075+0000 7f40017d1740 20 client.26239 trim_cache size 0 max 16384 2024-09-21T03:53:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.091+0000 7f40017d1740 1 Processor -- start 2024-09-21T03:53:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.091+0000 7f40017d1740 1 -- start start 2024-09-21T03:53:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.091+0000 7f40017d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e5fb2f0 0x55d53e9d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.091+0000 7f40017d1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6034d0 0x55d53e9cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.091+0000 7f40017d1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e6241b0 0x55d53e9cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.091+0000 7f40017d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd400fc50 con 0x55d53e5fb2f0 2024-09-21T03:53:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.091+0000 7f40017d1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd4011790 con 0x55d53e6241b0 2024-09-21T03:53:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.091+0000 7f40017d1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd4010320 con 0x55d53e6034d0 2024-09-21T03:53:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.091+0000 7f3ff2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6034d0 0x55d53e9cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.091+0000 7f3ff2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6034d0 0x55d53e9cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:38438/0 (socket says 172.21.15.16:38438) 2024-09-21T03:53:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.092+0000 7f3ff2a00640 1 -- 172.21.15.16:0/3303798803 learned_addr learned my addr 172.21.15.16:0/3303798803 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.092+0000 7f3ff3400640 1 --2- 172.21.15.16:0/3303798803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e5fb2f0 0x55d53e9d3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.092+0000 7f3ff3e00640 1 --2- 172.21.15.16:0/3303798803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e6241b0 0x55d53e9cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.092+0000 7f3ff2a00640 1 -- 172.21.15.16:0/3303798803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e6241b0 msgr2=0x55d53e9cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.092+0000 7f3ff2a00640 1 --2- 172.21.15.16:0/3303798803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e6241b0 0x55d53e9cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.092+0000 7f3ff2a00640 1 -- 172.21.15.16:0/3303798803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e5fb2f0 msgr2=0x55d53e9d3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.092+0000 7f3ff2a00640 1 --2- 172.21.15.16:0/3303798803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e5fb2f0 0x55d53e9d3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.092+0000 7f3ff2a00640 1 -- 172.21.15.16:0/3303798803 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d53e9d53f0 con 0x55d53e6034d0 2024-09-21T03:53:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.093+0000 7f3ff2a00640 1 --2- 172.21.15.16:0/3303798803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6034d0 0x55d53e9cefd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0030810 tx=0x7f3fe00056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae9ee4ef49177c0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.093+0000 7f3fdfe00640 1 -- 172.21.15.16:0/3303798803 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0051070 con 0x55d53e6034d0 2024-09-21T03:53:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.093+0000 7f3fdfe00640 1 -- 172.21.15.16:0/3303798803 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe0003ca0 con 0x55d53e6034d0 2024-09-21T03:53:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.093+0000 7f3fdfe00640 1 -- 172.21.15.16:0/3303798803 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0045040 con 0x55d53e6034d0 2024-09-21T03:53:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.094+0000 7f40017d1740 1 -- 172.21.15.16:0/3303798803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6034d0 msgr2=0x55d53e9cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.094+0000 7f40017d1740 1 --2- 172.21.15.16:0/3303798803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6034d0 0x55d53e9cefd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0030810 tx=0x7f3fe00056d0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.094+0000 7f40017d1740 1 -- 172.21.15.16:0/3303798803 shutdown_connections 2024-09-21T03:53:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.094+0000 7f40017d1740 1 --2- 172.21.15.16:0/3303798803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e6241b0 0x55d53e9cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.094+0000 7f40017d1740 1 --2- 172.21.15.16:0/3303798803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e6034d0 0x55d53e9cefd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.094+0000 7f40017d1740 1 --2- 172.21.15.16:0/3303798803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e5fb2f0 0x55d53e9d3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.094+0000 7f40017d1740 1 -- 172.21.15.16:0/3303798803 >> 172.21.15.16:0/3303798803 conn(0x55d53e618300 msgr2=0x55d53e9d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.095+0000 7f40017d1740 1 -- 172.21.15.16:0/3303798803 shutdown_connections 2024-09-21T03:53:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.095+0000 7f40017d1740 1 -- 172.21.15.16:0/3303798803 wait complete. 2024-09-21T03:53:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.095+0000 7f40017d1740 1 -- start start 2024-09-21T03:53:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.095+0000 7f40017d1740 1 Processor -- start 2024-09-21T03:53:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.096+0000 7f40017d1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e5fb2f0 0x55d53e9dc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.096+0000 7f40017d1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e6034d0 0x55d53e9db200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.096+0000 7f40017d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e6241b0 0x55d53e9db740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.096+0000 7f40017d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd40110c0 con 0x55d53e6241b0 2024-09-21T03:53:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.096+0000 7f40017d1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd40109f0 con 0x55d53e6034d0 2024-09-21T03:53:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.096+0000 7f40017d1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd400dc30 con 0x55d53e5fb2f0 2024-09-21T03:53:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.096+0000 7f40017d1740 10 client.0 _ll_register_callbacks cb 0x7f3fec00bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.096+0000 7f3ff3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e5fb2f0 0x55d53e9dc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.096+0000 7f3ff3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e5fb2f0 0x55d53e9dc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:38446/0 (socket says 172.21.15.16:38446) 2024-09-21T03:53:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.096+0000 7f3ff3400640 1 -- 172.21.15.16:0/1448048822 learned_addr learned my addr 172.21.15.16:0/1448048822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.096+0000 7f3ff2a00640 1 --2- 172.21.15.16:0/1448048822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e6034d0 0x55d53e9db200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.096+0000 7f3ff3e00640 1 --2- 172.21.15.16:0/1448048822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e6241b0 0x55d53e9db740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.097+0000 7f3ff3400640 1 -- 172.21.15.16:0/1448048822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e6034d0 msgr2=0x55d53e9db200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.097+0000 7f3ff3400640 1 --2- 172.21.15.16:0/1448048822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e6034d0 0x55d53e9db200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.097+0000 7f3ff3400640 1 -- 172.21.15.16:0/1448048822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e6241b0 msgr2=0x55d53e9db740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.097+0000 7f3ff3400640 1 --2- 172.21.15.16:0/1448048822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e6241b0 0x55d53e9db740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.097+0000 7f3ff3400640 1 -- 172.21.15.16:0/1448048822 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d53e9dbcb0 con 0x55d53e5fb2f0 2024-09-21T03:53:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.097+0000 7f3ff3400640 1 --2- 172.21.15.16:0/1448048822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e5fb2f0 0x55d53e9dc540 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4006010 tx=0x7f3fe4012f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6351a4c6fced24a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.097+0000 7f3fdd600640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:53:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.097+0000 7f3fdd600640 1 -- 172.21.15.16:0/1448048822 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4032060 con 0x55d53e5fb2f0 2024-09-21T03:53:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.097+0000 7f40017d1740 1 -- 172.21.15.16:0/1448048822 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe0003ca0 con 0x55d53e5fb2f0 2024-09-21T03:53:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.097+0000 7f3fdd600640 1 -- 172.21.15.16:0/1448048822 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f3fe4009810 con 0x55d53e5fb2f0 2024-09-21T03:53:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.097+0000 7f3fdd600640 1 -- 172.21.15.16:0/1448048822 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3fe4039050 con 0x55d53e5fb2f0 2024-09-21T03:53:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.098+0000 7f40017d1740 10 client.26245 fetch_fsmap learned FSMap version 30 2024-09-21T03:53:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.098+0000 7f40017d1740 1 -- 172.21.15.16:0/1448048822 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d53e9dab40 con 0x55d53e5fb2f0 2024-09-21T03:53:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.098+0000 7f3fdd600640 1 -- 172.21.15.16:0/1448048822 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe0003ca0 con 0x55d53e5fb2f0 2024-09-21T03:53:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.098+0000 7f3fdd600640 1 -- 172.21.15.16:0/1448048822 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3fe4003710 con 0x55d53e5fb2f0 2024-09-21T03:53:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.098+0000 7f40017d1740 10 client.26245 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:53:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.098+0000 7f40017d1740 10 client.26245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.098+0000 7f40017d1740 20 client.26245 populate_metadata read hostname 'smithi016' 2024-09-21T03:53:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.098+0000 7f40017d1740 1 --2- 172.21.15.16:0/1448048822 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d53e5b81c0 0x55d53ea57780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.098+0000 7f40017d1740 4 client.26245 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:53:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.098+0000 7f40017d1740 1 -- 172.21.15.16:0/1448048822 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d53e93c0b0 con 0x55d53e5b81c0 2024-09-21T03:53:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.098+0000 7f3ff2a00640 1 --2- 172.21.15.16:0/1448048822 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d53e5b81c0 0x55d53ea57780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.099+0000 7f3ff2a00640 1 --2- 172.21.15.16:0/1448048822 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d53e5b81c0 0x55d53ea57780 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a10b5536d9e437a2 server_cookie=f3b3e531804c60ec in_seq=0 out_seq=0 2024-09-21T03:53:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.099+0000 7f3fdd600640 10 client.26245 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.518+0000 7f3fdd600640 1 -- 172.21.15.16:0/1448048822 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d53e93c0b0 con 0x55d53e5b81c0 2024-09-21T03:53:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.518+0000 7f3fdd600640 10 client.26245 handle_command_reply: tid=0 2024-09-21T03:53:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.518+0000 7f40017d1740 1 client.26245 shutdown 2024-09-21T03:53:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.520+0000 7f40017d1740 1 -- 172.21.15.16:0/1448048822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e5fb2f0 msgr2=0x55d53e9dc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.520+0000 7f40017d1740 1 --2- 172.21.15.16:0/1448048822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e5fb2f0 0x55d53e9dc540 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4006010 tx=0x7f3fe4012f60 comp rx=0 tx=0).stop 2024-09-21T03:53:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.520+0000 7f40017d1740 1 -- 172.21.15.16:0/1448048822 shutdown_connections 2024-09-21T03:53:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.520+0000 7f40017d1740 1 --2- 172.21.15.16:0/1448048822 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d53e5b81c0 0x55d53ea57780 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.520+0000 7f40017d1740 1 --2- 172.21.15.16:0/1448048822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d53e6241b0 0x55d53e9db740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.520+0000 7f40017d1740 1 --2- 172.21.15.16:0/1448048822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d53e6034d0 0x55d53e9db200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.520+0000 7f40017d1740 1 --2- 172.21.15.16:0/1448048822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d53e5fb2f0 0x55d53e9dc540 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.520+0000 7f40017d1740 1 -- 172.21.15.16:0/1448048822 >> 172.21.15.16:0/1448048822 conn(0x55d53e618300 msgr2=0x55d53e9c7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.520+0000 7f3fdd600640 0 client.26245 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.521+0000 7f40017d1740 1 -- 172.21.15.16:0/1448048822 shutdown_connections 2024-09-21T03:53:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.521+0000 7f40017d1740 1 -- 172.21.15.16:0/1448048822 wait complete. 2024-09-21T03:53:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.521+0000 7f40017d1740 20 client.26245 trim_cache size 0 max 16384 2024-09-21T03:53:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:53:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:53:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:53:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:53:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.523+0000 7f3ff9000640 1 -- 172.21.15.16:0/2544285817 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fd80b6790 msgr2=0x7f3fd80b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.523+0000 7f3ff9000640 1 --2- 172.21.15.16:0/2544285817 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fd80b6790 0x7f3fd80b8ca0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00095f0 tx=0x7f3fe0030470 comp rx=0 tx=0).stop 2024-09-21T03:53:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.523+0000 7f3ff9000640 1 -- 172.21.15.16:0/2544285817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff4111c40 msgr2=0x7f3ff41a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.523+0000 7f3ff9000640 1 --2- 172.21.15.16:0/2544285817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff4111c40 0x7f3ff41a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00ff40 tx=0x7f3fec00ff70 comp rx=0 tx=0).stop 2024-09-21T03:53:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.524+0000 7f3ff9000640 1 -- 172.21.15.16:0/2544285817 shutdown_connections 2024-09-21T03:53:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.524+0000 7f3ff9000640 1 --2- 172.21.15.16:0/2544285817 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3fd80b6790 0x7f3fd80b8ca0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.524+0000 7f3ff9000640 1 --2- 172.21.15.16:0/2544285817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff4111c40 0x7f3ff41a3600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.524+0000 7f3ff9000640 1 --2- 172.21.15.16:0/2544285817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff410ad10 0x7f3ff41a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.524+0000 7f3ff9000640 1 --2- 172.21.15.16:0/2544285817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff410a3c0 0x7f3ff41a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.524+0000 7f3ff9000640 1 -- 172.21.15.16:0/2544285817 >> 172.21.15.16:0/2544285817 conn(0x7f3ff4070640 msgr2=0x7f3ff410d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.525+0000 7f3ff9000640 1 -- 172.21.15.16:0/2544285817 shutdown_connections 2024-09-21T03:53:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:06.525+0000 7f3ff9000640 1 -- 172.21.15.16:0/2544285817 wait complete. 2024-09-21T03:53:07.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:07 smithi016 ceph-mon[30375]: pgmap v622: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 245 MiB/s wr, 1.47k op/s 2024-09-21T03:53:07.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:07 smithi134 ceph-mon[35031]: pgmap v622: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 245 MiB/s wr, 1.47k op/s 2024-09-21T03:53:07.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:07 smithi123 ceph-mon[34673]: pgmap v622: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 245 MiB/s wr, 1.47k op/s 2024-09-21T03:53:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.959+0000 7f3efd800640 1 -- 172.21.15.16:0/3821424843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ef810dc20 msgr2=0x7f3ef8110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.959+0000 7f3efd800640 1 --2- 172.21.15.16:0/3821424843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ef810dc20 0x7f3ef8110110 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009be0 tx=0x7f3ef002e840 comp rx=0 tx=0).stop 2024-09-21T03:53:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.959+0000 7f3efd800640 1 -- 172.21.15.16:0/3821424843 shutdown_connections 2024-09-21T03:53:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.959+0000 7f3efd800640 1 --2- 172.21.15.16:0/3821424843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ef810dc20 0x7f3ef8110110 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.959+0000 7f3efd800640 1 --2- 172.21.15.16:0/3821424843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef8105110 0x7f3ef810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.959+0000 7f3efd800640 1 --2- 172.21.15.16:0/3821424843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ef81047c0 0x7f3ef8104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.959+0000 7f3efd800640 1 -- 172.21.15.16:0/3821424843 >> 172.21.15.16:0/3821424843 conn(0x7f3ef80fe2c0 msgr2=0x7f3ef8100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.959+0000 7f3efd800640 1 -- 172.21.15.16:0/3821424843 shutdown_connections 2024-09-21T03:53:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.959+0000 7f3efd800640 1 -- 172.21.15.16:0/3821424843 wait complete. 2024-09-21T03:53:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.960+0000 7f3efd800640 1 Processor -- start 2024-09-21T03:53:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.961+0000 7f3efd800640 1 -- start start 2024-09-21T03:53:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.961+0000 7f3efd800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ef81047c0 0x7f3ef81a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.961+0000 7f3efd800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ef8105110 0x7f3ef81a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.961+0000 7f3efd800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef810dc20 0x7f3ef81a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.961+0000 7f3efd800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef8111690 con 0x7f3ef81047c0 2024-09-21T03:53:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.961+0000 7f3efd800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef8111510 con 0x7f3ef810dc20 2024-09-21T03:53:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.961+0000 7f3efd800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef8111810 con 0x7f3ef8105110 2024-09-21T03:53:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.962+0000 7f3efce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef810dc20 0x7f3ef81a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.962+0000 7f3efce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef810dc20 0x7f3ef81a60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44664/0 (socket says 172.21.15.16:44664) 2024-09-21T03:53:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.962+0000 7f3efce00640 1 -- 172.21.15.16:0/1014263612 learned_addr learned my addr 172.21.15.16:0/1014263612 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.962+0000 7f3ef7400640 1 --2- 172.21.15.16:0/1014263612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ef8105110 0x7f3ef81a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.962+0000 7f3ef7e00640 1 --2- 172.21.15.16:0/1014263612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ef81047c0 0x7f3ef81a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.962+0000 7f3ef7400640 1 -- 172.21.15.16:0/1014263612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef810dc20 msgr2=0x7f3ef81a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.962+0000 7f3ef7400640 1 --2- 172.21.15.16:0/1014263612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef810dc20 0x7f3ef81a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.962+0000 7f3ef7400640 1 -- 172.21.15.16:0/1014263612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ef81047c0 msgr2=0x7f3ef81a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.962+0000 7f3ef7400640 1 --2- 172.21.15.16:0/1014263612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ef81047c0 0x7f3ef81a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.962+0000 7f3ef7400640 1 -- 172.21.15.16:0/1014263612 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef81a6820 con 0x7f3ef8105110 2024-09-21T03:53:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.963+0000 7f3ef7400640 1 --2- 172.21.15.16:0/1014263612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ef8105110 0x7f3ef81a0e50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3ef806aa90 tx=0x7f3ee800db30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.963+0000 7f3ef4c00640 1 -- 172.21.15.16:0/1014263612 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8017070 con 0x7f3ef8105110 2024-09-21T03:53:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.963+0000 7f3ef4c00640 1 -- 172.21.15.16:0/1014263612 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee8010460 con 0x7f3ef8105110 2024-09-21T03:53:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.963+0000 7f3ef4c00640 1 -- 172.21.15.16:0/1014263612 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee800f620 con 0x7f3ef8105110 2024-09-21T03:53:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.963+0000 7f3efd800640 1 -- 172.21.15.16:0/1014263612 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef81a6b10 con 0x7f3ef8105110 2024-09-21T03:53:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.963+0000 7f3efd800640 1 -- 172.21.15.16:0/1014263612 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef81ad270 con 0x7f3ef8105110 2024-09-21T03:53:07.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.968+0000 7f3f059f2740 1 Processor -- start 2024-09-21T03:53:07.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.968+0000 7f3f059f2740 1 -- start start 2024-09-21T03:53:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.969+0000 7f3f059f2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a456be520 0x556a45a65340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.969+0000 7f3f059f2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a45695660 0x556a45a65880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.969+0000 7f3f059f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a65dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.969+0000 7f3f059f2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a459dc290 con 0x556a45695660 2024-09-21T03:53:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.969+0000 7f3f059f2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556a459cb6c0 con 0x556a456be520 2024-09-21T03:53:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.969+0000 7f3f059f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556a459cf4d0 con 0x556a4569d840 2024-09-21T03:53:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.969+0000 7f3ef4c00640 1 -- 172.21.15.16:0/1014263612 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3ee801a070 con 0x7f3ef8105110 2024-09-21T03:53:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.970+0000 7f3efce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a456be520 0x556a45a65340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.970+0000 7f3efce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a456be520 0x556a45a65340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44666/0 (socket says 172.21.15.16:44666) 2024-09-21T03:53:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.970+0000 7f3efce00640 1 -- 172.21.15.16:0/3098888572 learned_addr learned my addr 172.21.15.16:0/3098888572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.970+0000 7f3ef7400640 1 --2- 172.21.15.16:0/3098888572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a65dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.970+0000 7f3efce00640 1 -- 172.21.15.16:0/3098888572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 msgr2=0x556a45a65dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.970+0000 7f3efce00640 1 --2- 172.21.15.16:0/3098888572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a65dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.970+0000 7f3efce00640 1 -- 172.21.15.16:0/3098888572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a45695660 msgr2=0x556a45a65880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.970+0000 7f3efce00640 1 --2- 172.21.15.16:0/3098888572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a45695660 0x556a45a65880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.970+0000 7f3efce00640 1 -- 172.21.15.16:0/3098888572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a45958460 con 0x556a456be520 2024-09-21T03:53:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.970+0000 7f3ef7400640 1 --2- 172.21.15.16:0/3098888572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a65dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.972+0000 7f3ef4c00640 1 --2- 172.21.15.16:0/1014263612 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3edc0b6590 0x7f3edc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:07.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.973+0000 7f3ef4c00640 1 -- 172.21.15.16:0/1014263612 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f3ee80e6f40 con 0x7f3ef8105110 2024-09-21T03:53:07.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.973+0000 7f3ef7e00640 1 --2- 172.21.15.16:0/1014263612 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3edc0b6590 0x7f3edc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:07.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.974+0000 7f3efce00640 1 --2- 172.21.15.16:0/3098888572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a456be520 0x556a45a65340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0035f90 tx=0x7f3ef0035fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8232d37424e7cca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:07.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.975+0000 7f3eed600640 1 -- 172.21.15.16:0/3098888572 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0043020 con 0x556a456be520 2024-09-21T03:53:07.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.975+0000 7f3eed600640 1 -- 172.21.15.16:0/3098888572 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef0035b70 con 0x556a456be520 2024-09-21T03:53:07.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.975+0000 7f3eed600640 1 -- 172.21.15.16:0/3098888572 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef003b040 con 0x556a456be520 2024-09-21T03:53:07.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.976+0000 7f3f059f2740 1 -- 172.21.15.16:0/3098888572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a456be520 msgr2=0x556a45a65340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:07.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.976+0000 7f3ef7e00640 1 --2- 172.21.15.16:0/1014263612 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3edc0b6590 0x7f3edc0b8aa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x556a45a63fc0 tx=0x7f3ee40097d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:07.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.976+0000 7f3f059f2740 1 --2- 172.21.15.16:0/3098888572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a456be520 0x556a45a65340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0035f90 tx=0x7f3ef0035fc0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.977+0000 7f3f059f2740 1 -- 172.21.15.16:0/3098888572 shutdown_connections 2024-09-21T03:53:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.977+0000 7f3f059f2740 1 --2- 172.21.15.16:0/3098888572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a65dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.977+0000 7f3f059f2740 1 --2- 172.21.15.16:0/3098888572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a45695660 0x556a45a65880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.977+0000 7f3f059f2740 1 --2- 172.21.15.16:0/3098888572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a456be520 0x556a45a65340 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.977+0000 7f3f059f2740 1 -- 172.21.15.16:0/3098888572 >> 172.21.15.16:0/3098888572 conn(0x556a456b2910 msgr2=0x556a45a63230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.978+0000 7f3f059f2740 1 -- 172.21.15.16:0/3098888572 shutdown_connections 2024-09-21T03:53:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.978+0000 7f3f059f2740 1 -- 172.21.15.16:0/3098888572 wait complete. 2024-09-21T03:53:07.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.979+0000 7f3f059f2740 1 -- start start 2024-09-21T03:53:07.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.979+0000 7f3f059f2740 1 Processor -- start 2024-09-21T03:53:07.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.980+0000 7f3f059f2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a45695660 0x556a45a70f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:07.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.980+0000 7f3f059f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a71510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:07.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.980+0000 7f3f059f2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a456be520 0x556a45a7cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:07.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.980+0000 7f3f059f2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a4582cbb0 con 0x556a45695660 2024-09-21T03:53:07.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.980+0000 7f3f059f2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556a459666c0 con 0x556a456be520 2024-09-21T03:53:07.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.980+0000 7f3f059f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556a45a73ac0 con 0x556a4569d840 2024-09-21T03:53:07.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.980+0000 7f3f059f2740 10 client.0 _ll_register_callbacks cb 0x556a45981490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:07.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.980+0000 7f3ef7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a71510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:07.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.980+0000 7f3ef7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a71510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:38474/0 (socket says 172.21.15.16:38474) 2024-09-21T03:53:07.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.980+0000 7f3ef7400640 1 -- 172.21.15.16:0/2966438953 learned_addr learned my addr 172.21.15.16:0/2966438953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:07.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.980+0000 7f3efce00640 1 --2- 172.21.15.16:0/2966438953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a456be520 0x556a45a7cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:07.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.980+0000 7f3ef7e00640 1 --2- 172.21.15.16:0/2966438953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a45695660 0x556a45a70f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:07.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.981+0000 7f3efce00640 1 -- 172.21.15.16:0/2966438953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 msgr2=0x556a45a71510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:07.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.981+0000 7f3efce00640 1 --2- 172.21.15.16:0/2966438953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a71510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.981+0000 7f3efce00640 1 -- 172.21.15.16:0/2966438953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a45695660 msgr2=0x556a45a70f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.982+0000 7f3efce00640 1 --2- 172.21.15.16:0/2966438953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a45695660 0x556a45a70f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.982+0000 7f3efce00640 1 -- 172.21.15.16:0/2966438953 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a45729b00 con 0x556a456be520 2024-09-21T03:53:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.982+0000 7f3ef7400640 1 --2- 172.21.15.16:0/2966438953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a71510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.982+0000 7f3ef7e00640 1 --2- 172.21.15.16:0/2966438953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a45695660 0x556a45a70f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.982+0000 7f3efce00640 1 --2- 172.21.15.16:0/2966438953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a456be520 0x556a45a7cdb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0039040 tx=0x7f3ef000f580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3180f82f4e83ff2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.982+0000 7f3ee2000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:53:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.983+0000 7f3ee2000640 1 -- 172.21.15.16:0/2966438953 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0043030 con 0x556a456be520 2024-09-21T03:53:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.983+0000 7f3ee2000640 1 -- 172.21.15.16:0/2966438953 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f3ef0004150 con 0x556a456be520 2024-09-21T03:53:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.983+0000 7f3f059f2740 1 -- 172.21.15.16:0/2966438953 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ef0035b70 con 0x556a456be520 2024-09-21T03:53:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.983+0000 7f3ee2000640 1 -- 172.21.15.16:0/2966438953 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ef003b040 con 0x556a456be520 2024-09-21T03:53:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.983+0000 7f3f059f2740 10 client.26315 fetch_fsmap learned FSMap version 30 2024-09-21T03:53:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.983+0000 7f3f059f2740 1 -- 172.21.15.16:0/2966438953 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556a457a0960 con 0x556a456be520 2024-09-21T03:53:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.984+0000 7f3ee2000640 1 -- 172.21.15.16:0/2966438953 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef0003970 con 0x556a456be520 2024-09-21T03:53:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.984+0000 7f3ee2000640 1 -- 172.21.15.16:0/2966438953 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3ef003e030 con 0x556a456be520 2024-09-21T03:53:07.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.984+0000 7f3f059f2740 10 client.26315 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:53:07.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.984+0000 7f3f059f2740 10 client.26315 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:53:07.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.984+0000 7f3f059f2740 20 client.26315 populate_metadata read hostname 'smithi016' 2024-09-21T03:53:07.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.984+0000 7f3f059f2740 1 --2- 172.21.15.16:0/2966438953 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556a4567cea0 0x556a45a82cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:07.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.984+0000 7f3f059f2740 4 client.26315 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:07.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.984+0000 7f3f059f2740 1 -- 172.21.15.16:0/2966438953 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3ef0035b70 con 0x556a4567cea0 2024-09-21T03:53:07.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.984+0000 7f3ef7e00640 1 --2- 172.21.15.16:0/2966438953 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556a4567cea0 0x556a45a82cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:07.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.985+0000 7f3ee2000640 10 client.26315 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:07.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.986+0000 7f3ef7e00640 1 --2- 172.21.15.16:0/2966438953 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556a4567cea0 0x556a45a82cb0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85b86dd507861347 server_cookie=a3f3dfcd30074d9f in_seq=0 out_seq=0 2024-09-21T03:53:07.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.987+0000 7f3ee2000640 1 -- 172.21.15.16:0/2966438953 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f3ef0035b70 con 0x556a4567cea0 2024-09-21T03:53:07.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.987+0000 7f3ee2000640 10 client.26315 handle_command_reply: tid=0 2024-09-21T03:53:07.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.990+0000 7f3f059f2740 1 client.26315 shutdown 2024-09-21T03:53:07.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.991+0000 7f3f059f2740 1 -- 172.21.15.16:0/2966438953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a456be520 msgr2=0x556a45a7cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:07.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.991+0000 7f3f059f2740 1 --2- 172.21.15.16:0/2966438953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a456be520 0x556a45a7cdb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0039040 tx=0x7f3ef000f580 comp rx=0 tx=0).stop 2024-09-21T03:53:07.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.994+0000 7f3f059f2740 1 -- 172.21.15.16:0/2966438953 shutdown_connections 2024-09-21T03:53:07.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.994+0000 7f3f059f2740 1 --2- 172.21.15.16:0/2966438953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a456be520 0x556a45a7cdb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.994+0000 7f3f059f2740 1 --2- 172.21.15.16:0/2966438953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a71510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.994+0000 7f3f059f2740 1 --2- 172.21.15.16:0/2966438953 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556a4567cea0 0x556a45a82cb0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.994+0000 7f3f059f2740 1 --2- 172.21.15.16:0/2966438953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a45695660 0x556a45a70f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:07.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.994+0000 7f3f059f2740 1 -- 172.21.15.16:0/2966438953 >> 172.21.15.16:0/2966438953 conn(0x556a456b2910 msgr2=0x556a45a68520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:07.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.994+0000 7f3ee2000640 0 client.26315 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:08.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.998+0000 7f3f059f2740 1 -- 172.21.15.16:0/2966438953 shutdown_connections 2024-09-21T03:53:08.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:07.999+0000 7f3f059f2740 1 -- 172.21.15.16:0/2966438953 wait complete. 2024-09-21T03:53:08.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.000+0000 7f3f059f2740 20 client.26315 trim_cache size 0 max 16384 2024-09-21T03:53:08.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.059+0000 7f3f059f2740 1 Processor -- start 2024-09-21T03:53:08.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.059+0000 7f3f059f2740 1 -- start start 2024-09-21T03:53:08.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.060+0000 7f3f059f2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a45695660 0x556a45a68440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.060+0000 7f3f059f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a68980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.060+0000 7f3f059f2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a456be520 0x556a45a6a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.060+0000 7f3f059f2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef81b6fe0 con 0x556a456be520 2024-09-21T03:53:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.060+0000 7f3f059f2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef81b8b20 con 0x556a45695660 2024-09-21T03:53:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.060+0000 7f3f059f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef81b76b0 con 0x556a4569d840 2024-09-21T03:53:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.062+0000 7f3efce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a456be520 0x556a45a6a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.062+0000 7f3efce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a456be520 0x556a45a6a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34638/0 (socket says 172.21.15.16:34638) 2024-09-21T03:53:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.062+0000 7f3efce00640 1 -- 172.21.15.16:0/3883980705 learned_addr learned my addr 172.21.15.16:0/3883980705 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.063+0000 7f3ef7e00640 1 --2- 172.21.15.16:0/3883980705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a45695660 0x556a45a68440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.063+0000 7f3ef7400640 1 --2- 172.21.15.16:0/3883980705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a68980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.063+0000 7f3efce00640 1 -- 172.21.15.16:0/3883980705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 msgr2=0x556a45a68980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.063+0000 7f3efce00640 1 --2- 172.21.15.16:0/3883980705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a68980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.063+0000 7f3efce00640 1 -- 172.21.15.16:0/3883980705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a45695660 msgr2=0x556a45a68440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.063+0000 7f3efce00640 1 --2- 172.21.15.16:0/3883980705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a45695660 0x556a45a68440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.063+0000 7f3efce00640 1 -- 172.21.15.16:0/3883980705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a45a6c690 con 0x556a456be520 2024-09-21T03:53:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.064+0000 7f3efce00640 1 --2- 172.21.15.16:0/3883980705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a456be520 0x556a45a6a420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0036960 tx=0x7f3ef0009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b03b2589358fcd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.064+0000 7f3ef7400640 1 --2- 172.21.15.16:0/3883980705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a68980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:53:08.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.065+0000 7f3eecc00640 1 -- 172.21.15.16:0/3883980705 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0043070 con 0x556a456be520 2024-09-21T03:53:08.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.065+0000 7f3eecc00640 1 -- 172.21.15.16:0/3883980705 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef0036470 con 0x556a456be520 2024-09-21T03:53:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.067+0000 7f3f059f2740 1 -- 172.21.15.16:0/3883980705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a456be520 msgr2=0x556a45a6a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.067+0000 7f3f059f2740 1 --2- 172.21.15.16:0/3883980705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a456be520 0x556a45a6a420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0036960 tx=0x7f3ef0009a60 comp rx=0 tx=0).stop 2024-09-21T03:53:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.068+0000 7f3f059f2740 1 -- 172.21.15.16:0/3883980705 shutdown_connections 2024-09-21T03:53:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.068+0000 7f3f059f2740 1 --2- 172.21.15.16:0/3883980705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a456be520 0x556a45a6a420 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.068+0000 7f3f059f2740 1 --2- 172.21.15.16:0/3883980705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a68980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.068+0000 7f3f059f2740 1 --2- 172.21.15.16:0/3883980705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a45695660 0x556a45a68440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.068+0000 7f3f059f2740 1 -- 172.21.15.16:0/3883980705 >> 172.21.15.16:0/3883980705 conn(0x556a456b2910 msgr2=0x556a45a6f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:08.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.070+0000 7f3f059f2740 1 -- 172.21.15.16:0/3883980705 shutdown_connections 2024-09-21T03:53:08.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.071+0000 7f3f059f2740 1 -- 172.21.15.16:0/3883980705 wait complete. 2024-09-21T03:53:08.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.072+0000 7f3f059f2740 1 -- start start 2024-09-21T03:53:08.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.073+0000 7f3f059f2740 1 Processor -- start 2024-09-21T03:53:08.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.081+0000 7f3f059f2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a45695660 0x556a45a83f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:08.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.082+0000 7f3ef7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a45695660 0x556a45a83f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:08.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.082+0000 7f3ef7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a45695660 0x556a45a83f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44700/0 (socket says 172.21.15.16:44700) 2024-09-21T03:53:08.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.082+0000 7f3ef7e00640 1 -- 172.21.15.16:0/1479957174 learned_addr learned my addr 172.21.15.16:0/1479957174 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:08.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.082+0000 7f3f059f2740 1 --2- 172.21.15.16:0/1479957174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:08.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.083+0000 7f3f059f2740 1 --2- 172.21.15.16:0/1479957174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a456be520 0x556a45aec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:08.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.087+0000 7f3ef7400640 1 --2- 172.21.15.16:0/1479957174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.088+0000 7f3f059f2740 1 -- 172.21.15.16:0/1479957174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef81b8450 con 0x556a456be520 2024-09-21T03:53:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.088+0000 7f3f059f2740 1 -- 172.21.15.16:0/1479957174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef81b7d80 con 0x556a45695660 2024-09-21T03:53:08.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.088+0000 7f3f059f2740 1 -- 172.21.15.16:0/1479957174 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef81b4fc0 con 0x556a4569d840 2024-09-21T03:53:08.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.089+0000 7f3f059f2740 10 client.0 _ll_register_callbacks cb 0x7f3ee80ea850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:08.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.090+0000 7f3ef7e00640 1 -- 172.21.15.16:0/1479957174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 msgr2=0x556a45a844b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:08.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.090+0000 7f3ef7e00640 1 --2- 172.21.15.16:0/1479957174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a844b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:08.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.090+0000 7f3ef7e00640 1 -- 172.21.15.16:0/1479957174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a456be520 msgr2=0x556a45aec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:08.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.090+0000 7f3ef7e00640 1 --2- 172.21.15.16:0/1479957174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a456be520 0x556a45aec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:08.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.090+0000 7f3ef7e00640 1 -- 172.21.15.16:0/1479957174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a45a84cd0 con 0x556a45695660 2024-09-21T03:53:08.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.091+0000 7f3ef7e00640 1 --2- 172.21.15.16:0/1479957174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a45695660 0x556a45a83f70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4004700 tx=0x7f3ee4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35c6f6765ba29c37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:08.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.091+0000 7f3ee2000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:53:08.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.092+0000 7f3ee2000640 1 -- 172.21.15.16:0/1479957174 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4021760 con 0x556a45695660 2024-09-21T03:53:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.093+0000 7f3f059f2740 1 -- 172.21.15.16:0/1479957174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ef0036470 con 0x556a45695660 2024-09-21T03:53:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.093+0000 7f3ee2000640 1 -- 172.21.15.16:0/1479957174 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f3ee4021900 con 0x556a45695660 2024-09-21T03:53:08.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.094+0000 7f3ee2000640 1 -- 172.21.15.16:0/1479957174 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ee4021b00 con 0x556a45695660 2024-09-21T03:53:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.096+0000 7f3f059f2740 10 client.26321 fetch_fsmap learned FSMap version 30 2024-09-21T03:53:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.096+0000 7f3f059f2740 1 -- 172.21.15.16:0/1479957174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556a45aed980 con 0x556a45695660 2024-09-21T03:53:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.097+0000 7f3ee2000640 1 -- 172.21.15.16:0/1479957174 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef0036470 con 0x556a45695660 2024-09-21T03:53:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.098+0000 7f3ee2000640 1 -- 172.21.15.16:0/1479957174 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3ee401fcc0 con 0x556a45695660 2024-09-21T03:53:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.098+0000 7f3f059f2740 10 client.26321 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:53:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.098+0000 7f3f059f2740 10 client.26321 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:53:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.098+0000 7f3f059f2740 20 client.26321 populate_metadata read hostname 'smithi016' 2024-09-21T03:53:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.098+0000 7f3f059f2740 1 --2- 172.21.15.16:0/1479957174 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556a45652250 0x556a45af0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:08.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.098+0000 7f3f059f2740 4 client.26321 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fd60747c", "release": true, "await": true}] 2024-09-21T03:53:08.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.098+0000 7f3f059f2740 1 -- 172.21.15.16:0/1479957174 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fd60747c", "release": true, "await": true}) -- 0x556a459de1e0 con 0x556a45652250 2024-09-21T03:53:08.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.100+0000 7f3ef7400640 1 --2- 172.21.15.16:0/1479957174 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556a45652250 0x556a45af0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:08.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.101+0000 7f3ef7400640 1 --2- 172.21.15.16:0/1479957174 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556a45652250 0x556a45af0190 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f6d9336e8027742 server_cookie=6e625013376697f6 in_seq=0 out_seq=0 2024-09-21T03:53:08.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.102+0000 7f3ee2000640 10 client.26321 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:08.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.107+0000 7f3ee2000640 1 -- 172.21.15.16:0/1479957174 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556a459de1e0 con 0x556a45652250 2024-09-21T03:53:08.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.107+0000 7f3ee2000640 10 client.26321 handle_command_reply: tid=0 2024-09-21T03:53:08.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.108+0000 7f3f059f2740 1 client.26321 shutdown 2024-09-21T03:53:08.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.112+0000 7f3f059f2740 1 -- 172.21.15.16:0/1479957174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a45695660 msgr2=0x556a45a83f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:08.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.112+0000 7f3f059f2740 1 --2- 172.21.15.16:0/1479957174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a45695660 0x556a45a83f70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4004700 tx=0x7f3ee4014000 comp rx=0 tx=0).stop 2024-09-21T03:53:08.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.112+0000 7f3f059f2740 1 -- 172.21.15.16:0/1479957174 shutdown_connections 2024-09-21T03:53:08.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.112+0000 7f3f059f2740 1 --2- 172.21.15.16:0/1479957174 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x556a45652250 0x556a45af0190 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:08.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.112+0000 7f3f059f2740 1 --2- 172.21.15.16:0/1479957174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a456be520 0x556a45aec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:08.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.112+0000 7f3f059f2740 1 --2- 172.21.15.16:0/1479957174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a4569d840 0x556a45a844b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:08.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.112+0000 7f3f059f2740 1 --2- 172.21.15.16:0/1479957174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556a45695660 0x556a45a83f70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:08.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.113+0000 7f3f059f2740 1 -- 172.21.15.16:0/1479957174 >> 172.21.15.16:0/1479957174 conn(0x556a456b2910 msgr2=0x556a45a6f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:08.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.113+0000 7f3ee2000640 0 client.26321 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:08.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.115+0000 7f3f059f2740 1 -- 172.21.15.16:0/1479957174 shutdown_connections 2024-09-21T03:53:08.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.115+0000 7f3f059f2740 1 -- 172.21.15.16:0/1479957174 wait complete. 2024-09-21T03:53:08.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.116+0000 7f3f059f2740 20 client.26321 trim_cache size 0 max 16384 2024-09-21T03:53:08.121 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:53:08.121 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:53:08.121 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:53:08.121 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 28, 2024-09-21T03:53:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:53:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "fd60747c": { 2024-09-21T03:53:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 28, 2024-09-21T03:53:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:53:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:53:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:53:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:53:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:53:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:53:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 139.2, 2024-09-21T03:53:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:53:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:53:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:53:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:53:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:53:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:53:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:53:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:53:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:53:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:53:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:53:08.123 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:53:08.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.118+0000 7f3efd800640 1 -- 172.21.15.16:0/1014263612 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3edc0b6590 msgr2=0x7f3edc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:08.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.118+0000 7f3efd800640 1 --2- 172.21.15.16:0/1014263612 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3edc0b6590 0x7f3edc0b8aa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x556a45a63fc0 tx=0x7f3ee40097d0 comp rx=0 tx=0).stop 2024-09-21T03:53:08.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.118+0000 7f3efd800640 1 -- 172.21.15.16:0/1014263612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ef8105110 msgr2=0x7f3ef81a0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:08.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.118+0000 7f3efd800640 1 --2- 172.21.15.16:0/1014263612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ef8105110 0x7f3ef81a0e50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3ef806aa90 tx=0x7f3ee800db30 comp rx=0 tx=0).stop 2024-09-21T03:53:08.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.119+0000 7f3efd800640 1 -- 172.21.15.16:0/1014263612 shutdown_connections 2024-09-21T03:53:08.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.119+0000 7f3efd800640 1 --2- 172.21.15.16:0/1014263612 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3edc0b6590 0x7f3edc0b8aa0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:08.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.119+0000 7f3efd800640 1 --2- 172.21.15.16:0/1014263612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef810dc20 0x7f3ef81a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:08.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.119+0000 7f3efd800640 1 --2- 172.21.15.16:0/1014263612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ef8105110 0x7f3ef81a0e50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:08.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.119+0000 7f3efd800640 1 --2- 172.21.15.16:0/1014263612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ef81047c0 0x7f3ef81a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:08.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.119+0000 7f3efd800640 1 -- 172.21.15.16:0/1014263612 >> 172.21.15.16:0/1014263612 conn(0x7f3ef80fe2c0 msgr2=0x7f3ef81076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:08.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.120+0000 7f3efd800640 1 -- 172.21.15.16:0/1014263612 shutdown_connections 2024-09-21T03:53:08.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:53:08.122+0000 7f3efd800640 1 -- 172.21.15.16:0/1014263612 wait complete. 2024-09-21T03:53:09.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:09 smithi016 ceph-mon[30375]: pgmap v623: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 236 MiB/s wr, 1.46k op/s 2024-09-21T03:53:09.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:09 smithi134 ceph-mon[35031]: pgmap v623: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 236 MiB/s wr, 1.46k op/s 2024-09-21T03:53:09.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:09 smithi123 ceph-mon[34673]: pgmap v623: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 236 MiB/s wr, 1.46k op/s 2024-09-21T03:53:11.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:11 smithi016 ceph-mon[30375]: pgmap v624: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 235 MiB/s wr, 1.55k op/s 2024-09-21T03:53:11.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:11 smithi134 ceph-mon[35031]: pgmap v624: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 235 MiB/s wr, 1.55k op/s 2024-09-21T03:53:11.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:11 smithi123 ceph-mon[34673]: pgmap v624: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 235 MiB/s wr, 1.55k op/s 2024-09-21T03:53:11.820 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fd60747c', total seconds elapsed: 32.2 2024-09-21T03:53:13.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:13 smithi134 ceph-mon[35031]: pgmap v625: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 227 MiB/s wr, 1.63k op/s 2024-09-21T03:53:13.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:13 smithi123 ceph-mon[34673]: pgmap v625: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 227 MiB/s wr, 1.63k op/s 2024-09-21T03:53:13.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:13 smithi016 ceph-mon[30375]: pgmap v625: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 227 MiB/s wr, 1.63k op/s 2024-09-21T03:53:15.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:15 smithi134 ceph-mon[35031]: pgmap v626: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 210 MiB/s wr, 1.61k op/s 2024-09-21T03:53:15.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:15 smithi123 ceph-mon[34673]: pgmap v626: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 210 MiB/s wr, 1.61k op/s 2024-09-21T03:53:15.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:15 smithi016 ceph-mon[30375]: pgmap v626: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 210 MiB/s wr, 1.61k op/s 2024-09-21T03:53:17.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:17 smithi134 ceph-mon[35031]: pgmap v627: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 209 MiB/s wr, 1.69k op/s 2024-09-21T03:53:17.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:17 smithi123 ceph-mon[34673]: pgmap v627: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 209 MiB/s wr, 1.69k op/s 2024-09-21T03:53:17.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:17 smithi016 ceph-mon[30375]: pgmap v627: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 209 MiB/s wr, 1.69k op/s 2024-09-21T03:53:19.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:19 smithi134 ceph-mon[35031]: pgmap v628: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 200 MiB/s wr, 1.68k op/s 2024-09-21T03:53:19.653 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:19 smithi016 ceph-mon[30375]: pgmap v628: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 200 MiB/s wr, 1.68k op/s 2024-09-21T03:53:19.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:19 smithi123 ceph-mon[34673]: pgmap v628: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 200 MiB/s wr, 1.68k op/s 2024-09-21T03:53:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.125+0000 7fe5b3000640 1 -- 172.21.15.16:0/1058257220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5ac1056f0 msgr2=0x7fe5ac105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.125+0000 7fe5b3000640 1 --2- 172.21.15.16:0/1058257220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5ac1056f0 0x7fe5ac105b60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009900 tx=0x7fe5a002a400 comp rx=0 tx=0).stop 2024-09-21T03:53:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.125+0000 7fe5b3000640 1 -- 172.21.15.16:0/1058257220 shutdown_connections 2024-09-21T03:53:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.125+0000 7fe5b3000640 1 --2- 172.21.15.16:0/1058257220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5ac107110 0x7fe5ac10dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.125+0000 7fe5b3000640 1 --2- 172.21.15.16:0/1058257220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5ac106770 0x7fe5ac106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.125+0000 7fe5b3000640 1 --2- 172.21.15.16:0/1058257220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5ac1056f0 0x7fe5ac105b60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.125+0000 7fe5b3000640 1 -- 172.21.15.16:0/1058257220 >> 172.21.15.16:0/1058257220 conn(0x7fe5ac101050 msgr2=0x7fe5ac1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.126+0000 7fe5b3000640 1 -- 172.21.15.16:0/1058257220 shutdown_connections 2024-09-21T03:53:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.127+0000 7fe5b3000640 1 -- 172.21.15.16:0/1058257220 wait complete. 2024-09-21T03:53:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.128+0000 7fe5b3000640 1 Processor -- start 2024-09-21T03:53:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.129+0000 7fe5b3000640 1 -- start start 2024-09-21T03:53:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.130+0000 7fe5b3000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5ac1056f0 0x7fe5ac19bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.130+0000 7fe5b3000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5ac106770 0x7fe5ac19c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.131+0000 7fe5b3000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5ac107110 0x7fe5ac19d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.131+0000 7fe5b3000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac111360 con 0x7fe5ac106770 2024-09-21T03:53:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.131+0000 7fe5b3000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac1111e0 con 0x7fe5ac1056f0 2024-09-21T03:53:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.131+0000 7fe5b3000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac1114e0 con 0x7fe5ac107110 2024-09-21T03:53:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.131+0000 7fe5b1c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5ac1056f0 0x7fe5ac19bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.131+0000 7fe5b1c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5ac1056f0 0x7fe5ac19bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:38230/0 (socket says 172.21.15.16:38230) 2024-09-21T03:53:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.131+0000 7fe5b1c00640 1 -- 172.21.15.16:0/2420787328 learned_addr learned my addr 172.21.15.16:0/2420787328 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.131+0000 7fe5b1c00640 1 -- 172.21.15.16:0/2420787328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5ac107110 msgr2=0x7fe5ac19d180 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:53:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.131+0000 7fe5b1c00640 1 --2- 172.21.15.16:0/2420787328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5ac107110 0x7fe5ac19d180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.131+0000 7fe5b1c00640 1 -- 172.21.15.16:0/2420787328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5ac106770 msgr2=0x7fe5ac19c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.131+0000 7fe5b1c00640 1 --2- 172.21.15.16:0/2420787328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5ac106770 0x7fe5ac19c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.131+0000 7fe5b1c00640 1 -- 172.21.15.16:0/2420787328 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ac19d9a0 con 0x7fe5ac1056f0 2024-09-21T03:53:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.132+0000 7fe5b1c00640 1 --2- 172.21.15.16:0/2420787328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5ac1056f0 0x7fe5ac19bf60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0031500 tx=0x7fe5a0003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.133+0000 7fe5aa000640 1 -- 172.21.15.16:0/2420787328 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0035070 con 0x7fe5ac1056f0 2024-09-21T03:53:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.133+0000 7fe5aa000640 1 -- 172.21.15.16:0/2420787328 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a00315a0 con 0x7fe5ac1056f0 2024-09-21T03:53:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.133+0000 7fe5aa000640 1 -- 172.21.15.16:0/2420787328 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0047620 con 0x7fe5ac1056f0 2024-09-21T03:53:20.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.133+0000 7fe5b3000640 1 -- 172.21.15.16:0/2420787328 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5ac19dc30 con 0x7fe5ac1056f0 2024-09-21T03:53:20.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.133+0000 7fe5b3000640 1 -- 172.21.15.16:0/2420787328 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5ac077800 con 0x7fe5ac1056f0 2024-09-21T03:53:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.147+0000 7fe5aa000640 1 -- 172.21.15.16:0/2420787328 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fe5a00319e0 con 0x7fe5ac1056f0 2024-09-21T03:53:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.148+0000 7fe5bb6d5740 1 Processor -- start 2024-09-21T03:53:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.148+0000 7fe5bb6d5740 1 -- start start 2024-09-21T03:53:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.148+0000 7fe5bb6d5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4421b0 0x55f4bb7e92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.148+0000 7fe5bb6d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4192f0 0x55f4bb7e9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.148+0000 7fe5bb6d5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4214d0 0x55f4bb7ea470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.148+0000 7fe5bb6d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f4bb75c6b0 con 0x55f4bb4192f0 2024-09-21T03:53:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.148+0000 7fe5bb6d5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f4bb74bae0 con 0x55f4bb4421b0 2024-09-21T03:53:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.148+0000 7fe5bb6d5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f4bb74f8f0 con 0x55f4bb4214d0 2024-09-21T03:53:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.149+0000 7fe5b1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4192f0 0x55f4bb7e9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.150+0000 7fe5b1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4192f0 0x55f4bb7e9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36514/0 (socket says 172.21.15.16:36514) 2024-09-21T03:53:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.150+0000 7fe5b1c00640 1 -- 172.21.15.16:0/3010265324 learned_addr learned my addr 172.21.15.16:0/3010265324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.151+0000 7fe5b1c00640 1 -- 172.21.15.16:0/3010265324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4214d0 msgr2=0x55f4bb7ea470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:53:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.151+0000 7fe5b1c00640 1 --2- 172.21.15.16:0/3010265324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4214d0 0x55f4bb7ea470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.151+0000 7fe5b1c00640 1 -- 172.21.15.16:0/3010265324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4421b0 msgr2=0x55f4bb7e92b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:53:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.151+0000 7fe5b1c00640 1 --2- 172.21.15.16:0/3010265324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4421b0 0x55f4bb7e92b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.151+0000 7fe5b1c00640 1 -- 172.21.15.16:0/3010265324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4bb718780 con 0x55f4bb4192f0 2024-09-21T03:53:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.152+0000 7fe5b1c00640 1 --2- 172.21.15.16:0/3010265324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4192f0 0x55f4bb7e9920 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0113e40 tx=0x7fe5a0113f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a002d18b320d6480 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.153+0000 7fe59ea00640 1 -- 172.21.15.16:0/3010265324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0118c50 con 0x55f4bb4192f0 2024-09-21T03:53:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.154+0000 7fe59ea00640 1 -- 172.21.15.16:0/3010265324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a011d070 con 0x55f4bb4192f0 2024-09-21T03:53:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.156+0000 7fe5aa000640 1 --2- 172.21.15.16:0/2420787328 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5940b66a0 0x7fe5940b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.156+0000 7fe59ea00640 1 -- 172.21.15.16:0/3010265324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0118c50 con 0x55f4bb4192f0 2024-09-21T03:53:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.157+0000 7fe5b2600640 1 --2- 172.21.15.16:0/2420787328 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5940b66a0 0x7fe5940b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.157+0000 7fe5aa000640 1 -- 172.21.15.16:0/2420787328 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fe5a010ff30 con 0x7fe5ac1056f0 2024-09-21T03:53:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.157+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/3010265324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4192f0 msgr2=0x55f4bb7e9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.157+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/3010265324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4192f0 0x55f4bb7e9920 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0113e40 tx=0x7fe5a0113f30 comp rx=0 tx=0).stop 2024-09-21T03:53:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.159+0000 7fe5b2600640 1 --2- 172.21.15.16:0/2420787328 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5940b66a0 0x7fe5940b8bb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40092f0 tx=0x7fe5a4009200 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.159+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/3010265324 shutdown_connections 2024-09-21T03:53:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.159+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/3010265324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4214d0 0x55f4bb7ea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.159+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/3010265324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4192f0 0x55f4bb7e9920 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.159+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/3010265324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4421b0 0x55f4bb7e92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.159+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/3010265324 >> 172.21.15.16:0/3010265324 conn(0x55f4bb436300 msgr2=0x55f4bb7e4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.159+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/3010265324 shutdown_connections 2024-09-21T03:53:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.160+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/3010265324 wait complete. 2024-09-21T03:53:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.161+0000 7fe5bb6d5740 1 -- start start 2024-09-21T03:53:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.161+0000 7fe5bb6d5740 1 Processor -- start 2024-09-21T03:53:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.162+0000 7fe5bb6d5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4192f0 0x55f4bb7f2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.162+0000 7fe5bb6d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4214d0 0x55f4bb7f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.162+0000 7fe5bb6d5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4421b0 0x55f4bb7f3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.162+0000 7fe5b1c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4192f0 0x55f4bb7f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.162+0000 7fe5b1c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4192f0 0x55f4bb7f2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:38242/0 (socket says 172.21.15.16:38242) 2024-09-21T03:53:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.163+0000 7fe5b2600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4421b0 0x55f4bb7f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.163+0000 7fe5b2600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4421b0 0x55f4bb7f3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:50612/0 (socket says 172.21.15.16:50612) 2024-09-21T03:53:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.163+0000 7fe5b1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4214d0 0x55f4bb7f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.163+0000 7fe5b1c00640 1 -- 172.21.15.16:0/1073578955 learned_addr learned my addr 172.21.15.16:0/1073578955 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.163+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/1073578955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f4bb683ab0 con 0x55f4bb4214d0 2024-09-21T03:53:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.163+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/1073578955 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f4bb6f2570 con 0x55f4bb4192f0 2024-09-21T03:53:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.163+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/1073578955 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f4bb7ead50 con 0x55f4bb4421b0 2024-09-21T03:53:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.164+0000 7fe5bb6d5740 10 client.0 _ll_register_callbacks cb 0x55f4bb701100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.164+0000 7fe5b1c00640 1 -- 172.21.15.16:0/1073578955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4421b0 msgr2=0x55f4bb7f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.164+0000 7fe5b1c00640 1 --2- 172.21.15.16:0/1073578955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4421b0 0x55f4bb7f3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.164+0000 7fe5b1c00640 1 -- 172.21.15.16:0/1073578955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4214d0 msgr2=0x55f4bb7f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.164+0000 7fe5b1c00640 1 --2- 172.21.15.16:0/1073578955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4214d0 0x55f4bb7f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.164+0000 7fe5b1c00640 1 -- 172.21.15.16:0/1073578955 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4bb4ad790 con 0x55f4bb4192f0 2024-09-21T03:53:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.164+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/1073578955 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a011d070 con 0x55f4bb4192f0 2024-09-21T03:53:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.164+0000 7fe5b1c00640 1 --2- 172.21.15.16:0/1073578955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4192f0 0x55f4bb7f2660 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00d2e90 tx=0x7fe5a0031d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a4aebd23055a171 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.164+0000 7fe593400640 10 client.26330 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:53:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.165+0000 7fe593400640 1 -- 172.21.15.16:0/1073578955 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a00d2270 con 0x55f4bb4192f0 2024-09-21T03:53:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.165+0000 7fe593400640 1 -- 172.21.15.16:0/1073578955 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fe5a0003b20 con 0x55f4bb4192f0 2024-09-21T03:53:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.165+0000 7fe593400640 1 -- 172.21.15.16:0/1073578955 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe5a00d1390 con 0x55f4bb4192f0 2024-09-21T03:53:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.165+0000 7fe5bb6d5740 10 client.26330 fetch_fsmap learned FSMap version 30 2024-09-21T03:53:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.165+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/1073578955 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55f4bb526620 con 0x55f4bb4192f0 2024-09-21T03:53:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.165+0000 7fe593400640 1 -- 172.21.15.16:0/1073578955 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe5a004f340 con 0x55f4bb4192f0 2024-09-21T03:53:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.165+0000 7fe5bb6d5740 10 client.26330 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:53:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.165+0000 7fe5bb6d5740 10 client.26330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.165+0000 7fe5bb6d5740 20 client.26330 populate_metadata read hostname 'smithi016' 2024-09-21T03:53:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.165+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/1073578955 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55f4bb400d80 0x55f4bb7ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.166+0000 7fe5bb6d5740 4 client.26330 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.166+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/1073578955 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4bb75e600 con 0x55f4bb400d80 2024-09-21T03:53:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.166+0000 7fe5b1200640 1 --2- 172.21.15.16:0/1073578955 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55f4bb400d80 0x55f4bb7ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.167+0000 7fe5b1200640 1 --2- 172.21.15.16:0/1073578955 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55f4bb400d80 0x55f4bb7ff9c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29ccfefbfbbd5705 server_cookie=1eaf5e7c9c0cf09 in_seq=0 out_seq=0 2024-09-21T03:53:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.167+0000 7fe593400640 10 client.26330 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.168+0000 7fe593400640 1 -- 172.21.15.16:0/1073578955 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f4bb75e600 con 0x55f4bb400d80 2024-09-21T03:53:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.168+0000 7fe593400640 10 client.26330 handle_command_reply: tid=0 2024-09-21T03:53:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.168+0000 7fe5bb6d5740 1 client.26330 shutdown 2024-09-21T03:53:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.168+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/1073578955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4192f0 msgr2=0x55f4bb7f2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.168+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/1073578955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4192f0 0x55f4bb7f2660 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00d2e90 tx=0x7fe5a0031d80 comp rx=0 tx=0).stop 2024-09-21T03:53:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.168+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/1073578955 shutdown_connections 2024-09-21T03:53:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.168+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/1073578955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4421b0 0x55f4bb7f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.168+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/1073578955 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55f4bb400d80 0x55f4bb7ff9c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.168+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/1073578955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4214d0 0x55f4bb7f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.168+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/1073578955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4192f0 0x55f4bb7f2660 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.168+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/1073578955 >> 172.21.15.16:0/1073578955 conn(0x55f4bb436300 msgr2=0x55f4bb7e3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.168+0000 7fe593400640 0 client.26330 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.169+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/1073578955 shutdown_connections 2024-09-21T03:53:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.169+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/1073578955 wait complete. 2024-09-21T03:53:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.169+0000 7fe5bb6d5740 20 client.26330 trim_cache size 0 max 16384 2024-09-21T03:53:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.202+0000 7fe5bb6d5740 1 Processor -- start 2024-09-21T03:53:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.207+0000 7fe5bb6d5740 1 -- start start 2024-09-21T03:53:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.207+0000 7fe5bb6d5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4192f0 0x55f4bb7edb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.207+0000 7fe5bb6d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4214d0 0x55f4bb7e4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.207+0000 7fe5bb6d5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4421b0 0x55f4bb7f1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.207+0000 7fe5bb6d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c00fc40 con 0x55f4bb4214d0 2024-09-21T03:53:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.207+0000 7fe5bb6d5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c011780 con 0x55f4bb4192f0 2024-09-21T03:53:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.207+0000 7fe5bb6d5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c010310 con 0x55f4bb4421b0 2024-09-21T03:53:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.209+0000 7fe5b1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4214d0 0x55f4bb7e4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.209+0000 7fe5b1c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4192f0 0x55f4bb7edb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.209+0000 7fe5b1c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4192f0 0x55f4bb7edb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:38244/0 (socket says 172.21.15.16:38244) 2024-09-21T03:53:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.209+0000 7fe5b1c00640 1 -- 172.21.15.16:0/1419379635 learned_addr learned my addr 172.21.15.16:0/1419379635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.209+0000 7fe5b2600640 1 --2- 172.21.15.16:0/1419379635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4421b0 0x55f4bb7f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.209+0000 7fe5b1c00640 1 -- 172.21.15.16:0/1419379635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4421b0 msgr2=0x55f4bb7f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.209+0000 7fe5b1c00640 1 --2- 172.21.15.16:0/1419379635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4421b0 0x55f4bb7f1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.209+0000 7fe5b1c00640 1 -- 172.21.15.16:0/1419379635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4214d0 msgr2=0x55f4bb7e4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.209+0000 7fe5b1c00640 1 --2- 172.21.15.16:0/1419379635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4214d0 0x55f4bb7e4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.209+0000 7fe5b1c00640 1 -- 172.21.15.16:0/1419379635 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4bb7f4450 con 0x55f4bb4192f0 2024-09-21T03:53:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.210+0000 7fe5b1c00640 1 --2- 172.21.15.16:0/1419379635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4192f0 0x55f4bb7edb50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0031440 tx=0x7fe5a00f9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6164905f1cf4049b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.211+0000 7fe59e000640 1 -- 172.21.15.16:0/1419379635 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a00d5420 con 0x55f4bb4192f0 2024-09-21T03:53:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.211+0000 7fe59e000640 1 -- 172.21.15.16:0/1419379635 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a011d070 con 0x55f4bb4192f0 2024-09-21T03:53:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.212+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/1419379635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4192f0 msgr2=0x55f4bb7edb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.212+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/1419379635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4192f0 0x55f4bb7edb50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0031440 tx=0x7fe5a00f9040 comp rx=0 tx=0).stop 2024-09-21T03:53:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.213+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/1419379635 shutdown_connections 2024-09-21T03:53:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.213+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/1419379635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4421b0 0x55f4bb7f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.213+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/1419379635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4214d0 0x55f4bb7e4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.213+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/1419379635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4192f0 0x55f4bb7edb50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.213+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/1419379635 >> 172.21.15.16:0/1419379635 conn(0x55f4bb436300 msgr2=0x55f4bb7e4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.213+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/1419379635 shutdown_connections 2024-09-21T03:53:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.214+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/1419379635 wait complete. 2024-09-21T03:53:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.214+0000 7fe5bb6d5740 1 -- start start 2024-09-21T03:53:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.214+0000 7fe5bb6d5740 1 Processor -- start 2024-09-21T03:53:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.217+0000 7fe5bb6d5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4192f0 0x55f4bb7f5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.218+0000 7fe5b1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4192f0 0x55f4bb7f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.218+0000 7fe5b1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4192f0 0x55f4bb7f5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:50622/0 (socket says 172.21.15.16:50622) 2024-09-21T03:53:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.218+0000 7fe5b1c00640 1 -- 172.21.15.16:0/2430965258 learned_addr learned my addr 172.21.15.16:0/2430965258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.218+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/2430965258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4214d0 0x55f4bb7f6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.219+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/2430965258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4421b0 0x55f4bb7f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.219+0000 7fe5b1200640 1 --2- 172.21.15.16:0/2430965258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4214d0 0x55f4bb7f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.220+0000 7fe5b2600640 1 --2- 172.21.15.16:0/2430965258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4421b0 0x55f4bb7f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.220+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/2430965258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c0110b0 con 0x55f4bb4214d0 2024-09-21T03:53:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.220+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/2430965258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c0109e0 con 0x55f4bb4421b0 2024-09-21T03:53:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.220+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/2430965258 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c00dc20 con 0x55f4bb4192f0 2024-09-21T03:53:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.221+0000 7fe5b2600640 1 -- 172.21.15.16:0/2430965258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4192f0 msgr2=0x55f4bb7f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.221+0000 7fe5b2600640 1 --2- 172.21.15.16:0/2430965258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4192f0 0x55f4bb7f5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.221+0000 7fe5b2600640 1 -- 172.21.15.16:0/2430965258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4214d0 msgr2=0x55f4bb7f6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.221+0000 7fe5bb6d5740 10 client.0 _ll_register_callbacks cb 0x7fe5a0113c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.222+0000 7fe5b2600640 1 --2- 172.21.15.16:0/2430965258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4214d0 0x55f4bb7f6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.222+0000 7fe5b2600640 1 -- 172.21.15.16:0/2430965258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4bb7f72d0 con 0x55f4bb4421b0 2024-09-21T03:53:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.222+0000 7fe5b1c00640 1 --2- 172.21.15.16:0/2430965258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4192f0 0x55f4bb7f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.223+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/2430965258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a011d070 con 0x55f4bb4421b0 2024-09-21T03:53:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.223+0000 7fe5b1200640 1 --2- 172.21.15.16:0/2430965258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4214d0 0x55f4bb7f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.223+0000 7fe5b2600640 1 --2- 172.21.15.16:0/2430965258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4421b0 0x55f4bb7f6b70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4011920 tx=0x7fe5a4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81e641fa607c93e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.223+0000 7fe593400640 10 client.26336 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:53:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.224+0000 7fe593400640 1 -- 172.21.15.16:0/2430965258 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4010d60 con 0x55f4bb4421b0 2024-09-21T03:53:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.224+0000 7fe593400640 1 -- 172.21.15.16:0/2430965258 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fe5a4004b20 con 0x55f4bb4421b0 2024-09-21T03:53:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.225+0000 7fe593400640 1 -- 172.21.15.16:0/2430965258 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe5a4004d60 con 0x55f4bb4421b0 2024-09-21T03:53:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.233+0000 7fe5bb6d5740 10 client.26336 fetch_fsmap learned FSMap version 30 2024-09-21T03:53:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.233+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/2430965258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55f4bb7fa5c0 con 0x55f4bb4421b0 2024-09-21T03:53:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.235+0000 7fe593400640 1 -- 172.21.15.16:0/2430965258 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe5a400f040 con 0x55f4bb4421b0 2024-09-21T03:53:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.235+0000 7fe5bb6d5740 10 client.26336 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:53:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.235+0000 7fe5bb6d5740 10 client.26336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.235+0000 7fe5bb6d5740 20 client.26336 populate_metadata read hostname 'smithi016' 2024-09-21T03:53:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.235+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/2430965258 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55f4bb3d61c0 0x55f4bb87d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.235+0000 7fe5bb6d5740 4 client.26336 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:53:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.235+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/2430965258 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4bb75a0b0 con 0x55f4bb3d61c0 2024-09-21T03:53:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.236+0000 7fe5b1c00640 1 --2- 172.21.15.16:0/2430965258 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55f4bb3d61c0 0x55f4bb87d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.237+0000 7fe593400640 10 client.26336 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.237+0000 7fe5b1c00640 1 --2- 172.21.15.16:0/2430965258 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55f4bb3d61c0 0x55f4bb87d600 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b2009ecbc588994 server_cookie=87ad4a93e543a17f in_seq=0 out_seq=0 2024-09-21T03:53:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.282+0000 7fe593400640 1 -- 172.21.15.16:0/2430965258 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f4bb75a0b0 con 0x55f4bb3d61c0 2024-09-21T03:53:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.282+0000 7fe593400640 10 client.26336 handle_command_reply: tid=0 2024-09-21T03:53:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.282+0000 7fe5bb6d5740 1 client.26336 shutdown 2024-09-21T03:53:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.282+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/2430965258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4421b0 msgr2=0x55f4bb7f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.282+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/2430965258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4421b0 0x55f4bb7f6b70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4011920 tx=0x7fe5a4004ab0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.283+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/2430965258 shutdown_connections 2024-09-21T03:53:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.283+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/2430965258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4bb4421b0 0x55f4bb7f6b70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.283+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/2430965258 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55f4bb3d61c0 0x55f4bb87d600 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.283+0000 7fe593400640 0 client.26336 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.283+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/2430965258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4bb4214d0 0x55f4bb7f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.283+0000 7fe5bb6d5740 1 --2- 172.21.15.16:0/2430965258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bb4192f0 0x55f4bb7f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.283+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/2430965258 >> 172.21.15.16:0/2430965258 conn(0x55f4bb436300 msgr2=0x55f4bb7ed920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.284+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/2430965258 shutdown_connections 2024-09-21T03:53:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.284+0000 7fe5bb6d5740 1 -- 172.21.15.16:0/2430965258 wait complete. 2024-09-21T03:53:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.286+0000 7fe5bb6d5740 20 client.26336 trim_cache size 0 max 16384 2024-09-21T03:53:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:53:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:53:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:53:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:53:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.293+0000 7fe5b3000640 1 -- 172.21.15.16:0/2420787328 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5940b66a0 msgr2=0x7fe5940b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.293+0000 7fe5b3000640 1 --2- 172.21.15.16:0/2420787328 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5940b66a0 0x7fe5940b8bb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40092f0 tx=0x7fe5a4009200 comp rx=0 tx=0).stop 2024-09-21T03:53:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.293+0000 7fe5b3000640 1 -- 172.21.15.16:0/2420787328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5ac1056f0 msgr2=0x7fe5ac19bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.293+0000 7fe5b3000640 1 --2- 172.21.15.16:0/2420787328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5ac1056f0 0x7fe5ac19bf60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0031500 tx=0x7fe5a0003820 comp rx=0 tx=0).stop 2024-09-21T03:53:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.299+0000 7fe5b3000640 1 -- 172.21.15.16:0/2420787328 shutdown_connections 2024-09-21T03:53:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.299+0000 7fe5b3000640 1 --2- 172.21.15.16:0/2420787328 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fe5940b66a0 0x7fe5940b8bb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.299+0000 7fe5b3000640 1 --2- 172.21.15.16:0/2420787328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5ac107110 0x7fe5ac19d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.299+0000 7fe5b3000640 1 --2- 172.21.15.16:0/2420787328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5ac106770 0x7fe5ac19c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.299+0000 7fe5b3000640 1 --2- 172.21.15.16:0/2420787328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5ac1056f0 0x7fe5ac19bf60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.299+0000 7fe5b3000640 1 -- 172.21.15.16:0/2420787328 >> 172.21.15.16:0/2420787328 conn(0x7fe5ac101050 msgr2=0x7fe5ac109a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.299+0000 7fe5b3000640 1 -- 172.21.15.16:0/2420787328 shutdown_connections 2024-09-21T03:53:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:20.300+0000 7fe5b3000640 1 -- 172.21.15.16:0/2420787328 wait complete. 2024-09-21T03:53:21.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:21 smithi134 ceph-mon[35031]: pgmap v629: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 195 MiB/s wr, 1.65k op/s 2024-09-21T03:53:21.616 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:21 smithi016 ceph-mon[30375]: pgmap v629: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 195 MiB/s wr, 1.65k op/s 2024-09-21T03:53:21.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:21 smithi123 ceph-mon[34673]: pgmap v629: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 195 MiB/s wr, 1.65k op/s 2024-09-21T03:53:23.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:23 smithi134 ceph-mon[35031]: pgmap v630: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 199 MiB/s wr, 1.61k op/s 2024-09-21T03:53:23.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:23 smithi123 ceph-mon[34673]: pgmap v630: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 199 MiB/s wr, 1.61k op/s 2024-09-21T03:53:23.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:23 smithi016 ceph-mon[30375]: pgmap v630: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 199 MiB/s wr, 1.61k op/s 2024-09-21T03:53:25.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:25 smithi134 ceph-mon[35031]: pgmap v631: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 194 MiB/s wr, 1.50k op/s 2024-09-21T03:53:25.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:25 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:53:25.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:25 smithi123 ceph-mon[34673]: pgmap v631: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 194 MiB/s wr, 1.50k op/s 2024-09-21T03:53:25.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:25 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:53:25.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:25 smithi016 ceph-mon[30375]: pgmap v631: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 194 MiB/s wr, 1.50k op/s 2024-09-21T03:53:25.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:25 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:53:27.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:27 smithi134 ceph-mon[35031]: pgmap v632: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 229 MiB/s wr, 1.67k op/s 2024-09-21T03:53:27.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:27 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:53:27.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:27 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:53:27.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:27 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:53:27.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:27 smithi123 ceph-mon[34673]: pgmap v632: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 229 MiB/s wr, 1.67k op/s 2024-09-21T03:53:27.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:27 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:53:27.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:27 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:53:27.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:27 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:53:27.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:27 smithi016 ceph-mon[30375]: pgmap v632: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 229 MiB/s wr, 1.67k op/s 2024-09-21T03:53:27.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:27 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:53:27.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:27 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:53:27.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:27 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:53:29.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:29 smithi123 ceph-mon[34673]: pgmap v633: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 225 MiB/s wr, 1.54k op/s 2024-09-21T03:53:29.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:29 smithi016 ceph-mon[30375]: pgmap v633: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 225 MiB/s wr, 1.54k op/s 2024-09-21T03:53:29.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:29 smithi134 ceph-mon[35031]: pgmap v633: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 225 MiB/s wr, 1.54k op/s 2024-09-21T03:53:30.854 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:30 smithi134 ceph-mon[35031]: pgmap v634: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 224 MiB/s wr, 1.46k op/s 2024-09-21T03:53:30.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:30 smithi123 ceph-mon[34673]: pgmap v634: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 224 MiB/s wr, 1.46k op/s 2024-09-21T03:53:30.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:30 smithi016 ceph-mon[30375]: pgmap v634: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 224 MiB/s wr, 1.46k op/s 2024-09-21T03:53:33.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:32 smithi134 ceph-mon[35031]: pgmap v635: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 225 MiB/s wr, 1.40k op/s 2024-09-21T03:53:33.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:32 smithi123 ceph-mon[34673]: pgmap v635: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 225 MiB/s wr, 1.40k op/s 2024-09-21T03:53:33.168 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:32 smithi016 ceph-mon[30375]: pgmap v635: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 225 MiB/s wr, 1.40k op/s 2024-09-21T03:53:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.940+0000 7f981fe00640 1 -- 172.21.15.16:0/1370268626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f982010cd20 msgr2=0x7f982010f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.940+0000 7f981fe00640 1 --2- 172.21.15.16:0/1370268626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f982010cd20 0x7f982010f140 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9810009900 tx=0x7f981002a410 comp rx=0 tx=0).stop 2024-09-21T03:53:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.940+0000 7f981fe00640 1 -- 172.21.15.16:0/1370268626 shutdown_connections 2024-09-21T03:53:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.940+0000 7f981fe00640 1 --2- 172.21.15.16:0/1370268626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982010f830 0x7f982010fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.940+0000 7f981fe00640 1 --2- 172.21.15.16:0/1370268626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f982010cd20 0x7f982010f140 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.940+0000 7f981fe00640 1 --2- 172.21.15.16:0/1370268626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982010a3c0 0x7f982010c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.940+0000 7f981fe00640 1 -- 172.21.15.16:0/1370268626 >> 172.21.15.16:0/1370268626 conn(0x7f9820070640 msgr2=0x7f9820072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.942+0000 7f981fe00640 1 -- 172.21.15.16:0/1370268626 shutdown_connections 2024-09-21T03:53:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.943+0000 7f981fe00640 1 -- 172.21.15.16:0/1370268626 wait complete. 2024-09-21T03:53:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.946+0000 7f981fe00640 1 Processor -- start 2024-09-21T03:53:33.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.947+0000 7f981fe00640 1 -- start start 2024-09-21T03:53:33.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.949+0000 7f981fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982010a3c0 0x7f98201a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.950+0000 7f981fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f982010cd20 0x7f98201a9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.950+0000 7f981ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982010a3c0 0x7f98201a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:33.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.950+0000 7f981ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982010a3c0 0x7f98201a8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:51700/0 (socket says 172.21.15.16:51700) 2024-09-21T03:53:33.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.952+0000 7f981fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982010f830 0x7f98201a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.952+0000 7f981ea00640 1 -- 172.21.15.16:0/2906056975 learned_addr learned my addr 172.21.15.16:0/2906056975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:33.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.952+0000 7f981fe00640 1 -- 172.21.15.16:0/2906056975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f982011e070 con 0x7f982010f830 2024-09-21T03:53:33.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.952+0000 7f981fe00640 1 -- 172.21.15.16:0/2906056975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f982011def0 con 0x7f982010a3c0 2024-09-21T03:53:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.952+0000 7f981fe00640 1 -- 172.21.15.16:0/2906056975 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f982011e1f0 con 0x7f982010cd20 2024-09-21T03:53:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.952+0000 7f981e000640 1 --2- 172.21.15.16:0/2906056975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f982010cd20 0x7f98201a9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.953+0000 7f981ea00640 1 -- 172.21.15.16:0/2906056975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f982010cd20 msgr2=0x7f98201a9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.953+0000 7f981ea00640 1 --2- 172.21.15.16:0/2906056975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f982010cd20 0x7f98201a9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.953+0000 7f981ea00640 1 -- 172.21.15.16:0/2906056975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982010f830 msgr2=0x7f98201a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.953+0000 7f981ea00640 1 --2- 172.21.15.16:0/2906056975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982010f830 0x7f98201a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.953+0000 7f981ea00640 1 -- 172.21.15.16:0/2906056975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98201aa740 con 0x7f982010a3c0 2024-09-21T03:53:33.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.954+0000 7f981ea00640 1 --2- 172.21.15.16:0/2906056975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982010a3c0 0x7f98201a8d00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f980c00b450 tx=0x7f980c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.955+0000 7f9817400640 1 -- 172.21.15.16:0/2906056975 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c015020 con 0x7f982010a3c0 2024-09-21T03:53:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.955+0000 7f9817400640 1 -- 172.21.15.16:0/2906056975 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f980c011d40 con 0x7f982010a3c0 2024-09-21T03:53:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.955+0000 7f9817400640 1 -- 172.21.15.16:0/2906056975 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c00f340 con 0x7f982010a3c0 2024-09-21T03:53:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.956+0000 7f981fe00640 1 -- 172.21.15.16:0/2906056975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98201aaa30 con 0x7f982010a3c0 2024-09-21T03:53:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.956+0000 7f981fe00640 1 -- 172.21.15.16:0/2906056975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98201b1120 con 0x7f982010a3c0 2024-09-21T03:53:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.965+0000 7f982c831740 1 Processor -- start 2024-09-21T03:53:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.965+0000 7f982c831740 1 -- start start 2024-09-21T03:53:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.966+0000 7f9817400640 1 -- 172.21.15.16:0/2906056975 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f980c018020 con 0x7f982010a3c0 2024-09-21T03:53:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.966+0000 7f982c831740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cdc1b0 0x55df560832b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.967+0000 7f982c831740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cb32f0 0x55df56083920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.967+0000 7f982c831740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cbb4d0 0x55df56084470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.967+0000 7f982c831740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df55ff66b0 con 0x55df55cdc1b0 2024-09-21T03:53:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.967+0000 7f982c831740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55df55fe5ae0 con 0x55df55cb32f0 2024-09-21T03:53:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.967+0000 7f982c831740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55df55fe98f0 con 0x55df55cbb4d0 2024-09-21T03:53:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.967+0000 7f981f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cdc1b0 0x55df560832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.967+0000 7f981e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cbb4d0 0x55df56084470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.968+0000 7f981f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cdc1b0 0x55df560832b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49416/0 (socket says 172.21.15.16:49416) 2024-09-21T03:53:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.968+0000 7f981f400640 1 -- 172.21.15.16:0/124975525 learned_addr learned my addr 172.21.15.16:0/124975525 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.968+0000 7f981ea00640 1 --2- 172.21.15.16:0/124975525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cb32f0 0x55df56083920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.969+0000 7f981e000640 1 -- 172.21.15.16:0/124975525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cb32f0 msgr2=0x55df56083920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.969+0000 7f981e000640 1 --2- 172.21.15.16:0/124975525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cb32f0 0x55df56083920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.969+0000 7f981e000640 1 -- 172.21.15.16:0/124975525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cdc1b0 msgr2=0x55df560832b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.969+0000 7f981e000640 1 --2- 172.21.15.16:0/124975525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cdc1b0 0x55df560832b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.969+0000 7f981e000640 1 -- 172.21.15.16:0/124975525 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df55fb2780 con 0x55df55cbb4d0 2024-09-21T03:53:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.969+0000 7f9817400640 1 --2- 172.21.15.16:0/2906056975 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f98040b6590 0x7f98040b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.969+0000 7f9817400640 1 -- 172.21.15.16:0/2906056975 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f980c0e6f20 con 0x7f982010a3c0 2024-09-21T03:53:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.969+0000 7f981e000640 1 --2- 172.21.15.16:0/124975525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cbb4d0 0x55df56084470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f981002aa00 tx=0x7f98100043d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f05dea858a4f475 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.970+0000 7f980b400640 1 -- 172.21.15.16:0/124975525 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9810035030 con 0x55df55cbb4d0 2024-09-21T03:53:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.970+0000 7f980b400640 1 -- 172.21.15.16:0/124975525 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9810040070 con 0x55df55cbb4d0 2024-09-21T03:53:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.969+0000 7f981f400640 1 --2- 172.21.15.16:0/2906056975 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f98040b6590 0x7f98040b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.971+0000 7f982c831740 1 -- 172.21.15.16:0/124975525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cbb4d0 msgr2=0x55df56084470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.971+0000 7f982c831740 1 --2- 172.21.15.16:0/124975525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cbb4d0 0x55df56084470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f981002aa00 tx=0x7f98100043d0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.972+0000 7f982c831740 1 -- 172.21.15.16:0/124975525 shutdown_connections 2024-09-21T03:53:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.972+0000 7f982c831740 1 --2- 172.21.15.16:0/124975525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cbb4d0 0x55df56084470 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.972+0000 7f982c831740 1 --2- 172.21.15.16:0/124975525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cb32f0 0x55df56083920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.972+0000 7f982c831740 1 --2- 172.21.15.16:0/124975525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cdc1b0 0x55df560832b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.972+0000 7f982c831740 1 -- 172.21.15.16:0/124975525 >> 172.21.15.16:0/124975525 conn(0x55df55cd0300 msgr2=0x55df5607ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.973+0000 7f982c831740 1 -- 172.21.15.16:0/124975525 shutdown_connections 2024-09-21T03:53:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.973+0000 7f982c831740 1 -- 172.21.15.16:0/124975525 wait complete. 2024-09-21T03:53:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.973+0000 7f981f400640 1 --2- 172.21.15.16:0/2906056975 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f98040b6590 0x7f98040b8aa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9818009fa0 tx=0x7f981800b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.974+0000 7f982c831740 1 -- start start 2024-09-21T03:53:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.974+0000 7f982c831740 1 Processor -- start 2024-09-21T03:53:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.983+0000 7f982c831740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cb32f0 0x55df5608c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.983+0000 7f982c831740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cbb4d0 0x55df5608d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.983+0000 7f982c831740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cdc1b0 0x55df5608db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.983+0000 7f982c831740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df55f1dab0 con 0x55df55cb32f0 2024-09-21T03:53:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.983+0000 7f982c831740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55df55f8c570 con 0x55df55cbb4d0 2024-09-21T03:53:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.983+0000 7f982c831740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55df56084fd0 con 0x55df55cdc1b0 2024-09-21T03:53:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.983+0000 7f982c831740 10 client.0 _ll_register_callbacks cb 0x55df55f9b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.985+0000 7f981e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cbb4d0 0x55df5608d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.985+0000 7f981e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cbb4d0 0x55df5608d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:51728/0 (socket says 172.21.15.16:51728) 2024-09-21T03:53:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.985+0000 7f981e000640 1 -- 172.21.15.16:0/645289853 learned_addr learned my addr 172.21.15.16:0/645289853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.985+0000 7f981ea00640 1 --2- 172.21.15.16:0/645289853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cb32f0 0x55df5608c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.985+0000 7f981f400640 1 --2- 172.21.15.16:0/645289853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cdc1b0 0x55df5608db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.987+0000 7f981e000640 1 -- 172.21.15.16:0/645289853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cdc1b0 msgr2=0x55df5608db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.987+0000 7f981e000640 1 --2- 172.21.15.16:0/645289853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cdc1b0 0x55df5608db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.987+0000 7f981e000640 1 -- 172.21.15.16:0/645289853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cb32f0 msgr2=0x55df5608c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:33.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.987+0000 7f981e000640 1 --2- 172.21.15.16:0/645289853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cb32f0 0x55df5608c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.987+0000 7f981e000640 1 -- 172.21.15.16:0/645289853 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df55d47790 con 0x55df55cbb4d0 2024-09-21T03:53:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.988+0000 7f981ea00640 1 --2- 172.21.15.16:0/645289853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cb32f0 0x55df5608c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:53:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.988+0000 7f981e000640 1 --2- 172.21.15.16:0/645289853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cbb4d0 0x55df5608d060 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9810003830 tx=0x7f9810009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e38d5fca9ee60b95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.988+0000 7f981f400640 1 --2- 172.21.15.16:0/645289853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cdc1b0 0x55df5608db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.988+0000 7f97ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:53:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.995+0000 7f97ffe00640 1 -- 172.21.15.16:0/645289853 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9810035070 con 0x55df55cbb4d0 2024-09-21T03:53:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.995+0000 7f982c831740 1 -- 172.21.15.16:0/645289853 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9810040070 con 0x55df55cbb4d0 2024-09-21T03:53:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.997+0000 7f97ffe00640 1 -- 172.21.15.16:0/645289853 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f9810045470 con 0x55df55cbb4d0 2024-09-21T03:53:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.997+0000 7f97ffe00640 1 -- 172.21.15.16:0/645289853 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9810003e20 con 0x55df55cbb4d0 2024-09-21T03:53:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.997+0000 7f982c831740 10 client.26345 fetch_fsmap learned FSMap version 30 2024-09-21T03:53:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.997+0000 7f982c831740 1 -- 172.21.15.16:0/645289853 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df55dc0620 con 0x55df55cbb4d0 2024-09-21T03:53:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.998+0000 7f97ffe00640 1 -- 172.21.15.16:0/645289853 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9810040070 con 0x55df55cbb4d0 2024-09-21T03:53:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.999+0000 7f97ffe00640 1 -- 172.21.15.16:0/645289853 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9810045670 con 0x55df55cbb4d0 2024-09-21T03:53:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.999+0000 7f982c831740 10 client.26345 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:53:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.999+0000 7f982c831740 10 client.26345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:33.999+0000 7f982c831740 20 client.26345 populate_metadata read hostname 'smithi016' 2024-09-21T03:53:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.000+0000 7f982c831740 1 --2- 172.21.15.16:0/645289853 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55df55c9ad80 0x55df56099c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.000+0000 7f982c831740 4 client.26345 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.000+0000 7f982c831740 1 -- 172.21.15.16:0/645289853 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df55ff8600 con 0x55df55c9ad80 2024-09-21T03:53:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.000+0000 7f981ea00640 1 --2- 172.21.15.16:0/645289853 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55df55c9ad80 0x55df56099c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.001+0000 7f981ea00640 1 --2- 172.21.15.16:0/645289853 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55df55c9ad80 0x55df56099c40 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2852c14a00d56a0d server_cookie=b8d32d4ffd3dd3e6 in_seq=0 out_seq=0 2024-09-21T03:53:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.001+0000 7f97ffe00640 10 client.26345 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.002+0000 7f97ffe00640 1 -- 172.21.15.16:0/645289853 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55df55ff8600 con 0x55df55c9ad80 2024-09-21T03:53:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.002+0000 7f97ffe00640 10 client.26345 handle_command_reply: tid=0 2024-09-21T03:53:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.003+0000 7f982c831740 1 client.26345 shutdown 2024-09-21T03:53:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.004+0000 7f982c831740 1 -- 172.21.15.16:0/645289853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cbb4d0 msgr2=0x55df5608d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.004+0000 7f982c831740 1 --2- 172.21.15.16:0/645289853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cbb4d0 0x55df5608d060 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9810003830 tx=0x7f9810009260 comp rx=0 tx=0).stop 2024-09-21T03:53:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.004+0000 7f982c831740 1 -- 172.21.15.16:0/645289853 shutdown_connections 2024-09-21T03:53:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.004+0000 7f982c831740 1 --2- 172.21.15.16:0/645289853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cdc1b0 0x55df5608db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.004+0000 7f982c831740 1 --2- 172.21.15.16:0/645289853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cbb4d0 0x55df5608d060 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.004+0000 7f982c831740 1 --2- 172.21.15.16:0/645289853 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55df55c9ad80 0x55df56099c40 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.004+0000 7f982c831740 1 --2- 172.21.15.16:0/645289853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cb32f0 0x55df5608c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.004+0000 7f982c831740 1 -- 172.21.15.16:0/645289853 >> 172.21.15.16:0/645289853 conn(0x55df55cd0300 msgr2=0x55df5607f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.004+0000 7f97ffe00640 0 client.26345 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.007+0000 7f982c831740 1 -- 172.21.15.16:0/645289853 shutdown_connections 2024-09-21T03:53:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.007+0000 7f982c831740 1 -- 172.21.15.16:0/645289853 wait complete. 2024-09-21T03:53:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.007+0000 7f982c831740 20 client.26345 trim_cache size 0 max 16384 2024-09-21T03:53:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.102+0000 7f982c831740 1 Processor -- start 2024-09-21T03:53:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.102+0000 7f982c831740 1 -- start start 2024-09-21T03:53:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.103+0000 7f982c831740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cb32f0 0x55df5608c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.103+0000 7f982c831740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cbb4d0 0x55df56094a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.103+0000 7f982c831740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cdc1b0 0x55df5607f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.104+0000 7f981ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cb32f0 0x55df5608c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.105+0000 7f982c831740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f980000fc90 con 0x55df55cdc1b0 2024-09-21T03:53:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.105+0000 7f982c831740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98000117d0 con 0x55df55cbb4d0 2024-09-21T03:53:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.105+0000 7f982c831740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9800010360 con 0x55df55cb32f0 2024-09-21T03:53:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.104+0000 7f981e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cbb4d0 0x55df56094a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.105+0000 7f981e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cbb4d0 0x55df56094a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:51734/0 (socket says 172.21.15.16:51734) 2024-09-21T03:53:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.105+0000 7f981ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cb32f0 0x55df5608c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52688/0 (socket says 172.21.15.16:52688) 2024-09-21T03:53:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.105+0000 7f981ea00640 1 -- 172.21.15.16:0/4190107387 learned_addr learned my addr 172.21.15.16:0/4190107387 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.105+0000 7f981f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cdc1b0 0x55df5607f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.106+0000 7f981ea00640 1 -- 172.21.15.16:0/4190107387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cbb4d0 msgr2=0x55df56094a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.106+0000 7f981ea00640 1 --2- 172.21.15.16:0/4190107387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cbb4d0 0x55df56094a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.106+0000 7f981ea00640 1 -- 172.21.15.16:0/4190107387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cdc1b0 msgr2=0x55df5607f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.106+0000 7f981ea00640 1 --2- 172.21.15.16:0/4190107387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cdc1b0 0x55df5607f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.106+0000 7f981ea00640 1 -- 172.21.15.16:0/4190107387 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df5608ded0 con 0x55df55cb32f0 2024-09-21T03:53:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.108+0000 7f981ea00640 1 --2- 172.21.15.16:0/4190107387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cb32f0 0x55df5608c2b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f980c000c00 tx=0x7f980c0eaa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=940d90b9bcfd672d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.110+0000 7f980aa00640 1 -- 172.21.15.16:0/4190107387 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c010eb0 con 0x55df55cb32f0 2024-09-21T03:53:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.110+0000 7f980aa00640 1 -- 172.21.15.16:0/4190107387 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f980c0ac810 con 0x55df55cb32f0 2024-09-21T03:53:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.110+0000 7f980aa00640 1 -- 172.21.15.16:0/4190107387 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c0b4b60 con 0x55df55cb32f0 2024-09-21T03:53:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.111+0000 7f982c831740 1 -- 172.21.15.16:0/4190107387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cb32f0 msgr2=0x55df5608c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.111+0000 7f982c831740 1 --2- 172.21.15.16:0/4190107387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cb32f0 0x55df5608c2b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f980c000c00 tx=0x7f980c0eaa00 comp rx=0 tx=0).stop 2024-09-21T03:53:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.112+0000 7f982c831740 1 -- 172.21.15.16:0/4190107387 shutdown_connections 2024-09-21T03:53:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.116+0000 7f982c831740 1 --2- 172.21.15.16:0/4190107387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cdc1b0 0x55df5607f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.116+0000 7f982c831740 1 --2- 172.21.15.16:0/4190107387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cbb4d0 0x55df56094a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.116+0000 7f982c831740 1 --2- 172.21.15.16:0/4190107387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cb32f0 0x55df5608c2b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.116+0000 7f982c831740 1 -- 172.21.15.16:0/4190107387 >> 172.21.15.16:0/4190107387 conn(0x55df55cd0300 msgr2=0x55df5608bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.117+0000 7f982c831740 1 -- 172.21.15.16:0/4190107387 shutdown_connections 2024-09-21T03:53:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.117+0000 7f982c831740 1 -- 172.21.15.16:0/4190107387 wait complete. 2024-09-21T03:53:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.117+0000 7f982c831740 1 -- start start 2024-09-21T03:53:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.118+0000 7f982c831740 1 Processor -- start 2024-09-21T03:53:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.119+0000 7f982c831740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cb32f0 0x55df56091b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.119+0000 7f982c831740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cbb4d0 0x55df56092240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.119+0000 7f982c831740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cdc1b0 0x55df56092db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.120+0000 7f982c831740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9800011100 con 0x55df55cbb4d0 2024-09-21T03:53:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.120+0000 7f981f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cdc1b0 0x55df56092db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.120+0000 7f981f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cdc1b0 0x55df56092db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52690/0 (socket says 172.21.15.16:52690) 2024-09-21T03:53:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.120+0000 7f981f400640 1 -- 172.21.15.16:0/4024953955 learned_addr learned my addr 172.21.15.16:0/4024953955 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.122+0000 7f981e000640 1 --2- 172.21.15.16:0/4024953955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cbb4d0 0x55df56092240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.123+0000 7f981ea00640 1 --2- 172.21.15.16:0/4024953955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cb32f0 0x55df56091b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.124+0000 7f982c831740 1 -- 172.21.15.16:0/4024953955 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9800010a30 con 0x55df55cb32f0 2024-09-21T03:53:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.124+0000 7f982c831740 1 -- 172.21.15.16:0/4024953955 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f980000dc70 con 0x55df55cdc1b0 2024-09-21T03:53:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.125+0000 7f982c831740 10 client.0 _ll_register_callbacks cb 0x7f9818009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.127+0000 7f981f400640 1 -- 172.21.15.16:0/4024953955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cb32f0 msgr2=0x55df56091b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.127+0000 7f981f400640 1 --2- 172.21.15.16:0/4024953955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cb32f0 0x55df56091b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.127+0000 7f981f400640 1 -- 172.21.15.16:0/4024953955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cbb4d0 msgr2=0x55df56092240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.127+0000 7f981f400640 1 --2- 172.21.15.16:0/4024953955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cbb4d0 0x55df56092240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.127+0000 7f981f400640 1 -- 172.21.15.16:0/4024953955 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df56093510 con 0x55df55cdc1b0 2024-09-21T03:53:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.128+0000 7f981f400640 1 --2- 172.21.15.16:0/4024953955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cdc1b0 0x55df56092db0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9818015dd0 tx=0x7f9818004bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d2a6b6523396201 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.128+0000 7f97ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:53:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.129+0000 7f97ffe00640 1 -- 172.21.15.16:0/4024953955 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9818026070 con 0x55df55cdc1b0 2024-09-21T03:53:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.130+0000 7f97ffe00640 1 -- 172.21.15.16:0/4024953955 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f9818022430 con 0x55df55cdc1b0 2024-09-21T03:53:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.130+0000 7f982c831740 1 -- 172.21.15.16:0/4024953955 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f980c0ac810 con 0x55df55cdc1b0 2024-09-21T03:53:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.130+0000 7f97ffe00640 1 -- 172.21.15.16:0/4024953955 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f981801b040 con 0x55df55cdc1b0 2024-09-21T03:53:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.130+0000 7f982c831740 10 client.26290 fetch_fsmap learned FSMap version 30 2024-09-21T03:53:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.131+0000 7f982c831740 1 -- 172.21.15.16:0/4024953955 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df56094900 con 0x55df55cdc1b0 2024-09-21T03:53:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.131+0000 7f97ffe00640 1 -- 172.21.15.16:0/4024953955 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f980c0ac810 con 0x55df55cdc1b0 2024-09-21T03:53:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.131+0000 7f97ffe00640 1 -- 172.21.15.16:0/4024953955 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f98180228f0 con 0x55df55cdc1b0 2024-09-21T03:53:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.132+0000 7f982c831740 10 client.26290 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:53:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.132+0000 7f982c831740 10 client.26290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.132+0000 7f982c831740 20 client.26290 populate_metadata read hostname 'smithi016' 2024-09-21T03:53:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.132+0000 7f982c831740 1 --2- 172.21.15.16:0/4024953955 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55df55c701c0 0x55df561068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.132+0000 7f982c831740 4 client.26290 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:53:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.132+0000 7f982c831740 1 -- 172.21.15.16:0/4024953955 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df55ff40b0 con 0x55df55c701c0 2024-09-21T03:53:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.132+0000 7f981ea00640 1 --2- 172.21.15.16:0/4024953955 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55df55c701c0 0x55df561068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.133+0000 7f981ea00640 1 --2- 172.21.15.16:0/4024953955 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55df55c701c0 0x55df561068a0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d89fd1e4e69e49e3 server_cookie=402975b1c3372238 in_seq=0 out_seq=0 2024-09-21T03:53:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.133+0000 7f97ffe00640 10 client.26290 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.148+0000 7f97ffe00640 1 -- 172.21.15.16:0/4024953955 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df55ff40b0 con 0x55df55c701c0 2024-09-21T03:53:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.148+0000 7f97ffe00640 10 client.26290 handle_command_reply: tid=0 2024-09-21T03:53:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.148+0000 7f982c831740 1 client.26290 shutdown 2024-09-21T03:53:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.148+0000 7f982c831740 1 -- 172.21.15.16:0/4024953955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cdc1b0 msgr2=0x55df56092db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.148+0000 7f982c831740 1 --2- 172.21.15.16:0/4024953955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cdc1b0 0x55df56092db0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9818015dd0 tx=0x7f9818004bc0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.149+0000 7f982c831740 1 -- 172.21.15.16:0/4024953955 shutdown_connections 2024-09-21T03:53:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.149+0000 7f982c831740 1 --2- 172.21.15.16:0/4024953955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df55cdc1b0 0x55df56092db0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.149+0000 7f982c831740 1 --2- 172.21.15.16:0/4024953955 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55df55c701c0 0x55df561068a0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.149+0000 7f982c831740 1 --2- 172.21.15.16:0/4024953955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df55cbb4d0 0x55df56092240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.149+0000 7f982c831740 1 --2- 172.21.15.16:0/4024953955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df55cb32f0 0x55df56091b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.149+0000 7f97ffe00640 0 client.26290 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.149+0000 7f982c831740 1 -- 172.21.15.16:0/4024953955 >> 172.21.15.16:0/4024953955 conn(0x55df55cd0300 msgr2=0x55df5607f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.149+0000 7f982c831740 1 -- 172.21.15.16:0/4024953955 shutdown_connections 2024-09-21T03:53:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.149+0000 7f982c831740 1 -- 172.21.15.16:0/4024953955 wait complete. 2024-09-21T03:53:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.149+0000 7f982c831740 20 client.26290 trim_cache size 0 max 16384 2024-09-21T03:53:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:53:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:53:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:53:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:53:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.150+0000 7f981fe00640 1 -- 172.21.15.16:0/2906056975 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f98040b6590 msgr2=0x7f98040b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.150+0000 7f981fe00640 1 --2- 172.21.15.16:0/2906056975 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f98040b6590 0x7f98040b8aa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9818009fa0 tx=0x7f981800b040 comp rx=0 tx=0).stop 2024-09-21T03:53:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.150+0000 7f981fe00640 1 -- 172.21.15.16:0/2906056975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982010a3c0 msgr2=0x7f98201a8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.150+0000 7f981fe00640 1 --2- 172.21.15.16:0/2906056975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982010a3c0 0x7f98201a8d00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f980c00b450 tx=0x7f980c00b910 comp rx=0 tx=0).stop 2024-09-21T03:53:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.151+0000 7f981fe00640 1 -- 172.21.15.16:0/2906056975 shutdown_connections 2024-09-21T03:53:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.151+0000 7f981fe00640 1 --2- 172.21.15.16:0/2906056975 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f98040b6590 0x7f98040b8aa0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.151+0000 7f981fe00640 1 --2- 172.21.15.16:0/2906056975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982010f830 0x7f98201a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.151+0000 7f981fe00640 1 --2- 172.21.15.16:0/2906056975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f982010cd20 0x7f98201a9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.151+0000 7f981fe00640 1 --2- 172.21.15.16:0/2906056975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982010a3c0 0x7f98201a8d00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.151+0000 7f981fe00640 1 -- 172.21.15.16:0/2906056975 >> 172.21.15.16:0/2906056975 conn(0x7f9820070640 msgr2=0x7f982010b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.151+0000 7f981fe00640 1 -- 172.21.15.16:0/2906056975 shutdown_connections 2024-09-21T03:53:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:34.151+0000 7f981fe00640 1 -- 172.21.15.16:0/2906056975 wait complete. 2024-09-21T03:53:35.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:34 smithi134 ceph-mon[35031]: pgmap v636: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 220 MiB/s wr, 1.30k op/s 2024-09-21T03:53:35.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:34 smithi123 ceph-mon[34673]: pgmap v636: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 220 MiB/s wr, 1.30k op/s 2024-09-21T03:53:35.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:34 smithi016 ceph-mon[30375]: pgmap v636: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 220 MiB/s wr, 1.30k op/s 2024-09-21T03:53:37.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:36 smithi134 ceph-mon[35031]: pgmap v637: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 246 MiB/s wr, 1.34k op/s 2024-09-21T03:53:37.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:36 smithi123 ceph-mon[34673]: pgmap v637: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 246 MiB/s wr, 1.34k op/s 2024-09-21T03:53:37.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:36 smithi016 ceph-mon[30375]: pgmap v637: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 246 MiB/s wr, 1.34k op/s 2024-09-21T03:53:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:38 smithi134 ceph-mon[35031]: pgmap v638: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 229 MiB/s wr, 1.14k op/s 2024-09-21T03:53:39.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:38 smithi123 ceph-mon[34673]: pgmap v638: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 229 MiB/s wr, 1.14k op/s 2024-09-21T03:53:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:38 smithi016 ceph-mon[30375]: pgmap v638: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 229 MiB/s wr, 1.14k op/s 2024-09-21T03:53:41.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:53:41.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:53:41.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:53:41.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:53:41.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:53:41.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:53:41.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:53:41.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:53:41.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:53:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:53:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:53:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:53:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:53:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:53:41.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:53:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:53:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:53:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:53:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:53:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:53:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:53:45.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:45 smithi134 ceph-mon[35031]: pgmap v640: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 223 MiB/s wr, 991 op/s 2024-09-21T03:53:45.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:45 smithi134 ceph-mon[35031]: pgmap v641: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 225 MiB/s wr, 982 op/s 2024-09-21T03:53:45.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:45 smithi123 ceph-mon[34673]: pgmap v640: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 223 MiB/s wr, 991 op/s 2024-09-21T03:53:45.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:45 smithi123 ceph-mon[34673]: pgmap v641: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 225 MiB/s wr, 982 op/s 2024-09-21T03:53:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:45 smithi016 ceph-mon[30375]: pgmap v640: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 223 MiB/s wr, 991 op/s 2024-09-21T03:53:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:45 smithi016 ceph-mon[30375]: pgmap v641: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 225 MiB/s wr, 982 op/s 2024-09-21T03:53:46.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:46 smithi134 ceph-mon[35031]: pgmap v642: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 1.11k op/s 2024-09-21T03:53:46.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:46 smithi123 ceph-mon[34673]: pgmap v642: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 1.11k op/s 2024-09-21T03:53:46.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:46 smithi016 ceph-mon[30375]: pgmap v642: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 1.11k op/s 2024-09-21T03:53:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.803+0000 7fedc7e00640 1 -- 172.21.15.16:0/2450115328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc8104570 msgr2=0x7fedc8104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.803+0000 7fedc7e00640 1 --2- 172.21.15.16:0/2450115328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc8104570 0x7fedc8104980 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009900 tx=0x7fedb402a400 comp rx=0 tx=0).stop 2024-09-21T03:53:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.803+0000 7fedc7e00640 1 -- 172.21.15.16:0/2450115328 shutdown_connections 2024-09-21T03:53:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.803+0000 7fedc7e00640 1 --2- 172.21.15.16:0/2450115328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedc8104f10 0x7fedc810b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.803+0000 7fedc7e00640 1 --2- 172.21.15.16:0/2450115328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc8104570 0x7fedc8104980 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.803+0000 7fedc7e00640 1 --2- 172.21.15.16:0/2450115328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc81034f0 0x7fedc8103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.803+0000 7fedc7e00640 1 -- 172.21.15.16:0/2450115328 >> 172.21.15.16:0/2450115328 conn(0x7fedc80fee50 msgr2=0x7fedc81012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.804+0000 7fedc7e00640 1 -- 172.21.15.16:0/2450115328 shutdown_connections 2024-09-21T03:53:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.804+0000 7fedc7e00640 1 -- 172.21.15.16:0/2450115328 wait complete. 2024-09-21T03:53:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.805+0000 7fedc7e00640 1 Processor -- start 2024-09-21T03:53:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.805+0000 7fedc7e00640 1 -- start start 2024-09-21T03:53:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.806+0000 7fedc7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc81034f0 0x7fedc8194c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.806+0000 7fedc7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc8104570 0x7fedc81952a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.806+0000 7fedc7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedc8104f10 0x7fedc8195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.806+0000 7fedc7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fedc810f170 con 0x7fedc81034f0 2024-09-21T03:53:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.806+0000 7fedc7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fedc810eff0 con 0x7fedc8104570 2024-09-21T03:53:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.806+0000 7fedc7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fedc810f2f0 con 0x7fedc8104f10 2024-09-21T03:53:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.806+0000 7fedc6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc81034f0 0x7fedc8194c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.806+0000 7fedcd800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedc8104f10 0x7fedc8195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.806+0000 7fedcd800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedc8104f10 0x7fedc8195de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:58908/0 (socket says 172.21.15.16:58908) 2024-09-21T03:53:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.806+0000 7fedcd800640 1 -- 172.21.15.16:0/2732407286 learned_addr learned my addr 172.21.15.16:0/2732407286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.806+0000 7fedc6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc8104570 0x7fedc81952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.807+0000 7fedcd800640 1 -- 172.21.15.16:0/2732407286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc8104570 msgr2=0x7fedc81952a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.807+0000 7fedcd800640 1 --2- 172.21.15.16:0/2732407286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc8104570 0x7fedc81952a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.807+0000 7fedcd800640 1 -- 172.21.15.16:0/2732407286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc81034f0 msgr2=0x7fedc8194c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.807+0000 7fedcd800640 1 --2- 172.21.15.16:0/2732407286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc81034f0 0x7fedc8194c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.807+0000 7fedcd800640 1 -- 172.21.15.16:0/2732407286 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc81963a0 con 0x7fedc8104f10 2024-09-21T03:53:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.807+0000 7fedc6a00640 1 --2- 172.21.15.16:0/2732407286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc81034f0 0x7fedc8194c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.808+0000 7fedcd800640 1 --2- 172.21.15.16:0/2732407286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedc8104f10 0x7fedc8195de0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fedc000eaa0 tx=0x7fedc000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.808+0000 7fedbf400640 1 -- 172.21.15.16:0/2732407286 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc000cdb0 con 0x7fedc8104f10 2024-09-21T03:53:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.808+0000 7fedbf400640 1 -- 172.21.15.16:0/2732407286 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedc0005020 con 0x7fedc8104f10 2024-09-21T03:53:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.808+0000 7fedbf400640 1 -- 172.21.15.16:0/2732407286 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc00116d0 con 0x7fedc8104f10 2024-09-21T03:53:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.808+0000 7fedc7e00640 1 -- 172.21.15.16:0/2732407286 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc8196690 con 0x7fedc8104f10 2024-09-21T03:53:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.808+0000 7fedc7e00640 1 -- 172.21.15.16:0/2732407286 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc8196b90 con 0x7fedc8104f10 2024-09-21T03:53:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.822+0000 7fedd48b2740 1 Processor -- start 2024-09-21T03:53:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.823+0000 7fedbf400640 1 -- 172.21.15.16:0/2732407286 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fedc00051d0 con 0x7fedc8104f10 2024-09-21T03:53:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.823+0000 7fedd48b2740 1 -- start start 2024-09-21T03:53:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.826+0000 7fedd48b2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3eea1b0 0x5567c42912b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.826+0000 7fedd48b2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ec12f0 0x5567c4291920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.826+0000 7fedd48b2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3ec94d0 0x5567c4292470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.826+0000 7fedd48b2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5567c42046b0 con 0x5567c3eea1b0 2024-09-21T03:53:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.826+0000 7fedd48b2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5567c41f3ae0 con 0x5567c3ec12f0 2024-09-21T03:53:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.826+0000 7fedd48b2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5567c41f78f0 con 0x5567c3ec94d0 2024-09-21T03:53:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.827+0000 7fedc6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ec12f0 0x5567c4291920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.827+0000 7fedc6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ec12f0 0x5567c4291920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:40958/0 (socket says 172.21.15.16:40958) 2024-09-21T03:53:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.827+0000 7fedc6a00640 1 -- 172.21.15.16:0/3227591559 learned_addr learned my addr 172.21.15.16:0/3227591559 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.830+0000 7fedc6000640 1 --2- 172.21.15.16:0/3227591559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3eea1b0 0x5567c42912b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.830+0000 7fedc6000640 1 --2- 172.21.15.16:0/3227591559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3ec94d0 0x5567c4292470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.833+0000 7fedc6000640 1 -- 172.21.15.16:0/3227591559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ec12f0 msgr2=0x5567c4291920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.833+0000 7fedc6000640 1 --2- 172.21.15.16:0/3227591559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ec12f0 0x5567c4291920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.833+0000 7fedc6000640 1 -- 172.21.15.16:0/3227591559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3eea1b0 msgr2=0x5567c42912b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.833+0000 7fedc6000640 1 --2- 172.21.15.16:0/3227591559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3eea1b0 0x5567c42912b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.833+0000 7fedc6000640 1 -- 172.21.15.16:0/3227591559 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c41c0780 con 0x5567c3ec94d0 2024-09-21T03:53:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.834+0000 7fedc6a00640 1 --2- 172.21.15.16:0/3227591559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ec12f0 0x5567c4291920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.834+0000 7fedbf400640 1 --2- 172.21.15.16:0/2732407286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fedac0b6510 0x7fedac0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.834+0000 7fedbf400640 1 -- 172.21.15.16:0/2732407286 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fedc00e7ab0 con 0x7fedc8104f10 2024-09-21T03:53:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.835+0000 7fedc6000640 1 --2- 172.21.15.16:0/3227591559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3ec94d0 0x5567c4292470 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fedb402a9f0 tx=0x7fedb402a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a016148592e4f07d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.836+0000 7fedb3e00640 1 -- 172.21.15.16:0/3227591559 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb4035030 con 0x5567c3ec94d0 2024-09-21T03:53:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.836+0000 7fedb3e00640 1 -- 172.21.15.16:0/3227591559 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedb4004360 con 0x5567c3ec94d0 2024-09-21T03:53:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.836+0000 7fedb3e00640 1 -- 172.21.15.16:0/3227591559 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb4031890 con 0x5567c3ec94d0 2024-09-21T03:53:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.836+0000 7fedc6a00640 1 --2- 172.21.15.16:0/2732407286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fedac0b6510 0x7fedac0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.838+0000 7fedd48b2740 1 -- 172.21.15.16:0/3227591559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3ec94d0 msgr2=0x5567c4292470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.838+0000 7fedd48b2740 1 --2- 172.21.15.16:0/3227591559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3ec94d0 0x5567c4292470 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fedb402a9f0 tx=0x7fedb402a400 comp rx=0 tx=0).stop 2024-09-21T03:53:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.840+0000 7fedd48b2740 1 -- 172.21.15.16:0/3227591559 shutdown_connections 2024-09-21T03:53:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.840+0000 7fedd48b2740 1 --2- 172.21.15.16:0/3227591559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3ec94d0 0x5567c4292470 secure :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fedb402a9f0 tx=0x7fedb402a400 comp rx=0 tx=0).stop 2024-09-21T03:53:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.840+0000 7fedd48b2740 1 --2- 172.21.15.16:0/3227591559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ec12f0 0x5567c4291920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.840+0000 7fedd48b2740 1 --2- 172.21.15.16:0/3227591559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3eea1b0 0x5567c42912b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.840+0000 7fedd48b2740 1 -- 172.21.15.16:0/3227591559 >> 172.21.15.16:0/3227591559 conn(0x5567c3ede300 msgr2=0x5567c428cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.841+0000 7fedd48b2740 1 -- 172.21.15.16:0/3227591559 shutdown_connections 2024-09-21T03:53:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.842+0000 7fedd48b2740 1 -- 172.21.15.16:0/3227591559 wait complete. 2024-09-21T03:53:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.843+0000 7fedc6a00640 1 --2- 172.21.15.16:0/2732407286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fedac0b6510 0x7fedac0b8a20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009ea0 tx=0x7fedb8005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.843+0000 7fedd48b2740 1 -- start start 2024-09-21T03:53:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.843+0000 7fedd48b2740 1 Processor -- start 2024-09-21T03:53:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.846+0000 7fedd48b2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 0x5567c429aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.846+0000 7fedd48b2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3eea1b0 0x5567c429b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.846+0000 7fedd48b2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ea8d80 0x5567c429bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.846+0000 7fedc6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3eea1b0 0x5567c429b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.846+0000 7fedc6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3eea1b0 0x5567c429b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:58924/0 (socket says 172.21.15.16:58924) 2024-09-21T03:53:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.849+0000 7fedd48b2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5567c412bab0 con 0x5567c3ec12f0 2024-09-21T03:53:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.849+0000 7fedd48b2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5567c419a570 con 0x5567c3ea8d80 2024-09-21T03:53:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.849+0000 7fedd48b2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5567c4292f30 con 0x5567c3eea1b0 2024-09-21T03:53:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.849+0000 7fedd48b2740 10 client.0 _ll_register_callbacks cb 0x5567c41a9100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.850+0000 7fedc6000640 1 -- 172.21.15.16:0/1160727917 learned_addr learned my addr 172.21.15.16:0/1160727917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.850+0000 7fedc6a00640 1 --2- 172.21.15.16:0/1160727917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 0x5567c429aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.851+0000 7fedcd800640 1 --2- 172.21.15.16:0/1160727917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ea8d80 0x5567c429bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.852+0000 7fedcd800640 1 -- 172.21.15.16:0/1160727917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3eea1b0 msgr2=0x5567c429b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.852+0000 7fedcd800640 1 --2- 172.21.15.16:0/1160727917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3eea1b0 0x5567c429b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.852+0000 7fedcd800640 1 -- 172.21.15.16:0/1160727917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 msgr2=0x5567c429aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.853+0000 7fedcd800640 1 --2- 172.21.15.16:0/1160727917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 0x5567c429aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.853+0000 7fedcd800640 1 -- 172.21.15.16:0/1160727917 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567c3f55790 con 0x5567c3ea8d80 2024-09-21T03:53:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.853+0000 7fedc6a00640 1 --2- 172.21.15.16:0/1160727917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 0x5567c429aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:53:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.854+0000 7fedcd800640 1 --2- 172.21.15.16:0/1160727917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ea8d80 0x5567c429bc60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fedc00e5250 tx=0x7fedc00ad000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5ddaae7e08a94be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.854+0000 7fedb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:53:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.854+0000 7fedb0c00640 1 -- 172.21.15.16:0/1160727917 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc00adf90 con 0x5567c3ea8d80 2024-09-21T03:53:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.854+0000 7fedb0c00640 1 -- 172.21.15.16:0/1160727917 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fedc00ac560 con 0x5567c3ea8d80 2024-09-21T03:53:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.855+0000 7fedd48b2740 1 -- 172.21.15.16:0/1160727917 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedb4004360 con 0x5567c3ea8d80 2024-09-21T03:53:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.856+0000 7fedb0c00640 1 -- 172.21.15.16:0/1160727917 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fedc00b2370 con 0x5567c3ea8d80 2024-09-21T03:53:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.856+0000 7fedd48b2740 10 client.26363 fetch_fsmap learned FSMap version 30 2024-09-21T03:53:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.856+0000 7fedd48b2740 1 -- 172.21.15.16:0/1160727917 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5567c3fce620 con 0x5567c3ea8d80 2024-09-21T03:53:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.857+0000 7fedb0c00640 1 -- 172.21.15.16:0/1160727917 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedb4004360 con 0x5567c3ea8d80 2024-09-21T03:53:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.857+0000 7fedb0c00640 1 -- 172.21.15.16:0/1160727917 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fedc000f040 con 0x5567c3ea8d80 2024-09-21T03:53:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.857+0000 7fedd48b2740 10 client.26363 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:53:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.857+0000 7fedd48b2740 10 client.26363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.857+0000 7fedd48b2740 20 client.26363 populate_metadata read hostname 'smithi016' 2024-09-21T03:53:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.857+0000 7fedd48b2740 1 --2- 172.21.15.16:0/1160727917 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5567c3e7e1c0 0x5567c42a8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.857+0000 7fedd48b2740 4 client.26363 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:53:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.857+0000 7fedd48b2740 1 -- 172.21.15.16:0/1160727917 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567c4206600 con 0x5567c3e7e1c0 2024-09-21T03:53:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.857+0000 7fedc6a00640 1 --2- 172.21.15.16:0/1160727917 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5567c3e7e1c0 0x5567c42a8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.859+0000 7fedc6a00640 1 --2- 172.21.15.16:0/1160727917 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5567c3e7e1c0 0x5567c42a8120 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a0b7374c3665651 server_cookie=e2050952693d61ad in_seq=0 out_seq=0 2024-09-21T03:53:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.859+0000 7fedb0c00640 10 client.26363 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.860+0000 7fedb0c00640 1 -- 172.21.15.16:0/1160727917 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5567c4206600 con 0x5567c3e7e1c0 2024-09-21T03:53:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.860+0000 7fedb0c00640 10 client.26363 handle_command_reply: tid=0 2024-09-21T03:53:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.860+0000 7fedd48b2740 1 client.26363 shutdown 2024-09-21T03:53:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.861+0000 7fedd48b2740 1 -- 172.21.15.16:0/1160727917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ea8d80 msgr2=0x5567c429bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:47.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.861+0000 7fedd48b2740 1 --2- 172.21.15.16:0/1160727917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ea8d80 0x5567c429bc60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fedc00e5250 tx=0x7fedc00ad000 comp rx=0 tx=0).stop 2024-09-21T03:53:47.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.861+0000 7fedd48b2740 1 -- 172.21.15.16:0/1160727917 shutdown_connections 2024-09-21T03:53:47.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.861+0000 7fedd48b2740 1 --2- 172.21.15.16:0/1160727917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ea8d80 0x5567c429bc60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.861+0000 7fedd48b2740 1 --2- 172.21.15.16:0/1160727917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3eea1b0 0x5567c429b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.861+0000 7fedd48b2740 1 --2- 172.21.15.16:0/1160727917 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5567c3e7e1c0 0x5567c42a8120 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.861+0000 7fedd48b2740 1 --2- 172.21.15.16:0/1160727917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 0x5567c429aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.861+0000 7fedd48b2740 1 -- 172.21.15.16:0/1160727917 >> 172.21.15.16:0/1160727917 conn(0x5567c3ede300 msgr2=0x5567c428b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.862+0000 7fedb0c00640 0 client.26363 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.862+0000 7fedd48b2740 1 -- 172.21.15.16:0/1160727917 shutdown_connections 2024-09-21T03:53:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.864+0000 7fedd48b2740 1 -- 172.21.15.16:0/1160727917 wait complete. 2024-09-21T03:53:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.864+0000 7fedd48b2740 20 client.26363 trim_cache size 0 max 16384 2024-09-21T03:53:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.896+0000 7fedd48b2740 1 Processor -- start 2024-09-21T03:53:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.897+0000 7fedd48b2740 1 -- start start 2024-09-21T03:53:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.897+0000 7fedd48b2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3ea8d80 0x5567c428c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.897+0000 7fedd48b2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 0x5567c4290ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.897+0000 7fedd48b2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3eea1b0 0x5567c4298fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.897+0000 7fedd48b2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fedc81aa490 con 0x5567c3ec12f0 2024-09-21T03:53:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.897+0000 7fedd48b2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fedc81abfd0 con 0x5567c3eea1b0 2024-09-21T03:53:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.897+0000 7fedd48b2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fedc81aab60 con 0x5567c3ea8d80 2024-09-21T03:53:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.898+0000 7fedcd800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3eea1b0 0x5567c4298fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.899+0000 7fedcd800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3eea1b0 0x5567c4298fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:40972/0 (socket says 172.21.15.16:40972) 2024-09-21T03:53:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.899+0000 7fedcd800640 1 -- 172.21.15.16:0/321052168 learned_addr learned my addr 172.21.15.16:0/321052168 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.898+0000 7fedc6a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3ea8d80 0x5567c428c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.898+0000 7fedc6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 0x5567c4290ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.900+0000 7fedcd800640 1 -- 172.21.15.16:0/321052168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3ea8d80 msgr2=0x5567c428c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.900+0000 7fedcd800640 1 --2- 172.21.15.16:0/321052168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3ea8d80 0x5567c428c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.900+0000 7fedcd800640 1 -- 172.21.15.16:0/321052168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 msgr2=0x5567c4290ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.900+0000 7fedcd800640 1 --2- 172.21.15.16:0/321052168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 0x5567c4290ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.900+0000 7fedcd800640 1 -- 172.21.15.16:0/321052168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c42a2f00 con 0x5567c3eea1b0 2024-09-21T03:53:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.900+0000 7fedc6000640 1 --2- 172.21.15.16:0/321052168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 0x5567c4290ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.900+0000 7fedc6a00640 1 --2- 172.21.15.16:0/321052168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3ea8d80 0x5567c428c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.901+0000 7fedcd800640 1 --2- 172.21.15.16:0/321052168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3eea1b0 0x5567c4298fc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fedc00a9e10 tx=0x7fedc00ad130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40907242143a15e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.901+0000 7fedb3400640 1 -- 172.21.15.16:0/321052168 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc00b72b0 con 0x5567c3eea1b0 2024-09-21T03:53:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.902+0000 7fedb3400640 1 -- 172.21.15.16:0/321052168 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedc0004ca0 con 0x5567c3eea1b0 2024-09-21T03:53:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.902+0000 7fedb3400640 1 -- 172.21.15.16:0/321052168 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc00b72b0 con 0x5567c3eea1b0 2024-09-21T03:53:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.903+0000 7fedd48b2740 1 -- 172.21.15.16:0/321052168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3eea1b0 msgr2=0x5567c4298fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.903+0000 7fedd48b2740 1 --2- 172.21.15.16:0/321052168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3eea1b0 0x5567c4298fc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fedc00a9e10 tx=0x7fedc00ad130 comp rx=0 tx=0).stop 2024-09-21T03:53:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.903+0000 7fedd48b2740 1 -- 172.21.15.16:0/321052168 shutdown_connections 2024-09-21T03:53:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.903+0000 7fedd48b2740 1 --2- 172.21.15.16:0/321052168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3eea1b0 0x5567c4298fc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.903+0000 7fedd48b2740 1 --2- 172.21.15.16:0/321052168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 0x5567c4290ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.903+0000 7fedd48b2740 1 --2- 172.21.15.16:0/321052168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3ea8d80 0x5567c428c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.903+0000 7fedd48b2740 1 -- 172.21.15.16:0/321052168 >> 172.21.15.16:0/321052168 conn(0x5567c3ede300 msgr2=0x5567c4299c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.903+0000 7fedd48b2740 1 -- 172.21.15.16:0/321052168 shutdown_connections 2024-09-21T03:53:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.903+0000 7fedd48b2740 1 -- 172.21.15.16:0/321052168 wait complete. 2024-09-21T03:53:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.904+0000 7fedd48b2740 1 -- start start 2024-09-21T03:53:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.904+0000 7fedd48b2740 1 Processor -- start 2024-09-21T03:53:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.905+0000 7fedd48b2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ea8d80 0x5567c42a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.905+0000 7fedd48b2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 0x5567c429e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.905+0000 7fedd48b2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3eea1b0 0x5567c429eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.905+0000 7fedd48b2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fedc81ab900 con 0x5567c3ec12f0 2024-09-21T03:53:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.905+0000 7fedd48b2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fedc81ab230 con 0x5567c3ea8d80 2024-09-21T03:53:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.905+0000 7fedd48b2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fedc81a8470 con 0x5567c3eea1b0 2024-09-21T03:53:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.905+0000 7fedd48b2740 10 client.0 _ll_register_callbacks cb 0x7fedc000cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:53:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.906+0000 7fedc6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ea8d80 0x5567c42a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.906+0000 7fedc6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ea8d80 0x5567c42a28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:40976/0 (socket says 172.21.15.16:40976) 2024-09-21T03:53:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.906+0000 7fedc6a00640 1 -- 172.21.15.16:0/2287128780 learned_addr learned my addr 172.21.15.16:0/2287128780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:53:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.907+0000 7fedc6000640 1 --2- 172.21.15.16:0/2287128780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 0x5567c429e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.907+0000 7fedcd800640 1 --2- 172.21.15.16:0/2287128780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3eea1b0 0x5567c429eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.908+0000 7fedc6a00640 1 -- 172.21.15.16:0/2287128780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3eea1b0 msgr2=0x5567c429eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.908+0000 7fedc6a00640 1 --2- 172.21.15.16:0/2287128780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3eea1b0 0x5567c429eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.908+0000 7fedc6a00640 1 -- 172.21.15.16:0/2287128780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 msgr2=0x5567c429e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:53:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.908+0000 7fedc6a00640 1 --2- 172.21.15.16:0/2287128780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 0x5567c429e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.908+0000 7fedc6a00640 1 -- 172.21.15.16:0/2287128780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567c429fa30 con 0x5567c3ea8d80 2024-09-21T03:53:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.908+0000 7fedc6000640 1 --2- 172.21.15.16:0/2287128780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 0x5567c429e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:53:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.909+0000 7fedb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:53:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.909+0000 7fedc6a00640 1 --2- 172.21.15.16:0/2287128780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ea8d80 0x5567c42a28b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5567c42a88c0 tx=0x7fedb8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43e036b0dacd6cbd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:53:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.909+0000 7fedb0c00640 1 -- 172.21.15.16:0/2287128780 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb8031da0 con 0x5567c3ea8d80 2024-09-21T03:53:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.909+0000 7fedb0c00640 1 -- 172.21.15.16:0/2287128780 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fedb803a030 con 0x5567c3ea8d80 2024-09-21T03:53:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.910+0000 7fedd48b2740 1 -- 172.21.15.16:0/2287128780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc0004ca0 con 0x5567c3ea8d80 2024-09-21T03:53:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.910+0000 7fedb0c00640 1 -- 172.21.15.16:0/2287128780 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fedb8031da0 con 0x5567c3ea8d80 2024-09-21T03:53:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.910+0000 7fedd48b2740 10 client.26369 fetch_fsmap learned FSMap version 30 2024-09-21T03:53:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.910+0000 7fedd48b2740 1 -- 172.21.15.16:0/2287128780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5567c43240e0 con 0x5567c3ea8d80 2024-09-21T03:53:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.910+0000 7fedb0c00640 1 -- 172.21.15.16:0/2287128780 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedb8003aa0 con 0x5567c3ea8d80 2024-09-21T03:53:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.911+0000 7fedb0c00640 1 -- 172.21.15.16:0/2287128780 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fedb8033860 con 0x5567c3ea8d80 2024-09-21T03:53:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.911+0000 7fedd48b2740 10 client.26369 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:53:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.911+0000 7fedd48b2740 10 client.26369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:53:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.911+0000 7fedd48b2740 20 client.26369 populate_metadata read hostname 'smithi016' 2024-09-21T03:53:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.911+0000 7fedd48b2740 1 --2- 172.21.15.16:0/2287128780 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5567c4324a50 0x5567c4326e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:53:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.911+0000 7fedd48b2740 4 client.26369 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:53:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.911+0000 7fedd48b2740 1 -- 172.21.15.16:0/2287128780 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fedc0004ca0 con 0x5567c4324a50 2024-09-21T03:53:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.911+0000 7fedc6000640 1 --2- 172.21.15.16:0/2287128780 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5567c4324a50 0x5567c4326e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:53:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.912+0000 7fedc6000640 1 --2- 172.21.15.16:0/2287128780 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5567c4324a50 0x5567c4326e70 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fd0194e0f1b9303 server_cookie=60e05094ca14f40a in_seq=0 out_seq=0 2024-09-21T03:53:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.912+0000 7fedb0c00640 10 client.26369 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.932+0000 7fedb0c00640 1 -- 172.21.15.16:0/2287128780 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fedc0004ca0 con 0x5567c4324a50 2024-09-21T03:53:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.932+0000 7fedb0c00640 10 client.26369 handle_command_reply: tid=0 2024-09-21T03:53:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.932+0000 7fedd48b2740 1 client.26369 shutdown 2024-09-21T03:53:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.932+0000 7fedd48b2740 1 -- 172.21.15.16:0/2287128780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ea8d80 msgr2=0x5567c42a28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.932+0000 7fedd48b2740 1 --2- 172.21.15.16:0/2287128780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ea8d80 0x5567c42a28b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5567c42a88c0 tx=0x7fedb8004730 comp rx=0 tx=0).stop 2024-09-21T03:53:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.932+0000 7fedd48b2740 1 -- 172.21.15.16:0/2287128780 shutdown_connections 2024-09-21T03:53:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.932+0000 7fedd48b2740 1 --2- 172.21.15.16:0/2287128780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567c3eea1b0 0x5567c429eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.932+0000 7fedd48b2740 1 --2- 172.21.15.16:0/2287128780 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5567c4324a50 0x5567c4326e70 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.933+0000 7fedd48b2740 1 --2- 172.21.15.16:0/2287128780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3ec12f0 0x5567c429e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.933+0000 7fedd48b2740 1 --2- 172.21.15.16:0/2287128780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567c3ea8d80 0x5567c42a28b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.933+0000 7fedd48b2740 1 -- 172.21.15.16:0/2287128780 >> 172.21.15.16:0/2287128780 conn(0x5567c3ede300 msgr2=0x5567c4290ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.933+0000 7fedb0c00640 0 client.26369 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:53:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.933+0000 7fedd48b2740 1 -- 172.21.15.16:0/2287128780 shutdown_connections 2024-09-21T03:53:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.933+0000 7fedd48b2740 1 -- 172.21.15.16:0/2287128780 wait complete. 2024-09-21T03:53:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.933+0000 7fedd48b2740 20 client.26369 trim_cache size 0 max 16384 2024-09-21T03:53:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:53:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:53:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:53:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:53:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.934+0000 7fedc7e00640 1 -- 172.21.15.16:0/2732407286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fedac0b6510 msgr2=0x7fedac0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.935+0000 7fedc7e00640 1 --2- 172.21.15.16:0/2732407286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fedac0b6510 0x7fedac0b8a20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009ea0 tx=0x7fedb8005f90 comp rx=0 tx=0).stop 2024-09-21T03:53:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.935+0000 7fedc7e00640 1 -- 172.21.15.16:0/2732407286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedc8104f10 msgr2=0x7fedc8195de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:53:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.935+0000 7fedc7e00640 1 --2- 172.21.15.16:0/2732407286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedc8104f10 0x7fedc8195de0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fedc000eaa0 tx=0x7fedc000ef60 comp rx=0 tx=0).stop 2024-09-21T03:53:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.935+0000 7fedc7e00640 1 -- 172.21.15.16:0/2732407286 shutdown_connections 2024-09-21T03:53:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.935+0000 7fedc7e00640 1 --2- 172.21.15.16:0/2732407286 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fedac0b6510 0x7fedac0b8a20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.935+0000 7fedc7e00640 1 --2- 172.21.15.16:0/2732407286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedc8104f10 0x7fedc8195de0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.935+0000 7fedc7e00640 1 --2- 172.21.15.16:0/2732407286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc8104570 0x7fedc81952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.935+0000 7fedc7e00640 1 --2- 172.21.15.16:0/2732407286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc81034f0 0x7fedc8194c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:53:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.935+0000 7fedc7e00640 1 -- 172.21.15.16:0/2732407286 >> 172.21.15.16:0/2732407286 conn(0x7fedc80fee50 msgr2=0x7fedc8105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:53:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.936+0000 7fedc7e00640 1 -- 172.21.15.16:0/2732407286 shutdown_connections 2024-09-21T03:53:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:53:47.936+0000 7fedc7e00640 1 -- 172.21.15.16:0/2732407286 wait complete. 2024-09-21T03:53:49.054 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:48 smithi134 ceph-mon[35031]: pgmap v643: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 245 MiB/s wr, 1.10k op/s 2024-09-21T03:53:49.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:48 smithi123 ceph-mon[34673]: pgmap v643: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 245 MiB/s wr, 1.10k op/s 2024-09-21T03:53:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:48 smithi016 ceph-mon[30375]: pgmap v643: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 245 MiB/s wr, 1.10k op/s 2024-09-21T03:53:51.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:50 smithi134 ceph-mon[35031]: pgmap v644: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 230 MiB/s wr, 1.06k op/s 2024-09-21T03:53:51.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:50 smithi123 ceph-mon[34673]: pgmap v644: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 230 MiB/s wr, 1.06k op/s 2024-09-21T03:53:51.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:50 smithi016 ceph-mon[30375]: pgmap v644: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 230 MiB/s wr, 1.06k op/s 2024-09-21T03:53:53.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:52 smithi134 ceph-mon[35031]: pgmap v645: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 246 MiB/s wr, 1.16k op/s 2024-09-21T03:53:53.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:52 smithi123 ceph-mon[34673]: pgmap v645: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 246 MiB/s wr, 1.16k op/s 2024-09-21T03:53:53.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:52 smithi016 ceph-mon[30375]: pgmap v645: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 246 MiB/s wr, 1.16k op/s 2024-09-21T03:53:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:54 smithi134 ceph-mon[35031]: pgmap v646: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 236 MiB/s wr, 1.16k op/s 2024-09-21T03:53:55.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:54 smithi123 ceph-mon[34673]: pgmap v646: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 236 MiB/s wr, 1.16k op/s 2024-09-21T03:53:55.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:54 smithi016 ceph-mon[30375]: pgmap v646: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 236 MiB/s wr, 1.16k op/s 2024-09-21T03:53:57.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:56 smithi134 ceph-mon[35031]: pgmap v647: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 265 MiB/s wr, 1.31k op/s 2024-09-21T03:53:57.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:56 smithi123 ceph-mon[34673]: pgmap v647: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 265 MiB/s wr, 1.31k op/s 2024-09-21T03:53:57.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:56 smithi016 ceph-mon[30375]: pgmap v647: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 265 MiB/s wr, 1.31k op/s 2024-09-21T03:53:59.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:53:58 smithi134 ceph-mon[35031]: pgmap v648: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 245 MiB/s wr, 1.21k op/s 2024-09-21T03:53:59.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:53:58 smithi123 ceph-mon[34673]: pgmap v648: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 245 MiB/s wr, 1.21k op/s 2024-09-21T03:53:59.167 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:53:58 smithi016 ceph-mon[30375]: pgmap v648: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 245 MiB/s wr, 1.21k op/s 2024-09-21T03:54:01.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:00 smithi134 ceph-mon[35031]: pgmap v649: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 229 MiB/s wr, 1.13k op/s 2024-09-21T03:54:01.168 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:00 smithi016 ceph-mon[30375]: pgmap v649: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 229 MiB/s wr, 1.13k op/s 2024-09-21T03:54:01.168 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:00 smithi123 ceph-mon[34673]: pgmap v649: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 229 MiB/s wr, 1.13k op/s 2024-09-21T03:54:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.525+0000 7fb0a1e00640 1 -- 172.21.15.16:0/4291164432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb09c06b710 msgr2=0x7fb09c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.525+0000 7fb0a1e00640 1 --2- 172.21.15.16:0/4291164432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb09c06b710 0x7fb09c103f70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb08801a910 tx=0x7fb08803b450 comp rx=0 tx=0).stop 2024-09-21T03:54:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.526+0000 7fb0a1e00640 1 -- 172.21.15.16:0/4291164432 shutdown_connections 2024-09-21T03:54:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.526+0000 7fb0a1e00640 1 --2- 172.21.15.16:0/4291164432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb09c107070 0x7fb09c107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.526+0000 7fb0a1e00640 1 --2- 172.21.15.16:0/4291164432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb09c1044b0 0x7fb09c106930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.526+0000 7fb0a1e00640 1 --2- 172.21.15.16:0/4291164432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb09c06b710 0x7fb09c103f70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.526+0000 7fb0a1e00640 1 -- 172.21.15.16:0/4291164432 >> 172.21.15.16:0/4291164432 conn(0x7fb09c0fdd80 msgr2=0x7fb09c100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.526+0000 7fb0a1e00640 1 -- 172.21.15.16:0/4291164432 shutdown_connections 2024-09-21T03:54:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.527+0000 7fb0a1e00640 1 -- 172.21.15.16:0/4291164432 wait complete. 2024-09-21T03:54:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.529+0000 7fb0a1e00640 1 Processor -- start 2024-09-21T03:54:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.530+0000 7fb0a1e00640 1 -- start start 2024-09-21T03:54:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.530+0000 7fb0a1e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb09c06b710 0x7fb09c06d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.531+0000 7fb0a1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb09c1044b0 0x7fb09c06d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.531+0000 7fb0a1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb09c107070 0x7fb09c06e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.531+0000 7fb0a1e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c10b040 con 0x7fb09c107070 2024-09-21T03:54:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.531+0000 7fb0a1e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c10aec0 con 0x7fb09c1044b0 2024-09-21T03:54:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.531+0000 7fb0a1e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c10b1c0 con 0x7fb09c06b710 2024-09-21T03:54:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.531+0000 7fb0a0a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb09c06b710 0x7fb09c06d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.531+0000 7fb09be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb09c1044b0 0x7fb09c06d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.531+0000 7fb0a0a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb09c06b710 0x7fb09c06d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:59430/0 (socket says 172.21.15.16:59430) 2024-09-21T03:54:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.531+0000 7fb0a0a00640 1 -- 172.21.15.16:0/2123065951 learned_addr learned my addr 172.21.15.16:0/2123065951 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.531+0000 7fb09be00640 1 -- 172.21.15.16:0/2123065951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb09c06b710 msgr2=0x7fb09c06d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.531+0000 7fb09be00640 1 --2- 172.21.15.16:0/2123065951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb09c06b710 0x7fb09c06d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.531+0000 7fb09be00640 1 -- 172.21.15.16:0/2123065951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb09c107070 msgr2=0x7fb09c06e250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:54:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.532+0000 7fb0a1400640 1 --2- 172.21.15.16:0/2123065951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb09c107070 0x7fb09c06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.532+0000 7fb09be00640 1 --2- 172.21.15.16:0/2123065951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb09c107070 0x7fb09c06e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.532+0000 7fb09be00640 1 -- 172.21.15.16:0/2123065951 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09c06e790 con 0x7fb09c1044b0 2024-09-21T03:54:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.532+0000 7fb0a1400640 1 --2- 172.21.15.16:0/2123065951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb09c107070 0x7fb09c06e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.532+0000 7fb09be00640 1 --2- 172.21.15.16:0/2123065951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb09c1044b0 0x7fb09c06d6a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb090009fd0 tx=0x7fb09000b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.533+0000 7fb099600640 1 -- 172.21.15.16:0/2123065951 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090015020 con 0x7fb09c1044b0 2024-09-21T03:54:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.533+0000 7fb099600640 1 -- 172.21.15.16:0/2123065951 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb090011ce0 con 0x7fb09c1044b0 2024-09-21T03:54:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.533+0000 7fb099600640 1 -- 172.21.15.16:0/2123065951 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090010de0 con 0x7fb09c1044b0 2024-09-21T03:54:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.533+0000 7fb0a1e00640 1 -- 172.21.15.16:0/2123065951 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09c06ea80 con 0x7fb09c1044b0 2024-09-21T03:54:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.533+0000 7fb0a1e00640 1 -- 172.21.15.16:0/2123065951 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb09c06f000 con 0x7fb09c1044b0 2024-09-21T03:54:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.539+0000 7fb099600640 1 -- 172.21.15.16:0/2123065951 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb090018020 con 0x7fb09c1044b0 2024-09-21T03:54:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.541+0000 7fb0aa5c3740 1 Processor -- start 2024-09-21T03:54:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.542+0000 7fb0aa5c3740 1 -- start start 2024-09-21T03:54:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.542+0000 7fb0aa5c3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58d81b0 0x5646a5c7f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.542+0000 7fb0aa5c3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 0x5646a5c7f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.542+0000 7fb0aa5c3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58b74d0 0x5646a5c80470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.542+0000 7fb0aa5c3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5646a5bf26b0 con 0x5646a58b74d0 2024-09-21T03:54:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.542+0000 7fb0aa5c3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5646a5be1ae0 con 0x5646a58af2f0 2024-09-21T03:54:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.542+0000 7fb0aa5c3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5646a5be58f0 con 0x5646a58d81b0 2024-09-21T03:54:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.543+0000 7fb099600640 1 --2- 172.21.15.16:0/2123065951 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb0840b65d0 0x7fb0840b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.543+0000 7fb0a1400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58d81b0 0x5646a5c7f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.543+0000 7fb099600640 1 -- 172.21.15.16:0/2123065951 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb0900e6d50 con 0x7fb09c1044b0 2024-09-21T03:54:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.544+0000 7fb0a1400640 1 --2- 172.21.15.16:0/2123065951 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb0840b65d0 0x7fb0840b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.544+0000 7fb0a1400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58d81b0 0x5646a5c7f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:59444/0 (socket says 172.21.15.16:59444) 2024-09-21T03:54:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.544+0000 7fb0a1400640 1 -- 172.21.15.16:0/2641398953 learned_addr learned my addr 172.21.15.16:0/2641398953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.544+0000 7fb0a1400640 1 -- 172.21.15.16:0/2641398953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 msgr2=0x5646a5c7f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.544+0000 7fb0a0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 0x5646a5c7f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.544+0000 7fb0a1400640 1 --2- 172.21.15.16:0/2641398953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 0x5646a5c7f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.544+0000 7fb0a1400640 1 -- 172.21.15.16:0/2641398953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58b74d0 msgr2=0x5646a5c80470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.544+0000 7fb0a1400640 1 --2- 172.21.15.16:0/2641398953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58b74d0 0x5646a5c80470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.544+0000 7fb0a1400640 1 -- 172.21.15.16:0/2641398953 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a5bae780 con 0x5646a58d81b0 2024-09-21T03:54:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.545+0000 7fb0a0a00640 1 --2- 172.21.15.16:0/2641398953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 0x5646a5c7f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.545+0000 7fb0a1400640 1 --2- 172.21.15.16:0/2641398953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58d81b0 0x5646a5c7f2b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb094011e60 tx=0x7fb0940137c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e062c9ff69f78c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.545+0000 7fb0a1400640 1 --2- 172.21.15.16:0/2123065951 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb0840b65d0 0x7fb0840b8ae0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb094013cc0 tx=0x7fb09401a000 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.545+0000 7fb08dc00640 1 -- 172.21.15.16:0/2641398953 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0940118f0 con 0x5646a58d81b0 2024-09-21T03:54:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.546+0000 7fb08dc00640 1 -- 172.21.15.16:0/2641398953 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb094021070 con 0x5646a58d81b0 2024-09-21T03:54:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.546+0000 7fb08dc00640 1 -- 172.21.15.16:0/2641398953 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09401ce50 con 0x5646a58d81b0 2024-09-21T03:54:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.547+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/2641398953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58d81b0 msgr2=0x5646a5c7f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.547+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/2641398953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58d81b0 0x5646a5c7f2b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb094011e60 tx=0x7fb0940137c0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.548+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/2641398953 shutdown_connections 2024-09-21T03:54:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.548+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/2641398953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58b74d0 0x5646a5c80470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.548+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/2641398953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 0x5646a5c7f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.548+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/2641398953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58d81b0 0x5646a5c7f2b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.548+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/2641398953 >> 172.21.15.16:0/2641398953 conn(0x5646a58cc300 msgr2=0x5646a5c7ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.548+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/2641398953 shutdown_connections 2024-09-21T03:54:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.548+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/2641398953 wait complete. 2024-09-21T03:54:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.549+0000 7fb0aa5c3740 1 -- start start 2024-09-21T03:54:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.549+0000 7fb0aa5c3740 1 Processor -- start 2024-09-21T03:54:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.550+0000 7fb0aa5c3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 0x5646a5c88a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.550+0000 7fb0aa5c3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58b74d0 0x5646a5c89150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.550+0000 7fb0aa5c3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58d81b0 0x5646a5c89cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.551+0000 7fb0a0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 0x5646a5c88a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.551+0000 7fb0a0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 0x5646a5c88a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42736/0 (socket says 172.21.15.16:42736) 2024-09-21T03:54:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.551+0000 7fb0a0a00640 1 -- 172.21.15.16:0/4020296908 learned_addr learned my addr 172.21.15.16:0/4020296908 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.551+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/4020296908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5646a5b19ab0 con 0x5646a58b74d0 2024-09-21T03:54:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.551+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/4020296908 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5646a5b88570 con 0x5646a58af2f0 2024-09-21T03:54:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.551+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/4020296908 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5646a5c80f30 con 0x5646a58d81b0 2024-09-21T03:54:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.551+0000 7fb0a1400640 1 --2- 172.21.15.16:0/4020296908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58d81b0 0x5646a5c89cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.552+0000 7fb09be00640 1 --2- 172.21.15.16:0/4020296908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58b74d0 0x5646a5c89150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.552+0000 7fb0aa5c3740 10 client.0 _ll_register_callbacks cb 0x5646a5b97100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.552+0000 7fb0a1400640 1 -- 172.21.15.16:0/4020296908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 msgr2=0x5646a5c88a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.552+0000 7fb0a1400640 1 --2- 172.21.15.16:0/4020296908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 0x5646a5c88a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.552+0000 7fb0a1400640 1 -- 172.21.15.16:0/4020296908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58b74d0 msgr2=0x5646a5c89150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.552+0000 7fb0a1400640 1 --2- 172.21.15.16:0/4020296908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58b74d0 0x5646a5c89150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.552+0000 7fb0a1400640 1 -- 172.21.15.16:0/4020296908 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a5943790 con 0x5646a58d81b0 2024-09-21T03:54:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.552+0000 7fb0a0a00640 1 --2- 172.21.15.16:0/4020296908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 0x5646a5c88a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.553+0000 7fb082000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:54:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.553+0000 7fb0a1400640 1 --2- 172.21.15.16:0/4020296908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58d81b0 0x5646a5c89cc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb094004a80 tx=0x7fb094007e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8adc878afd55d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.554+0000 7fb082000640 1 -- 172.21.15.16:0/4020296908 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09402a070 con 0x5646a58d81b0 2024-09-21T03:54:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.554+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/4020296908 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb094021070 con 0x5646a58d81b0 2024-09-21T03:54:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.555+0000 7fb082000640 1 -- 172.21.15.16:0/4020296908 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb09401cbd0 con 0x5646a58d81b0 2024-09-21T03:54:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.556+0000 7fb082000640 1 -- 172.21.15.16:0/4020296908 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb09402a070 con 0x5646a58d81b0 2024-09-21T03:54:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.556+0000 7fb0aa5c3740 10 client.26317 fetch_fsmap learned FSMap version 30 2024-09-21T03:54:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.556+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/4020296908 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5646a59bc620 con 0x5646a58d81b0 2024-09-21T03:54:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.557+0000 7fb082000640 1 -- 172.21.15.16:0/4020296908 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb094021070 con 0x5646a58d81b0 2024-09-21T03:54:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.560+0000 7fb082000640 1 -- 172.21.15.16:0/4020296908 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb09401b030 con 0x5646a58d81b0 2024-09-21T03:54:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.565+0000 7fb0aa5c3740 10 client.26317 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:54:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.565+0000 7fb0aa5c3740 10 client.26317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.565+0000 7fb0aa5c3740 20 client.26317 populate_metadata read hostname 'smithi016' 2024-09-21T03:54:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.565+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/4020296908 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5646a5896d80 0x5646a5c95ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.566+0000 7fb0aa5c3740 4 client.26317 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:01.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.566+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/4020296908 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646a5bf4600 con 0x5646a5896d80 2024-09-21T03:54:01.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.566+0000 7fb0a0a00640 1 --2- 172.21.15.16:0/4020296908 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5646a5896d80 0x5646a5c95ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.568+0000 7fb0a0a00640 1 --2- 172.21.15.16:0/4020296908 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5646a5896d80 0x5646a5c95ed0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b147e7ce0a6a2e95 server_cookie=996e03f0a5d3df97 in_seq=0 out_seq=0 2024-09-21T03:54:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.568+0000 7fb082000640 10 client.26317 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.569+0000 7fb082000640 1 -- 172.21.15.16:0/4020296908 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5646a5bf4600 con 0x5646a5896d80 2024-09-21T03:54:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.570+0000 7fb082000640 10 client.26317 handle_command_reply: tid=0 2024-09-21T03:54:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.570+0000 7fb0aa5c3740 1 client.26317 shutdown 2024-09-21T03:54:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.570+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/4020296908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58d81b0 msgr2=0x5646a5c89cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.570+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/4020296908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58d81b0 0x5646a5c89cc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb094004a80 tx=0x7fb094007e70 comp rx=0 tx=0).stop 2024-09-21T03:54:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.571+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/4020296908 shutdown_connections 2024-09-21T03:54:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.571+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/4020296908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58d81b0 0x5646a5c89cc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.571+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/4020296908 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5646a5896d80 0x5646a5c95ed0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.571+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/4020296908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58b74d0 0x5646a5c89150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.571+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/4020296908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 0x5646a5c88a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.571+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/4020296908 >> 172.21.15.16:0/4020296908 conn(0x5646a58cc300 msgr2=0x5646a5c79360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.571+0000 7fb082000640 0 client.26317 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.571+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/4020296908 shutdown_connections 2024-09-21T03:54:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.571+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/4020296908 wait complete. 2024-09-21T03:54:01.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.579+0000 7fb0aa5c3740 20 client.26317 trim_cache size 0 max 16384 2024-09-21T03:54:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.638+0000 7fb0aa5c3740 1 Processor -- start 2024-09-21T03:54:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.638+0000 7fb0aa5c3740 1 -- start start 2024-09-21T03:54:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.638+0000 7fb0aa5c3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58af2f0 0x5646a5c75b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.639+0000 7fb0aa5c3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58b74d0 0x5646a5c87cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.639+0000 7fb0aa5c3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58d81b0 0x5646a5c7b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.639+0000 7fb0aa5c3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1b2e00 con 0x5646a58b74d0 2024-09-21T03:54:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.639+0000 7fb0aa5c3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1b4940 con 0x5646a58d81b0 2024-09-21T03:54:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.639+0000 7fb0aa5c3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1b34d0 con 0x5646a58af2f0 2024-09-21T03:54:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.639+0000 7fb0a1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58d81b0 0x5646a5c7b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.639+0000 7fb0a0a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58af2f0 0x5646a5c75b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.639+0000 7fb0a1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58d81b0 0x5646a5c7b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42744/0 (socket says 172.21.15.16:42744) 2024-09-21T03:54:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.639+0000 7fb0a1400640 1 -- 172.21.15.16:0/1955415606 learned_addr learned my addr 172.21.15.16:0/1955415606 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.640+0000 7fb09be00640 1 --2- 172.21.15.16:0/1955415606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58b74d0 0x5646a5c87cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.640+0000 7fb0a0a00640 1 -- 172.21.15.16:0/1955415606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58d81b0 msgr2=0x5646a5c7b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.640+0000 7fb0a0a00640 1 --2- 172.21.15.16:0/1955415606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58d81b0 0x5646a5c7b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.640+0000 7fb0a0a00640 1 -- 172.21.15.16:0/1955415606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58b74d0 msgr2=0x5646a5c87cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.640+0000 7fb0a0a00640 1 --2- 172.21.15.16:0/1955415606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58b74d0 0x5646a5c87cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.640+0000 7fb0a0a00640 1 -- 172.21.15.16:0/1955415606 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a5c8a220 con 0x5646a58af2f0 2024-09-21T03:54:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.640+0000 7fb0a1400640 1 --2- 172.21.15.16:0/1955415606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58d81b0 0x5646a5c7b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:54:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.640+0000 7fb0a0a00640 1 --2- 172.21.15.16:0/1955415606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58af2f0 0x5646a5c75b80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x5646a5c964e0 tx=0x7fb08801a890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c95329224019a8d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.641+0000 7fb08d200640 1 -- 172.21.15.16:0/1955415606 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088046070 con 0x5646a58af2f0 2024-09-21T03:54:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.641+0000 7fb08d200640 1 -- 172.21.15.16:0/1955415606 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0880053d0 con 0x5646a58af2f0 2024-09-21T03:54:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.641+0000 7fb08d200640 1 -- 172.21.15.16:0/1955415606 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088040b60 con 0x5646a58af2f0 2024-09-21T03:54:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.642+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/1955415606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58af2f0 msgr2=0x5646a5c75b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.642+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/1955415606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58af2f0 0x5646a5c75b80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x5646a5c964e0 tx=0x7fb08801a890 comp rx=0 tx=0).stop 2024-09-21T03:54:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.643+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/1955415606 shutdown_connections 2024-09-21T03:54:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.643+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/1955415606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58d81b0 0x5646a5c7b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.643+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/1955415606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58b74d0 0x5646a5c87cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.643+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/1955415606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58af2f0 0x5646a5c75b80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.643+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/1955415606 >> 172.21.15.16:0/1955415606 conn(0x5646a58cc300 msgr2=0x5646a5c7ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.644+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/1955415606 shutdown_connections 2024-09-21T03:54:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.644+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/1955415606 wait complete. 2024-09-21T03:54:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.644+0000 7fb0aa5c3740 1 -- start start 2024-09-21T03:54:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.645+0000 7fb0aa5c3740 1 Processor -- start 2024-09-21T03:54:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.646+0000 7fb0aa5c3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 0x5646a5c8ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.646+0000 7fb0aa5c3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58b74d0 0x5646a5c8ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.646+0000 7fb0aa5c3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58d81b0 0x5646a5c8f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.646+0000 7fb0aa5c3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1b4270 con 0x5646a58d81b0 2024-09-21T03:54:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.646+0000 7fb0aa5c3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1b3ba0 con 0x5646a58af2f0 2024-09-21T03:54:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.646+0000 7fb0aa5c3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1b0d80 con 0x5646a58b74d0 2024-09-21T03:54:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.646+0000 7fb0aa5c3740 10 client.0 _ll_register_callbacks cb 0x7fb094011cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.646+0000 7fb0a0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 0x5646a5c8ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.646+0000 7fb0a0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 0x5646a5c8ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42748/0 (socket says 172.21.15.16:42748) 2024-09-21T03:54:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.646+0000 7fb0a0a00640 1 -- 172.21.15.16:0/1519535589 learned_addr learned my addr 172.21.15.16:0/1519535589 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.647+0000 7fb09be00640 1 --2- 172.21.15.16:0/1519535589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58b74d0 0x5646a5c8ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.647+0000 7fb0a0a00640 1 -- 172.21.15.16:0/1519535589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58b74d0 msgr2=0x5646a5c8ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.647+0000 7fb0a0a00640 1 --2- 172.21.15.16:0/1519535589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58b74d0 0x5646a5c8ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.647+0000 7fb0a0a00640 1 -- 172.21.15.16:0/1519535589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58d81b0 msgr2=0x5646a5c8f180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:54:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.647+0000 7fb0a0a00640 1 --2- 172.21.15.16:0/1519535589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58d81b0 0x5646a5c8f180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.647+0000 7fb0a0a00640 1 -- 172.21.15.16:0/1519535589 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a5c8f6c0 con 0x5646a58af2f0 2024-09-21T03:54:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.647+0000 7fb0a0a00640 1 --2- 172.21.15.16:0/1519535589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 0x5646a5c8ff80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb088004a30 tx=0x7fb088004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=410255787c6aa79d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.648+0000 7fb082000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:54:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.648+0000 7fb082000640 1 -- 172.21.15.16:0/1519535589 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088046040 con 0x5646a58af2f0 2024-09-21T03:54:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.648+0000 7fb082000640 1 -- 172.21.15.16:0/1519535589 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb088055040 con 0x5646a58af2f0 2024-09-21T03:54:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.649+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/1519535589 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0880053d0 con 0x5646a58af2f0 2024-09-21T03:54:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.650+0000 7fb082000640 1 -- 172.21.15.16:0/1519535589 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb088040590 con 0x5646a58af2f0 2024-09-21T03:54:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.650+0000 7fb0aa5c3740 10 client.26381 fetch_fsmap learned FSMap version 30 2024-09-21T03:54:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.650+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/1519535589 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5646a5c8e460 con 0x5646a58af2f0 2024-09-21T03:54:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.650+0000 7fb082000640 1 -- 172.21.15.16:0/1519535589 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0880053d0 con 0x5646a58af2f0 2024-09-21T03:54:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.654+0000 7fb082000640 1 -- 172.21.15.16:0/1519535589 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb08804b040 con 0x5646a58af2f0 2024-09-21T03:54:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.654+0000 7fb0aa5c3740 10 client.26381 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:54:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.654+0000 7fb0aa5c3740 10 client.26381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.654+0000 7fb0aa5c3740 20 client.26381 populate_metadata read hostname 'smithi016' 2024-09-21T03:54:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.654+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/1519535589 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5646a586c1c0 0x5646a5d02d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.654+0000 7fb0aa5c3740 4 client.26381 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:54:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.654+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/1519535589 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646a5bf00b0 con 0x5646a586c1c0 2024-09-21T03:54:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.656+0000 7fb09be00640 1 --2- 172.21.15.16:0/1519535589 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5646a586c1c0 0x5646a5d02d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.656+0000 7fb082000640 10 client.26381 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.658+0000 7fb09be00640 1 --2- 172.21.15.16:0/1519535589 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5646a586c1c0 0x5646a5d02d90 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b76dd018374673a server_cookie=309fcdb0d473bd57 in_seq=0 out_seq=0 2024-09-21T03:54:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.679+0000 7fb082000640 1 -- 172.21.15.16:0/1519535589 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646a5bf00b0 con 0x5646a586c1c0 2024-09-21T03:54:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.679+0000 7fb082000640 10 client.26381 handle_command_reply: tid=0 2024-09-21T03:54:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.679+0000 7fb0aa5c3740 1 client.26381 shutdown 2024-09-21T03:54:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.685+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/1519535589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 msgr2=0x5646a5c8ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.685+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/1519535589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 0x5646a5c8ff80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb088004a30 tx=0x7fb088004a60 comp rx=0 tx=0).stop 2024-09-21T03:54:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.688+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/1519535589 shutdown_connections 2024-09-21T03:54:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.688+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/1519535589 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5646a586c1c0 0x5646a5d02d90 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.688+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/1519535589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646a58d81b0 0x5646a5c8f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.688+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/1519535589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a58b74d0 0x5646a5c8ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.688+0000 7fb0aa5c3740 1 --2- 172.21.15.16:0/1519535589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a58af2f0 0x5646a5c8ff80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.688+0000 7fb082000640 0 client.26381 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.688+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/1519535589 >> 172.21.15.16:0/1519535589 conn(0x5646a58cc300 msgr2=0x5646a5c87cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.694+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/1519535589 shutdown_connections 2024-09-21T03:54:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.694+0000 7fb0aa5c3740 1 -- 172.21.15.16:0/1519535589 wait complete. 2024-09-21T03:54:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.694+0000 7fb0aa5c3740 20 client.26381 trim_cache size 0 max 16384 2024-09-21T03:54:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:54:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:54:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:54:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:54:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.698+0000 7fb0a1e00640 1 -- 172.21.15.16:0/2123065951 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb0840b65d0 msgr2=0x7fb0840b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.698+0000 7fb0a1e00640 1 --2- 172.21.15.16:0/2123065951 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb0840b65d0 0x7fb0840b8ae0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb094013cc0 tx=0x7fb09401a000 comp rx=0 tx=0).stop 2024-09-21T03:54:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.698+0000 7fb0a1e00640 1 -- 172.21.15.16:0/2123065951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb09c1044b0 msgr2=0x7fb09c06d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.698+0000 7fb0a1e00640 1 --2- 172.21.15.16:0/2123065951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb09c1044b0 0x7fb09c06d6a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb090009fd0 tx=0x7fb09000b870 comp rx=0 tx=0).stop 2024-09-21T03:54:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.699+0000 7fb0a1e00640 1 -- 172.21.15.16:0/2123065951 shutdown_connections 2024-09-21T03:54:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.699+0000 7fb0a1e00640 1 --2- 172.21.15.16:0/2123065951 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb0840b65d0 0x7fb0840b8ae0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.699+0000 7fb0a1e00640 1 --2- 172.21.15.16:0/2123065951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb09c107070 0x7fb09c06e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.699+0000 7fb0a1e00640 1 --2- 172.21.15.16:0/2123065951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb09c1044b0 0x7fb09c06d6a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.699+0000 7fb0a1e00640 1 --2- 172.21.15.16:0/2123065951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb09c06b710 0x7fb09c06d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.699+0000 7fb0a1e00640 1 -- 172.21.15.16:0/2123065951 >> 172.21.15.16:0/2123065951 conn(0x7fb09c0fdd80 msgr2=0x7fb09c1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.699+0000 7fb0a1e00640 1 -- 172.21.15.16:0/2123065951 shutdown_connections 2024-09-21T03:54:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:01.699+0000 7fb0a1e00640 1 -- 172.21.15.16:0/2123065951 wait complete. 2024-09-21T03:54:03.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:02 smithi123 ceph-mon[34673]: pgmap v650: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 240 MiB/s wr, 1.17k op/s 2024-09-21T03:54:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:02 smithi016 ceph-mon[30375]: pgmap v650: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 240 MiB/s wr, 1.17k op/s 2024-09-21T03:54:03.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:02 smithi134 ceph-mon[35031]: pgmap v650: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 240 MiB/s wr, 1.17k op/s 2024-09-21T03:54:05.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:04 smithi123 ceph-mon[34673]: pgmap v651: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 237 MiB/s wr, 1.16k op/s 2024-09-21T03:54:05.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:04 smithi016 ceph-mon[30375]: pgmap v651: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 237 MiB/s wr, 1.16k op/s 2024-09-21T03:54:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:04 smithi134 ceph-mon[35031]: pgmap v651: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 237 MiB/s wr, 1.16k op/s 2024-09-21T03:54:07.151 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:06 smithi016 ceph-mon[30375]: pgmap v652: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 262 MiB/s wr, 1.27k op/s 2024-09-21T03:54:07.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:06 smithi123 ceph-mon[34673]: pgmap v652: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 262 MiB/s wr, 1.27k op/s 2024-09-21T03:54:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:06 smithi134 ceph-mon[35031]: pgmap v652: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 262 MiB/s wr, 1.27k op/s 2024-09-21T03:54:09.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:08 smithi134 ceph-mon[35031]: pgmap v653: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 241 MiB/s wr, 1.18k op/s 2024-09-21T03:54:09.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:08 smithi123 ceph-mon[34673]: pgmap v653: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 241 MiB/s wr, 1.18k op/s 2024-09-21T03:54:09.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:08 smithi016 ceph-mon[30375]: pgmap v653: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 241 MiB/s wr, 1.18k op/s 2024-09-21T03:54:11.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:10 smithi134 ceph-mon[35031]: pgmap v654: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 228 MiB/s wr, 1.11k op/s 2024-09-21T03:54:11.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:10 smithi123 ceph-mon[34673]: pgmap v654: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 228 MiB/s wr, 1.11k op/s 2024-09-21T03:54:11.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:10 smithi016 ceph-mon[30375]: pgmap v654: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 228 MiB/s wr, 1.11k op/s 2024-09-21T03:54:13.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:12 smithi134 ceph-mon[35031]: pgmap v655: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 240 MiB/s wr, 1.16k op/s 2024-09-21T03:54:13.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:12 smithi123 ceph-mon[34673]: pgmap v655: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 240 MiB/s wr, 1.16k op/s 2024-09-21T03:54:13.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:12 smithi016 ceph-mon[30375]: pgmap v655: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 240 MiB/s wr, 1.16k op/s 2024-09-21T03:54:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.100+0000 7f8acc000640 1 -- 172.21.15.16:0/199844006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ac4104610 msgr2=0x7f8ac4106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.100+0000 7f8acc000640 1 --2- 172.21.15.16:0/199844006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ac4104610 0x7f8ac4106a90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8009900 tx=0x7f8ab802a440 comp rx=0 tx=0).stop 2024-09-21T03:54:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.100+0000 7f8acc000640 1 -- 172.21.15.16:0/199844006 shutdown_connections 2024-09-21T03:54:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.100+0000 7f8acc000640 1 --2- 172.21.15.16:0/199844006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac41071d0 0x7f8ac41076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.100+0000 7f8acc000640 1 --2- 172.21.15.16:0/199844006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ac4104610 0x7f8ac4106a90 secure :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8009900 tx=0x7f8ab802a440 comp rx=0 tx=0).stop 2024-09-21T03:54:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.100+0000 7f8acc000640 1 --2- 172.21.15.16:0/199844006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ac406b710 0x7f8ac41040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.100+0000 7f8acc000640 1 -- 172.21.15.16:0/199844006 >> 172.21.15.16:0/199844006 conn(0x7f8ac40fdee0 msgr2=0x7f8ac4100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.103+0000 7f8acc000640 1 -- 172.21.15.16:0/199844006 shutdown_connections 2024-09-21T03:54:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.103+0000 7f8acc000640 1 -- 172.21.15.16:0/199844006 wait complete. 2024-09-21T03:54:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.105+0000 7f8acc000640 1 Processor -- start 2024-09-21T03:54:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.106+0000 7f8acc000640 1 -- start start 2024-09-21T03:54:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.109+0000 7f8acc000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ac406b710 0x7f8ac40749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.110+0000 7f8ac9600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ac406b710 0x7f8ac40749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.110+0000 7f8ac9600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ac406b710 0x7f8ac40749d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:36228/0 (socket says 172.21.15.16:36228) 2024-09-21T03:54:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.111+0000 7f8ac9600640 1 -- 172.21.15.16:0/1545129444 learned_addr learned my addr 172.21.15.16:0/1545129444 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.111+0000 7f8acc000640 1 --2- 172.21.15.16:0/1545129444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ac41071d0 0x7f8ac4075040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.112+0000 7f8acc000640 1 --2- 172.21.15.16:0/1545129444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac40736d0 0x7f8ac4073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.112+0000 7f8ac8c00640 1 --2- 172.21.15.16:0/1545129444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ac41071d0 0x7f8ac4075040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.113+0000 7f8aca000640 1 --2- 172.21.15.16:0/1545129444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac40736d0 0x7f8ac4073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.114+0000 7f8acc000640 1 -- 172.21.15.16:0/1545129444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac4115a60 con 0x7f8ac41071d0 2024-09-21T03:54:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.117+0000 7f8acc000640 1 -- 172.21.15.16:0/1545129444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41158e0 con 0x7f8ac40736d0 2024-09-21T03:54:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.117+0000 7f8acc000640 1 -- 172.21.15.16:0/1545129444 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac4115be0 con 0x7f8ac406b710 2024-09-21T03:54:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.118+0000 7f8aca000640 1 -- 172.21.15.16:0/1545129444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ac406b710 msgr2=0x7f8ac40749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.118+0000 7f8aca000640 1 --2- 172.21.15.16:0/1545129444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ac406b710 0x7f8ac40749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.118+0000 7f8aca000640 1 -- 172.21.15.16:0/1545129444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ac41071d0 msgr2=0x7f8ac4075040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.120+0000 7f8aca000640 1 --2- 172.21.15.16:0/1545129444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ac41071d0 0x7f8ac4075040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.120+0000 7f8aca000640 1 -- 172.21.15.16:0/1545129444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac40743f0 con 0x7f8ac40736d0 2024-09-21T03:54:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.121+0000 7f8aca000640 1 --2- 172.21.15.16:0/1545129444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac40736d0 0x7f8ac4073bd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00be00 tx=0x7f8abc00bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.122+0000 7f8ac2000640 1 -- 172.21.15.16:0/1545129444 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc019070 con 0x7f8ac40736d0 2024-09-21T03:54:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.122+0000 7f8ac2000640 1 -- 172.21.15.16:0/1545129444 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8abc0148f0 con 0x7f8ac40736d0 2024-09-21T03:54:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.122+0000 7f8ac2000640 1 -- 172.21.15.16:0/1545129444 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc012830 con 0x7f8ac40736d0 2024-09-21T03:54:15.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.122+0000 7f8acc000640 1 -- 172.21.15.16:0/1545129444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac40722e0 con 0x7f8ac40736d0 2024-09-21T03:54:15.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.122+0000 7f8acc000640 1 -- 172.21.15.16:0/1545129444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac4072860 con 0x7f8ac40736d0 2024-09-21T03:54:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.128+0000 7f8ad3040740 1 Processor -- start 2024-09-21T03:54:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.132+0000 7f8ad3040740 1 -- start start 2024-09-21T03:54:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.134+0000 7f8ac2000640 1 -- 172.21.15.16:0/1545129444 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8abc01c070 con 0x7f8ac40736d0 2024-09-21T03:54:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.134+0000 7f8ad3040740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 0x563b6f2fd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.134+0000 7f8ad3040740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef2d2f0 0x563b6f2fd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.134+0000 7f8ad3040740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef354d0 0x563b6f2fe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.134+0000 7f8ad3040740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b6f2706b0 con 0x563b6ef2d2f0 2024-09-21T03:54:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.134+0000 7f8ad3040740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563b6f25fae0 con 0x563b6ef354d0 2024-09-21T03:54:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.134+0000 7f8ad3040740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563b6f2638f0 con 0x563b6ef561b0 2024-09-21T03:54:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.135+0000 7f8ac9600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef2d2f0 0x563b6f2fd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.136+0000 7f8ac9600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef2d2f0 0x563b6f2fd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56088/0 (socket says 172.21.15.16:56088) 2024-09-21T03:54:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.136+0000 7f8ac9600640 1 -- 172.21.15.16:0/2280128958 learned_addr learned my addr 172.21.15.16:0/2280128958 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.138+0000 7f8ac9600640 1 -- 172.21.15.16:0/2280128958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 msgr2=0x563b6f2fd0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:54:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.138+0000 7f8ac9600640 1 --2- 172.21.15.16:0/2280128958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 0x563b6f2fd0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.138+0000 7f8ac9600640 1 -- 172.21.15.16:0/2280128958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef354d0 msgr2=0x563b6f2fe290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:54:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.138+0000 7f8ac9600640 1 --2- 172.21.15.16:0/2280128958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef354d0 0x563b6f2fe290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.138+0000 7f8ac9600640 1 -- 172.21.15.16:0/2280128958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b6f22c780 con 0x563b6ef2d2f0 2024-09-21T03:54:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.139+0000 7f8ac9600640 1 --2- 172.21.15.16:0/2280128958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef2d2f0 0x563b6f2fd740 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0009b00 tx=0x7f8ab000e1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab5bf50f9728d1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:15.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.140+0000 7f8ab6600640 1 -- 172.21.15.16:0/2280128958 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0017070 con 0x563b6ef2d2f0 2024-09-21T03:54:15.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.140+0000 7f8ab6600640 1 -- 172.21.15.16:0/2280128958 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ab000ec10 con 0x563b6ef2d2f0 2024-09-21T03:54:15.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.140+0000 7f8ab6600640 1 -- 172.21.15.16:0/2280128958 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0012630 con 0x563b6ef2d2f0 2024-09-21T03:54:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.140+0000 7f8ac2000640 1 --2- 172.21.15.16:0/1545129444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8aac0b6650 0x7f8aac0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.141+0000 7f8ac2000640 1 -- 172.21.15.16:0/1545129444 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f8abc0e6d60 con 0x7f8ac40736d0 2024-09-21T03:54:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.141+0000 7f8ac8c00640 1 --2- 172.21.15.16:0/1545129444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8aac0b6650 0x7f8aac0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.141+0000 7f8ad3040740 1 -- 172.21.15.16:0/2280128958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef2d2f0 msgr2=0x563b6f2fd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.141+0000 7f8ad3040740 1 --2- 172.21.15.16:0/2280128958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef2d2f0 0x563b6f2fd740 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0009b00 tx=0x7f8ab000e1d0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.142+0000 7f8ad3040740 1 -- 172.21.15.16:0/2280128958 shutdown_connections 2024-09-21T03:54:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.142+0000 7f8ad3040740 1 --2- 172.21.15.16:0/2280128958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef354d0 0x563b6f2fe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.142+0000 7f8ad3040740 1 --2- 172.21.15.16:0/2280128958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef2d2f0 0x563b6f2fd740 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.142+0000 7f8ad3040740 1 --2- 172.21.15.16:0/2280128958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 0x563b6f2fd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.142+0000 7f8ad3040740 1 -- 172.21.15.16:0/2280128958 >> 172.21.15.16:0/2280128958 conn(0x563b6ef4a300 msgr2=0x563b6f2f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.142+0000 7f8ad3040740 1 -- 172.21.15.16:0/2280128958 shutdown_connections 2024-09-21T03:54:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.143+0000 7f8ad3040740 1 -- 172.21.15.16:0/2280128958 wait complete. 2024-09-21T03:54:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.143+0000 7f8ad3040740 1 -- start start 2024-09-21T03:54:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.144+0000 7f8ad3040740 1 Processor -- start 2024-09-21T03:54:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.145+0000 7f8ad3040740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 0x563b6f306860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.146+0000 7f8ad3040740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef354d0 0x563b6f306f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.146+0000 7f8ad3040740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 0x563b6f307a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.147+0000 7f8ad3040740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b6f197ab0 con 0x563b6ef354d0 2024-09-21T03:54:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.147+0000 7f8ad3040740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563b6f206570 con 0x563b6ef2d2f0 2024-09-21T03:54:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.147+0000 7f8ad3040740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563b6f2fedb0 con 0x563b6ef561b0 2024-09-21T03:54:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.147+0000 7f8aca000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 0x563b6f307a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.147+0000 7f8aca000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 0x563b6f307a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:36250/0 (socket says 172.21.15.16:36250) 2024-09-21T03:54:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.147+0000 7f8aca000640 1 -- 172.21.15.16:0/2643886052 learned_addr learned my addr 172.21.15.16:0/2643886052 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.146+0000 7f8ac9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 0x563b6f306860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.147+0000 7f8ad3040740 10 client.0 _ll_register_callbacks cb 0x563b6f215100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.146+0000 7f8ac8c00640 1 --2- 172.21.15.16:0/1545129444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8aac0b6650 0x7f8aac0b8b60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8009850 tx=0x7f8ab8031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.148+0000 7f8ac9600640 1 -- 172.21.15.16:0/2643886052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 msgr2=0x563b6f307a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.148+0000 7f8ac9600640 1 --2- 172.21.15.16:0/2643886052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 0x563b6f307a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.148+0000 7f8ac9600640 1 -- 172.21.15.16:0/2643886052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef354d0 msgr2=0x563b6f306f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.148+0000 7f8ac9600640 1 --2- 172.21.15.16:0/2643886052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef354d0 0x563b6f306f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.148+0000 7f8ac9600640 1 -- 172.21.15.16:0/2643886052 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b6efc1790 con 0x563b6ef2d2f0 2024-09-21T03:54:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.150+0000 7f8ac9600640 1 --2- 172.21.15.16:0/2643886052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 0x563b6f306860 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0009b00 tx=0x7f8ab0013a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53cb9de66aa9f4ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.150+0000 7f8aaaa00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:54:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.151+0000 7f8aaaa00640 1 -- 172.21.15.16:0/2643886052 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0017040 con 0x563b6ef2d2f0 2024-09-21T03:54:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.151+0000 7f8ad3040740 1 -- 172.21.15.16:0/2643886052 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ab000ec10 con 0x563b6ef2d2f0 2024-09-21T03:54:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.154+0000 7f8aaaa00640 1 -- 172.21.15.16:0/2643886052 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f8ab000edc0 con 0x563b6ef2d2f0 2024-09-21T03:54:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.154+0000 7f8aaaa00640 1 -- 172.21.15.16:0/2643886052 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ab002f050 con 0x563b6ef2d2f0 2024-09-21T03:54:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.154+0000 7f8ad3040740 10 client.26387 fetch_fsmap learned FSMap version 30 2024-09-21T03:54:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.154+0000 7f8ad3040740 1 -- 172.21.15.16:0/2643886052 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563b6f03a620 con 0x563b6ef2d2f0 2024-09-21T03:54:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.155+0000 7f8aaaa00640 1 -- 172.21.15.16:0/2643886052 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ab000ec10 con 0x563b6ef2d2f0 2024-09-21T03:54:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.157+0000 7f8aaaa00640 1 -- 172.21.15.16:0/2643886052 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8ab001c930 con 0x563b6ef2d2f0 2024-09-21T03:54:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.158+0000 7f8ad3040740 10 client.26387 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:54:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.158+0000 7f8ad3040740 10 client.26387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.158+0000 7f8ad3040740 20 client.26387 populate_metadata read hostname 'smithi016' 2024-09-21T03:54:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.158+0000 7f8ad3040740 1 --2- 172.21.15.16:0/2643886052 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x563b6ef14d80 0x563b6f314180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.158+0000 7f8ad3040740 4 client.26387 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.158+0000 7f8ad3040740 1 -- 172.21.15.16:0/2643886052 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b6f272600 con 0x563b6ef14d80 2024-09-21T03:54:15.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.159+0000 7f8ac8c00640 1 --2- 172.21.15.16:0/2643886052 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x563b6ef14d80 0x563b6f314180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.160+0000 7f8aaaa00640 10 client.26387 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:15.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.160+0000 7f8ac8c00640 1 --2- 172.21.15.16:0/2643886052 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x563b6ef14d80 0x563b6f314180 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c65461de827ed0a server_cookie=44a494468f743ce3 in_seq=0 out_seq=0 2024-09-21T03:54:15.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.161+0000 7f8aaaa00640 1 -- 172.21.15.16:0/2643886052 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563b6f272600 con 0x563b6ef14d80 2024-09-21T03:54:15.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.161+0000 7f8aaaa00640 10 client.26387 handle_command_reply: tid=0 2024-09-21T03:54:15.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.161+0000 7f8ad3040740 1 client.26387 shutdown 2024-09-21T03:54:15.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.162+0000 7f8ad3040740 1 -- 172.21.15.16:0/2643886052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 msgr2=0x563b6f306860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.162+0000 7f8ad3040740 1 --2- 172.21.15.16:0/2643886052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 0x563b6f306860 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0009b00 tx=0x7f8ab0013a00 comp rx=0 tx=0).stop 2024-09-21T03:54:15.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.162+0000 7f8ad3040740 1 -- 172.21.15.16:0/2643886052 shutdown_connections 2024-09-21T03:54:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.162+0000 7f8ad3040740 1 --2- 172.21.15.16:0/2643886052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 0x563b6f307a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.162+0000 7f8ad3040740 1 --2- 172.21.15.16:0/2643886052 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x563b6ef14d80 0x563b6f314180 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.163+0000 7f8ad3040740 1 --2- 172.21.15.16:0/2643886052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef354d0 0x563b6f306f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.163+0000 7f8ad3040740 1 --2- 172.21.15.16:0/2643886052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 0x563b6f306860 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.163+0000 7f8ad3040740 1 -- 172.21.15.16:0/2643886052 >> 172.21.15.16:0/2643886052 conn(0x563b6ef4a300 msgr2=0x563b6f2f9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.163+0000 7f8aaaa00640 0 client.26387 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.164+0000 7f8ad3040740 1 -- 172.21.15.16:0/2643886052 shutdown_connections 2024-09-21T03:54:15.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.164+0000 7f8ad3040740 1 -- 172.21.15.16:0/2643886052 wait complete. 2024-09-21T03:54:15.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.165+0000 7f8ad3040740 20 client.26387 trim_cache size 0 max 16384 2024-09-21T03:54:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.218+0000 7f8ad3040740 1 Processor -- start 2024-09-21T03:54:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.218+0000 7f8ad3040740 1 -- start start 2024-09-21T03:54:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.221+0000 7f8ad3040740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 0x563b6f3053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.221+0000 7f8ad3040740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef354d0 0x563b6f2f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.221+0000 7f8ac9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 0x563b6f3053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.221+0000 7f8ac9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 0x563b6f3053e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:45966/0 (socket says 172.21.15.16:45966) 2024-09-21T03:54:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.222+0000 7f8ad3040740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 0x563b6f3011c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.222+0000 7f8ad3040740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41bbbc0 con 0x563b6ef354d0 2024-09-21T03:54:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.222+0000 7f8ad3040740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41bd700 con 0x563b6ef2d2f0 2024-09-21T03:54:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.222+0000 7f8ad3040740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41bc290 con 0x563b6ef561b0 2024-09-21T03:54:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.223+0000 7f8ac8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef354d0 0x563b6f2f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.223+0000 7f8ac8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef354d0 0x563b6f2f8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56108/0 (socket says 172.21.15.16:56108) 2024-09-21T03:54:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.223+0000 7f8ac8c00640 1 -- 172.21.15.16:0/3414369866 learned_addr learned my addr 172.21.15.16:0/3414369866 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.223+0000 7f8aca000640 1 --2- 172.21.15.16:0/3414369866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 0x563b6f3011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.223+0000 7f8ac9600640 1 -- 172.21.15.16:0/3414369866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 msgr2=0x563b6f3011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.224+0000 7f8ac9600640 1 --2- 172.21.15.16:0/3414369866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 0x563b6f3011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.224+0000 7f8ac9600640 1 -- 172.21.15.16:0/3414369866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef354d0 msgr2=0x563b6f2f8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.224+0000 7f8ac9600640 1 --2- 172.21.15.16:0/3414369866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef354d0 0x563b6f2f8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.224+0000 7f8ac9600640 1 -- 172.21.15.16:0/3414369866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b6f2fd750 con 0x563b6ef2d2f0 2024-09-21T03:54:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.224+0000 7f8ac8c00640 1 --2- 172.21.15.16:0/3414369866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef354d0 0x563b6f2f8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.225+0000 7f8ac9600640 1 --2- 172.21.15.16:0/3414369866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 0x563b6f3053e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000afd0 tx=0x7f8ab0013920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5761f3a6623c46 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.227+0000 7f8ab5c00640 1 -- 172.21.15.16:0/3414369866 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0017040 con 0x563b6ef2d2f0 2024-09-21T03:54:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.227+0000 7f8ab5c00640 1 -- 172.21.15.16:0/3414369866 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ab0005e40 con 0x563b6ef2d2f0 2024-09-21T03:54:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.227+0000 7f8ab5c00640 1 -- 172.21.15.16:0/3414369866 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0012be0 con 0x563b6ef2d2f0 2024-09-21T03:54:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.228+0000 7f8ad3040740 1 -- 172.21.15.16:0/3414369866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 msgr2=0x563b6f3053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.228+0000 7f8ad3040740 1 --2- 172.21.15.16:0/3414369866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 0x563b6f3053e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000afd0 tx=0x7f8ab0013920 comp rx=0 tx=0).stop 2024-09-21T03:54:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.231+0000 7f8ad3040740 1 -- 172.21.15.16:0/3414369866 shutdown_connections 2024-09-21T03:54:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.231+0000 7f8ad3040740 1 --2- 172.21.15.16:0/3414369866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 0x563b6f3011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.231+0000 7f8ad3040740 1 --2- 172.21.15.16:0/3414369866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef354d0 0x563b6f2f8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.231+0000 7f8ad3040740 1 --2- 172.21.15.16:0/3414369866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 0x563b6f3053e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.231+0000 7f8ad3040740 1 -- 172.21.15.16:0/3414369866 >> 172.21.15.16:0/3414369866 conn(0x563b6ef4a300 msgr2=0x563b6f2f84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.232+0000 7f8ad3040740 1 -- 172.21.15.16:0/3414369866 shutdown_connections 2024-09-21T03:54:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.232+0000 7f8ad3040740 1 -- 172.21.15.16:0/3414369866 wait complete. 2024-09-21T03:54:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.234+0000 7f8ad3040740 1 -- start start 2024-09-21T03:54:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.234+0000 7f8ad3040740 1 Processor -- start 2024-09-21T03:54:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.236+0000 7f8ad3040740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 0x563b6f30d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.236+0000 7f8ad3040740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef354d0 0x563b6f30d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.236+0000 7f8ad3040740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 0x563b6f37ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.236+0000 7f8ad3040740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41bd030 con 0x563b6ef354d0 2024-09-21T03:54:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.236+0000 7f8ad3040740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41bc960 con 0x563b6ef2d2f0 2024-09-21T03:54:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.236+0000 7f8ad3040740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41b9c50 con 0x563b6ef561b0 2024-09-21T03:54:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.236+0000 7f8ad3040740 10 client.0 _ll_register_callbacks cb 0x7f8ab0009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.236+0000 7f8ac9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 0x563b6f30d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.237+0000 7f8ac9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 0x563b6f30d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:45978/0 (socket says 172.21.15.16:45978) 2024-09-21T03:54:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.237+0000 7f8ac9600640 1 -- 172.21.15.16:0/3488970642 learned_addr learned my addr 172.21.15.16:0/3488970642 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.237+0000 7f8aca000640 1 --2- 172.21.15.16:0/3488970642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 0x563b6f37ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.237+0000 7f8ac8c00640 1 --2- 172.21.15.16:0/3488970642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef354d0 0x563b6f30d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.237+0000 7f8aca000640 1 -- 172.21.15.16:0/3488970642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 msgr2=0x563b6f30d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.237+0000 7f8aca000640 1 --2- 172.21.15.16:0/3488970642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 0x563b6f30d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.237+0000 7f8aca000640 1 -- 172.21.15.16:0/3488970642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef354d0 msgr2=0x563b6f30d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.237+0000 7f8aca000640 1 --2- 172.21.15.16:0/3488970642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef354d0 0x563b6f30d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.237+0000 7f8aca000640 1 -- 172.21.15.16:0/3488970642 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b6f37cf50 con 0x563b6ef561b0 2024-09-21T03:54:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.238+0000 7f8aca000640 1 --2- 172.21.15.16:0/3488970642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 0x563b6f37ca10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00bdd0 tx=0x7f8abc0ac5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ed4d8ee948acc6e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.238+0000 7f8aaaa00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:54:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.238+0000 7f8aaaa00640 1 -- 172.21.15.16:0/3488970642 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc014640 con 0x563b6ef561b0 2024-09-21T03:54:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.238+0000 7f8aaaa00640 1 -- 172.21.15.16:0/3488970642 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f8abc0abdf0 con 0x563b6ef561b0 2024-09-21T03:54:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.238+0000 7f8ad3040740 1 -- 172.21.15.16:0/3488970642 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563b6f26e0b0 con 0x563b6ef561b0 2024-09-21T03:54:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.238+0000 7f8aaaa00640 1 -- 172.21.15.16:0/3488970642 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8abc0c7050 con 0x563b6ef561b0 2024-09-21T03:54:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.238+0000 7f8ad3040740 10 client.26335 fetch_fsmap learned FSMap version 30 2024-09-21T03:54:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.238+0000 7f8ad3040740 1 -- 172.21.15.16:0/3488970642 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563b6f37e150 con 0x563b6ef561b0 2024-09-21T03:54:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.239+0000 7f8aaaa00640 1 -- 172.21.15.16:0/3488970642 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8abc0ca080 con 0x563b6ef561b0 2024-09-21T03:54:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.239+0000 7f8aaaa00640 1 -- 172.21.15.16:0/3488970642 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8abc0ceaf0 con 0x563b6ef561b0 2024-09-21T03:54:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.239+0000 7f8ad3040740 10 client.26335 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:54:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.239+0000 7f8ad3040740 10 client.26335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.239+0000 7f8ad3040740 20 client.26335 populate_metadata read hostname 'smithi016' 2024-09-21T03:54:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.239+0000 7f8ad3040740 1 --2- 172.21.15.16:0/3488970642 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x563b6eeea1c0 0x563b6f380b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.239+0000 7f8ad3040740 4 client.26335 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:54:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.239+0000 7f8ad3040740 1 -- 172.21.15.16:0/3488970642 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b6f26b4e0 con 0x563b6eeea1c0 2024-09-21T03:54:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.239+0000 7f8ac9600640 1 --2- 172.21.15.16:0/3488970642 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x563b6eeea1c0 0x563b6f380b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.240+0000 7f8ac9600640 1 --2- 172.21.15.16:0/3488970642 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x563b6eeea1c0 0x563b6f380b40 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64c23e5b8610888c server_cookie=e6d3d6a1a0146e17 in_seq=0 out_seq=0 2024-09-21T03:54:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.240+0000 7f8aaaa00640 10 client.26335 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.271+0000 7f8aaaa00640 1 -- 172.21.15.16:0/3488970642 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563b6f26b4e0 con 0x563b6eeea1c0 2024-09-21T03:54:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.271+0000 7f8aaaa00640 10 client.26335 handle_command_reply: tid=0 2024-09-21T03:54:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.271+0000 7f8ad3040740 1 client.26335 shutdown 2024-09-21T03:54:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.271+0000 7f8ad3040740 1 -- 172.21.15.16:0/3488970642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 msgr2=0x563b6f37ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.271+0000 7f8ad3040740 1 --2- 172.21.15.16:0/3488970642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 0x563b6f37ca10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00bdd0 tx=0x7f8abc0ac5a0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.272+0000 7f8ad3040740 1 -- 172.21.15.16:0/3488970642 shutdown_connections 2024-09-21T03:54:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.272+0000 7f8ad3040740 1 --2- 172.21.15.16:0/3488970642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b6ef561b0 0x563b6f37ca10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.272+0000 7f8ad3040740 1 --2- 172.21.15.16:0/3488970642 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x563b6eeea1c0 0x563b6f380b40 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.272+0000 7f8ad3040740 1 --2- 172.21.15.16:0/3488970642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6ef354d0 0x563b6f30d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.272+0000 7f8ad3040740 1 --2- 172.21.15.16:0/3488970642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6ef2d2f0 0x563b6f30d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.272+0000 7f8ad3040740 1 -- 172.21.15.16:0/3488970642 >> 172.21.15.16:0/3488970642 conn(0x563b6ef4a300 msgr2=0x563b6f3011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.272+0000 7f8aaaa00640 0 client.26335 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.272+0000 7f8ad3040740 1 -- 172.21.15.16:0/3488970642 shutdown_connections 2024-09-21T03:54:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.272+0000 7f8ad3040740 1 -- 172.21.15.16:0/3488970642 wait complete. 2024-09-21T03:54:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.273+0000 7f8ad3040740 20 client.26335 trim_cache size 0 max 16384 2024-09-21T03:54:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:54:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:54:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:54:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:54:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.273+0000 7f8acc000640 1 -- 172.21.15.16:0/1545129444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8aac0b6650 msgr2=0x7f8aac0b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.273+0000 7f8acc000640 1 --2- 172.21.15.16:0/1545129444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8aac0b6650 0x7f8aac0b8b60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8009850 tx=0x7f8ab8031400 comp rx=0 tx=0).stop 2024-09-21T03:54:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.273+0000 7f8acc000640 1 -- 172.21.15.16:0/1545129444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac40736d0 msgr2=0x7f8ac4073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.273+0000 7f8acc000640 1 --2- 172.21.15.16:0/1545129444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac40736d0 0x7f8ac4073bd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00be00 tx=0x7f8abc00bef0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.274+0000 7f8acc000640 1 -- 172.21.15.16:0/1545129444 shutdown_connections 2024-09-21T03:54:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.274+0000 7f8acc000640 1 --2- 172.21.15.16:0/1545129444 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8aac0b6650 0x7f8aac0b8b60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.274+0000 7f8acc000640 1 --2- 172.21.15.16:0/1545129444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac40736d0 0x7f8ac4073bd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.274+0000 7f8acc000640 1 --2- 172.21.15.16:0/1545129444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ac41071d0 0x7f8ac4075040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.274+0000 7f8acc000640 1 --2- 172.21.15.16:0/1545129444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ac406b710 0x7f8ac40749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.274+0000 7f8acc000640 1 -- 172.21.15.16:0/1545129444 >> 172.21.15.16:0/1545129444 conn(0x7f8ac40fdee0 msgr2=0x7f8ac4105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.275+0000 7f8acc000640 1 -- 172.21.15.16:0/1545129444 shutdown_connections 2024-09-21T03:54:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:15.276+0000 7f8acc000640 1 -- 172.21.15.16:0/1545129444 wait complete. 2024-09-21T03:54:15.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:14 smithi134 ceph-mon[35031]: pgmap v656: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 235 MiB/s wr, 1.14k op/s 2024-09-21T03:54:15.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:14 smithi123 ceph-mon[34673]: pgmap v656: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 235 MiB/s wr, 1.14k op/s 2024-09-21T03:54:15.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:14 smithi016 ceph-mon[30375]: pgmap v656: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 235 MiB/s wr, 1.14k op/s 2024-09-21T03:54:17.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:16 smithi134 ceph-mon[35031]: pgmap v657: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 254 MiB/s wr, 1.23k op/s 2024-09-21T03:54:17.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:16 smithi123 ceph-mon[34673]: pgmap v657: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 254 MiB/s wr, 1.23k op/s 2024-09-21T03:54:17.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:16 smithi016 ceph-mon[30375]: pgmap v657: 129 pgs: 129 active+clean; 10 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 254 MiB/s wr, 1.23k op/s 2024-09-21T03:54:19.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:18 smithi134 ceph-mon[35031]: pgmap v658: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 240 MiB/s wr, 1.16k op/s 2024-09-21T03:54:19.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:18 smithi123 ceph-mon[34673]: pgmap v658: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 240 MiB/s wr, 1.16k op/s 2024-09-21T03:54:19.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:18 smithi016 ceph-mon[30375]: pgmap v658: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 240 MiB/s wr, 1.16k op/s 2024-09-21T03:54:20.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:20 smithi134 ceph-mon[35031]: pgmap v659: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 227 MiB/s wr, 1.09k op/s 2024-09-21T03:54:20.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:20 smithi123 ceph-mon[34673]: pgmap v659: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 227 MiB/s wr, 1.09k op/s 2024-09-21T03:54:20.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:20 smithi016 ceph-mon[30375]: pgmap v659: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 227 MiB/s wr, 1.09k op/s 2024-09-21T03:54:23.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:22 smithi134 ceph-mon[35031]: pgmap v660: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 243 MiB/s wr, 1.17k op/s 2024-09-21T03:54:23.108 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:22 smithi123 ceph-mon[34673]: pgmap v660: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 243 MiB/s wr, 1.17k op/s 2024-09-21T03:54:23.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:22 smithi016 ceph-mon[30375]: pgmap v660: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 243 MiB/s wr, 1.17k op/s 2024-09-21T03:54:23.321 INFO:tasks.quiescer.fs.[cephfs]:Total duration (42.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T03:54:23.321 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.05 2024-09-21T03:54:23.321 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:54:24.557 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:25.040 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:24 smithi016 ceph-mon[30375]: pgmap v661: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 233 MiB/s wr, 1.12k op/s 2024-09-21T03:54:25.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:24 smithi134 ceph-mon[35031]: pgmap v661: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 233 MiB/s wr, 1.12k op/s 2024-09-21T03:54:25.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:24 smithi123 ceph-mon[34673]: pgmap v661: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 233 MiB/s wr, 1.12k op/s 2024-09-21T03:54:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.808+0000 7f28d1800640 1 -- 172.21.15.16:0/2630457169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28cc104a30 msgr2=0x7f28cc104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.808+0000 7f28d1800640 1 --2- 172.21.15.16:0/2630457169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28cc104a30 0x7f28cc104e40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009ca0 tx=0x7f28bc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.808+0000 7f28d1800640 1 -- 172.21.15.16:0/2630457169 shutdown_connections 2024-09-21T03:54:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.808+0000 7f28d1800640 1 --2- 172.21.15.16:0/2630457169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc10de90 0x7f28cc110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.808+0000 7f28d1800640 1 --2- 172.21.15.16:0/2630457169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28cc105380 0x7f28cc10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.808+0000 7f28d1800640 1 --2- 172.21.15.16:0/2630457169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28cc104a30 0x7f28cc104e40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.808+0000 7f28d1800640 1 -- 172.21.15.16:0/2630457169 >> 172.21.15.16:0/2630457169 conn(0x7f28cc0796b0 msgr2=0x7f28cc079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:26.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.810+0000 7f28d1800640 1 -- 172.21.15.16:0/2630457169 shutdown_connections 2024-09-21T03:54:26.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.810+0000 7f28d1800640 1 -- 172.21.15.16:0/2630457169 wait complete. 2024-09-21T03:54:26.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.816+0000 7f28d1800640 1 Processor -- start 2024-09-21T03:54:26.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.817+0000 7f28d1800640 1 -- start start 2024-09-21T03:54:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.817+0000 7f28d1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc104a30 0x7f28cc1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.817+0000 7f28d1800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28cc105380 0x7f28cc1a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.817+0000 7f28d1800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28cc10de90 0x7f28cc1ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.817+0000 7f28d1800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc113b40 con 0x7f28cc104a30 2024-09-21T03:54:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.817+0000 7f28d1800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc1139c0 con 0x7f28cc105380 2024-09-21T03:54:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.817+0000 7f28d1800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc113cc0 con 0x7f28cc10de90 2024-09-21T03:54:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.818+0000 7f28ca000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28cc105380 0x7f28cc1a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.818+0000 7f28ca000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28cc105380 0x7f28cc1a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:32806/0 (socket says 172.21.15.16:32806) 2024-09-21T03:54:26.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.818+0000 7f28ca000640 1 -- 172.21.15.16:0/3436884830 learned_addr learned my addr 172.21.15.16:0/3436884830 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:26.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.818+0000 7f28cb400640 1 --2- 172.21.15.16:0/3436884830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28cc10de90 0x7f28cc1ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.819+0000 7f28ca000640 1 -- 172.21.15.16:0/3436884830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28cc10de90 msgr2=0x7f28cc1ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:26.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.819+0000 7f28ca000640 1 --2- 172.21.15.16:0/3436884830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28cc10de90 0x7f28cc1ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.819+0000 7f28ca000640 1 -- 172.21.15.16:0/3436884830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc104a30 msgr2=0x7f28cc1a1820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:54:26.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.819+0000 7f28caa00640 1 --2- 172.21.15.16:0/3436884830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc104a30 0x7f28cc1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.819+0000 7f28ca000640 1 --2- 172.21.15.16:0/3436884830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc104a30 0x7f28cc1a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:26.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.820+0000 7f28ca000640 1 -- 172.21.15.16:0/3436884830 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc1a25e0 con 0x7f28cc105380 2024-09-21T03:54:26.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.820+0000 7f28caa00640 1 --2- 172.21.15.16:0/3436884830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc104a30 0x7f28cc1a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:26.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.820+0000 7f28ca000640 1 --2- 172.21.15.16:0/3436884830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28cc105380 0x7f28cc1a1d60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f28b400e9f0 tx=0x7f28b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:26.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.821+0000 7f28c3400640 1 -- 172.21.15.16:0/3436884830 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b400ccc0 con 0x7f28cc105380 2024-09-21T03:54:26.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.821+0000 7f28c3400640 1 -- 172.21.15.16:0/3436884830 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b40108e0 con 0x7f28cc105380 2024-09-21T03:54:26.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.821+0000 7f28c3400640 1 -- 172.21.15.16:0/3436884830 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4018860 con 0x7f28cc105380 2024-09-21T03:54:26.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.822+0000 7f28d1800640 1 -- 172.21.15.16:0/3436884830 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28cc1acea0 con 0x7f28cc105380 2024-09-21T03:54:26.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.822+0000 7f28d1800640 1 -- 172.21.15.16:0/3436884830 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28cc1ad300 con 0x7f28cc105380 2024-09-21T03:54:26.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.824+0000 7f28d1800640 1 -- 172.21.15.16:0/3436884830 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28cc06c240 con 0x7f28cc105380 2024-09-21T03:54:26.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.830+0000 7f28c3400640 1 -- 172.21.15.16:0/3436884830 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f28b4010430 con 0x7f28cc105380 2024-09-21T03:54:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.837+0000 7f28c3400640 1 --2- 172.21.15.16:0/3436884830 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f28b00b6660 0x7f28b00b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.837+0000 7f28c3400640 1 -- 172.21.15.16:0/3436884830 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f28b40e7f20 con 0x7f28cc105380 2024-09-21T03:54:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.838+0000 7f28c3400640 1 -- 172.21.15.16:0/3436884830 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f28b40e8420 con 0x7f28cc105380 2024-09-21T03:54:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.838+0000 7f28caa00640 1 --2- 172.21.15.16:0/3436884830 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f28b00b6660 0x7f28b00b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:26.840+0000 7f28caa00640 1 --2- 172.21.15.16:0/3436884830 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f28b00b6660 0x7f28b00b8b70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f28bc006010 tx=0x7f28bc031f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:27.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:26 smithi134 ceph-mon[35031]: pgmap v662: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 263 MiB/s wr, 1.26k op/s 2024-09-21T03:54:27.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:26 smithi123 ceph-mon[34673]: pgmap v662: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 263 MiB/s wr, 1.26k op/s 2024-09-21T03:54:27.169 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:26 smithi016 ceph-mon[30375]: pgmap v662: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 263 MiB/s wr, 1.26k op/s 2024-09-21T03:54:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:27.393+0000 7f28d1800640 1 -- 172.21.15.16:0/3436884830 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28cc10d230 con 0x7f28cc105380 2024-09-21T03:54:27.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:27.394+0000 7f28c3400640 1 -- 172.21.15.16:0/3436884830 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f28b40ae480 con 0x7f28cc105380 2024-09-21T03:54:27.396 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:54:27.397 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:54:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:27.397+0000 7f28d1800640 1 -- 172.21.15.16:0/3436884830 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f28b00b6660 msgr2=0x7f28b00b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:27.397+0000 7f28d1800640 1 --2- 172.21.15.16:0/3436884830 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f28b00b6660 0x7f28b00b8b70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f28bc006010 tx=0x7f28bc031f90 comp rx=0 tx=0).stop 2024-09-21T03:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:27.397+0000 7f28d1800640 1 -- 172.21.15.16:0/3436884830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28cc105380 msgr2=0x7f28cc1a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:27.397+0000 7f28d1800640 1 --2- 172.21.15.16:0/3436884830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28cc105380 0x7f28cc1a1d60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f28b400e9f0 tx=0x7f28b400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:27.397+0000 7f28d1800640 1 -- 172.21.15.16:0/3436884830 shutdown_connections 2024-09-21T03:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:27.397+0000 7f28d1800640 1 --2- 172.21.15.16:0/3436884830 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f28b00b6660 0x7f28b00b8b70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:27.397+0000 7f28d1800640 1 --2- 172.21.15.16:0/3436884830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28cc10de90 0x7f28cc1ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:27.397+0000 7f28d1800640 1 --2- 172.21.15.16:0/3436884830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28cc105380 0x7f28cc1a1d60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:27.397+0000 7f28d1800640 1 --2- 172.21.15.16:0/3436884830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc104a30 0x7f28cc1a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:27.397+0000 7f28d1800640 1 -- 172.21.15.16:0/3436884830 >> 172.21.15.16:0/3436884830 conn(0x7f28cc0796b0 msgr2=0x7f28cc102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:27.397+0000 7f28d1800640 1 -- 172.21.15.16:0/3436884830 shutdown_connections 2024-09-21T03:54:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:27.397+0000 7f28d1800640 1 -- 172.21.15.16:0/3436884830 wait complete. 2024-09-21T03:54:28.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:27 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3436884830' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:54:28.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:27 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3436884830' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:54:28.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:27 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3436884830' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:54:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.671+0000 7f11d0800640 1 -- 172.21.15.16:0/117582184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c81050e0 msgr2=0x7f11c810ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.671+0000 7f11d0800640 1 --2- 172.21.15.16:0/117582184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c81050e0 0x7f11c810ba80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009900 tx=0x7f11b802a400 comp rx=0 tx=0).stop 2024-09-21T03:54:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.671+0000 7f11d0800640 1 -- 172.21.15.16:0/117582184 shutdown_connections 2024-09-21T03:54:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.671+0000 7f11d0800640 1 --2- 172.21.15.16:0/117582184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c81050e0 0x7f11c810ba80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.671+0000 7f11d0800640 1 --2- 172.21.15.16:0/117582184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11c8104740 0x7f11c8104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.671+0000 7f11d0800640 1 --2- 172.21.15.16:0/117582184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c81036c0 0x7f11c8103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.671+0000 7f11d0800640 1 -- 172.21.15.16:0/117582184 >> 172.21.15.16:0/117582184 conn(0x7f11c80ff020 msgr2=0x7f11c81014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.671+0000 7f11d0800640 1 -- 172.21.15.16:0/117582184 shutdown_connections 2024-09-21T03:54:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.671+0000 7f11d0800640 1 -- 172.21.15.16:0/117582184 wait complete. 2024-09-21T03:54:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.673+0000 7f11d0800640 1 Processor -- start 2024-09-21T03:54:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.673+0000 7f11d0800640 1 -- start start 2024-09-21T03:54:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.673+0000 7f11d0800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11c81036c0 0x7f11c81a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.673+0000 7f11d0800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c8104740 0x7f11c81a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.674+0000 7f11d0800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c81050e0 0x7f11c81a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.674+0000 7f11d0800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11c810f340 con 0x7f11c8104740 2024-09-21T03:54:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.674+0000 7f11d0800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f11c810f1c0 con 0x7f11c81036c0 2024-09-21T03:54:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.674+0000 7f11d0800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f11c810f4c0 con 0x7f11c81050e0 2024-09-21T03:54:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.674+0000 7f11cde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11c81036c0 0x7f11c81a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.674+0000 7f11cde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11c81036c0 0x7f11c81a0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:32820/0 (socket says 172.21.15.16:32820) 2024-09-21T03:54:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.674+0000 7f11cde00640 1 -- 172.21.15.16:0/1056741807 learned_addr learned my addr 172.21.15.16:0/1056741807 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.674+0000 7f11ce800640 1 --2- 172.21.15.16:0/1056741807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c81050e0 0x7f11c81a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.674+0000 7f11cd400640 1 --2- 172.21.15.16:0/1056741807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c8104740 0x7f11c81a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.674+0000 7f11cde00640 1 -- 172.21.15.16:0/1056741807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c81050e0 msgr2=0x7f11c81a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.674+0000 7f11cde00640 1 --2- 172.21.15.16:0/1056741807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c81050e0 0x7f11c81a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.674+0000 7f11cde00640 1 -- 172.21.15.16:0/1056741807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c8104740 msgr2=0x7f11c81a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.674+0000 7f11cde00640 1 --2- 172.21.15.16:0/1056741807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c8104740 0x7f11c81a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.674+0000 7f11cde00640 1 -- 172.21.15.16:0/1056741807 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c81a1ee0 con 0x7f11c81036c0 2024-09-21T03:54:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.674+0000 7f11cd400640 1 --2- 172.21.15.16:0/1056741807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c8104740 0x7f11c81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.675+0000 7f11cde00640 1 --2- 172.21.15.16:0/1056741807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11c81036c0 0x7f11c81a0510 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f11c400cce0 tx=0x7f11c400a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.675+0000 7f11c2a00640 1 -- 172.21.15.16:0/1056741807 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4004b20 con 0x7f11c81036c0 2024-09-21T03:54:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.675+0000 7f11c2a00640 1 -- 172.21.15.16:0/1056741807 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c4004cc0 con 0x7f11c81036c0 2024-09-21T03:54:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.675+0000 7f11c2a00640 1 -- 172.21.15.16:0/1056741807 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c400f6b0 con 0x7f11c81036c0 2024-09-21T03:54:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.675+0000 7f11d0800640 1 -- 172.21.15.16:0/1056741807 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c81a21d0 con 0x7f11c81036c0 2024-09-21T03:54:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.675+0000 7f11d0800640 1 -- 172.21.15.16:0/1056741807 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c81a88c0 con 0x7f11c81036c0 2024-09-21T03:54:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.679+0000 7f11d7843740 1 Processor -- start 2024-09-21T03:54:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.679+0000 7f11d7843740 1 -- start start 2024-09-21T03:54:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.680+0000 7f11d7843740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd1360d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.680+0000 7f11d7843740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd662f0 0x55c1dd136740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.680+0000 7f11d7843740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd137290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.681+0000 7f11d7843740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c1dd0a96b0 con 0x55c1dcd8f1b0 2024-09-21T03:54:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.681+0000 7f11d7843740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c1dd098ae0 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.681+0000 7f11d7843740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c1dd09c8f0 con 0x55c1dcd662f0 2024-09-21T03:54:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.681+0000 7f11ce800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd1360d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.681+0000 7f11cd400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd137290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.681+0000 7f11ce800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd1360d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36744/0 (socket says 172.21.15.16:36744) 2024-09-21T03:54:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.681+0000 7f11ce800640 1 -- 172.21.15.16:0/1717655713 learned_addr learned my addr 172.21.15.16:0/1717655713 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.681+0000 7f11cd400640 1 -- 172.21.15.16:0/1717655713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd662f0 msgr2=0x55c1dd136740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:54:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.681+0000 7f11cd400640 1 --2- 172.21.15.16:0/1717655713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd662f0 0x55c1dd136740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.681+0000 7f11cd400640 1 -- 172.21.15.16:0/1717655713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd8f1b0 msgr2=0x55c1dd1360d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.681+0000 7f11cd400640 1 --2- 172.21.15.16:0/1717655713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd1360d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.681+0000 7f11cd400640 1 -- 172.21.15.16:0/1717655713 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1dd065780 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.682+0000 7f11c2a00640 1 -- 172.21.15.16:0/1056741807 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f11c400f850 con 0x7f11c81036c0 2024-09-21T03:54:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.682+0000 7f11cd400640 1 --2- 172.21.15.16:0/1717655713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd137290 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00bc80 tx=0x7f11bc00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0c487e08a59da7e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.682+0000 7f11b7000640 1 -- 172.21.15.16:0/1717655713 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc015070 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.682+0000 7f11b7000640 1 -- 172.21.15.16:0/1717655713 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11bc01a910 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.682+0000 7f11b7000640 1 -- 172.21.15.16:0/1717655713 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc011900 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.683+0000 7f11d7843740 1 -- 172.21.15.16:0/1717655713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd6e4d0 msgr2=0x55c1dd137290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.683+0000 7f11d7843740 1 --2- 172.21.15.16:0/1717655713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd137290 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00bc80 tx=0x7f11bc00f730 comp rx=0 tx=0).stop 2024-09-21T03:54:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.683+0000 7f11d7843740 1 -- 172.21.15.16:0/1717655713 shutdown_connections 2024-09-21T03:54:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.683+0000 7f11d7843740 1 --2- 172.21.15.16:0/1717655713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd137290 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.683+0000 7f11d7843740 1 --2- 172.21.15.16:0/1717655713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd662f0 0x55c1dd136740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.683+0000 7f11d7843740 1 --2- 172.21.15.16:0/1717655713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd1360d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.683+0000 7f11d7843740 1 -- 172.21.15.16:0/1717655713 >> 172.21.15.16:0/1717655713 conn(0x55c1dcd83300 msgr2=0x55c1dd131b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.684+0000 7f11d7843740 1 -- 172.21.15.16:0/1717655713 shutdown_connections 2024-09-21T03:54:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.684+0000 7f11d7843740 1 -- 172.21.15.16:0/1717655713 wait complete. 2024-09-21T03:54:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.684+0000 7f11c2a00640 1 --2- 172.21.15.16:0/1056741807 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f11b00b64a0 0x7f11b00b89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.685+0000 7f11d7843740 1 -- start start 2024-09-21T03:54:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.685+0000 7f11c2a00640 1 -- 172.21.15.16:0/1056741807 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f11c40e77b0 con 0x7f11c81036c0 2024-09-21T03:54:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.685+0000 7f11d7843740 1 Processor -- start 2024-09-21T03:54:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.685+0000 7f11ce800640 1 --2- 172.21.15.16:0/1056741807 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f11b00b64a0 0x7f11b00b89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:28.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.687+0000 7f11ce800640 1 --2- 172.21.15.16:0/1056741807 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f11b00b64a0 0x7f11b00b89b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009230 tx=0x7f11b80314c0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:28.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.687+0000 7f11d7843740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd662f0 0x55c1dd13f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.688+0000 7f11d7843740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd13ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.688+0000 7f11d7843740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd140a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.688+0000 7f11d7843740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c1dcfd0ab0 con 0x55c1dcd662f0 2024-09-21T03:54:28.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.688+0000 7f11d7843740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c1dd03f570 con 0x55c1dcd8f1b0 2024-09-21T03:54:28.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.688+0000 7f11d7843740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c1dd137db0 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.688+0000 7f11d7843740 10 client.0 _ll_register_callbacks cb 0x55c1dd04e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:28.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.688+0000 7f11ce800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:28.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.688+0000 7f11cde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd662f0 0x55c1dd13f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.688+0000 7f11cd400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd13ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.688+0000 7f11cd400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd13ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:60042/0 (socket says 172.21.15.16:60042) 2024-09-21T03:54:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.688+0000 7f11ce800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd140a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:32834/0 (socket says 172.21.15.16:32834) 2024-09-21T03:54:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.688+0000 7f11cde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd662f0 0x55c1dd13f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36754/0 (socket says 172.21.15.16:36754) 2024-09-21T03:54:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.688+0000 7f11cd400640 1 -- 172.21.15.16:0/1997790900 learned_addr learned my addr 172.21.15.16:0/1997790900 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.689+0000 7f11cd400640 1 -- 172.21.15.16:0/1997790900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd8f1b0 msgr2=0x55c1dd140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.689+0000 7f11cd400640 1 --2- 172.21.15.16:0/1997790900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd140a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.689+0000 7f11cd400640 1 -- 172.21.15.16:0/1997790900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd662f0 msgr2=0x55c1dd13f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.689+0000 7f11cd400640 1 --2- 172.21.15.16:0/1997790900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd662f0 0x55c1dd13f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.689+0000 7f11cd400640 1 -- 172.21.15.16:0/1997790900 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1dcdfa790 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.689+0000 7f11ce800640 1 --2- 172.21.15.16:0/1997790900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd140a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:54:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.690+0000 7f11cd400640 1 --2- 172.21.15.16:0/1997790900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd13ff00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f11bc005890 tx=0x7f11bc00bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5d159384f2ecd96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.690+0000 7f11ab400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:54:28.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.690+0000 7f11ab400640 1 -- 172.21.15.16:0/1997790900 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc015050 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.690+0000 7f11ab400640 1 -- 172.21.15.16:0/1997790900 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f11bc01acc0 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.691+0000 7f11d7843740 1 -- 172.21.15.16:0/1997790900 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11bc01a910 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.691+0000 7f11ab400640 1 -- 172.21.15.16:0/1997790900 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f11bc004030 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.691+0000 7f11d7843740 10 client.26350 fetch_fsmap learned FSMap version 30 2024-09-21T03:54:28.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.691+0000 7f11d7843740 1 -- 172.21.15.16:0/1997790900 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c1dce73620 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.692+0000 7f11ab400640 1 -- 172.21.15.16:0/1997790900 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11bc01a910 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.692+0000 7f11ab400640 1 -- 172.21.15.16:0/1997790900 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f11bc023070 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.692+0000 7f11d7843740 10 client.26350 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:54:28.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.692+0000 7f11d7843740 10 client.26350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:28.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.692+0000 7f11d7843740 20 client.26350 populate_metadata read hostname 'smithi016' 2024-09-21T03:54:28.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.692+0000 7f11d7843740 1 --2- 172.21.15.16:0/1997790900 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c1dcd4dd80 0x55c1dd14cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.692+0000 7f11d7843740 4 client.26350 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:28.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.692+0000 7f11d7843740 1 -- 172.21.15.16:0/1997790900 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1dd0ab600 con 0x55c1dcd4dd80 2024-09-21T03:54:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.693+0000 7f11cde00640 1 --2- 172.21.15.16:0/1997790900 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c1dcd4dd80 0x55c1dd14cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.693+0000 7f11cde00640 1 --2- 172.21.15.16:0/1997790900 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c1dcd4dd80 0x55c1dd14cbf0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e78e9ea2ea6fe32 server_cookie=3d94431cab33ffd9 in_seq=0 out_seq=0 2024-09-21T03:54:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.693+0000 7f11ab400640 10 client.26350 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.694+0000 7f11ab400640 1 -- 172.21.15.16:0/1997790900 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c1dd0ab600 con 0x55c1dcd4dd80 2024-09-21T03:54:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.694+0000 7f11ab400640 10 client.26350 handle_command_reply: tid=0 2024-09-21T03:54:28.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.694+0000 7f11d7843740 1 client.26350 shutdown 2024-09-21T03:54:28.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.694+0000 7f11d7843740 1 -- 172.21.15.16:0/1997790900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd6e4d0 msgr2=0x55c1dd13ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:28.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.694+0000 7f11d7843740 1 --2- 172.21.15.16:0/1997790900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd13ff00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f11bc005890 tx=0x7f11bc00bfa0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.695+0000 7f11d7843740 1 -- 172.21.15.16:0/1997790900 shutdown_connections 2024-09-21T03:54:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.695+0000 7f11d7843740 1 --2- 172.21.15.16:0/1997790900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd140a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.695+0000 7f11d7843740 1 --2- 172.21.15.16:0/1997790900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd13ff00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.695+0000 7f11d7843740 1 --2- 172.21.15.16:0/1997790900 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c1dcd4dd80 0x55c1dd14cbf0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.695+0000 7f11ab400640 0 client.26350 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.695+0000 7f11d7843740 1 --2- 172.21.15.16:0/1997790900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd662f0 0x55c1dd13f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.695+0000 7f11d7843740 1 -- 172.21.15.16:0/1997790900 >> 172.21.15.16:0/1997790900 conn(0x55c1dcd83300 msgr2=0x55c1dd1300e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.695+0000 7f11d7843740 1 -- 172.21.15.16:0/1997790900 shutdown_connections 2024-09-21T03:54:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.696+0000 7f11d7843740 1 -- 172.21.15.16:0/1997790900 wait complete. 2024-09-21T03:54:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.696+0000 7f11d7843740 20 client.26350 trim_cache size 0 max 16384 2024-09-21T03:54:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.742+0000 7f11d7843740 1 Processor -- start 2024-09-21T03:54:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.743+0000 7f11d7843740 1 -- start start 2024-09-21T03:54:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.743+0000 7f11d7843740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd662f0 0x55c1dd1314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.743+0000 7f11d7843740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd13f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.744+0000 7f11d7843740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd13fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.744+0000 7f11d7843740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81b2d70 con 0x55c1dcd662f0 2024-09-21T03:54:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.744+0000 7f11d7843740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81b48b0 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.744+0000 7f11d7843740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81b3440 con 0x55c1dcd8f1b0 2024-09-21T03:54:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.745+0000 7f11cd400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd13f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.745+0000 7f11cd400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd13f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:32836/0 (socket says 172.21.15.16:32836) 2024-09-21T03:54:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.745+0000 7f11cd400640 1 -- 172.21.15.16:0/4020857135 learned_addr learned my addr 172.21.15.16:0/4020857135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.745+0000 7f11ce800640 1 --2- 172.21.15.16:0/4020857135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd13fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.747+0000 7f11cde00640 1 --2- 172.21.15.16:0/4020857135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd662f0 0x55c1dd1314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.750+0000 7f11ce800640 1 -- 172.21.15.16:0/4020857135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd6e4d0 msgr2=0x55c1dd13f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.750+0000 7f11ce800640 1 --2- 172.21.15.16:0/4020857135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd13f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.750+0000 7f11ce800640 1 -- 172.21.15.16:0/4020857135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd662f0 msgr2=0x55c1dd1314a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.750+0000 7f11ce800640 1 --2- 172.21.15.16:0/4020857135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd662f0 0x55c1dd1314a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.750+0000 7f11ce800640 1 -- 172.21.15.16:0/4020857135 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1dd140d70 con 0x55c1dcd8f1b0 2024-09-21T03:54:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.751+0000 7f11ce800640 1 --2- 172.21.15.16:0/4020857135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd13fd80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f11b802a920 tx=0x7f11b8040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=459338eef2b2100d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.754+0000 7f11b6600640 1 -- 172.21.15.16:0/4020857135 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8035040 con 0x55c1dcd8f1b0 2024-09-21T03:54:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.755+0000 7f11b6600640 1 -- 172.21.15.16:0/4020857135 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11b8031a50 con 0x55c1dcd8f1b0 2024-09-21T03:54:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.755+0000 7f11b6600640 1 -- 172.21.15.16:0/4020857135 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b80528b0 con 0x55c1dcd8f1b0 2024-09-21T03:54:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.757+0000 7f11d7843740 1 -- 172.21.15.16:0/4020857135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd8f1b0 msgr2=0x55c1dd13fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.758+0000 7f11d7843740 1 --2- 172.21.15.16:0/4020857135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd13fd80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f11b802a920 tx=0x7f11b8040000 comp rx=0 tx=0).stop 2024-09-21T03:54:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.759+0000 7f11d7843740 1 -- 172.21.15.16:0/4020857135 shutdown_connections 2024-09-21T03:54:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.759+0000 7f11d7843740 1 --2- 172.21.15.16:0/4020857135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd13fd80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.759+0000 7f11d7843740 1 --2- 172.21.15.16:0/4020857135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd13f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.759+0000 7f11d7843740 1 --2- 172.21.15.16:0/4020857135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd662f0 0x55c1dd1314a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.760+0000 7f11d7843740 1 -- 172.21.15.16:0/4020857135 >> 172.21.15.16:0/4020857135 conn(0x55c1dcd83300 msgr2=0x55c1dd13ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.760+0000 7f11d7843740 1 -- 172.21.15.16:0/4020857135 shutdown_connections 2024-09-21T03:54:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.760+0000 7f11d7843740 1 -- 172.21.15.16:0/4020857135 wait complete. 2024-09-21T03:54:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.762+0000 7f11d7843740 1 -- start start 2024-09-21T03:54:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.762+0000 7f11d7843740 1 Processor -- start 2024-09-21T03:54:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.763+0000 7f11d7843740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd662f0 0x55c1dd142c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.763+0000 7f11d7843740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd1432d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.763+0000 7f11d7843740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd143e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.763+0000 7f11d7843740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81b41e0 con 0x55c1dcd662f0 2024-09-21T03:54:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.763+0000 7f11d7843740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81b3b10 con 0x55c1dcd8f1b0 2024-09-21T03:54:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.763+0000 7f11d7843740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81b0d50 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.763+0000 7f11d7843740 10 client.0 _ll_register_callbacks cb 0x7f11b8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.763+0000 7f11cd400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd1432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.763+0000 7f11ce800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.763+0000 7f11cd400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd1432d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:60070/0 (socket says 172.21.15.16:60070) 2024-09-21T03:54:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.763+0000 7f11cd400640 1 -- 172.21.15.16:0/1247304995 learned_addr learned my addr 172.21.15.16:0/1247304995 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.764+0000 7f11cd400640 1 -- 172.21.15.16:0/1247304995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd8f1b0 msgr2=0x55c1dd143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.764+0000 7f11cd400640 1 --2- 172.21.15.16:0/1247304995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.764+0000 7f11cd400640 1 -- 172.21.15.16:0/1247304995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd662f0 msgr2=0x55c1dd142c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.764+0000 7f11cd400640 1 --2- 172.21.15.16:0/1247304995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd662f0 0x55c1dd142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.764+0000 7f11cd400640 1 -- 172.21.15.16:0/1247304995 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1dd1445a0 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.764+0000 7f11cd400640 1 --2- 172.21.15.16:0/1247304995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd1432d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f11bc005890 tx=0x7f11bc003e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=404a0d197a1cdcc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.764+0000 7f11ab400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:54:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.764+0000 7f11ab400640 1 -- 172.21.15.16:0/1247304995 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc015070 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.765+0000 7f11d7843740 1 -- 172.21.15.16:0/1247304995 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11b8031a50 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.765+0000 7f11ab400640 1 -- 172.21.15.16:0/1247304995 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f11bc0043e0 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.765+0000 7f11ab400640 1 -- 172.21.15.16:0/1247304995 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f11bc01a910 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.765+0000 7f11d7843740 10 client.26356 fetch_fsmap learned FSMap version 30 2024-09-21T03:54:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.765+0000 7f11d7843740 1 -- 172.21.15.16:0/1247304995 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c1dd1478a0 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.766+0000 7f11ab400640 1 -- 172.21.15.16:0/1247304995 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11b8031a50 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.766+0000 7f11ab400640 1 -- 172.21.15.16:0/1247304995 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f11bc023040 con 0x55c1dcd6e4d0 2024-09-21T03:54:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.766+0000 7f11d7843740 10 client.26356 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:54:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.766+0000 7f11d7843740 10 client.26356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.766+0000 7f11d7843740 20 client.26356 populate_metadata read hostname 'smithi016' 2024-09-21T03:54:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.766+0000 7f11d7843740 1 --2- 172.21.15.16:0/1247304995 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c1dcd231c0 0x55c1dd1ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.766+0000 7f11d7843740 4 client.26356 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:54:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.766+0000 7f11d7843740 1 -- 172.21.15.16:0/1247304995 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c1dd0a70b0 con 0x55c1dcd231c0 2024-09-21T03:54:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.767+0000 7f11cde00640 1 --2- 172.21.15.16:0/1247304995 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c1dcd231c0 0x55c1dd1ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.768+0000 7f11cde00640 1 --2- 172.21.15.16:0/1247304995 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c1dcd231c0 0x55c1dd1ca9c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fab95a4352e3a41 server_cookie=78be9be0f0def623 in_seq=0 out_seq=0 2024-09-21T03:54:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.768+0000 7f11ab400640 10 client.26356 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.785+0000 7f11ab400640 1 -- 172.21.15.16:0/1247304995 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c1dd0a70b0 con 0x55c1dcd231c0 2024-09-21T03:54:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.785+0000 7f11ab400640 10 client.26356 handle_command_reply: tid=0 2024-09-21T03:54:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.786+0000 7f11d7843740 1 client.26356 shutdown 2024-09-21T03:54:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.786+0000 7f11d7843740 1 -- 172.21.15.16:0/1247304995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd6e4d0 msgr2=0x55c1dd1432d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.786+0000 7f11d7843740 1 --2- 172.21.15.16:0/1247304995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd1432d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f11bc005890 tx=0x7f11bc003e90 comp rx=0 tx=0).stop 2024-09-21T03:54:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.786+0000 7f11d7843740 1 -- 172.21.15.16:0/1247304995 shutdown_connections 2024-09-21T03:54:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.786+0000 7f11d7843740 1 --2- 172.21.15.16:0/1247304995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1dcd8f1b0 0x55c1dd143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.786+0000 7f11d7843740 1 --2- 172.21.15.16:0/1247304995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1dcd6e4d0 0x55c1dd1432d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.786+0000 7f11d7843740 1 --2- 172.21.15.16:0/1247304995 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55c1dcd231c0 0x55c1dd1ca9c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.786+0000 7f11ab400640 0 client.26356 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.786+0000 7f11d7843740 1 --2- 172.21.15.16:0/1247304995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1dcd662f0 0x55c1dd142c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.786+0000 7f11d7843740 1 -- 172.21.15.16:0/1247304995 >> 172.21.15.16:0/1247304995 conn(0x55c1dcd83300 msgr2=0x55c1dd13ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.787+0000 7f11d7843740 1 -- 172.21.15.16:0/1247304995 shutdown_connections 2024-09-21T03:54:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.787+0000 7f11d7843740 1 -- 172.21.15.16:0/1247304995 wait complete. 2024-09-21T03:54:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.787+0000 7f11d7843740 20 client.26356 trim_cache size 0 max 16384 2024-09-21T03:54:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:54:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:54:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:54:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:54:28.791 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:28 smithi016 ceph-mon[30375]: pgmap v663: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 248 MiB/s wr, 1.19k op/s 2024-09-21T03:54:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.790+0000 7f11d0800640 1 -- 172.21.15.16:0/1056741807 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f11b00b64a0 msgr2=0x7f11b00b89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.790+0000 7f11d0800640 1 --2- 172.21.15.16:0/1056741807 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f11b00b64a0 0x7f11b00b89b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009230 tx=0x7f11b80314c0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.790+0000 7f11d0800640 1 -- 172.21.15.16:0/1056741807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11c81036c0 msgr2=0x7f11c81a0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.790+0000 7f11d0800640 1 --2- 172.21.15.16:0/1056741807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11c81036c0 0x7f11c81a0510 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f11c400cce0 tx=0x7f11c400a620 comp rx=0 tx=0).stop 2024-09-21T03:54:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.790+0000 7f11d0800640 1 -- 172.21.15.16:0/1056741807 shutdown_connections 2024-09-21T03:54:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.790+0000 7f11d0800640 1 --2- 172.21.15.16:0/1056741807 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f11b00b64a0 0x7f11b00b89b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.790+0000 7f11d0800640 1 --2- 172.21.15.16:0/1056741807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c81050e0 0x7f11c81a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.790+0000 7f11d0800640 1 --2- 172.21.15.16:0/1056741807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c8104740 0x7f11c81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.790+0000 7f11d0800640 1 --2- 172.21.15.16:0/1056741807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11c81036c0 0x7f11c81a0510 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.790+0000 7f11d0800640 1 -- 172.21.15.16:0/1056741807 >> 172.21.15.16:0/1056741807 conn(0x7f11c80ff020 msgr2=0x7f11c8105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.791+0000 7f11d0800640 1 -- 172.21.15.16:0/1056741807 shutdown_connections 2024-09-21T03:54:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:28.791+0000 7f11d0800640 1 -- 172.21.15.16:0/1056741807 wait complete. 2024-09-21T03:54:29.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:28 smithi134 ceph-mon[35031]: pgmap v663: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 248 MiB/s wr, 1.19k op/s 2024-09-21T03:54:29.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:28 smithi123 ceph-mon[34673]: pgmap v663: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 248 MiB/s wr, 1.19k op/s 2024-09-21T03:54:30.305 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0' 2024-09-21T03:54:30.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0 2024-09-21T03:54:31.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:30 smithi134 ceph-mon[35031]: pgmap v664: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 232 MiB/s wr, 1.11k op/s 2024-09-21T03:54:31.147 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:31.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:30 smithi123 ceph-mon[34673]: pgmap v664: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 232 MiB/s wr, 1.11k op/s 2024-09-21T03:54:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:30 smithi016 ceph-mon[30375]: pgmap v664: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 232 MiB/s wr, 1.11k op/s 2024-09-21T03:54:33.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:32 smithi134 ceph-mon[35031]: pgmap v665: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 236 MiB/s wr, 1.13k op/s 2024-09-21T03:54:33.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:32 smithi123 ceph-mon[34673]: pgmap v665: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 236 MiB/s wr, 1.13k op/s 2024-09-21T03:54:33.169 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:32 smithi016 ceph-mon[30375]: pgmap v665: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 236 MiB/s wr, 1.13k op/s 2024-09-21T03:54:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.824+0000 7f3cbe400640 1 -- 172.21.15.16:0/1911264196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb8105dc0 msgr2=0x7f3cb81061d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.824+0000 7f3cbe400640 1 --2- 172.21.15.16:0/1911264196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb8105dc0 0x7f3cb81061d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8009900 tx=0x7f3ca802a410 comp rx=0 tx=0).stop 2024-09-21T03:54:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.824+0000 7f3cbe400640 1 -- 172.21.15.16:0/1911264196 shutdown_connections 2024-09-21T03:54:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.824+0000 7f3cbe400640 1 --2- 172.21.15.16:0/1911264196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb810c050 0x7f3cb8112880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.824+0000 7f3cbe400640 1 --2- 172.21.15.16:0/1911264196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cb810b700 0x7f3cb810bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.824+0000 7f3cbe400640 1 --2- 172.21.15.16:0/1911264196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb8105dc0 0x7f3cb81061d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.824+0000 7f3cbe400640 1 -- 172.21.15.16:0/1911264196 >> 172.21.15.16:0/1911264196 conn(0x7f3cb81017d0 msgr2=0x7f3cb8103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:33.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.825+0000 7f3cbe400640 1 -- 172.21.15.16:0/1911264196 shutdown_connections 2024-09-21T03:54:33.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.825+0000 7f3cbe400640 1 -- 172.21.15.16:0/1911264196 wait complete. 2024-09-21T03:54:33.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.826+0000 7f3cbe400640 1 Processor -- start 2024-09-21T03:54:33.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.826+0000 7f3cbe400640 1 -- start start 2024-09-21T03:54:33.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.826+0000 7f3cbe400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb8105dc0 0x7f3cb819c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.826+0000 7f3cbe400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cb810b700 0x7f3cb819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.826+0000 7f3cbe400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb810c050 0x7f3cb81a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.826+0000 7f3cbe400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb8115d60 con 0x7f3cb8105dc0 2024-09-21T03:54:33.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.826+0000 7f3cbe400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb8115be0 con 0x7f3cb810c050 2024-09-21T03:54:33.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.826+0000 7f3cbe400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb8115ee0 con 0x7f3cb810b700 2024-09-21T03:54:33.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.827+0000 7f3cb6a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cb810b700 0x7f3cb819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.827+0000 7f3cb7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb810c050 0x7f3cb81a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.827+0000 7f3cb7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb810c050 0x7f3cb81a1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:53036/0 (socket says 172.21.15.16:53036) 2024-09-21T03:54:33.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.827+0000 7f3cb7e00640 1 -- 172.21.15.16:0/2233246904 learned_addr learned my addr 172.21.15.16:0/2233246904 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:33.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.827+0000 7f3cb6a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cb810b700 0x7f3cb819cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39356/0 (socket says 172.21.15.16:39356) 2024-09-21T03:54:33.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.827+0000 7f3cb7400640 1 --2- 172.21.15.16:0/2233246904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb8105dc0 0x7f3cb819c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.828+0000 7f3cb6a00640 1 -- 172.21.15.16:0/2233246904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb810c050 msgr2=0x7f3cb81a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:33.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.828+0000 7f3cb6a00640 1 --2- 172.21.15.16:0/2233246904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb810c050 0x7f3cb81a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.828+0000 7f3cb6a00640 1 -- 172.21.15.16:0/2233246904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb8105dc0 msgr2=0x7f3cb819c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:33.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.828+0000 7f3cb6a00640 1 --2- 172.21.15.16:0/2233246904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb8105dc0 0x7f3cb819c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.828+0000 7f3cb6a00640 1 -- 172.21.15.16:0/2233246904 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb81a26e0 con 0x7f3cb810b700 2024-09-21T03:54:33.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.828+0000 7f3cb7400640 1 --2- 172.21.15.16:0/2233246904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb8105dc0 0x7f3cb819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:33.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.828+0000 7f3cb6a00640 1 --2- 172.21.15.16:0/2233246904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cb810b700 0x7f3cb819cd10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3cb806bad0 tx=0x7f3ca400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:33.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.829+0000 7f3cafe00640 1 -- 172.21.15.16:0/2233246904 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca400cd00 con 0x7f3cb810b700 2024-09-21T03:54:33.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.829+0000 7f3cafe00640 1 -- 172.21.15.16:0/2233246904 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca40035f0 con 0x7f3cb810b700 2024-09-21T03:54:33.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.829+0000 7f3cbe400640 1 -- 172.21.15.16:0/2233246904 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb81a29d0 con 0x7f3cb810b700 2024-09-21T03:54:33.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.829+0000 7f3cafe00640 1 -- 172.21.15.16:0/2233246904 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4010690 con 0x7f3cb810b700 2024-09-21T03:54:33.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.829+0000 7f3cbe400640 1 -- 172.21.15.16:0/2233246904 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb81a9130 con 0x7f3cb810b700 2024-09-21T03:54:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.835+0000 7f3cc500b740 1 Processor -- start 2024-09-21T03:54:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.835+0000 7f3cc500b740 1 -- start start 2024-09-21T03:54:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.838+0000 7f3cc500b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac7911850 0x555ac7cb89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.840+0000 7f3cafe00640 1 -- 172.21.15.16:0/2233246904 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3ca40037a0 con 0x7f3cb810b700 2024-09-21T03:54:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.840+0000 7f3cc500b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac78e8990 0x555ac7cb8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.840+0000 7f3cc500b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac78f0b70 0x555ac7cb9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.840+0000 7f3cc500b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555ac7c2ea90 con 0x555ac78f0b70 2024-09-21T03:54:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.840+0000 7f3cc500b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555ac7c1dec0 con 0x555ac78e8990 2024-09-21T03:54:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.840+0000 7f3cc500b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555ac7c21cd0 con 0x555ac7911850 2024-09-21T03:54:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.841+0000 7f3cb7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac78e8990 0x555ac7cb8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.841+0000 7f3cb7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac78e8990 0x555ac7cb8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:53052/0 (socket says 172.21.15.16:53052) 2024-09-21T03:54:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.841+0000 7f3cb7400640 1 -- 172.21.15.16:0/414158817 learned_addr learned my addr 172.21.15.16:0/414158817 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.841+0000 7f3cb7e00640 1 --2- 172.21.15.16:0/414158817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac7911850 0x555ac7cb89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.842+0000 7f3cb7400640 1 -- 172.21.15.16:0/414158817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac7911850 msgr2=0x555ac7cb89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.842+0000 7f3cb7400640 1 --2- 172.21.15.16:0/414158817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac7911850 0x555ac7cb89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.842+0000 7f3cb7400640 1 -- 172.21.15.16:0/414158817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac78f0b70 msgr2=0x555ac7cb9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:33.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.842+0000 7f3cb7400640 1 --2- 172.21.15.16:0/414158817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac78f0b70 0x555ac7cb9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.842+0000 7f3cb7400640 1 -- 172.21.15.16:0/414158817 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ac7bc39d0 con 0x555ac78e8990 2024-09-21T03:54:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.843+0000 7f3cafe00640 1 --2- 172.21.15.16:0/2233246904 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3c9c0b6610 0x7f3c9c0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.843+0000 7f3cafe00640 1 -- 172.21.15.16:0/2233246904 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f3ca40e7330 con 0x7f3cb810b700 2024-09-21T03:54:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.844+0000 7f3cb7400640 1 --2- 172.21.15.16:0/414158817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac78e8990 0x555ac7cb8ef0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8009900 tx=0x7f3ca80042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d04bee1910d9538 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.845+0000 7f3cb7e00640 1 --2- 172.21.15.16:0/2233246904 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3c9c0b6610 0x7f3c9c0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.845+0000 7f3cacc00640 1 -- 172.21.15.16:0/414158817 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8035030 con 0x555ac78e8990 2024-09-21T03:54:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.845+0000 7f3cacc00640 1 -- 172.21.15.16:0/414158817 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca80044d0 con 0x555ac78e8990 2024-09-21T03:54:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.845+0000 7f3cacc00640 1 -- 172.21.15.16:0/414158817 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca80035f0 con 0x555ac78e8990 2024-09-21T03:54:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.846+0000 7f3cc500b740 1 -- 172.21.15.16:0/414158817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac78e8990 msgr2=0x555ac7cb8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.846+0000 7f3cc500b740 1 --2- 172.21.15.16:0/414158817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac78e8990 0x555ac7cb8ef0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8009900 tx=0x7f3ca80042e0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.853+0000 7f3cb7e00640 1 --2- 172.21.15.16:0/2233246904 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3c9c0b6610 0x7f3c9c0b8b20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0005c40 tx=0x7f3cb0009500 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.853+0000 7f3cc500b740 1 -- 172.21.15.16:0/414158817 shutdown_connections 2024-09-21T03:54:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.853+0000 7f3cc500b740 1 --2- 172.21.15.16:0/414158817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac78f0b70 0x555ac7cb9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.853+0000 7f3cc500b740 1 --2- 172.21.15.16:0/414158817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac78e8990 0x555ac7cb8ef0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.853+0000 7f3cc500b740 1 --2- 172.21.15.16:0/414158817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac7911850 0x555ac7cb89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.853+0000 7f3cc500b740 1 -- 172.21.15.16:0/414158817 >> 172.21.15.16:0/414158817 conn(0x555ac7905c40 msgr2=0x555ac7cb68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.854+0000 7f3cc500b740 1 -- 172.21.15.16:0/414158817 shutdown_connections 2024-09-21T03:54:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.854+0000 7f3cc500b740 1 -- 172.21.15.16:0/414158817 wait complete. 2024-09-21T03:54:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.855+0000 7f3cc500b740 1 -- start start 2024-09-21T03:54:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.855+0000 7f3cc500b740 1 Processor -- start 2024-09-21T03:54:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.856+0000 7f3cc500b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac78e8990 0x555ac7cc49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.857+0000 7f3cc500b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac78f0b70 0x555ac7cc4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.857+0000 7f3cc500b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac7911850 0x555ac7cc9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.857+0000 7f3cb7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac78e8990 0x555ac7cc49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.857+0000 7f3cc500b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555ac7a7f7b0 con 0x555ac7911850 2024-09-21T03:54:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.857+0000 7f3cc500b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555ac7bc2650 con 0x555ac78f0b70 2024-09-21T03:54:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.857+0000 7f3cc500b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555ac7cbeec0 con 0x555ac78e8990 2024-09-21T03:54:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.857+0000 7f3cb6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac78f0b70 0x555ac7cc4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.858+0000 7f3cb6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac78f0b70 0x555ac7cc4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:53054/0 (socket says 172.21.15.16:53054) 2024-09-21T03:54:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.858+0000 7f3cb6a00640 1 -- 172.21.15.16:0/1772891370 learned_addr learned my addr 172.21.15.16:0/1772891370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.858+0000 7f3cc500b740 10 client.0 _ll_register_callbacks cb 0x555ac79290a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.859+0000 7f3cb6a00640 1 -- 172.21.15.16:0/1772891370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac78e8990 msgr2=0x555ac7cc49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.859+0000 7f3cb6a00640 1 --2- 172.21.15.16:0/1772891370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac78e8990 0x555ac7cc49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.859+0000 7f3cb6a00640 1 -- 172.21.15.16:0/1772891370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac7911850 msgr2=0x555ac7cc9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.859+0000 7f3cb6a00640 1 --2- 172.21.15.16:0/1772891370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac7911850 0x555ac7cc9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.859+0000 7f3cb6a00640 1 -- 172.21.15.16:0/1772891370 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ac798bba0 con 0x555ac78f0b70 2024-09-21T03:54:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.859+0000 7f3ca1600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:54:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.860+0000 7f3cb6a00640 1 --2- 172.21.15.16:0/1772891370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac78f0b70 0x555ac7cc4f60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4003e40 tx=0x7f3ca40acaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=751f0d8fbeec5c02 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.860+0000 7f3ca1600640 1 -- 172.21.15.16:0/1772891370 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca40ad7b0 con 0x555ac78f0b70 2024-09-21T03:54:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.860+0000 7f3ca1600640 1 -- 172.21.15.16:0/1772891370 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f3ca40a8c10 con 0x555ac78f0b70 2024-09-21T03:54:33.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.860+0000 7f3cc500b740 1 -- 172.21.15.16:0/1772891370 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ca80044d0 con 0x555ac78f0b70 2024-09-21T03:54:33.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.861+0000 7f3ca1600640 1 -- 172.21.15.16:0/1772891370 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ca40ad7b0 con 0x555ac78f0b70 2024-09-21T03:54:33.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.861+0000 7f3cc500b740 10 client.26429 fetch_fsmap learned FSMap version 30 2024-09-21T03:54:33.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.861+0000 7f3cc500b740 1 -- 172.21.15.16:0/1772891370 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555ac7ce4070 con 0x555ac78f0b70 2024-09-21T03:54:33.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.861+0000 7f3ca1600640 1 -- 172.21.15.16:0/1772891370 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca80044d0 con 0x555ac78f0b70 2024-09-21T03:54:33.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.862+0000 7f3ca1600640 1 -- 172.21.15.16:0/1772891370 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3ca40981a0 con 0x555ac78f0b70 2024-09-21T03:54:33.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.863+0000 7f3cc500b740 10 client.26429 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:54:33.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.863+0000 7f3cc500b740 10 client.26429 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:54:33.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.863+0000 7f3cc500b740 20 client.26429 populate_metadata read hostname 'smithi016' 2024-09-21T03:54:33.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.863+0000 7f3cc500b740 1 --2- 172.21.15.16:0/1772891370 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x555ac78d01d0 0x555ac7ce75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.863+0000 7f3cc500b740 4 client.26429 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:33.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.863+0000 7f3cc500b740 1 -- 172.21.15.16:0/1772891370 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ac7c309e0 con 0x555ac78d01d0 2024-09-21T03:54:33.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.863+0000 7f3cb7400640 1 --2- 172.21.15.16:0/1772891370 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x555ac78d01d0 0x555ac7ce75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.864+0000 7f3ca1600640 10 client.26429 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:33.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.864+0000 7f3cb7400640 1 --2- 172.21.15.16:0/1772891370 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x555ac78d01d0 0x555ac7ce75c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f0bd9b969ebb12 server_cookie=c52bb64c5ceef0b4 in_seq=0 out_seq=0 2024-09-21T03:54:33.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.865+0000 7f3ca1600640 1 -- 172.21.15.16:0/1772891370 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555ac7c309e0 con 0x555ac78d01d0 2024-09-21T03:54:33.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.865+0000 7f3ca1600640 10 client.26429 handle_command_reply: tid=0 2024-09-21T03:54:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.866+0000 7f3cc500b740 1 client.26429 shutdown 2024-09-21T03:54:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.867+0000 7f3cc500b740 1 -- 172.21.15.16:0/1772891370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac78f0b70 msgr2=0x555ac7cc4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.867+0000 7f3cc500b740 1 --2- 172.21.15.16:0/1772891370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac78f0b70 0x555ac7cc4f60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4003e40 tx=0x7f3ca40acaf0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.867+0000 7f3cc500b740 1 -- 172.21.15.16:0/1772891370 shutdown_connections 2024-09-21T03:54:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.867+0000 7f3cc500b740 1 --2- 172.21.15.16:0/1772891370 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x555ac78d01d0 0x555ac7ce75c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.867+0000 7f3cc500b740 1 --2- 172.21.15.16:0/1772891370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac7911850 0x555ac7cc9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.867+0000 7f3cc500b740 1 --2- 172.21.15.16:0/1772891370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac78f0b70 0x555ac7cc4f60 secure :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4003e40 tx=0x7f3ca40acaf0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.867+0000 7f3cc500b740 1 --2- 172.21.15.16:0/1772891370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac78e8990 0x555ac7cc49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.867+0000 7f3cc500b740 1 -- 172.21.15.16:0/1772891370 >> 172.21.15.16:0/1772891370 conn(0x555ac7905c40 msgr2=0x555ac7cbbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.867+0000 7f3ca1600640 0 client.26429 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.870+0000 7f3cc500b740 1 -- 172.21.15.16:0/1772891370 shutdown_connections 2024-09-21T03:54:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.870+0000 7f3cc500b740 1 -- 172.21.15.16:0/1772891370 wait complete. 2024-09-21T03:54:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.871+0000 7f3cc500b740 20 client.26429 trim_cache size 0 max 16384 2024-09-21T03:54:33.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.916+0000 7f3cc500b740 1 Processor -- start 2024-09-21T03:54:33.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.916+0000 7f3cc500b740 1 -- start start 2024-09-21T03:54:33.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.917+0000 7f3cc500b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac78e8990 0x555ac7cc1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.917+0000 7f3cc500b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac7911850 0x555ac7cc1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.917+0000 7f3cc500b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac78a5580 0x555ac7cecd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.917+0000 7f3cc500b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb80724c0 con 0x555ac78e8990 2024-09-21T03:54:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.917+0000 7f3cc500b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb8074000 con 0x555ac7911850 2024-09-21T03:54:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.917+0000 7f3cc500b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb8072b90 con 0x555ac78a5580 2024-09-21T03:54:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.919+0000 7f3cb6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac7911850 0x555ac7cc1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.919+0000 7f3cb6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac7911850 0x555ac7cc1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:53062/0 (socket says 172.21.15.16:53062) 2024-09-21T03:54:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.919+0000 7f3cb6a00640 1 -- 172.21.15.16:0/2721225470 learned_addr learned my addr 172.21.15.16:0/2721225470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.920+0000 7f3cb6a00640 1 -- 172.21.15.16:0/2721225470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac78a5580 msgr2=0x555ac7cecd80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:54:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.920+0000 7f3cb6a00640 1 --2- 172.21.15.16:0/2721225470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac78a5580 0x555ac7cecd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.920+0000 7f3cb6a00640 1 -- 172.21.15.16:0/2721225470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac78e8990 msgr2=0x555ac7cc1990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:54:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.920+0000 7f3cb6a00640 1 --2- 172.21.15.16:0/2721225470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac78e8990 0x555ac7cc1990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.920+0000 7f3cb6a00640 1 -- 172.21.15.16:0/2721225470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ac7cb7220 con 0x555ac7911850 2024-09-21T03:54:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.922+0000 7f3cb6a00640 1 --2- 172.21.15.16:0/2721225470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac7911850 0x555ac7cc1ed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40a9630 tx=0x7f3ca400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2e56c097c9dd21f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.922+0000 7f3ca3e00640 1 -- 172.21.15.16:0/2721225470 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca40ace80 con 0x555ac7911850 2024-09-21T03:54:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.922+0000 7f3ca3e00640 1 -- 172.21.15.16:0/2721225470 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca40b1b90 con 0x555ac7911850 2024-09-21T03:54:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.922+0000 7f3ca3e00640 1 -- 172.21.15.16:0/2721225470 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca40b6010 con 0x555ac7911850 2024-09-21T03:54:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.924+0000 7f3cc500b740 1 -- 172.21.15.16:0/2721225470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac7911850 msgr2=0x555ac7cc1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.924+0000 7f3cc500b740 1 --2- 172.21.15.16:0/2721225470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac7911850 0x555ac7cc1ed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40a9630 tx=0x7f3ca400cf90 comp rx=0 tx=0).stop 2024-09-21T03:54:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.924+0000 7f3cc500b740 1 -- 172.21.15.16:0/2721225470 shutdown_connections 2024-09-21T03:54:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.924+0000 7f3cc500b740 1 --2- 172.21.15.16:0/2721225470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac78a5580 0x555ac7cecd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.924+0000 7f3cc500b740 1 --2- 172.21.15.16:0/2721225470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac7911850 0x555ac7cc1ed0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.924+0000 7f3cc500b740 1 --2- 172.21.15.16:0/2721225470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac78e8990 0x555ac7cc1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.924+0000 7f3cc500b740 1 -- 172.21.15.16:0/2721225470 >> 172.21.15.16:0/2721225470 conn(0x555ac7905c40 msgr2=0x555ac7cc3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.924+0000 7f3cc500b740 1 -- 172.21.15.16:0/2721225470 shutdown_connections 2024-09-21T03:54:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.924+0000 7f3cc500b740 1 -- 172.21.15.16:0/2721225470 wait complete. 2024-09-21T03:54:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.925+0000 7f3cc500b740 1 -- start start 2024-09-21T03:54:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.925+0000 7f3cc500b740 1 Processor -- start 2024-09-21T03:54:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.928+0000 7f3cc500b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac78a5580 0x555ac7cf53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.929+0000 7f3cc500b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac78e8990 0x555ac7cf58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.929+0000 7f3cc500b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac7911850 0x555ac7cd66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.929+0000 7f3cc500b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb8073930 con 0x555ac78e8990 2024-09-21T03:54:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.929+0000 7f3cc500b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb8073260 con 0x555ac7911850 2024-09-21T03:54:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.929+0000 7f3cc500b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb80704a0 con 0x555ac78a5580 2024-09-21T03:54:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.929+0000 7f3cc500b740 10 client.0 _ll_register_callbacks cb 0x7f3ca4003d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.930+0000 7f3cb7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac7911850 0x555ac7cd66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.930+0000 7f3cb7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac7911850 0x555ac7cd66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:53074/0 (socket says 172.21.15.16:53074) 2024-09-21T03:54:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.930+0000 7f3cb7e00640 1 -- 172.21.15.16:0/3774985831 learned_addr learned my addr 172.21.15.16:0/3774985831 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.930+0000 7f3cb7400640 1 --2- 172.21.15.16:0/3774985831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac78a5580 0x555ac7cf53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.930+0000 7f3cb7e00640 1 -- 172.21.15.16:0/3774985831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac78a5580 msgr2=0x555ac7cf53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.930+0000 7f3cb7e00640 1 --2- 172.21.15.16:0/3774985831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac78a5580 0x555ac7cf53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.931+0000 7f3cb7e00640 1 -- 172.21.15.16:0/3774985831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac78e8990 msgr2=0x555ac7cf58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.931+0000 7f3cb7e00640 1 --2- 172.21.15.16:0/3774985831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac78e8990 0x555ac7cf58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.931+0000 7f3cb7e00640 1 -- 172.21.15.16:0/3774985831 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ac7cd6c00 con 0x555ac7911850 2024-09-21T03:54:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.931+0000 7f3cb7400640 1 --2- 172.21.15.16:0/3774985831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac78a5580 0x555ac7cf53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:54:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.933+0000 7f3cb7e00640 1 --2- 172.21.15.16:0/3774985831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac7911850 0x555ac7cd66c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0020b20 tx=0x7f3cb0020b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=284b58b6b42db4e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.933+0000 7f3ca1600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:54:33.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.934+0000 7f3ca1600640 1 -- 172.21.15.16:0/3774985831 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0035070 con 0x555ac7911850 2024-09-21T03:54:33.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.934+0000 7f3ca1600640 1 -- 172.21.15.16:0/3774985831 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f3cb0004b20 con 0x555ac7911850 2024-09-21T03:54:33.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.934+0000 7f3cc500b740 1 -- 172.21.15.16:0/3774985831 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ca40b1b90 con 0x555ac7911850 2024-09-21T03:54:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.935+0000 7f3ca1600640 1 -- 172.21.15.16:0/3774985831 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3cb0004d20 con 0x555ac7911850 2024-09-21T03:54:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.935+0000 7f3cc500b740 10 client.26435 fetch_fsmap learned FSMap version 30 2024-09-21T03:54:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.935+0000 7f3cc500b740 1 -- 172.21.15.16:0/3774985831 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555ac7cd7ec0 con 0x555ac7911850 2024-09-21T03:54:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.935+0000 7f3ca1600640 1 -- 172.21.15.16:0/3774985831 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca40b1b90 con 0x555ac7911850 2024-09-21T03:54:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.936+0000 7f3ca1600640 1 -- 172.21.15.16:0/3774985831 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3cb0043b10 con 0x555ac7911850 2024-09-21T03:54:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.937+0000 7f3cc500b740 10 client.26435 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:54:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.937+0000 7f3cc500b740 10 client.26435 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:54:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.937+0000 7f3cc500b740 20 client.26435 populate_metadata read hostname 'smithi016' 2024-09-21T03:54:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.937+0000 7f3cc500b740 1 --2- 172.21.15.16:0/3774985831 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x555ac7cd86c0 0x555ac7cdaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.937+0000 7f3cc500b740 4 client.26435 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}] 2024-09-21T03:54:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.937+0000 7f3cc500b740 1 -- 172.21.15.16:0/3774985831 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}) -- 0x555ac7c2c490 con 0x555ac7cd86c0 2024-09-21T03:54:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.939+0000 7f3cb7400640 1 --2- 172.21.15.16:0/3774985831 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x555ac7cd86c0 0x555ac7cdaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.940+0000 7f3cb7400640 1 --2- 172.21.15.16:0/3774985831 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x555ac7cd86c0 0x555ac7cdaae0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69b4906bf41ab7e4 server_cookie=496eae23dd1c1454 in_seq=0 out_seq=0 2024-09-21T03:54:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:33.940+0000 7f3ca1600640 10 client.26435 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:34.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.081+0000 7f3ca1600640 1 -- 172.21.15.16:0/3774985831 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555ac7c2c490 con 0x555ac7cd86c0 2024-09-21T03:54:34.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.081+0000 7f3ca1600640 10 client.26435 handle_command_reply: tid=0 2024-09-21T03:54:34.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.081+0000 7f3cc500b740 1 client.26435 shutdown 2024-09-21T03:54:34.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.081+0000 7f3cc500b740 1 -- 172.21.15.16:0/3774985831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac7911850 msgr2=0x555ac7cd66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:34.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.081+0000 7f3cc500b740 1 --2- 172.21.15.16:0/3774985831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac7911850 0x555ac7cd66c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0020b20 tx=0x7f3cb0020b50 comp rx=0 tx=0).stop 2024-09-21T03:54:34.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.081+0000 7f3cc500b740 1 -- 172.21.15.16:0/3774985831 shutdown_connections 2024-09-21T03:54:34.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.082+0000 7f3cc500b740 1 --2- 172.21.15.16:0/3774985831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ac7911850 0x555ac7cd66c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:34.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.082+0000 7f3cc500b740 1 --2- 172.21.15.16:0/3774985831 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x555ac7cd86c0 0x555ac7cdaae0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:34.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.082+0000 7f3cc500b740 1 --2- 172.21.15.16:0/3774985831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac78e8990 0x555ac7cf58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:34.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.082+0000 7f3cc500b740 1 --2- 172.21.15.16:0/3774985831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ac78a5580 0x555ac7cf53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:34.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.082+0000 7f3cc500b740 1 -- 172.21.15.16:0/3774985831 >> 172.21.15.16:0/3774985831 conn(0x555ac7905c40 msgr2=0x555ac7cc2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:34.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.082+0000 7f3ca1600640 0 client.26435 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.082+0000 7f3cc500b740 1 -- 172.21.15.16:0/3774985831 shutdown_connections 2024-09-21T03:54:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.083+0000 7f3cc500b740 1 -- 172.21.15.16:0/3774985831 wait complete. 2024-09-21T03:54:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.084+0000 7f3cc500b740 20 client.26435 trim_cache size 0 max 16384 2024-09-21T03:54:34.087 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:54:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:54:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:54:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 30, 2024-09-21T03:54:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:54:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "b02854af": { 2024-09-21T03:54:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 30, 2024-09-21T03:54:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:54:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:54:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:54:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:54:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:54:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:54:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 141.1, 2024-09-21T03:54:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:54:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:54:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:54:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:54:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:54:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:54:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:54:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:54:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:54:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:54:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:54:34.089 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:54:34.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.086+0000 7f3cbe400640 1 -- 172.21.15.16:0/2233246904 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3c9c0b6610 msgr2=0x7f3c9c0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:34.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.086+0000 7f3cbe400640 1 --2- 172.21.15.16:0/2233246904 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3c9c0b6610 0x7f3c9c0b8b20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0005c40 tx=0x7f3cb0009500 comp rx=0 tx=0).stop 2024-09-21T03:54:34.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.086+0000 7f3cbe400640 1 -- 172.21.15.16:0/2233246904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cb810b700 msgr2=0x7f3cb819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.086+0000 7f3cbe400640 1 --2- 172.21.15.16:0/2233246904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cb810b700 0x7f3cb819cd10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3cb806bad0 tx=0x7f3ca400ee50 comp rx=0 tx=0).stop 2024-09-21T03:54:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.087+0000 7f3cbe400640 1 -- 172.21.15.16:0/2233246904 shutdown_connections 2024-09-21T03:54:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.087+0000 7f3cbe400640 1 --2- 172.21.15.16:0/2233246904 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3c9c0b6610 0x7f3c9c0b8b20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.087+0000 7f3cbe400640 1 --2- 172.21.15.16:0/2233246904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb810c050 0x7f3cb81a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.087+0000 7f3cbe400640 1 --2- 172.21.15.16:0/2233246904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cb810b700 0x7f3cb819cd10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.087+0000 7f3cbe400640 1 --2- 172.21.15.16:0/2233246904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb8105dc0 0x7f3cb819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.087+0000 7f3cbe400640 1 -- 172.21.15.16:0/2233246904 >> 172.21.15.16:0/2233246904 conn(0x7f3cb81017d0 msgr2=0x7f3cb8104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.088+0000 7f3cbe400640 1 -- 172.21.15.16:0/2233246904 shutdown_connections 2024-09-21T03:54:34.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:34.088+0000 7f3cbe400640 1 -- 172.21.15.16:0/2233246904 wait complete. 2024-09-21T03:54:35.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:34 smithi134 ceph-mon[35031]: pgmap v666: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 226 MiB/s wr, 1.08k op/s 2024-09-21T03:54:35.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:34 smithi123 ceph-mon[34673]: pgmap v666: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 226 MiB/s wr, 1.08k op/s 2024-09-21T03:54:35.168 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:34 smithi016 ceph-mon[30375]: pgmap v666: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 226 MiB/s wr, 1.08k op/s 2024-09-21T03:54:37.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:36 smithi134 ceph-mon[35031]: pgmap v667: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 250 MiB/s wr, 1.20k op/s 2024-09-21T03:54:37.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:36 smithi123 ceph-mon[34673]: pgmap v667: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 250 MiB/s wr, 1.20k op/s 2024-09-21T03:54:37.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:36 smithi016 ceph-mon[30375]: pgmap v667: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 250 MiB/s wr, 1.20k op/s 2024-09-21T03:54:38.310 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b02854af', quiesce took 15.0 seconds. Will release after: 6.050000000000001 2024-09-21T03:54:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:38 smithi134 ceph-mon[35031]: pgmap v668: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 MiB/s wr, 1.13k op/s 2024-09-21T03:54:39.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:38 smithi123 ceph-mon[34673]: pgmap v668: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 MiB/s wr, 1.13k op/s 2024-09-21T03:54:39.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:38 smithi016 ceph-mon[30375]: pgmap v668: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 MiB/s wr, 1.13k op/s 2024-09-21T03:54:41.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:40 smithi123 ceph-mon[34673]: pgmap v669: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 222 MiB/s wr, 1.09k op/s 2024-09-21T03:54:41.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:54:41.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:40 smithi016 ceph-mon[30375]: pgmap v669: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 222 MiB/s wr, 1.09k op/s 2024-09-21T03:54:41.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:54:41.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:40 smithi134 ceph-mon[35031]: pgmap v669: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 222 MiB/s wr, 1.09k op/s 2024-09-21T03:54:41.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:54:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.134+0000 7f1b79e00640 1 -- 172.21.15.16:0/3674279002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b74102400 msgr2=0x7f1b74102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.134+0000 7f1b79e00640 1 --2- 172.21.15.16:0/3674279002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b74102400 0x7f1b74102870 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009900 tx=0x7f1b6002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.135+0000 7f1b79e00640 1 -- 172.21.15.16:0/3674279002 shutdown_connections 2024-09-21T03:54:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.135+0000 7f1b79e00640 1 --2- 172.21.15.16:0/3674279002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b74102fb0 0x7f1b74107a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.135+0000 7f1b79e00640 1 --2- 172.21.15.16:0/3674279002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b74102400 0x7f1b74102870 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.135+0000 7f1b79e00640 1 --2- 172.21.15.16:0/3674279002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b741019d0 0x7f1b74101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.135+0000 7f1b79e00640 1 -- 172.21.15.16:0/3674279002 >> 172.21.15.16:0/3674279002 conn(0x7f1b740fd690 msgr2=0x7f1b740ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.136+0000 7f1b79e00640 1 -- 172.21.15.16:0/3674279002 shutdown_connections 2024-09-21T03:54:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.137+0000 7f1b79e00640 1 -- 172.21.15.16:0/3674279002 wait complete. 2024-09-21T03:54:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.138+0000 7f1b79e00640 1 Processor -- start 2024-09-21T03:54:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.142+0000 7f1b79e00640 1 -- start start 2024-09-21T03:54:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.142+0000 7f1b79e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b741019d0 0x7f1b741a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.142+0000 7f1b79e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b74102400 0x7f1b741a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.142+0000 7f1b79e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b74102fb0 0x7f1b741a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.142+0000 7f1b79e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b74115980 con 0x7f1b74102fb0 2024-09-21T03:54:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.142+0000 7f1b79e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b74115800 con 0x7f1b74102400 2024-09-21T03:54:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.142+0000 7f1b79e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1b74115b00 con 0x7f1b741019d0 2024-09-21T03:54:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.143+0000 7f1b72a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b741019d0 0x7f1b741a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.143+0000 7f1b72000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b74102400 0x7f1b741a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.143+0000 7f1b72a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b741019d0 0x7f1b741a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:57010/0 (socket says 172.21.15.16:57010) 2024-09-21T03:54:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.143+0000 7f1b72a00640 1 -- 172.21.15.16:0/828013057 learned_addr learned my addr 172.21.15.16:0/828013057 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.143+0000 7f1b73400640 1 --2- 172.21.15.16:0/828013057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b74102fb0 0x7f1b741a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.144+0000 7f1b72a00640 1 -- 172.21.15.16:0/828013057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b74102400 msgr2=0x7f1b741a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.144+0000 7f1b72a00640 1 --2- 172.21.15.16:0/828013057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b74102400 0x7f1b741a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.144+0000 7f1b72a00640 1 -- 172.21.15.16:0/828013057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b74102fb0 msgr2=0x7f1b741a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.144+0000 7f1b72a00640 1 --2- 172.21.15.16:0/828013057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b74102fb0 0x7f1b741a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.144+0000 7f1b72a00640 1 -- 172.21.15.16:0/828013057 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b741a1fb0 con 0x7f1b741019d0 2024-09-21T03:54:42.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.145+0000 7f1b72a00640 1 --2- 172.21.15.16:0/828013057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b741019d0 0x7f1b741a0570 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400e9f0 tx=0x7f1b6400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:42.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.145+0000 7f1b78a00640 1 -- 172.21.15.16:0/828013057 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6400cd80 con 0x7f1b741019d0 2024-09-21T03:54:42.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.145+0000 7f1b78a00640 1 -- 172.21.15.16:0/828013057 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b640108e0 con 0x7f1b741019d0 2024-09-21T03:54:42.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.145+0000 7f1b78a00640 1 -- 172.21.15.16:0/828013057 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64018980 con 0x7f1b741019d0 2024-09-21T03:54:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.147+0000 7f1b79e00640 1 -- 172.21.15.16:0/828013057 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b741a22a0 con 0x7f1b741019d0 2024-09-21T03:54:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.147+0000 7f1b79e00640 1 -- 172.21.15.16:0/828013057 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b74109100 con 0x7f1b741019d0 2024-09-21T03:54:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.150+0000 7f1b80dd0740 1 Processor -- start 2024-09-21T03:54:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.150+0000 7f1b80dd0740 1 -- start start 2024-09-21T03:54:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.151+0000 7f1b80dd0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31451b0 0x562ce34ec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.151+0000 7f1b80dd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce311c2f0 0x562ce34ec740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.151+0000 7f1b80dd0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31244d0 0x562ce34ed290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.151+0000 7f1b80dd0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562ce345f6b0 con 0x562ce31451b0 2024-09-21T03:54:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.151+0000 7f1b80dd0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562ce344eae0 con 0x562ce31244d0 2024-09-21T03:54:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.151+0000 7f1b80dd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562ce34528f0 con 0x562ce311c2f0 2024-09-21T03:54:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.151+0000 7f1b72000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31244d0 0x562ce34ed290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.151+0000 7f1b72000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31244d0 0x562ce34ed290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:39668/0 (socket says 172.21.15.16:39668) 2024-09-21T03:54:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.151+0000 7f1b72000640 1 -- 172.21.15.16:0/1316185137 learned_addr learned my addr 172.21.15.16:0/1316185137 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.151+0000 7f1b73400640 1 --2- 172.21.15.16:0/1316185137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31451b0 0x562ce34ec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.152+0000 7f1b72000640 1 -- 172.21.15.16:0/1316185137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce311c2f0 msgr2=0x562ce34ec740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:54:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.152+0000 7f1b72000640 1 --2- 172.21.15.16:0/1316185137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce311c2f0 0x562ce34ec740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.152+0000 7f1b72000640 1 -- 172.21.15.16:0/1316185137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31451b0 msgr2=0x562ce34ec0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.152+0000 7f1b72000640 1 --2- 172.21.15.16:0/1316185137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31451b0 0x562ce34ec0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.152+0000 7f1b72000640 1 -- 172.21.15.16:0/1316185137 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce341b780 con 0x562ce31244d0 2024-09-21T03:54:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.153+0000 7f1b78a00640 1 -- 172.21.15.16:0/828013057 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1b64010430 con 0x7f1b741019d0 2024-09-21T03:54:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.154+0000 7f1b72000640 1 --2- 172.21.15.16:0/1316185137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31244d0 0x562ce34ed290 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1b7406abb0 tx=0x7f1b600040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=744187e8237ec08e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.154+0000 7f1b5fe00640 1 -- 172.21.15.16:0/1316185137 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60035030 con 0x562ce31244d0 2024-09-21T03:54:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.155+0000 7f1b5fe00640 1 -- 172.21.15.16:0/1316185137 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b60040070 con 0x562ce31244d0 2024-09-21T03:54:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.155+0000 7f1b5fe00640 1 -- 172.21.15.16:0/1316185137 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60031920 con 0x562ce31244d0 2024-09-21T03:54:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.155+0000 7f1b78a00640 1 --2- 172.21.15.16:0/828013057 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1b580b66e0 0x7f1b580b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.155+0000 7f1b78a00640 1 -- 172.21.15.16:0/828013057 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f1b640e7c10 con 0x7f1b741019d0 2024-09-21T03:54:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.156+0000 7f1b80dd0740 1 -- 172.21.15.16:0/1316185137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31244d0 msgr2=0x562ce34ed290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.156+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/1316185137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31244d0 0x562ce34ed290 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1b7406abb0 tx=0x7f1b600040c0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.156+0000 7f1b80dd0740 1 -- 172.21.15.16:0/1316185137 shutdown_connections 2024-09-21T03:54:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.156+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/1316185137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31244d0 0x562ce34ed290 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.156+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/1316185137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce311c2f0 0x562ce34ec740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.156+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/1316185137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31451b0 0x562ce34ec0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.156+0000 7f1b80dd0740 1 -- 172.21.15.16:0/1316185137 >> 172.21.15.16:0/1316185137 conn(0x562ce3139300 msgr2=0x562ce34e7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.157+0000 7f1b80dd0740 1 -- 172.21.15.16:0/1316185137 shutdown_connections 2024-09-21T03:54:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.157+0000 7f1b80dd0740 1 -- 172.21.15.16:0/1316185137 wait complete. 2024-09-21T03:54:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.157+0000 7f1b73400640 1 --2- 172.21.15.16:0/828013057 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1b580b66e0 0x7f1b580b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.157+0000 7f1b80dd0740 1 -- start start 2024-09-21T03:54:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.158+0000 7f1b73400640 1 --2- 172.21.15.16:0/828013057 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1b580b66e0 0x7f1b580b8bf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c005fd0 tx=0x7f1b6c0097e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.159+0000 7f1b80dd0740 1 Processor -- start 2024-09-21T03:54:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.160+0000 7f1b80dd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce311c2f0 0x562ce34f5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.160+0000 7f1b80dd0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31244d0 0x562ce34f5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.161+0000 7f1b80dd0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31451b0 0x562ce34f6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.161+0000 7f1b80dd0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562ce3386ab0 con 0x562ce31244d0 2024-09-21T03:54:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.161+0000 7f1b80dd0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562ce33f5570 con 0x562ce31451b0 2024-09-21T03:54:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.161+0000 7f1b80dd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562ce34edd50 con 0x562ce311c2f0 2024-09-21T03:54:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.161+0000 7f1b72a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce311c2f0 0x562ce34f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.161+0000 7f1b72a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce311c2f0 0x562ce34f5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:57016/0 (socket says 172.21.15.16:57016) 2024-09-21T03:54:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.161+0000 7f1b72a00640 1 -- 172.21.15.16:0/3788474184 learned_addr learned my addr 172.21.15.16:0/3788474184 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.161+0000 7f1b80dd0740 10 client.0 _ll_register_callbacks cb 0x562ce3404100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.161+0000 7f1b72000640 1 --2- 172.21.15.16:0/3788474184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31244d0 0x562ce34f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.162+0000 7f1b73400640 1 --2- 172.21.15.16:0/3788474184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31451b0 0x562ce34f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.162+0000 7f1b72a00640 1 -- 172.21.15.16:0/3788474184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31451b0 msgr2=0x562ce34f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.162+0000 7f1b72a00640 1 --2- 172.21.15.16:0/3788474184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31451b0 0x562ce34f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.162+0000 7f1b72a00640 1 -- 172.21.15.16:0/3788474184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31244d0 msgr2=0x562ce34f5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.162+0000 7f1b72a00640 1 --2- 172.21.15.16:0/3788474184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31244d0 0x562ce34f5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.162+0000 7f1b72a00640 1 -- 172.21.15.16:0/3788474184 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce31b0790 con 0x562ce311c2f0 2024-09-21T03:54:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.163+0000 7f1b73400640 1 --2- 172.21.15.16:0/3788474184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31451b0 0x562ce34f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.163+0000 7f1b72000640 1 --2- 172.21.15.16:0/3788474184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31244d0 0x562ce34f5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:54:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.163+0000 7f1b72a00640 1 --2- 172.21.15.16:0/3788474184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce311c2f0 0x562ce34f5730 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1b640aac00 tx=0x7f1b640ad3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a100ff4c0cf5835 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.163+0000 7f1b5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:54:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.163+0000 7f1b5cc00640 1 -- 172.21.15.16:0/3788474184 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b640acdf0 con 0x562ce311c2f0 2024-09-21T03:54:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.163+0000 7f1b5cc00640 1 -- 172.21.15.16:0/3788474184 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f1b64018b20 con 0x562ce311c2f0 2024-09-21T03:54:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.163+0000 7f1b80dd0740 1 -- 172.21.15.16:0/3788474184 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b60040070 con 0x562ce311c2f0 2024-09-21T03:54:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.164+0000 7f1b5cc00640 1 -- 172.21.15.16:0/3788474184 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1b640ad4a0 con 0x562ce311c2f0 2024-09-21T03:54:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.164+0000 7f1b80dd0740 10 client.26380 fetch_fsmap learned FSMap version 30 2024-09-21T03:54:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.164+0000 7f1b80dd0740 1 -- 172.21.15.16:0/3788474184 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562ce3229620 con 0x562ce311c2f0 2024-09-21T03:54:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.166+0000 7f1b5cc00640 1 -- 172.21.15.16:0/3788474184 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b60040070 con 0x562ce311c2f0 2024-09-21T03:54:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.166+0000 7f1b5cc00640 1 -- 172.21.15.16:0/3788474184 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1b640a92e0 con 0x562ce311c2f0 2024-09-21T03:54:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.168+0000 7f1b80dd0740 10 client.26380 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:54:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.168+0000 7f1b80dd0740 10 client.26380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.168+0000 7f1b80dd0740 20 client.26380 populate_metadata read hostname 'smithi016' 2024-09-21T03:54:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.168+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/3788474184 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562ce3103d80 0x562ce3502de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.168+0000 7f1b80dd0740 4 client.26380 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.168+0000 7f1b80dd0740 1 -- 172.21.15.16:0/3788474184 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ce3461600 con 0x562ce3103d80 2024-09-21T03:54:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.168+0000 7f1b72000640 1 --2- 172.21.15.16:0/3788474184 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562ce3103d80 0x562ce3502de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.170+0000 7f1b72000640 1 --2- 172.21.15.16:0/3788474184 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562ce3103d80 0x562ce3502de0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe36fb079e90040 server_cookie=d079883273d69c6e in_seq=0 out_seq=0 2024-09-21T03:54:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.170+0000 7f1b5cc00640 10 client.26380 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.171+0000 7f1b5cc00640 1 -- 172.21.15.16:0/3788474184 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562ce3461600 con 0x562ce3103d80 2024-09-21T03:54:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.171+0000 7f1b5cc00640 10 client.26380 handle_command_reply: tid=0 2024-09-21T03:54:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.171+0000 7f1b80dd0740 1 client.26380 shutdown 2024-09-21T03:54:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.171+0000 7f1b80dd0740 1 -- 172.21.15.16:0/3788474184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce311c2f0 msgr2=0x562ce34f5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.171+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/3788474184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce311c2f0 0x562ce34f5730 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1b640aac00 tx=0x7f1b640ad3d0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.171+0000 7f1b80dd0740 1 -- 172.21.15.16:0/3788474184 shutdown_connections 2024-09-21T03:54:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.171+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/3788474184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31451b0 0x562ce34f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.171+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/3788474184 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562ce3103d80 0x562ce3502de0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.172+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/3788474184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31244d0 0x562ce34f5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.172+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/3788474184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce311c2f0 0x562ce34f5730 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.172+0000 7f1b80dd0740 1 -- 172.21.15.16:0/3788474184 >> 172.21.15.16:0/3788474184 conn(0x562ce3139300 msgr2=0x562ce34e60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.172+0000 7f1b5cc00640 0 client.26380 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.172+0000 7f1b80dd0740 1 -- 172.21.15.16:0/3788474184 shutdown_connections 2024-09-21T03:54:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.173+0000 7f1b80dd0740 1 -- 172.21.15.16:0/3788474184 wait complete. 2024-09-21T03:54:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.174+0000 7f1b80dd0740 20 client.26380 trim_cache size 0 max 16384 2024-09-21T03:54:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.228+0000 7f1b80dd0740 1 Processor -- start 2024-09-21T03:54:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.228+0000 7f1b80dd0740 1 -- start start 2024-09-21T03:54:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.229+0000 7f1b80dd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce311c2f0 0x562ce34f4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.229+0000 7f1b80dd0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31244d0 0x562ce34f0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.229+0000 7f1b80dd0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31451b0 0x562ce34f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.229+0000 7f1b80dd0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741bb5e0 con 0x562ce31451b0 2024-09-21T03:54:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.229+0000 7f1b80dd0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741bd120 con 0x562ce31244d0 2024-09-21T03:54:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.229+0000 7f1b80dd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741bbcb0 con 0x562ce311c2f0 2024-09-21T03:54:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.229+0000 7f1b72000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31244d0 0x562ce34f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.229+0000 7f1b72000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31244d0 0x562ce34f0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:39688/0 (socket says 172.21.15.16:39688) 2024-09-21T03:54:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.229+0000 7f1b72000640 1 -- 172.21.15.16:0/2875911051 learned_addr learned my addr 172.21.15.16:0/2875911051 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.230+0000 7f1b72a00640 1 --2- 172.21.15.16:0/2875911051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce311c2f0 0x562ce34f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.230+0000 7f1b73400640 1 --2- 172.21.15.16:0/2875911051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31451b0 0x562ce34f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.230+0000 7f1b72000640 1 -- 172.21.15.16:0/2875911051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce311c2f0 msgr2=0x562ce34f4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.230+0000 7f1b72000640 1 --2- 172.21.15.16:0/2875911051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce311c2f0 0x562ce34f4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.230+0000 7f1b72000640 1 -- 172.21.15.16:0/2875911051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31451b0 msgr2=0x562ce34f06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.230+0000 7f1b72000640 1 --2- 172.21.15.16:0/2875911051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31451b0 0x562ce34f06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.230+0000 7f1b72000640 1 -- 172.21.15.16:0/2875911051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce3103fb0 con 0x562ce31244d0 2024-09-21T03:54:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.231+0000 7f1b72000640 1 --2- 172.21.15.16:0/2875911051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31244d0 0x562ce34f0160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009900 tx=0x7f1b6002a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc95b731a0095fae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.231+0000 7f1b5f400640 1 -- 172.21.15.16:0/2875911051 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60035070 con 0x562ce31244d0 2024-09-21T03:54:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.231+0000 7f1b5f400640 1 -- 172.21.15.16:0/2875911051 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b60040030 con 0x562ce31244d0 2024-09-21T03:54:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.231+0000 7f1b5f400640 1 -- 172.21.15.16:0/2875911051 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60031cd0 con 0x562ce31244d0 2024-09-21T03:54:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.233+0000 7f1b80dd0740 1 -- 172.21.15.16:0/2875911051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31244d0 msgr2=0x562ce34f0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.233+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/2875911051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31244d0 0x562ce34f0160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009900 tx=0x7f1b6002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.233+0000 7f1b80dd0740 1 -- 172.21.15.16:0/2875911051 shutdown_connections 2024-09-21T03:54:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.233+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/2875911051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31451b0 0x562ce34f06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.233+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/2875911051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce31244d0 0x562ce34f0160 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.233+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/2875911051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce311c2f0 0x562ce34f4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.233+0000 7f1b80dd0740 1 -- 172.21.15.16:0/2875911051 >> 172.21.15.16:0/2875911051 conn(0x562ce3139300 msgr2=0x562ce34f3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.233+0000 7f1b80dd0740 1 -- 172.21.15.16:0/2875911051 shutdown_connections 2024-09-21T03:54:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.233+0000 7f1b80dd0740 1 -- 172.21.15.16:0/2875911051 wait complete. 2024-09-21T03:54:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.233+0000 7f1b80dd0740 1 -- start start 2024-09-21T03:54:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.234+0000 7f1b80dd0740 1 Processor -- start 2024-09-21T03:54:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.234+0000 7f1b80dd0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce311c2f0 0x562ce34fa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.234+0000 7f1b80dd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce31244d0 0x562ce34fb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.234+0000 7f1b80dd0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31451b0 0x562ce34fbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.234+0000 7f1b80dd0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741bca50 con 0x562ce31451b0 2024-09-21T03:54:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.234+0000 7f1b80dd0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741bc380 con 0x562ce311c2f0 2024-09-21T03:54:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.234+0000 7f1b80dd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1b74111530 con 0x562ce31244d0 2024-09-21T03:54:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.234+0000 7f1b80dd0740 10 client.0 _ll_register_callbacks cb 0x7f1b640056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.235+0000 7f1b72a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce311c2f0 0x562ce34fa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.235+0000 7f1b72a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce311c2f0 0x562ce34fa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:39700/0 (socket says 172.21.15.16:39700) 2024-09-21T03:54:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.235+0000 7f1b72a00640 1 -- 172.21.15.16:0/2684932364 learned_addr learned my addr 172.21.15.16:0/2684932364 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.235+0000 7f1b72000640 1 --2- 172.21.15.16:0/2684932364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce31244d0 0x562ce34fb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.235+0000 7f1b73400640 1 --2- 172.21.15.16:0/2684932364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31451b0 0x562ce34fbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.235+0000 7f1b72a00640 1 -- 172.21.15.16:0/2684932364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce31244d0 msgr2=0x562ce34fb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.235+0000 7f1b72a00640 1 --2- 172.21.15.16:0/2684932364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce31244d0 0x562ce34fb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.235+0000 7f1b72a00640 1 -- 172.21.15.16:0/2684932364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31451b0 msgr2=0x562ce34fbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.235+0000 7f1b72a00640 1 --2- 172.21.15.16:0/2684932364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31451b0 0x562ce34fbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.235+0000 7f1b72a00640 1 -- 172.21.15.16:0/2684932364 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce34fc380 con 0x562ce311c2f0 2024-09-21T03:54:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.235+0000 7f1b72000640 1 --2- 172.21.15.16:0/2684932364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce31244d0 0x562ce34fb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.235+0000 7f1b73400640 1 --2- 172.21.15.16:0/2684932364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31451b0 0x562ce34fbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.235+0000 7f1b72a00640 1 --2- 172.21.15.16:0/2684932364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce311c2f0 0x562ce34fa9f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1b640ad640 tx=0x7f1b640b73b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=299afdb3d6a0de5c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.235+0000 7f1b5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:54:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.235+0000 7f1b5cc00640 1 -- 172.21.15.16:0/2684932364 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b640ad760 con 0x562ce311c2f0 2024-09-21T03:54:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.236+0000 7f1b5cc00640 1 -- 172.21.15.16:0/2684932364 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f1b640c3b40 con 0x562ce311c2f0 2024-09-21T03:54:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.236+0000 7f1b80dd0740 1 -- 172.21.15.16:0/2684932364 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b60040030 con 0x562ce311c2f0 2024-09-21T03:54:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.236+0000 7f1b5cc00640 1 -- 172.21.15.16:0/2684932364 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1b640a9100 con 0x562ce311c2f0 2024-09-21T03:54:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.236+0000 7f1b80dd0740 10 client.26450 fetch_fsmap learned FSMap version 30 2024-09-21T03:54:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.236+0000 7f1b80dd0740 1 -- 172.21.15.16:0/2684932364 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562ce34fd6e0 con 0x562ce311c2f0 2024-09-21T03:54:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.236+0000 7f1b5cc00640 1 -- 172.21.15.16:0/2684932364 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b640ae090 con 0x562ce311c2f0 2024-09-21T03:54:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.236+0000 7f1b5cc00640 1 -- 172.21.15.16:0/2684932364 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1b640ac860 con 0x562ce311c2f0 2024-09-21T03:54:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.236+0000 7f1b80dd0740 10 client.26450 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:54:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.236+0000 7f1b80dd0740 10 client.26450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.236+0000 7f1b80dd0740 20 client.26450 populate_metadata read hostname 'smithi016' 2024-09-21T03:54:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.236+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/2684932364 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562ce30d91c0 0x562ce356fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.236+0000 7f1b80dd0740 4 client.26450 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:54:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.236+0000 7f1b80dd0740 1 -- 172.21.15.16:0/2684932364 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1b60040030 con 0x562ce30d91c0 2024-09-21T03:54:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.237+0000 7f1b72000640 1 --2- 172.21.15.16:0/2684932364 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562ce30d91c0 0x562ce356fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.238+0000 7f1b5cc00640 10 client.26450 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.238+0000 7f1b72000640 1 --2- 172.21.15.16:0/2684932364 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562ce30d91c0 0x562ce356fb90 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2030db734e6bec8f server_cookie=fdc7e26d24a9529b in_seq=0 out_seq=0 2024-09-21T03:54:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.252+0000 7f1b5cc00640 1 -- 172.21.15.16:0/2684932364 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f1b60040030 con 0x562ce30d91c0 2024-09-21T03:54:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.252+0000 7f1b5cc00640 10 client.26450 handle_command_reply: tid=0 2024-09-21T03:54:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.252+0000 7f1b80dd0740 1 client.26450 shutdown 2024-09-21T03:54:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.252+0000 7f1b80dd0740 1 -- 172.21.15.16:0/2684932364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce311c2f0 msgr2=0x562ce34fa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.252+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/2684932364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce311c2f0 0x562ce34fa9f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1b640ad640 tx=0x7f1b640b73b0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.252+0000 7f1b80dd0740 1 -- 172.21.15.16:0/2684932364 shutdown_connections 2024-09-21T03:54:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.252+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/2684932364 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562ce30d91c0 0x562ce356fb90 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.252+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/2684932364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce31451b0 0x562ce34fbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.252+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/2684932364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce31244d0 0x562ce34fb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.252+0000 7f1b5cc00640 0 client.26450 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.252+0000 7f1b80dd0740 1 --2- 172.21.15.16:0/2684932364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ce311c2f0 0x562ce34fa9f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.252+0000 7f1b80dd0740 1 -- 172.21.15.16:0/2684932364 >> 172.21.15.16:0/2684932364 conn(0x562ce3139300 msgr2=0x562ce34f0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.252+0000 7f1b80dd0740 1 -- 172.21.15.16:0/2684932364 shutdown_connections 2024-09-21T03:54:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.252+0000 7f1b80dd0740 1 -- 172.21.15.16:0/2684932364 wait complete. 2024-09-21T03:54:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.253+0000 7f1b80dd0740 20 client.26450 trim_cache size 0 max 16384 2024-09-21T03:54:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:54:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:54:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:54:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:54:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.253+0000 7f1b79e00640 1 -- 172.21.15.16:0/828013057 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1b580b66e0 msgr2=0x7f1b580b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.253+0000 7f1b79e00640 1 --2- 172.21.15.16:0/828013057 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1b580b66e0 0x7f1b580b8bf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c005fd0 tx=0x7f1b6c0097e0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.253+0000 7f1b79e00640 1 -- 172.21.15.16:0/828013057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b741019d0 msgr2=0x7f1b741a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.253+0000 7f1b79e00640 1 --2- 172.21.15.16:0/828013057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b741019d0 0x7f1b741a0570 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400e9f0 tx=0x7f1b6400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.254+0000 7f1b79e00640 1 -- 172.21.15.16:0/828013057 shutdown_connections 2024-09-21T03:54:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.254+0000 7f1b79e00640 1 --2- 172.21.15.16:0/828013057 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1b580b66e0 0x7f1b580b8bf0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.254+0000 7f1b79e00640 1 --2- 172.21.15.16:0/828013057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b74102fb0 0x7f1b741a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.254+0000 7f1b79e00640 1 --2- 172.21.15.16:0/828013057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b74102400 0x7f1b741a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.254+0000 7f1b79e00640 1 --2- 172.21.15.16:0/828013057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b741019d0 0x7f1b741a0570 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.254+0000 7f1b79e00640 1 -- 172.21.15.16:0/828013057 >> 172.21.15.16:0/828013057 conn(0x7f1b740fd690 msgr2=0x7f1b74103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.255+0000 7f1b79e00640 1 -- 172.21.15.16:0/828013057 shutdown_connections 2024-09-21T03:54:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:42.255+0000 7f1b79e00640 1 -- 172.21.15.16:0/828013057 wait complete. 2024-09-21T03:54:43.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:42 smithi123 ceph-mon[34673]: pgmap v670: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 237 MiB/s wr, 1.20k op/s 2024-09-21T03:54:43.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:42 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:54:43.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:42 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:54:43.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:42 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:54:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:42 smithi016 ceph-mon[30375]: pgmap v670: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 237 MiB/s wr, 1.20k op/s 2024-09-21T03:54:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:42 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:54:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:42 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:54:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:42 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:54:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:42 smithi134 ceph-mon[35031]: pgmap v670: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 237 MiB/s wr, 1.20k op/s 2024-09-21T03:54:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:42 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:54:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:42 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:54:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:42 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:54:44.361 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:54:44.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:44 smithi016 ceph-mon[30375]: pgmap v671: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 234 MiB/s wr, 1.22k op/s 2024-09-21T03:54:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:45.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:44 smithi123 ceph-mon[34673]: pgmap v671: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 234 MiB/s wr, 1.22k op/s 2024-09-21T03:54:45.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:44 smithi134 ceph-mon[35031]: pgmap v671: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 234 MiB/s wr, 1.22k op/s 2024-09-21T03:54:47.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:46 smithi123 ceph-mon[34673]: pgmap v672: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 258 MiB/s wr, 1.44k op/s 2024-09-21T03:54:47.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:46 smithi016 ceph-mon[30375]: pgmap v672: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 258 MiB/s wr, 1.44k op/s 2024-09-21T03:54:47.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:46 smithi134 ceph-mon[35031]: pgmap v672: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 258 MiB/s wr, 1.44k op/s 2024-09-21T03:54:48.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.518+0000 7f4db5000640 1 -- 172.21.15.16:0/160724271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db010fd10 msgr2=0x7f4db0110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:48.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.519+0000 7f4db5000640 1 --2- 172.21.15.16:0/160724271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db010fd10 0x7f4db0110120 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009900 tx=0x7f4d9802b550 comp rx=0 tx=0).stop 2024-09-21T03:54:48.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.520+0000 7f4db5000640 1 -- 172.21.15.16:0/160724271 shutdown_connections 2024-09-21T03:54:48.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.520+0000 7f4db5000640 1 --2- 172.21.15.16:0/160724271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db0110660 0x7f4db0116e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:48.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.520+0000 7f4db5000640 1 --2- 172.21.15.16:0/160724271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db010fd10 0x7f4db0110120 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:48.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.520+0000 7f4db5000640 1 --2- 172.21.15.16:0/160724271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db010a650 0x7f4db010aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:48.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.520+0000 7f4db5000640 1 -- 172.21.15.16:0/160724271 >> 172.21.15.16:0/160724271 conn(0x7f4db006e9f0 msgr2=0x7f4db006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:48.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.522+0000 7f4db5000640 1 -- 172.21.15.16:0/160724271 shutdown_connections 2024-09-21T03:54:48.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.522+0000 7f4db5000640 1 -- 172.21.15.16:0/160724271 wait complete. 2024-09-21T03:54:48.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.526+0000 7f4db5000640 1 Processor -- start 2024-09-21T03:54:48.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.526+0000 7f4db5000640 1 -- start start 2024-09-21T03:54:48.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.527+0000 7f4db5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db010a650 0x7f4db011bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:48.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.527+0000 7f4db5000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db010fd10 0x7f4db011c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:48.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.527+0000 7f4daf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db010a650 0x7f4db011bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:48.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.527+0000 7f4daf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db010a650 0x7f4db011bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:39722/0 (socket says 172.21.15.16:39722) 2024-09-21T03:54:48.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.529+0000 7f4daf400640 1 -- 172.21.15.16:0/2819619378 learned_addr learned my addr 172.21.15.16:0/2819619378 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:48.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.529+0000 7f4db5000640 1 --2- 172.21.15.16:0/2819619378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0110660 0x7f4db0121560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:48.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.529+0000 7f4db5000640 1 -- 172.21.15.16:0/2819619378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0119a80 con 0x7f4db0110660 2024-09-21T03:54:48.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.529+0000 7f4db5000640 1 -- 172.21.15.16:0/2819619378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0119900 con 0x7f4db010a650 2024-09-21T03:54:48.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.529+0000 7f4db5000640 1 -- 172.21.15.16:0/2819619378 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4db011a250 con 0x7f4db010fd10 2024-09-21T03:54:48.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.529+0000 7f4daea00640 1 --2- 172.21.15.16:0/2819619378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db010fd10 0x7f4db011c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:48.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.530+0000 7f4dafe00640 1 --2- 172.21.15.16:0/2819619378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0110660 0x7f4db0121560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.530+0000 7f4daea00640 1 -- 172.21.15.16:0/2819619378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db010a650 msgr2=0x7f4db011bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.531+0000 7f4daea00640 1 --2- 172.21.15.16:0/2819619378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db010a650 0x7f4db011bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.531+0000 7f4daea00640 1 -- 172.21.15.16:0/2819619378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0110660 msgr2=0x7f4db0121560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.531+0000 7f4daea00640 1 --2- 172.21.15.16:0/2819619378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0110660 0x7f4db0121560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.531+0000 7f4daea00640 1 -- 172.21.15.16:0/2819619378 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db0121aa0 con 0x7f4db010fd10 2024-09-21T03:54:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.532+0000 7f4dafe00640 1 --2- 172.21.15.16:0/2819619378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0110660 0x7f4db0121560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.532+0000 7f4daea00640 1 --2- 172.21.15.16:0/2819619378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db010fd10 0x7f4db011c100 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009230 tx=0x7f4d98003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:48.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.533+0000 7f4da7e00640 1 -- 172.21.15.16:0/2819619378 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9803f070 con 0x7f4db010fd10 2024-09-21T03:54:48.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.533+0000 7f4db5000640 1 -- 172.21.15.16:0/2819619378 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db0121d30 con 0x7f4db010fd10 2024-09-21T03:54:48.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.534+0000 7f4db5000640 1 -- 172.21.15.16:0/2819619378 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db0122230 con 0x7f4db010fd10 2024-09-21T03:54:48.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.536+0000 7f4da7e00640 1 -- 172.21.15.16:0/2819619378 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d98032cf0 con 0x7f4db010fd10 2024-09-21T03:54:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.537+0000 7f4da7e00640 1 -- 172.21.15.16:0/2819619378 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9803f070 con 0x7f4db010fd10 2024-09-21T03:54:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.538+0000 7f4db5000640 1 -- 172.21.15.16:0/2819619378 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d8c005290 con 0x7f4db010fd10 2024-09-21T03:54:48.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.543+0000 7f4da7e00640 1 -- 172.21.15.16:0/2819619378 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f4d98042070 con 0x7f4db010fd10 2024-09-21T03:54:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.546+0000 7f4da7e00640 1 --2- 172.21.15.16:0/2819619378 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4d940b67c0 0x7f4d940b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.546+0000 7f4da7e00640 1 -- 172.21.15.16:0/2819619378 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f4d98107b90 con 0x7f4db010fd10 2024-09-21T03:54:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.547+0000 7f4daf400640 1 --2- 172.21.15.16:0/2819619378 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4d940b67c0 0x7f4d940b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.548+0000 7f4daf400640 1 --2- 172.21.15.16:0/2819619378 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4d940b67c0 0x7f4d940b8cd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009a40 tx=0x7f4da00091c0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:48.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:48.551+0000 7f4da7e00640 1 -- 172.21.15.16:0/2819619378 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4d980ce070 con 0x7f4db010fd10 2024-09-21T03:54:49.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:49.057+0000 7f4db5000640 1 -- 172.21.15.16:0/2819619378 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d8c005580 con 0x7f4db010fd10 2024-09-21T03:54:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:49.058+0000 7f4da7e00640 1 -- 172.21.15.16:0/2819619378 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f4d980ce070 con 0x7f4db010fd10 2024-09-21T03:54:49.060 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:54:49.060 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:54:49.061 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:54:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:49.061+0000 7f4db5000640 1 -- 172.21.15.16:0/2819619378 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4d940b67c0 msgr2=0x7f4d940b8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:49.061+0000 7f4db5000640 1 --2- 172.21.15.16:0/2819619378 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4d940b67c0 0x7f4d940b8cd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009a40 tx=0x7f4da00091c0 comp rx=0 tx=0).stop 2024-09-21T03:54:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:49.061+0000 7f4db5000640 1 -- 172.21.15.16:0/2819619378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db010fd10 msgr2=0x7f4db011c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:49.061+0000 7f4db5000640 1 --2- 172.21.15.16:0/2819619378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db010fd10 0x7f4db011c100 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009230 tx=0x7f4d98003cd0 comp rx=0 tx=0).stop 2024-09-21T03:54:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:49.061+0000 7f4db5000640 1 -- 172.21.15.16:0/2819619378 shutdown_connections 2024-09-21T03:54:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:49.061+0000 7f4db5000640 1 --2- 172.21.15.16:0/2819619378 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f4d940b67c0 0x7f4d940b8cd0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:49.061+0000 7f4db5000640 1 --2- 172.21.15.16:0/2819619378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0110660 0x7f4db0121560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:49.061+0000 7f4db5000640 1 --2- 172.21.15.16:0/2819619378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db010fd10 0x7f4db011c100 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:49.061+0000 7f4db5000640 1 --2- 172.21.15.16:0/2819619378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db010a650 0x7f4db011bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:49.061+0000 7f4db5000640 1 -- 172.21.15.16:0/2819619378 >> 172.21.15.16:0/2819619378 conn(0x7f4db006e9f0 msgr2=0x7f4db010dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:49.061+0000 7f4db5000640 1 -- 172.21.15.16:0/2819619378 shutdown_connections 2024-09-21T03:54:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:49.061+0000 7f4db5000640 1 -- 172.21.15.16:0/2819619378 wait complete. 2024-09-21T03:54:49.153 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:48 smithi134 ceph-mon[35031]: pgmap v673: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 246 MiB/s wr, 1.51k op/s 2024-09-21T03:54:49.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:48 smithi123 ceph-mon[34673]: pgmap v673: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 246 MiB/s wr, 1.51k op/s 2024-09-21T03:54:49.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:48 smithi016 ceph-mon[30375]: pgmap v673: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 246 MiB/s wr, 1.51k op/s 2024-09-21T03:54:49.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:49 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2819619378' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:54:50.008 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db --set-id b02854af --release --await' 2024-09-21T03:54:50.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db --set-id b02854af --release --await 2024-09-21T03:54:50.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:49 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2819619378' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:54:50.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:49 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2819619378' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:54:51.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:50 smithi123 ceph-mon[34673]: pgmap v674: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 224 MiB/s wr, 1.49k op/s 2024-09-21T03:54:51.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:50 smithi016 ceph-mon[30375]: pgmap v674: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 224 MiB/s wr, 1.49k op/s 2024-09-21T03:54:51.324 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:51.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:50 smithi134 ceph-mon[35031]: pgmap v674: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 224 MiB/s wr, 1.49k op/s 2024-09-21T03:54:53.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:52 smithi123 ceph-mon[34673]: pgmap v675: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 230 MiB/s wr, 1.65k op/s 2024-09-21T03:54:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:52 smithi016 ceph-mon[30375]: pgmap v675: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 230 MiB/s wr, 1.65k op/s 2024-09-21T03:54:53.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.329+0000 7f1f6f000640 1 -- 172.21.15.16:0/1598780613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f68109c40 msgr2=0x7f1f68110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:53.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.329+0000 7f1f6f000640 1 --2- 172.21.15.16:0/1598780613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f68109c40 0x7f1f68110470 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009be0 tx=0x7f1f6002a6d0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.330+0000 7f1f6f000640 1 -- 172.21.15.16:0/1598780613 shutdown_connections 2024-09-21T03:54:53.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.330+0000 7f1f6f000640 1 --2- 172.21.15.16:0/1598780613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f68109c40 0x7f1f68110470 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.330+0000 7f1f6f000640 1 --2- 172.21.15.16:0/1598780613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f681092f0 0x7f1f68109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.330+0000 7f1f6f000640 1 --2- 172.21.15.16:0/1598780613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f681039b0 0x7f1f68103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.330+0000 7f1f6f000640 1 -- 172.21.15.16:0/1598780613 >> 172.21.15.16:0/1598780613 conn(0x7f1f680ff440 msgr2=0x7f1f68101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:53.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.330+0000 7f1f6f000640 1 -- 172.21.15.16:0/1598780613 shutdown_connections 2024-09-21T03:54:53.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.330+0000 7f1f6f000640 1 -- 172.21.15.16:0/1598780613 wait complete. 2024-09-21T03:54:53.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.332+0000 7f1f6f000640 1 Processor -- start 2024-09-21T03:54:53.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.332+0000 7f1f6f000640 1 -- start start 2024-09-21T03:54:53.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.333+0000 7f1f6f000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f681039b0 0x7f1f6819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.333+0000 7f1f6f000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f681092f0 0x7f1f6819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.333+0000 7f1f6f000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f68109c40 0x7f1f681a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.333+0000 7f1f6f000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68113900 con 0x7f1f681039b0 2024-09-21T03:54:53.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.333+0000 7f1f6f000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68113780 con 0x7f1f68109c40 2024-09-21T03:54:53.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.333+0000 7f1f6f000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68113a80 con 0x7f1f681092f0 2024-09-21T03:54:53.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.333+0000 7f1f6e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f68109c40 0x7f1f681a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:53.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.333+0000 7f1f6e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f68109c40 0x7f1f681a1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44262/0 (socket says 172.21.15.16:44262) 2024-09-21T03:54:53.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.333+0000 7f1f6e600640 1 -- 172.21.15.16:0/1393683622 learned_addr learned my addr 172.21.15.16:0/1393683622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:53.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.334+0000 7f1f6e600640 1 -- 172.21.15.16:0/1393683622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f681092f0 msgr2=0x7f1f6819ca70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:54:53.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.334+0000 7f1f6e600640 1 --2- 172.21.15.16:0/1393683622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f681092f0 0x7f1f6819ca70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.334+0000 7f1f6e600640 1 -- 172.21.15.16:0/1393683622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f681039b0 msgr2=0x7f1f6819c4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:54:53.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.334+0000 7f1f6e600640 1 --2- 172.21.15.16:0/1393683622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f681039b0 0x7f1f6819c4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.334+0000 7f1f6e600640 1 -- 172.21.15.16:0/1393683622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f681a2440 con 0x7f1f68109c40 2024-09-21T03:54:53.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.335+0000 7f1f6e600640 1 --2- 172.21.15.16:0/1393683622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f68109c40 0x7f1f681a1d10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009b30 tx=0x7f1f60003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.336+0000 7f1f66000640 1 -- 172.21.15.16:0/1393683622 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f60040020 con 0x7f1f68109c40 2024-09-21T03:54:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.336+0000 7f1f66000640 1 -- 172.21.15.16:0/1393683622 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f60030970 con 0x7f1f68109c40 2024-09-21T03:54:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.336+0000 7f1f66000640 1 -- 172.21.15.16:0/1393683622 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f60036040 con 0x7f1f68109c40 2024-09-21T03:54:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.336+0000 7f1f6f000640 1 -- 172.21.15.16:0/1393683622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f681a26d0 con 0x7f1f68109c40 2024-09-21T03:54:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.336+0000 7f1f6f000640 1 -- 172.21.15.16:0/1393683622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f681a2b60 con 0x7f1f68109c40 2024-09-21T03:54:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.343+0000 7f1f66000640 1 -- 172.21.15.16:0/1393683622 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f1f60043020 con 0x7f1f68109c40 2024-09-21T03:54:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.346+0000 7f1f66000640 1 --2- 172.21.15.16:0/1393683622 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1f500b6660 0x7f1f500b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.348+0000 7f1f6dc00640 1 --2- 172.21.15.16:0/1393683622 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1f500b6660 0x7f1f500b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.349+0000 7f1f66000640 1 -- 172.21.15.16:0/1393683622 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f1f60108300 con 0x7f1f68109c40 2024-09-21T03:54:53.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:52 smithi134 ceph-mon[35031]: pgmap v675: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 230 MiB/s wr, 1.65k op/s 2024-09-21T03:54:53.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.351+0000 7f1f6dc00640 1 --2- 172.21.15.16:0/1393683622 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1f500b6660 0x7f1f500b8b70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c005fd0 tx=0x7f1f5c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:53.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.364+0000 7f1f7727e740 1 Processor -- start 2024-09-21T03:54:53.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.365+0000 7f1f7727e740 1 -- start start 2024-09-21T03:54:53.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.366+0000 7f1f7727e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618188520 0x55e61852f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.366+0000 7f1f7727e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e61815f660 0x55e61852f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.366+0000 7f1f7727e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e618167840 0x55e61852fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.366+0000 7f1f7727e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e6184a6290 con 0x55e618188520 2024-09-21T03:54:53.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.366+0000 7f1f7727e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e6184956c0 con 0x55e61815f660 2024-09-21T03:54:53.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.366+0000 7f1f7727e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e6184994d0 con 0x55e618167840 2024-09-21T03:54:53.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.367+0000 7f1f6d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e61815f660 0x55e61852f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:53.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.367+0000 7f1f6d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e61815f660 0x55e61852f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44274/0 (socket says 172.21.15.16:44274) 2024-09-21T03:54:53.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.367+0000 7f1f6d200640 1 -- 172.21.15.16:0/1497411096 learned_addr learned my addr 172.21.15.16:0/1497411096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:53.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.367+0000 7f1f6e600640 1 --2- 172.21.15.16:0/1497411096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e618167840 0x55e61852fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:53.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.367+0000 7f1f6d200640 1 -- 172.21.15.16:0/1497411096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e618167840 msgr2=0x55e61852fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:53.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.368+0000 7f1f6d200640 1 --2- 172.21.15.16:0/1497411096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e618167840 0x55e61852fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.368+0000 7f1f6d200640 1 -- 172.21.15.16:0/1497411096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618188520 msgr2=0x55e61852f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:53.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.368+0000 7f1f6d200640 1 --2- 172.21.15.16:0/1497411096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618188520 0x55e61852f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.368+0000 7f1f6d200640 1 -- 172.21.15.16:0/1497411096 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e618422460 con 0x55e61815f660 2024-09-21T03:54:53.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.369+0000 7f1f6d200640 1 --2- 172.21.15.16:0/1497411096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e61815f660 0x55e61852f880 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1f54014fc0 tx=0x7f1f54009ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=664c6eb1aebe84fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:53.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.369+0000 7f1f5aa00640 1 -- 172.21.15.16:0/1497411096 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54013030 con 0x55e61815f660 2024-09-21T03:54:53.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.369+0000 7f1f5aa00640 1 -- 172.21.15.16:0/1497411096 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f540054d0 con 0x55e61815f660 2024-09-21T03:54:53.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.370+0000 7f1f5aa00640 1 -- 172.21.15.16:0/1497411096 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f540144d0 con 0x55e61815f660 2024-09-21T03:54:53.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.370+0000 7f1f7727e740 1 -- 172.21.15.16:0/1497411096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e61815f660 msgr2=0x55e61852f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:53.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.370+0000 7f1f7727e740 1 --2- 172.21.15.16:0/1497411096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e61815f660 0x55e61852f880 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1f54014fc0 tx=0x7f1f54009ac0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.371+0000 7f1f7727e740 1 -- 172.21.15.16:0/1497411096 shutdown_connections 2024-09-21T03:54:53.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.371+0000 7f1f7727e740 1 --2- 172.21.15.16:0/1497411096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e618167840 0x55e61852fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.371+0000 7f1f7727e740 1 --2- 172.21.15.16:0/1497411096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e61815f660 0x55e61852f880 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.371+0000 7f1f7727e740 1 --2- 172.21.15.16:0/1497411096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618188520 0x55e61852f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.371+0000 7f1f7727e740 1 -- 172.21.15.16:0/1497411096 >> 172.21.15.16:0/1497411096 conn(0x55e61817c910 msgr2=0x55e61852d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:53.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.371+0000 7f1f7727e740 1 -- 172.21.15.16:0/1497411096 shutdown_connections 2024-09-21T03:54:53.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.371+0000 7f1f7727e740 1 -- 172.21.15.16:0/1497411096 wait complete. 2024-09-21T03:54:53.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.372+0000 7f1f7727e740 1 -- start start 2024-09-21T03:54:53.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.372+0000 7f1f7727e740 1 Processor -- start 2024-09-21T03:54:53.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.373+0000 7f1f7727e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e61815f660 0x55e61854be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.373+0000 7f1f7727e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e618167840 0x55e61854c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.373+0000 7f1f7727e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618188520 0x55e61853e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.373+0000 7f1f7727e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e6182f6bb0 con 0x55e618188520 2024-09-21T03:54:53.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.373+0000 7f1f7727e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e6184306c0 con 0x55e618167840 2024-09-21T03:54:53.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.373+0000 7f1f7727e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e61854eac0 con 0x55e61815f660 2024-09-21T03:54:53.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.373+0000 7f1f7727e740 10 client.0 _ll_register_callbacks cb 0x55e61844b490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:53.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.374+0000 7f1f6e600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618188520 0x55e61853e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:53.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.374+0000 7f1f6dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e61815f660 0x55e61854be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:53.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.374+0000 7f1f6dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e61815f660 0x55e61854be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34116/0 (socket says 172.21.15.16:34116) 2024-09-21T03:54:53.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.374+0000 7f1f6dc00640 1 -- 172.21.15.16:0/2304300038 learned_addr learned my addr 172.21.15.16:0/2304300038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:53.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.374+0000 7f1f6d200640 1 --2- 172.21.15.16:0/2304300038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e618167840 0x55e61854c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:53.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.375+0000 7f1f6d200640 1 -- 172.21.15.16:0/2304300038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e61815f660 msgr2=0x55e61854be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:53.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.375+0000 7f1f6d200640 1 --2- 172.21.15.16:0/2304300038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e61815f660 0x55e61854be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.375+0000 7f1f6d200640 1 -- 172.21.15.16:0/2304300038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618188520 msgr2=0x55e61853e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:53.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.375+0000 7f1f6d200640 1 --2- 172.21.15.16:0/2304300038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618188520 0x55e61853e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.375+0000 7f1f6d200640 1 -- 172.21.15.16:0/2304300038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6181f3b00 con 0x55e618167840 2024-09-21T03:54:53.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.375+0000 7f1f6e600640 1 --2- 172.21.15.16:0/2304300038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618188520 0x55e61853e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:53.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.377+0000 7f1f4f400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:54:53.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.377+0000 7f1f6d200640 1 --2- 172.21.15.16:0/2304300038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e618167840 0x55e61854c360 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55e61852dfc0 tx=0x7f1f540048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b43091ae70992df6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:53.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.378+0000 7f1f4f400640 1 -- 172.21.15.16:0/2304300038 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54004460 con 0x55e618167840 2024-09-21T03:54:53.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.378+0000 7f1f4f400640 1 -- 172.21.15.16:0/2304300038 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f1f540035f0 con 0x55e618167840 2024-09-21T03:54:53.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.378+0000 7f1f7727e740 1 -- 172.21.15.16:0/2304300038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f540054d0 con 0x55e618167840 2024-09-21T03:54:53.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.379+0000 7f1f4f400640 1 -- 172.21.15.16:0/2304300038 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1f5403e050 con 0x55e618167840 2024-09-21T03:54:53.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.379+0000 7f1f7727e740 10 client.26468 fetch_fsmap learned FSMap version 30 2024-09-21T03:54:53.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.379+0000 7f1f7727e740 1 -- 172.21.15.16:0/2304300038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e61826a960 con 0x55e618167840 2024-09-21T03:54:53.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.379+0000 7f1f4f400640 1 -- 172.21.15.16:0/2304300038 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f540054d0 con 0x55e618167840 2024-09-21T03:54:53.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.379+0000 7f1f4f400640 1 -- 172.21.15.16:0/2304300038 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1f5400b5e0 con 0x55e618167840 2024-09-21T03:54:53.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.379+0000 7f1f7727e740 10 client.26468 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:54:53.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.379+0000 7f1f7727e740 10 client.26468 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:54:53.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.379+0000 7f1f7727e740 20 client.26468 populate_metadata read hostname 'smithi016' 2024-09-21T03:54:53.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.380+0000 7f1f7727e740 1 --2- 172.21.15.16:0/2304300038 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e618146ea0 0x55e618544460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.380+0000 7f1f7727e740 4 client.26468 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:53.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.380+0000 7f1f7727e740 1 -- 172.21.15.16:0/2304300038 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6184a81e0 con 0x55e618146ea0 2024-09-21T03:54:53.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.380+0000 7f1f6dc00640 1 --2- 172.21.15.16:0/2304300038 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e618146ea0 0x55e618544460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:53.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.381+0000 7f1f4f400640 10 client.26468 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:53.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.385+0000 7f1f6dc00640 1 --2- 172.21.15.16:0/2304300038 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e618146ea0 0x55e618544460 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=803ea758a1c0f82c server_cookie=6a8b4bc8140cf558 in_seq=0 out_seq=0 2024-09-21T03:54:53.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.386+0000 7f1f4f400640 1 -- 172.21.15.16:0/2304300038 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e6184a81e0 con 0x55e618146ea0 2024-09-21T03:54:53.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.386+0000 7f1f4f400640 10 client.26468 handle_command_reply: tid=0 2024-09-21T03:54:53.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.387+0000 7f1f7727e740 1 client.26468 shutdown 2024-09-21T03:54:53.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.388+0000 7f1f7727e740 1 -- 172.21.15.16:0/2304300038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e618167840 msgr2=0x55e61854c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:53.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.388+0000 7f1f7727e740 1 --2- 172.21.15.16:0/2304300038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e618167840 0x55e61854c360 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55e61852dfc0 tx=0x7f1f540048f0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.389+0000 7f1f7727e740 1 -- 172.21.15.16:0/2304300038 shutdown_connections 2024-09-21T03:54:53.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.389+0000 7f1f7727e740 1 --2- 172.21.15.16:0/2304300038 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e618146ea0 0x55e618544460 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.389+0000 7f1f7727e740 1 --2- 172.21.15.16:0/2304300038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618188520 0x55e61853e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.389+0000 7f1f7727e740 1 --2- 172.21.15.16:0/2304300038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e618167840 0x55e61854c360 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.389+0000 7f1f4f400640 0 client.26468 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:53.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.389+0000 7f1f7727e740 1 --2- 172.21.15.16:0/2304300038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e61815f660 0x55e61854be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.389+0000 7f1f7727e740 1 -- 172.21.15.16:0/2304300038 >> 172.21.15.16:0/2304300038 conn(0x55e61817c910 msgr2=0x55e61852b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:53.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.389+0000 7f1f7727e740 1 -- 172.21.15.16:0/2304300038 shutdown_connections 2024-09-21T03:54:53.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.390+0000 7f1f7727e740 1 -- 172.21.15.16:0/2304300038 wait complete. 2024-09-21T03:54:53.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.390+0000 7f1f7727e740 20 client.26468 trim_cache size 0 max 16384 2024-09-21T03:54:53.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.421+0000 7f1f7727e740 1 Processor -- start 2024-09-21T03:54:53.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.421+0000 7f1f7727e740 1 -- start start 2024-09-21T03:54:53.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.421+0000 7f1f7727e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e61815f660 0x55e61854ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.421+0000 7f1f7727e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e618167840 0x55e61852d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.421+0000 7f1f7727e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618188520 0x55e618548e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.421+0000 7f1f7727e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f680726c0 con 0x55e618188520 2024-09-21T03:54:53.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.421+0000 7f1f7727e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68074200 con 0x55e61815f660 2024-09-21T03:54:53.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.421+0000 7f1f7727e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68072d90 con 0x55e618167840 2024-09-21T03:54:53.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.422+0000 7f1f6d200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e618167840 0x55e61852d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:53.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.423+0000 7f1f6dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e61815f660 0x55e61854ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:53.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.423+0000 7f1f6dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e61815f660 0x55e61854ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44300/0 (socket says 172.21.15.16:44300) 2024-09-21T03:54:53.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.423+0000 7f1f6dc00640 1 -- 172.21.15.16:0/2390976890 learned_addr learned my addr 172.21.15.16:0/2390976890 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:53.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.423+0000 7f1f6dc00640 1 -- 172.21.15.16:0/2390976890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e618167840 msgr2=0x55e61852d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:53.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.423+0000 7f1f6dc00640 1 --2- 172.21.15.16:0/2390976890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e618167840 0x55e61852d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.423+0000 7f1f6dc00640 1 -- 172.21.15.16:0/2390976890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618188520 msgr2=0x55e618548e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:53.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.423+0000 7f1f6dc00640 1 --2- 172.21.15.16:0/2390976890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618188520 0x55e618548e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.423+0000 7f1f6dc00640 1 -- 172.21.15.16:0/2390976890 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e618532170 con 0x55e61815f660 2024-09-21T03:54:53.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.424+0000 7f1f6dc00640 1 --2- 172.21.15.16:0/2390976890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e61815f660 0x55e61854ab10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00e720 tx=0x7f1f5c011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3b3aa9a423ed39f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:53.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.424+0000 7f1f5a000640 1 -- 172.21.15.16:0/2390976890 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c02e470 con 0x55e61815f660 2024-09-21T03:54:53.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.424+0000 7f1f5a000640 1 -- 172.21.15.16:0/2390976890 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f5c029070 con 0x55e61815f660 2024-09-21T03:54:53.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.424+0000 7f1f5a000640 1 -- 172.21.15.16:0/2390976890 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c01c8b0 con 0x55e61815f660 2024-09-21T03:54:53.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.426+0000 7f1f7727e740 1 -- 172.21.15.16:0/2390976890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e61815f660 msgr2=0x55e61854ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:53.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.426+0000 7f1f7727e740 1 --2- 172.21.15.16:0/2390976890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e61815f660 0x55e61854ab10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00e720 tx=0x7f1f5c011000 comp rx=0 tx=0).stop 2024-09-21T03:54:53.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.426+0000 7f1f7727e740 1 -- 172.21.15.16:0/2390976890 shutdown_connections 2024-09-21T03:54:53.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.426+0000 7f1f7727e740 1 --2- 172.21.15.16:0/2390976890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618188520 0x55e618548e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.426+0000 7f1f7727e740 1 --2- 172.21.15.16:0/2390976890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e618167840 0x55e61852d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.426+0000 7f1f7727e740 1 --2- 172.21.15.16:0/2390976890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e61815f660 0x55e61854ab10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.426+0000 7f1f7727e740 1 -- 172.21.15.16:0/2390976890 >> 172.21.15.16:0/2390976890 conn(0x55e61817c910 msgr2=0x55e61852cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.427+0000 7f1f7727e740 1 -- 172.21.15.16:0/2390976890 shutdown_connections 2024-09-21T03:54:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.427+0000 7f1f7727e740 1 -- 172.21.15.16:0/2390976890 wait complete. 2024-09-21T03:54:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.428+0000 7f1f7727e740 1 -- start start 2024-09-21T03:54:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.428+0000 7f1f7727e740 1 Processor -- start 2024-09-21T03:54:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.429+0000 7f1f7727e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e61815f660 0x55e618535b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.430+0000 7f1f7727e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e618167840 0x55e618536080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.430+0000 7f1f7727e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e618188520 0x55e6185d9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.430+0000 7f1f6d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e618167840 0x55e618536080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:53.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.430+0000 7f1f6d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e618167840 0x55e618536080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44310/0 (socket says 172.21.15.16:44310) 2024-09-21T03:54:53.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.431+0000 7f1f6d200640 1 -- 172.21.15.16:0/134244690 learned_addr learned my addr 172.21.15.16:0/134244690 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.431+0000 7f1f6e600640 1 --2- 172.21.15.16:0/134244690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e618188520 0x55e6185d9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.431+0000 7f1f7727e740 1 -- 172.21.15.16:0/134244690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68073b30 con 0x55e61815f660 2024-09-21T03:54:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.431+0000 7f1f7727e740 1 -- 172.21.15.16:0/134244690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68073460 con 0x55e618167840 2024-09-21T03:54:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.431+0000 7f1f7727e740 1 -- 172.21.15.16:0/134244690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1f680706e0 con 0x55e618188520 2024-09-21T03:54:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.431+0000 7f1f7727e740 10 client.0 _ll_register_callbacks cb 0x7f1f60003e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.432+0000 7f1f6e600640 1 -- 172.21.15.16:0/134244690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e618167840 msgr2=0x55e618536080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.432+0000 7f1f6e600640 1 --2- 172.21.15.16:0/134244690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e618167840 0x55e618536080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.432+0000 7f1f6e600640 1 -- 172.21.15.16:0/134244690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e61815f660 msgr2=0x55e618535b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.432+0000 7f1f6e600640 1 --2- 172.21.15.16:0/134244690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e61815f660 0x55e618535b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.432+0000 7f1f6e600640 1 -- 172.21.15.16:0/134244690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6185d9580 con 0x55e618188520 2024-09-21T03:54:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.433+0000 7f1f6e600640 1 --2- 172.21.15.16:0/134244690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e618188520 0x55e6185d9040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1f60105aa0 tx=0x7f1f600cd850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a885b34b2184241 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.434+0000 7f1f4f400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:54:53.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.434+0000 7f1f4f400640 1 -- 172.21.15.16:0/134244690 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f600f5070 con 0x55e618188520 2024-09-21T03:54:53.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.434+0000 7f1f4f400640 1 -- 172.21.15.16:0/134244690 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f1f60004d40 con 0x55e618188520 2024-09-21T03:54:53.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.434+0000 7f1f7727e740 1 -- 172.21.15.16:0/134244690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f5c029070 con 0x55e618188520 2024-09-21T03:54:53.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.435+0000 7f1f4f400640 1 -- 172.21.15.16:0/134244690 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1f60108120 con 0x55e618188520 2024-09-21T03:54:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.435+0000 7f1f7727e740 10 client.26404 fetch_fsmap learned FSMap version 30 2024-09-21T03:54:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.435+0000 7f1f7727e740 1 -- 172.21.15.16:0/134244690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e6185da7a0 con 0x55e618188520 2024-09-21T03:54:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.435+0000 7f1f4f400640 1 -- 172.21.15.16:0/134244690 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f5c029070 con 0x55e618188520 2024-09-21T03:54:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.435+0000 7f1f4f400640 1 -- 172.21.15.16:0/134244690 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1f60004500 con 0x55e618188520 2024-09-21T03:54:53.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.435+0000 7f1f7727e740 10 client.26404 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:54:53.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.436+0000 7f1f7727e740 10 client.26404 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:54:53.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.436+0000 7f1f7727e740 20 client.26404 populate_metadata read hostname 'smithi016' 2024-09-21T03:54:53.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.436+0000 7f1f7727e740 1 --2- 172.21.15.16:0/134244690 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e61811c250 0x55e6185dcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:53.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.437+0000 7f1f7727e740 4 client.26404 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b02854af", "release": true, "await": true}] 2024-09-21T03:54:53.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.437+0000 7f1f7727e740 1 -- 172.21.15.16:0/134244690 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b02854af", "release": true, "await": true}) -- 0x55e61854c710 con 0x55e61811c250 2024-09-21T03:54:53.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.438+0000 7f1f6dc00640 1 --2- 172.21.15.16:0/134244690 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e61811c250 0x55e6185dcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:53.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.440+0000 7f1f6dc00640 1 --2- 172.21.15.16:0/134244690 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e61811c250 0x55e6185dcf40 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec916fcace42f67a server_cookie=d542287e7fdbdb4e in_seq=0 out_seq=0 2024-09-21T03:54:53.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.440+0000 7f1f4f400640 10 client.26404 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:53.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.445+0000 7f1f4f400640 1 -- 172.21.15.16:0/134244690 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e61854c710 con 0x55e61811c250 2024-09-21T03:54:53.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.445+0000 7f1f4f400640 10 client.26404 handle_command_reply: tid=0 2024-09-21T03:54:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.446+0000 7f1f7727e740 1 client.26404 shutdown 2024-09-21T03:54:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.446+0000 7f1f7727e740 1 -- 172.21.15.16:0/134244690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e618188520 msgr2=0x55e6185d9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.446+0000 7f1f7727e740 1 --2- 172.21.15.16:0/134244690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e618188520 0x55e6185d9040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1f60105aa0 tx=0x7f1f600cd850 comp rx=0 tx=0).stop 2024-09-21T03:54:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.446+0000 7f1f7727e740 1 -- 172.21.15.16:0/134244690 shutdown_connections 2024-09-21T03:54:53.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.446+0000 7f1f7727e740 1 --2- 172.21.15.16:0/134244690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e618188520 0x55e6185d9040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.446+0000 7f1f7727e740 1 --2- 172.21.15.16:0/134244690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e618167840 0x55e618536080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.446+0000 7f1f7727e740 1 --2- 172.21.15.16:0/134244690 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e61811c250 0x55e6185dcf40 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.446+0000 7f1f7727e740 1 --2- 172.21.15.16:0/134244690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e61815f660 0x55e618535b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.446+0000 7f1f7727e740 1 -- 172.21.15.16:0/134244690 >> 172.21.15.16:0/134244690 conn(0x55e61817c910 msgr2=0x55e61852b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:53.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.447+0000 7f1f4f400640 0 client.26404 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:53.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.447+0000 7f1f7727e740 1 -- 172.21.15.16:0/134244690 shutdown_connections 2024-09-21T03:54:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.448+0000 7f1f7727e740 1 -- 172.21.15.16:0/134244690 wait complete. 2024-09-21T03:54:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.448+0000 7f1f7727e740 20 client.26404 trim_cache size 0 max 16384 2024-09-21T03:54:53.454 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:54:53.454 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:54:53.454 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:54:53.454 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 32, 2024-09-21T03:54:53.454 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:54:53.454 INFO:teuthology.orchestra.run.smithi016.stdout: "b02854af": { 2024-09-21T03:54:53.454 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 32, 2024-09-21T03:54:53.454 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:54:53.454 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:54:53.454 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:54:53.455 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:54:53.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:54:53.455 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:54:53.455 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 141.1, 2024-09-21T03:54:53.455 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:54:53.455 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:54:53.455 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:54:53.455 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:54:53.455 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:54:53.455 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:54:53.455 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:54:53.455 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:54:53.455 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:54:53.455 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:54:53.455 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:54:53.456 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:54:53.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.452+0000 7f1f6f000640 1 -- 172.21.15.16:0/1393683622 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1f500b6660 msgr2=0x7f1f500b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:53.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.452+0000 7f1f6f000640 1 --2- 172.21.15.16:0/1393683622 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1f500b6660 0x7f1f500b8b70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c005fd0 tx=0x7f1f5c009260 comp rx=0 tx=0).stop 2024-09-21T03:54:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.452+0000 7f1f6f000640 1 -- 172.21.15.16:0/1393683622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f68109c40 msgr2=0x7f1f681a1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.452+0000 7f1f6f000640 1 --2- 172.21.15.16:0/1393683622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f68109c40 0x7f1f681a1d10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009b30 tx=0x7f1f60003c10 comp rx=0 tx=0).stop 2024-09-21T03:54:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.452+0000 7f1f6f000640 1 -- 172.21.15.16:0/1393683622 shutdown_connections 2024-09-21T03:54:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.452+0000 7f1f6f000640 1 --2- 172.21.15.16:0/1393683622 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f1f500b6660 0x7f1f500b8b70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.452+0000 7f1f6f000640 1 --2- 172.21.15.16:0/1393683622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f68109c40 0x7f1f681a1d10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.452+0000 7f1f6f000640 1 --2- 172.21.15.16:0/1393683622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f681092f0 0x7f1f6819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.452+0000 7f1f6f000640 1 --2- 172.21.15.16:0/1393683622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f681039b0 0x7f1f6819c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.452+0000 7f1f6f000640 1 -- 172.21.15.16:0/1393683622 >> 172.21.15.16:0/1393683622 conn(0x7f1f680ff440 msgr2=0x7f1f681068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.454+0000 7f1f6f000640 1 -- 172.21.15.16:0/1393683622 shutdown_connections 2024-09-21T03:54:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:54:53.454+0000 7f1f6f000640 1 -- 172.21.15.16:0/1393683622 wait complete. 2024-09-21T03:54:54.390 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b02854af', total seconds elapsed: 31.1 2024-09-21T03:54:54.390 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.05 2024-09-21T03:54:54.390 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:54:55.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:54 smithi134 ceph-mon[35031]: pgmap v676: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 218 MiB/s wr, 1.75k op/s 2024-09-21T03:54:55.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:54 smithi123 ceph-mon[34673]: pgmap v676: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 218 MiB/s wr, 1.75k op/s 2024-09-21T03:54:55.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:54 smithi016 ceph-mon[30375]: pgmap v676: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 218 MiB/s wr, 1.75k op/s 2024-09-21T03:54:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.118+0000 7f2422e00640 1 -- 172.21.15.16:0/3921556489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f241c075cf0 msgr2=0x7f241c076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.118+0000 7f2422e00640 1 --2- 172.21.15.16:0/3921556489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f241c075cf0 0x7f241c076100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2410009900 tx=0x7f241002a410 comp rx=0 tx=0).stop 2024-09-21T03:54:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.118+0000 7f2422e00640 1 -- 172.21.15.16:0/3921556489 shutdown_connections 2024-09-21T03:54:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.118+0000 7f2422e00640 1 --2- 172.21.15.16:0/3921556489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f241c10fc60 0x7f241c112080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.118+0000 7f2422e00640 1 --2- 172.21.15.16:0/3921556489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f241c075cf0 0x7f241c076100 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.118+0000 7f2422e00640 1 --2- 172.21.15.16:0/3921556489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f241c0770f0 0x7f241c0757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.118+0000 7f2422e00640 1 -- 172.21.15.16:0/3921556489 >> 172.21.15.16:0/3921556489 conn(0x7f241c0fee20 msgr2=0x7f241c1012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.119+0000 7f2422e00640 1 -- 172.21.15.16:0/3921556489 shutdown_connections 2024-09-21T03:54:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.119+0000 7f2422e00640 1 -- 172.21.15.16:0/3921556489 wait complete. 2024-09-21T03:54:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.120+0000 7f2422e00640 1 Processor -- start 2024-09-21T03:54:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.121+0000 7f2422e00640 1 -- start start 2024-09-21T03:54:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.121+0000 7f2422e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f241c075cf0 0x7f241c10c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.121+0000 7f2422e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f241c0770f0 0x7f241c10d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.121+0000 7f2422e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f241c10fc60 0x7f241c10dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.121+0000 7f2422e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1157a0 con 0x7f241c0770f0 2024-09-21T03:54:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.121+0000 7f2422e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f241c115620 con 0x7f241c075cf0 2024-09-21T03:54:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.121+0000 7f2422e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f241c115920 con 0x7f241c10fc60 2024-09-21T03:54:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.122+0000 7f2421a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f241c075cf0 0x7f241c10c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.123+0000 7f2421a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f241c075cf0 0x7f241c10c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44336/0 (socket says 172.21.15.16:44336) 2024-09-21T03:54:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.123+0000 7f2421a00640 1 -- 172.21.15.16:0/2812354068 learned_addr learned my addr 172.21.15.16:0/2812354068 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.123+0000 7f2421000640 1 --2- 172.21.15.16:0/2812354068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f241c0770f0 0x7f241c10d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.123+0000 7f2422400640 1 --2- 172.21.15.16:0/2812354068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f241c10fc60 0x7f241c10dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.124+0000 7f2422400640 1 -- 172.21.15.16:0/2812354068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f241c075cf0 msgr2=0x7f241c10c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.124+0000 7f2422400640 1 --2- 172.21.15.16:0/2812354068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f241c075cf0 0x7f241c10c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.124+0000 7f2422400640 1 -- 172.21.15.16:0/2812354068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f241c0770f0 msgr2=0x7f241c10d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.124+0000 7f2422400640 1 --2- 172.21.15.16:0/2812354068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f241c0770f0 0x7f241c10d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.124+0000 7f2422400640 1 -- 172.21.15.16:0/2812354068 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c10e3f0 con 0x7f241c10fc60 2024-09-21T03:54:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.125+0000 7f2421000640 1 --2- 172.21.15.16:0/2812354068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f241c0770f0 0x7f241c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:54:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.126+0000 7f2422400640 1 --2- 172.21.15.16:0/2812354068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f241c10fc60 0x7f241c10dbd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f241400e940 tx=0x7f241400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.130+0000 7f241a000640 1 -- 172.21.15.16:0/2812354068 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414004d20 con 0x7f241c10fc60 2024-09-21T03:54:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.130+0000 7f2422e00640 1 -- 172.21.15.16:0/2812354068 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241c10e6e0 con 0x7f241c10fc60 2024-09-21T03:54:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.130+0000 7f2422e00640 1 -- 172.21.15.16:0/2812354068 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241c10eda0 con 0x7f241c10fc60 2024-09-21T03:54:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.134+0000 7f241a000640 1 -- 172.21.15.16:0/2812354068 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2414005020 con 0x7f241c10fc60 2024-09-21T03:54:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.134+0000 7f241a000640 1 -- 172.21.15.16:0/2812354068 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414004d20 con 0x7f241c10fc60 2024-09-21T03:54:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.141+0000 7f242b5f4740 1 Processor -- start 2024-09-21T03:54:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.144+0000 7f241a000640 1 -- 172.21.15.16:0/2812354068 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f241400cb20 con 0x7f241c10fc60 2024-09-21T03:54:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.147+0000 7f241a000640 1 --2- 172.21.15.16:0/2812354068 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24040b6940 0x7f24040b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.148+0000 7f242b5f4740 1 -- start start 2024-09-21T03:54:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.148+0000 7f242b5f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1f11b0 0x55ec0b5982b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.148+0000 7f242b5f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1c82f0 0x55ec0b598920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.148+0000 7f242b5f4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1d04d0 0x55ec0b599470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.149+0000 7f2421000640 1 --2- 172.21.15.16:0/2812354068 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24040b6940 0x7f24040b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.150+0000 7f241a000640 1 -- 172.21.15.16:0/2812354068 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f2414004b20 con 0x7f241c10fc60 2024-09-21T03:54:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.150+0000 7f2422400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1d04d0 0x55ec0b599470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.150+0000 7f2422400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1d04d0 0x55ec0b599470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44352/0 (socket says 172.21.15.16:44352) 2024-09-21T03:54:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.150+0000 7f2422400640 1 -- 172.21.15.16:0/2575232915 learned_addr learned my addr 172.21.15.16:0/2575232915 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.156+0000 7f242b5f4740 1 -- 172.21.15.16:0/2575232915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ec0b50b6b0 con 0x55ec0b1f11b0 2024-09-21T03:54:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.156+0000 7f242b5f4740 1 -- 172.21.15.16:0/2575232915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ec0b4faae0 con 0x55ec0b1d04d0 2024-09-21T03:54:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.156+0000 7f242b5f4740 1 -- 172.21.15.16:0/2575232915 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ec0b4fe8f0 con 0x55ec0b1c82f0 2024-09-21T03:54:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.150+0000 7f2421000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1c82f0 0x55ec0b598920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.157+0000 7f2422400640 1 -- 172.21.15.16:0/2575232915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1c82f0 msgr2=0x55ec0b598920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.157+0000 7f2422400640 1 --2- 172.21.15.16:0/2575232915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1c82f0 0x55ec0b598920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.157+0000 7f2422400640 1 -- 172.21.15.16:0/2575232915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1f11b0 msgr2=0x55ec0b5982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.157+0000 7f2422400640 1 --2- 172.21.15.16:0/2575232915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1f11b0 0x55ec0b5982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.157+0000 7f2421000640 1 --2- 172.21.15.16:0/2812354068 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24040b6940 0x7f24040b8e50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2410000c00 tx=0x7f2410004510 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.157+0000 7f2422400640 1 -- 172.21.15.16:0/2575232915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec0b4c7780 con 0x55ec0b1d04d0 2024-09-21T03:54:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.158+0000 7f2422400640 1 --2- 172.21.15.16:0/2575232915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1d04d0 0x55ec0b599470 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f24140a9d20 tx=0x7f24140b5d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fec64ad09d9e6ae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.159+0000 7f240aa00640 1 -- 172.21.15.16:0/2575232915 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24140a9670 con 0x55ec0b1d04d0 2024-09-21T03:54:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.159+0000 7f240aa00640 1 -- 172.21.15.16:0/2575232915 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2414018ec0 con 0x55ec0b1d04d0 2024-09-21T03:54:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.159+0000 7f240aa00640 1 -- 172.21.15.16:0/2575232915 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24140a9810 con 0x55ec0b1d04d0 2024-09-21T03:54:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.167+0000 7f242b5f4740 1 -- 172.21.15.16:0/2575232915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1d04d0 msgr2=0x55ec0b599470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.167+0000 7f242b5f4740 1 --2- 172.21.15.16:0/2575232915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1d04d0 0x55ec0b599470 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f24140a9d20 tx=0x7f24140b5d70 comp rx=0 tx=0).stop 2024-09-21T03:54:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.168+0000 7f242b5f4740 1 -- 172.21.15.16:0/2575232915 shutdown_connections 2024-09-21T03:54:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.168+0000 7f242b5f4740 1 --2- 172.21.15.16:0/2575232915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1d04d0 0x55ec0b599470 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.168+0000 7f242b5f4740 1 --2- 172.21.15.16:0/2575232915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1c82f0 0x55ec0b598920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.168+0000 7f242b5f4740 1 --2- 172.21.15.16:0/2575232915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1f11b0 0x55ec0b5982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.168+0000 7f242b5f4740 1 -- 172.21.15.16:0/2575232915 >> 172.21.15.16:0/2575232915 conn(0x55ec0b1e5300 msgr2=0x55ec0b593d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.168+0000 7f242b5f4740 1 -- 172.21.15.16:0/2575232915 shutdown_connections 2024-09-21T03:54:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.168+0000 7f242b5f4740 1 -- 172.21.15.16:0/2575232915 wait complete. 2024-09-21T03:54:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.169+0000 7f242b5f4740 1 -- start start 2024-09-21T03:54:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.169+0000 7f242b5f4740 1 Processor -- start 2024-09-21T03:54:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.171+0000 7f242b5f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1c82f0 0x55ec0b5a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.172+0000 7f242b5f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1d04d0 0x55ec0b5a2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.172+0000 7f2421a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1c82f0 0x55ec0b5a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.172+0000 7f2421a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1c82f0 0x55ec0b5a19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34160/0 (socket says 172.21.15.16:34160) 2024-09-21T03:54:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.172+0000 7f2421a00640 1 -- 172.21.15.16:0/3921152751 learned_addr learned my addr 172.21.15.16:0/3921152751 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.172+0000 7f242b5f4740 1 --2- 172.21.15.16:0/3921152751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1f11b0 0x55ec0b5a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.173+0000 7f2422400640 1 --2- 172.21.15.16:0/3921152751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1f11b0 0x55ec0b5a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.173+0000 7f242b5f4740 1 -- 172.21.15.16:0/3921152751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ec0b432ab0 con 0x55ec0b1d04d0 2024-09-21T03:54:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.173+0000 7f2421000640 1 --2- 172.21.15.16:0/3921152751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1d04d0 0x55ec0b5a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.174+0000 7f242b5f4740 1 -- 172.21.15.16:0/3921152751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ec0b4a1570 con 0x55ec0b1f11b0 2024-09-21T03:54:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.174+0000 7f242b5f4740 1 -- 172.21.15.16:0/3921152751 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ec0b599f00 con 0x55ec0b1c82f0 2024-09-21T03:54:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.175+0000 7f2422400640 1 -- 172.21.15.16:0/3921152751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1c82f0 msgr2=0x55ec0b5a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.175+0000 7f2422400640 1 --2- 172.21.15.16:0/3921152751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1c82f0 0x55ec0b5a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.175+0000 7f2422400640 1 -- 172.21.15.16:0/3921152751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1d04d0 msgr2=0x55ec0b5a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.175+0000 7f2422400640 1 --2- 172.21.15.16:0/3921152751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1d04d0 0x55ec0b5a2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.175+0000 7f2422400640 1 -- 172.21.15.16:0/3921152751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec0b25c790 con 0x55ec0b1f11b0 2024-09-21T03:54:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.176+0000 7f2422400640 1 --2- 172.21.15.16:0/3921152751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1f11b0 0x55ec0b5a2bd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2414005d70 tx=0x7f24140a98a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fdcacb4822ca8c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.176+0000 7f23ff400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:54:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.176+0000 7f242b5f4740 10 client.0 _ll_register_callbacks cb 0x55ec0b4b0100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.177+0000 7f242b5f4740 1 -- 172.21.15.16:0/3921152751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2414018ec0 con 0x55ec0b1f11b0 2024-09-21T03:54:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.177+0000 7f23ff400640 1 -- 172.21.15.16:0/3921152751 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241401f030 con 0x55ec0b1f11b0 2024-09-21T03:54:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.177+0000 7f23ff400640 1 -- 172.21.15.16:0/3921152751 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f24140b6840 con 0x55ec0b1f11b0 2024-09-21T03:54:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.177+0000 7f23ff400640 1 -- 172.21.15.16:0/3921152751 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24140a99e0 con 0x55ec0b1f11b0 2024-09-21T03:54:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.178+0000 7f242b5f4740 10 client.26486 fetch_fsmap learned FSMap version 30 2024-09-21T03:54:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.178+0000 7f242b5f4740 1 -- 172.21.15.16:0/3921152751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55ec0b2d5620 con 0x55ec0b1f11b0 2024-09-21T03:54:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.179+0000 7f23ff400640 1 -- 172.21.15.16:0/3921152751 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f24140b4ce0 con 0x55ec0b1f11b0 2024-09-21T03:54:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.179+0000 7f242b5f4740 10 client.26486 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:54:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.179+0000 7f242b5f4740 10 client.26486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.179+0000 7f242b5f4740 20 client.26486 populate_metadata read hostname 'smithi016' 2024-09-21T03:54:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.179+0000 7f242b5f4740 1 --2- 172.21.15.16:0/3921152751 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55ec0b1afd80 0x55ec0b5ae970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.179+0000 7f242b5f4740 4 client.26486 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:54:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.179+0000 7f242b5f4740 1 -- 172.21.15.16:0/3921152751 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec0b50d600 con 0x55ec0b1afd80 2024-09-21T03:54:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.180+0000 7f2421a00640 1 --2- 172.21.15.16:0/3921152751 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55ec0b1afd80 0x55ec0b5ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.181+0000 7f2421a00640 1 --2- 172.21.15.16:0/3921152751 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55ec0b1afd80 0x55ec0b5ae970 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23582f3ed7ab4b1e server_cookie=df70d464d35089e1 in_seq=0 out_seq=0 2024-09-21T03:54:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.181+0000 7f23ff400640 10 client.26486 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.182+0000 7f23ff400640 1 -- 172.21.15.16:0/3921152751 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ec0b50d600 con 0x55ec0b1afd80 2024-09-21T03:54:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.182+0000 7f23ff400640 10 client.26486 handle_command_reply: tid=0 2024-09-21T03:54:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.182+0000 7f242b5f4740 1 client.26486 shutdown 2024-09-21T03:54:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.187+0000 7f242b5f4740 1 -- 172.21.15.16:0/3921152751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1f11b0 msgr2=0x55ec0b5a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.187+0000 7f242b5f4740 1 --2- 172.21.15.16:0/3921152751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1f11b0 0x55ec0b5a2bd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2414005d70 tx=0x7f24140a98a0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.187+0000 7f242b5f4740 1 -- 172.21.15.16:0/3921152751 shutdown_connections 2024-09-21T03:54:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.187+0000 7f242b5f4740 1 --2- 172.21.15.16:0/3921152751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1f11b0 0x55ec0b5a2bd0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.188+0000 7f242b5f4740 1 --2- 172.21.15.16:0/3921152751 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55ec0b1afd80 0x55ec0b5ae970 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.188+0000 7f242b5f4740 1 --2- 172.21.15.16:0/3921152751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1d04d0 0x55ec0b5a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.188+0000 7f242b5f4740 1 --2- 172.21.15.16:0/3921152751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1c82f0 0x55ec0b5a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.188+0000 7f23ff400640 0 client.26486 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.188+0000 7f242b5f4740 1 -- 172.21.15.16:0/3921152751 >> 172.21.15.16:0/3921152751 conn(0x55ec0b1e5300 msgr2=0x55ec0b592360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.188+0000 7f242b5f4740 1 -- 172.21.15.16:0/3921152751 shutdown_connections 2024-09-21T03:54:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.189+0000 7f242b5f4740 1 -- 172.21.15.16:0/3921152751 wait complete. 2024-09-21T03:54:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.189+0000 7f242b5f4740 20 client.26486 trim_cache size 0 max 16384 2024-09-21T03:54:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.227+0000 7f242b5f4740 1 Processor -- start 2024-09-21T03:54:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.229+0000 7f242b5f4740 1 -- start start 2024-09-21T03:54:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.229+0000 7f242b5f4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1c82f0 0x55ec0b58eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.229+0000 7f242b5f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1d04d0 0x55ec0b5a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.229+0000 7f242b5f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1f11b0 0x55ec0b592360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.229+0000 7f242b5f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f240000fc40 con 0x55ec0b1d04d0 2024-09-21T03:54:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.229+0000 7f242b5f4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2400011780 con 0x55ec0b1c82f0 2024-09-21T03:54:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.229+0000 7f242b5f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2400010310 con 0x55ec0b1f11b0 2024-09-21T03:54:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.230+0000 7f2421a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1c82f0 0x55ec0b58eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.230+0000 7f2422400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1f11b0 0x55ec0b592360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.230+0000 7f2422400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1f11b0 0x55ec0b592360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34172/0 (socket says 172.21.15.16:34172) 2024-09-21T03:54:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.230+0000 7f2422400640 1 -- 172.21.15.16:0/2465392044 learned_addr learned my addr 172.21.15.16:0/2465392044 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.231+0000 7f2421a00640 1 -- 172.21.15.16:0/2465392044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1f11b0 msgr2=0x55ec0b592360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.231+0000 7f2421a00640 1 --2- 172.21.15.16:0/2465392044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1f11b0 0x55ec0b592360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.231+0000 7f2421a00640 1 -- 172.21.15.16:0/2465392044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1d04d0 msgr2=0x55ec0b5a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.232+0000 7f2421a00640 1 --2- 172.21.15.16:0/2465392044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1d04d0 0x55ec0b5a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.232+0000 7f2421a00640 1 -- 172.21.15.16:0/2465392044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec0b59fbb0 con 0x55ec0b1c82f0 2024-09-21T03:54:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.232+0000 7f2422400640 1 --2- 172.21.15.16:0/2465392044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1f11b0 0x55ec0b592360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:54:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.232+0000 7f2421a00640 1 --2- 172.21.15.16:0/2465392044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1c82f0 0x55ec0b58eae0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55ec0b5aeed0 tx=0x7f240c013830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbf41abffe436d9b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.233+0000 7f240a000640 1 -- 172.21.15.16:0/2465392044 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c0130d0 con 0x55ec0b1c82f0 2024-09-21T03:54:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.233+0000 7f240a000640 1 -- 172.21.15.16:0/2465392044 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f240c013270 con 0x55ec0b1c82f0 2024-09-21T03:54:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.233+0000 7f240a000640 1 -- 172.21.15.16:0/2465392044 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c011440 con 0x55ec0b1c82f0 2024-09-21T03:54:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.236+0000 7f242b5f4740 1 -- 172.21.15.16:0/2465392044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1c82f0 msgr2=0x55ec0b58eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.236+0000 7f242b5f4740 1 --2- 172.21.15.16:0/2465392044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1c82f0 0x55ec0b58eae0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55ec0b5aeed0 tx=0x7f240c013830 comp rx=0 tx=0).stop 2024-09-21T03:54:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.236+0000 7f242b5f4740 1 -- 172.21.15.16:0/2465392044 shutdown_connections 2024-09-21T03:54:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.236+0000 7f242b5f4740 1 --2- 172.21.15.16:0/2465392044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1f11b0 0x55ec0b592360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.236+0000 7f242b5f4740 1 --2- 172.21.15.16:0/2465392044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1d04d0 0x55ec0b5a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.236+0000 7f242b5f4740 1 --2- 172.21.15.16:0/2465392044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1c82f0 0x55ec0b58eae0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.237+0000 7f242b5f4740 1 -- 172.21.15.16:0/2465392044 >> 172.21.15.16:0/2465392044 conn(0x55ec0b1e5300 msgr2=0x55ec0b59ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.237+0000 7f242b5f4740 1 -- 172.21.15.16:0/2465392044 shutdown_connections 2024-09-21T03:54:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.237+0000 7f242b5f4740 1 -- 172.21.15.16:0/2465392044 wait complete. 2024-09-21T03:54:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.237+0000 7f242b5f4740 1 -- start start 2024-09-21T03:54:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.238+0000 7f242b5f4740 1 Processor -- start 2024-09-21T03:54:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.239+0000 7f242b5f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1c82f0 0x55ec0b5a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.239+0000 7f242b5f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1d04d0 0x55ec0b5a8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.240+0000 7f2421a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1c82f0 0x55ec0b5a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.240+0000 7f2421a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1c82f0 0x55ec0b5a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33772/0 (socket says 172.21.15.16:33772) 2024-09-21T03:54:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.240+0000 7f242b5f4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1f11b0 0x55ec0b5a8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.240+0000 7f242b5f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24000110b0 con 0x55ec0b1c82f0 2024-09-21T03:54:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.240+0000 7f242b5f4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24000109e0 con 0x55ec0b1f11b0 2024-09-21T03:54:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.240+0000 7f242b5f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f240000dc20 con 0x55ec0b1d04d0 2024-09-21T03:54:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.240+0000 7f242b5f4740 10 client.0 _ll_register_callbacks cb 0x7f2410009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:54:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.241+0000 7f2421a00640 1 -- 172.21.15.16:0/3123416788 learned_addr learned my addr 172.21.15.16:0/3123416788 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:54:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.241+0000 7f2422400640 1 --2- 172.21.15.16:0/3123416788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1f11b0 0x55ec0b5a8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.242+0000 7f2421a00640 1 -- 172.21.15.16:0/3123416788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1d04d0 msgr2=0x55ec0b5a8180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:54:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.242+0000 7f2421a00640 1 --2- 172.21.15.16:0/3123416788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1d04d0 0x55ec0b5a8180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.242+0000 7f2421a00640 1 -- 172.21.15.16:0/3123416788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1f11b0 msgr2=0x55ec0b5a8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:54:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.242+0000 7f2421a00640 1 --2- 172.21.15.16:0/3123416788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1f11b0 0x55ec0b5a8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.242+0000 7f2421a00640 1 -- 172.21.15.16:0/3123416788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec0b5a6d20 con 0x55ec0b1c82f0 2024-09-21T03:54:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.242+0000 7f2422400640 1 --2- 172.21.15.16:0/3123416788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1f11b0 0x55ec0b5a8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:54:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.243+0000 7f23ff400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:54:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.243+0000 7f2421a00640 1 --2- 172.21.15.16:0/3123416788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1c82f0 0x55ec0b5a94a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f240c0130d0 tx=0x7f240c013100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9247d9238ce4505a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:54:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.244+0000 7f23ff400640 1 -- 172.21.15.16:0/3123416788 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c004880 con 0x55ec0b1c82f0 2024-09-21T03:54:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.244+0000 7f23ff400640 1 -- 172.21.15.16:0/3123416788 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f240c01f070 con 0x55ec0b1c82f0 2024-09-21T03:54:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.245+0000 7f242b5f4740 1 -- 172.21.15.16:0/3123416788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f240c013270 con 0x55ec0b1c82f0 2024-09-21T03:54:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.245+0000 7f23ff400640 1 -- 172.21.15.16:0/3123416788 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f240c004880 con 0x55ec0b1c82f0 2024-09-21T03:54:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.245+0000 7f242b5f4740 10 client.16257 fetch_fsmap learned FSMap version 30 2024-09-21T03:54:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.245+0000 7f242b5f4740 1 -- 172.21.15.16:0/3123416788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ec0b6194f0 con 0x55ec0b1c82f0 2024-09-21T03:54:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.246+0000 7f23ff400640 1 -- 172.21.15.16:0/3123416788 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f240c013270 con 0x55ec0b1c82f0 2024-09-21T03:54:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.247+0000 7f23ff400640 1 -- 172.21.15.16:0/3123416788 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f240c023650 con 0x55ec0b1c82f0 2024-09-21T03:54:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.247+0000 7f242b5f4740 10 client.16257 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:54:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.247+0000 7f242b5f4740 10 client.16257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:54:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.247+0000 7f242b5f4740 20 client.16257 populate_metadata read hostname 'smithi016' 2024-09-21T03:54:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.247+0000 7f242b5f4740 1 --2- 172.21.15.16:0/3123416788 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55ec0b1851c0 0x55ec0b61bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:54:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.247+0000 7f242b5f4740 4 client.16257 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:54:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.247+0000 7f242b5f4740 1 -- 172.21.15.16:0/3123416788 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ec0b58f1c0 con 0x55ec0b1851c0 2024-09-21T03:54:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.248+0000 7f2421000640 1 --2- 172.21.15.16:0/3123416788 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55ec0b1851c0 0x55ec0b61bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:54:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.252+0000 7f2421000640 1 --2- 172.21.15.16:0/3123416788 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55ec0b1851c0 0x55ec0b61bf00 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a86f5f1fadfb3e1 server_cookie=26785e33b7004025 in_seq=0 out_seq=0 2024-09-21T03:54:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.252+0000 7f23ff400640 10 client.16257 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.599+0000 7f23ff400640 1 -- 172.21.15.16:0/3123416788 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ec0b58f1c0 con 0x55ec0b1851c0 2024-09-21T03:54:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.599+0000 7f23ff400640 10 client.16257 handle_command_reply: tid=0 2024-09-21T03:54:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.600+0000 7f242b5f4740 1 client.16257 shutdown 2024-09-21T03:54:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.603+0000 7f242b5f4740 1 -- 172.21.15.16:0/3123416788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1c82f0 msgr2=0x55ec0b5a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.603+0000 7f242b5f4740 1 --2- 172.21.15.16:0/3123416788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1c82f0 0x55ec0b5a94a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f240c0130d0 tx=0x7f240c013100 comp rx=0 tx=0).stop 2024-09-21T03:54:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.604+0000 7f242b5f4740 1 -- 172.21.15.16:0/3123416788 shutdown_connections 2024-09-21T03:54:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.604+0000 7f242b5f4740 1 --2- 172.21.15.16:0/3123416788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec0b1f11b0 0x55ec0b5a8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.604+0000 7f242b5f4740 1 --2- 172.21.15.16:0/3123416788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0b1d04d0 0x55ec0b5a8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.605+0000 7f242b5f4740 1 --2- 172.21.15.16:0/3123416788 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55ec0b1851c0 0x55ec0b61bf00 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.605+0000 7f242b5f4740 1 --2- 172.21.15.16:0/3123416788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ec0b1c82f0 0x55ec0b5a94a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.605+0000 7f242b5f4740 1 -- 172.21.15.16:0/3123416788 >> 172.21.15.16:0/3123416788 conn(0x55ec0b1e5300 msgr2=0x55ec0b59caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.605+0000 7f23ff400640 0 client.16257 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:54:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.605+0000 7f242b5f4740 1 -- 172.21.15.16:0/3123416788 shutdown_connections 2024-09-21T03:54:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.605+0000 7f242b5f4740 1 -- 172.21.15.16:0/3123416788 wait complete. 2024-09-21T03:54:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.605+0000 7f242b5f4740 20 client.16257 trim_cache size 0 max 16384 2024-09-21T03:54:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:54:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:54:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:54:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:54:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.611+0000 7f2422e00640 1 -- 172.21.15.16:0/2812354068 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24040b6940 msgr2=0x7f24040b8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.611+0000 7f2422e00640 1 --2- 172.21.15.16:0/2812354068 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24040b6940 0x7f24040b8e50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2410000c00 tx=0x7f2410004510 comp rx=0 tx=0).stop 2024-09-21T03:54:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.611+0000 7f2422e00640 1 -- 172.21.15.16:0/2812354068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f241c10fc60 msgr2=0x7f241c10dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:54:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.611+0000 7f2422e00640 1 --2- 172.21.15.16:0/2812354068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f241c10fc60 0x7f241c10dbd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f241400e940 tx=0x7f241400ee00 comp rx=0 tx=0).stop 2024-09-21T03:54:56.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.611+0000 7f2422e00640 1 -- 172.21.15.16:0/2812354068 shutdown_connections 2024-09-21T03:54:56.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.611+0000 7f2422e00640 1 --2- 172.21.15.16:0/2812354068 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24040b6940 0x7f24040b8e50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.611+0000 7f2422e00640 1 --2- 172.21.15.16:0/2812354068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f241c10fc60 0x7f241c10dbd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.611+0000 7f2422e00640 1 --2- 172.21.15.16:0/2812354068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f241c0770f0 0x7f241c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.611+0000 7f2422e00640 1 --2- 172.21.15.16:0/2812354068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f241c075cf0 0x7f241c10c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:54:56.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.611+0000 7f2422e00640 1 -- 172.21.15.16:0/2812354068 >> 172.21.15.16:0/2812354068 conn(0x7f241c0fee20 msgr2=0x7f241c0ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:54:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.614+0000 7f2422e00640 1 -- 172.21.15.16:0/2812354068 shutdown_connections 2024-09-21T03:54:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:54:56.614+0000 7f2422e00640 1 -- 172.21.15.16:0/2812354068 wait complete. 2024-09-21T03:54:57.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:56 smithi134 ceph-mon[35031]: pgmap v677: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 221 MiB/s wr, 1.95k op/s 2024-09-21T03:54:57.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:56 smithi123 ceph-mon[34673]: pgmap v677: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 221 MiB/s wr, 1.95k op/s 2024-09-21T03:54:57.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:56 smithi016 ceph-mon[30375]: pgmap v677: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 221 MiB/s wr, 1.95k op/s 2024-09-21T03:54:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:54:58 smithi016 ceph-mon[30375]: pgmap v678: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 193 MiB/s wr, 1.88k op/s 2024-09-21T03:54:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:54:58 smithi134 ceph-mon[35031]: pgmap v678: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 193 MiB/s wr, 1.88k op/s 2024-09-21T03:54:59.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:54:58 smithi123 ceph-mon[34673]: pgmap v678: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 193 MiB/s wr, 1.88k op/s 2024-09-21T03:55:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:00 smithi134 ceph-mon[35031]: pgmap v679: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 170 MiB/s wr, 1.76k op/s 2024-09-21T03:55:01.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:00 smithi123 ceph-mon[34673]: pgmap v679: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 170 MiB/s wr, 1.76k op/s 2024-09-21T03:55:01.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:00 smithi016 ceph-mon[30375]: pgmap v679: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 170 MiB/s wr, 1.76k op/s 2024-09-21T03:55:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.452+0000 7f3c42a00640 1 -- 172.21.15.16:0/2286847769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c340a4e40 msgr2=0x7f3c340a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.452+0000 7f3c42a00640 1 --2- 172.21.15.16:0/2286847769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c340a4e40 0x7f3c340a5250 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009900 tx=0x7f3c3002a410 comp rx=0 tx=0).stop 2024-09-21T03:55:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.454+0000 7f3c42a00640 1 -- 172.21.15.16:0/2286847769 shutdown_connections 2024-09-21T03:55:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.454+0000 7f3c42a00640 1 --2- 172.21.15.16:0/2286847769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c340b7b60 0x7f3c340b9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.454+0000 7f3c42a00640 1 --2- 172.21.15.16:0/2286847769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c340a5790 0x7f3c340b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.454+0000 7f3c42a00640 1 --2- 172.21.15.16:0/2286847769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c340a4e40 0x7f3c340a5250 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.454+0000 7f3c42a00640 1 -- 172.21.15.16:0/2286847769 >> 172.21.15.16:0/2286847769 conn(0x7f3c3401cc20 msgr2=0x7f3c3401d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.454+0000 7f3c42a00640 1 -- 172.21.15.16:0/2286847769 shutdown_connections 2024-09-21T03:55:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.454+0000 7f3c42a00640 1 -- 172.21.15.16:0/2286847769 wait complete. 2024-09-21T03:55:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.456+0000 7f3c42a00640 1 Processor -- start 2024-09-21T03:55:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.456+0000 7f3c42a00640 1 -- start start 2024-09-21T03:55:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.456+0000 7f3c42a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c340a4e40 0x7f3c34144130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.457+0000 7f3c42a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c340a5790 0x7f3c341446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.457+0000 7f3c42a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c340b7b60 0x7f3c34149970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.457+0000 7f3c42a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c340bd680 con 0x7f3c340b7b60 2024-09-21T03:55:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.457+0000 7f3c42a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3c340bd500 con 0x7f3c340a4e40 2024-09-21T03:55:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.457+0000 7f3c42a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c340bd800 con 0x7f3c340a5790 2024-09-21T03:55:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.457+0000 7f3c40c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c340a5790 0x7f3c341446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.457+0000 7f3c40c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c340a5790 0x7f3c341446d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:34208/0 (socket says 172.21.15.16:34208) 2024-09-21T03:55:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.457+0000 7f3c40c00640 1 -- 172.21.15.16:0/1707308072 learned_addr learned my addr 172.21.15.16:0/1707308072 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.458+0000 7f3c42000640 1 --2- 172.21.15.16:0/1707308072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c340b7b60 0x7f3c34149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.459+0000 7f3c41600640 1 --2- 172.21.15.16:0/1707308072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c340a4e40 0x7f3c34144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.460+0000 7f3c42000640 1 -- 172.21.15.16:0/1707308072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c340a5790 msgr2=0x7f3c341446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.460+0000 7f3c42000640 1 --2- 172.21.15.16:0/1707308072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c340a5790 0x7f3c341446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.460+0000 7f3c42000640 1 -- 172.21.15.16:0/1707308072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c340a4e40 msgr2=0x7f3c34144130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.460+0000 7f3c42000640 1 --2- 172.21.15.16:0/1707308072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c340a4e40 0x7f3c34144130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.460+0000 7f3c42000640 1 -- 172.21.15.16:0/1707308072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3414a0a0 con 0x7f3c340b7b60 2024-09-21T03:55:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.461+0000 7f3c42000640 1 --2- 172.21.15.16:0/1707308072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c340b7b60 0x7f3c34149970 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c069e80 tx=0x7f3c3c06ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.462+0000 7f3c3a000640 1 -- 172.21.15.16:0/1707308072 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c07b070 con 0x7f3c340b7b60 2024-09-21T03:55:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.466+0000 7f3c3a000640 1 -- 172.21.15.16:0/1707308072 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c3c074e00 con 0x7f3c340b7b60 2024-09-21T03:55:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.466+0000 7f3c3a000640 1 -- 172.21.15.16:0/1707308072 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c077460 con 0x7f3c340b7b60 2024-09-21T03:55:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.466+0000 7f3c42a00640 1 -- 172.21.15.16:0/1707308072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3414a390 con 0x7f3c340b7b60 2024-09-21T03:55:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.466+0000 7f3c42a00640 1 -- 172.21.15.16:0/1707308072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c34150af0 con 0x7f3c340b7b60 2024-09-21T03:55:01.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.471+0000 7f3c2f400640 1 -- 172.21.15.16:0/1707308072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c340a61c0 con 0x7f3c340b7b60 2024-09-21T03:55:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.482+0000 7f3c3a000640 1 -- 172.21.15.16:0/1707308072 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f3c3c074950 con 0x7f3c340b7b60 2024-09-21T03:55:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.485+0000 7f3c3a000640 1 --2- 172.21.15.16:0/1707308072 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3c240b6590 0x7f3c240b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.485+0000 7f3c3a000640 1 -- 172.21.15.16:0/1707308072 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f3c3c14c5f0 con 0x7f3c340b7b60 2024-09-21T03:55:01.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.490+0000 7f3c41600640 1 --2- 172.21.15.16:0/1707308072 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3c240b6590 0x7f3c240b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:01.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.491+0000 7f3c41600640 1 --2- 172.21.15.16:0/1707308072 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3c240b6590 0x7f3c240b8aa0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3c30006010 tx=0x7f3c30005a20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:01.505+0000 7f3c3a000640 1 -- 172.21.15.16:0/1707308072 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3c3c085020 con 0x7f3c340b7b60 2024-09-21T03:55:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:02.080+0000 7f3c2f400640 1 -- 172.21.15.16:0/1707308072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c34145260 con 0x7f3c340b7b60 2024-09-21T03:55:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:02.081+0000 7f3c3a000640 1 -- 172.21.15.16:0/1707308072 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f3c3c112ad0 con 0x7f3c340b7b60 2024-09-21T03:55:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:55:02.084 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:55:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:55:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:02.085+0000 7f3c2f400640 1 -- 172.21.15.16:0/1707308072 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3c240b6590 msgr2=0x7f3c240b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:02.085+0000 7f3c2f400640 1 --2- 172.21.15.16:0/1707308072 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3c240b6590 0x7f3c240b8aa0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3c30006010 tx=0x7f3c30005a20 comp rx=0 tx=0).stop 2024-09-21T03:55:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:02.085+0000 7f3c2f400640 1 -- 172.21.15.16:0/1707308072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c340b7b60 msgr2=0x7f3c34149970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:02.085+0000 7f3c2f400640 1 --2- 172.21.15.16:0/1707308072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c340b7b60 0x7f3c34149970 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c069e80 tx=0x7f3c3c06ce70 comp rx=0 tx=0).stop 2024-09-21T03:55:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:02.085+0000 7f3c2f400640 1 -- 172.21.15.16:0/1707308072 shutdown_connections 2024-09-21T03:55:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:02.085+0000 7f3c2f400640 1 --2- 172.21.15.16:0/1707308072 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f3c240b6590 0x7f3c240b8aa0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:02.085+0000 7f3c2f400640 1 --2- 172.21.15.16:0/1707308072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c340b7b60 0x7f3c34149970 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:02.085+0000 7f3c2f400640 1 --2- 172.21.15.16:0/1707308072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c340a5790 0x7f3c341446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:02.085+0000 7f3c2f400640 1 --2- 172.21.15.16:0/1707308072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c340a4e40 0x7f3c34144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:02.085+0000 7f3c2f400640 1 -- 172.21.15.16:0/1707308072 >> 172.21.15.16:0/1707308072 conn(0x7f3c3401cc20 msgr2=0x7f3c340a7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:02.085+0000 7f3c2f400640 1 -- 172.21.15.16:0/1707308072 shutdown_connections 2024-09-21T03:55:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:02.085+0000 7f3c2f400640 1 -- 172.21.15.16:0/1707308072 wait complete. 2024-09-21T03:55:03.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:02 smithi134 ceph-mon[35031]: pgmap v680: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 178 MiB/s wr, 1.84k op/s 2024-09-21T03:55:03.396 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:02 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1707308072' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:55:03.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:02 smithi123 ceph-mon[34673]: pgmap v680: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 178 MiB/s wr, 1.84k op/s 2024-09-21T03:55:03.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:02 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1707308072' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:55:03.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:02 smithi016 ceph-mon[30375]: pgmap v680: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 178 MiB/s wr, 1.84k op/s 2024-09-21T03:55:03.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:02 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1707308072' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:55:05.334 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:04 smithi016 ceph-mon[30375]: pgmap v681: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 184 MiB/s wr, 1.86k op/s 2024-09-21T03:55:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:04 smithi134 ceph-mon[35031]: pgmap v681: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 184 MiB/s wr, 1.86k op/s 2024-09-21T03:55:05.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:04 smithi123 ceph-mon[34673]: pgmap v681: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 184 MiB/s wr, 1.86k op/s 2024-09-21T03:55:05.436 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0' 2024-09-21T03:55:05.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0 2024-09-21T03:55:06.758 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:06 smithi134 ceph-mon[35031]: pgmap v682: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 191 MiB/s wr, 1.82k op/s 2024-09-21T03:55:07.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:06 smithi123 ceph-mon[34673]: pgmap v682: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 191 MiB/s wr, 1.82k op/s 2024-09-21T03:55:07.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:06 smithi016 ceph-mon[30375]: pgmap v682: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 191 MiB/s wr, 1.82k op/s 2024-09-21T03:55:09.169 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:08 smithi016 ceph-mon[30375]: pgmap v683: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 199 MiB/s wr, 1.73k op/s 2024-09-21T03:55:09.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:08 smithi134 ceph-mon[35031]: pgmap v683: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 199 MiB/s wr, 1.73k op/s 2024-09-21T03:55:09.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:08 smithi123 ceph-mon[34673]: pgmap v683: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 199 MiB/s wr, 1.73k op/s 2024-09-21T03:55:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.148+0000 7f8863e00640 1 -- 172.21.15.16:0/3410950685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8864103e60 msgr2=0x7f886410a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.149+0000 7f8863e00640 1 --2- 172.21.15.16:0/3410950685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8864103e60 0x7f886410a800 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f885000d910 tx=0x7f885002c390 comp rx=0 tx=0).stop 2024-09-21T03:55:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.149+0000 7f8863e00640 1 -- 172.21.15.16:0/3410950685 shutdown_connections 2024-09-21T03:55:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.149+0000 7f8863e00640 1 --2- 172.21.15.16:0/3410950685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8864103e60 0x7f886410a800 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.149+0000 7f8863e00640 1 --2- 172.21.15.16:0/3410950685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88641034c0 0x7f88641038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.149+0000 7f8863e00640 1 --2- 172.21.15.16:0/3410950685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8864102440 0x7f88641028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.149+0000 7f8863e00640 1 -- 172.21.15.16:0/3410950685 >> 172.21.15.16:0/3410950685 conn(0x7f88640fdd80 msgr2=0x7f8864100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.149+0000 7f8863e00640 1 -- 172.21.15.16:0/3410950685 shutdown_connections 2024-09-21T03:55:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.149+0000 7f8863e00640 1 -- 172.21.15.16:0/3410950685 wait complete. 2024-09-21T03:55:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.151+0000 7f8863e00640 1 Processor -- start 2024-09-21T03:55:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.151+0000 7f8863e00640 1 -- start start 2024-09-21T03:55:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.152+0000 7f8863e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8864102440 0x7f8864197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.152+0000 7f8863e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88641034c0 0x7f8864198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.152+0000 7f8863e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8864103e60 0x7f8864198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.152+0000 7f8863e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f886410d000 con 0x7f8864102440 2024-09-21T03:55:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.152+0000 7f8863e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f886410ce80 con 0x7f8864103e60 2024-09-21T03:55:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.152+0000 7f8863e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f886410d180 con 0x7f88641034c0 2024-09-21T03:55:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.153+0000 7f8862000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88641034c0 0x7f8864198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.153+0000 7f8863400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8864103e60 0x7f8864198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.153+0000 7f8863400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8864103e60 0x7f8864198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:33668/0 (socket says 172.21.15.16:33668) 2024-09-21T03:55:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.153+0000 7f8863400640 1 -- 172.21.15.16:0/3017402806 learned_addr learned my addr 172.21.15.16:0/3017402806 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.153+0000 7f8863400640 1 -- 172.21.15.16:0/3017402806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88641034c0 msgr2=0x7f8864198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.153+0000 7f8863400640 1 --2- 172.21.15.16:0/3017402806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88641034c0 0x7f8864198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.153+0000 7f8863400640 1 -- 172.21.15.16:0/3017402806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8864102440 msgr2=0x7f8864197bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:55:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.153+0000 7f8863400640 1 --2- 172.21.15.16:0/3017402806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8864102440 0x7f8864197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.153+0000 7f8863400640 1 -- 172.21.15.16:0/3017402806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8864199610 con 0x7f8864103e60 2024-09-21T03:55:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.154+0000 7f8863400640 1 --2- 172.21.15.16:0/3017402806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8864103e60 0x7f8864198df0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8850004850 tx=0x7f8850034a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.154+0000 7f885b400640 1 -- 172.21.15.16:0/3017402806 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850034d30 con 0x7f8864103e60 2024-09-21T03:55:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.154+0000 7f885b400640 1 -- 172.21.15.16:0/3017402806 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88500035f0 con 0x7f8864103e60 2024-09-21T03:55:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.154+0000 7f885b400640 1 -- 172.21.15.16:0/3017402806 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885003c420 con 0x7f8864103e60 2024-09-21T03:55:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.154+0000 7f8863e00640 1 -- 172.21.15.16:0/3017402806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88641998a0 con 0x7f8864103e60 2024-09-21T03:55:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.154+0000 7f8863e00640 1 -- 172.21.15.16:0/3017402806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8864199de0 con 0x7f8864103e60 2024-09-21T03:55:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.161+0000 7f885b400640 1 -- 172.21.15.16:0/3017402806 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8850043030 con 0x7f8864103e60 2024-09-21T03:55:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.161+0000 7f8870a2f740 1 Processor -- start 2024-09-21T03:55:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.162+0000 7f8870a2f740 1 -- start start 2024-09-21T03:55:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.162+0000 7f8870a2f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1c1b1b0 0x5604c1fc22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.163+0000 7f885b400640 1 --2- 172.21.15.16:0/3017402806 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f88480b65d0 0x7f88480b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.163+0000 7f8870a2f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1bf22f0 0x5604c1fc2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.163+0000 7f8870a2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 0x5604c1fc3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.163+0000 7f8870a2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5604c1f356b0 con 0x5604c1bfa4d0 2024-09-21T03:55:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.163+0000 7f8870a2f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5604c1f24ae0 con 0x5604c1bf22f0 2024-09-21T03:55:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.163+0000 7f8870a2f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5604c1f288f0 con 0x5604c1c1b1b0 2024-09-21T03:55:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.163+0000 7f885b400640 1 -- 172.21.15.16:0/3017402806 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f8850107450 con 0x7f8864103e60 2024-09-21T03:55:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.163+0000 7f8862000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1c1b1b0 0x5604c1fc22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.163+0000 7f8862000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1c1b1b0 0x5604c1fc22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:44994/0 (socket says 172.21.15.16:44994) 2024-09-21T03:55:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.163+0000 7f8862000640 1 -- 172.21.15.16:0/2500070982 learned_addr learned my addr 172.21.15.16:0/2500070982 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.163+0000 7f8863400640 1 --2- 172.21.15.16:0/2500070982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 0x5604c1fc3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.163+0000 7f8862a00640 1 --2- 172.21.15.16:0/3017402806 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f88480b65d0 0x7f88480b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.163+0000 7f8862000640 1 --2- 172.21.15.16:0/2500070982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1bf22f0 0x5604c1fc2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.165+0000 7f8863400640 1 -- 172.21.15.16:0/2500070982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1c1b1b0 msgr2=0x5604c1fc22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.165+0000 7f8863400640 1 --2- 172.21.15.16:0/2500070982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1c1b1b0 0x5604c1fc22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.165+0000 7f8863400640 1 -- 172.21.15.16:0/2500070982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1bf22f0 msgr2=0x5604c1fc2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.165+0000 7f8863400640 1 --2- 172.21.15.16:0/2500070982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1bf22f0 0x5604c1fc2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.165+0000 7f8863400640 1 -- 172.21.15.16:0/2500070982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604c1ef1780 con 0x5604c1bfa4d0 2024-09-21T03:55:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.165+0000 7f8862000640 1 --2- 172.21.15.16:0/2500070982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1c1b1b0 0x5604c1fc22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:55:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.165+0000 7f8863400640 1 --2- 172.21.15.16:0/2500070982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 0x5604c1fc3470 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8850006010 tx=0x7f885002c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a70e7e54b4e8a26 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.165+0000 7f8862a00640 1 --2- 172.21.15.16:0/3017402806 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f88480b65d0 0x7f88480b8ae0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f885c00ab90 tx=0x7f885c009250 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.166+0000 7f884fe00640 1 -- 172.21.15.16:0/2500070982 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88500ac810 con 0x5604c1bfa4d0 2024-09-21T03:55:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.166+0000 7f884fe00640 1 -- 172.21.15.16:0/2500070982 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8850030020 con 0x5604c1bfa4d0 2024-09-21T03:55:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.166+0000 7f884fe00640 1 -- 172.21.15.16:0/2500070982 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850104c50 con 0x5604c1bfa4d0 2024-09-21T03:55:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.167+0000 7f8870a2f740 1 -- 172.21.15.16:0/2500070982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 msgr2=0x5604c1fc3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.167+0000 7f8870a2f740 1 --2- 172.21.15.16:0/2500070982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 0x5604c1fc3470 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8850006010 tx=0x7f885002c390 comp rx=0 tx=0).stop 2024-09-21T03:55:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.167+0000 7f8870a2f740 1 -- 172.21.15.16:0/2500070982 shutdown_connections 2024-09-21T03:55:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.167+0000 7f8870a2f740 1 --2- 172.21.15.16:0/2500070982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 0x5604c1fc3470 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.167+0000 7f8870a2f740 1 --2- 172.21.15.16:0/2500070982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1bf22f0 0x5604c1fc2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.167+0000 7f8870a2f740 1 --2- 172.21.15.16:0/2500070982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1c1b1b0 0x5604c1fc22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.167+0000 7f8870a2f740 1 -- 172.21.15.16:0/2500070982 >> 172.21.15.16:0/2500070982 conn(0x5604c1c0f300 msgr2=0x5604c1fbdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.167+0000 7f8870a2f740 1 -- 172.21.15.16:0/2500070982 shutdown_connections 2024-09-21T03:55:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.168+0000 7f8870a2f740 1 -- 172.21.15.16:0/2500070982 wait complete. 2024-09-21T03:55:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.168+0000 7f8870a2f740 1 -- start start 2024-09-21T03:55:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.168+0000 7f8870a2f740 1 Processor -- start 2024-09-21T03:55:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.169+0000 7f8870a2f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1bf22f0 0x5604c1fcb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.169+0000 7f8870a2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 0x5604c1fcbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.169+0000 7f8870a2f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1c1b1b0 0x5604c1fcc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.169+0000 7f8870a2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5604c1e5cab0 con 0x5604c1bfa4d0 2024-09-21T03:55:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.169+0000 7f8870a2f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5604c1ecb570 con 0x5604c1bf22f0 2024-09-21T03:55:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.169+0000 7f8870a2f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5604c1f356b0 con 0x5604c1c1b1b0 2024-09-21T03:55:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.169+0000 7f8870a2f740 10 client.0 _ll_register_callbacks cb 0x5604c1eda100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.169+0000 7f8862a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1bf22f0 0x5604c1fcb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.169+0000 7f8863400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1c1b1b0 0x5604c1fcc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.169+0000 7f8862a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1bf22f0 0x5604c1fcb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:33682/0 (socket says 172.21.15.16:33682) 2024-09-21T03:55:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.169+0000 7f8862a00640 1 -- 172.21.15.16:0/2615346509 learned_addr learned my addr 172.21.15.16:0/2615346509 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.169+0000 7f8862000640 1 --2- 172.21.15.16:0/2615346509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 0x5604c1fcbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.170+0000 7f8863400640 1 -- 172.21.15.16:0/2615346509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1bf22f0 msgr2=0x5604c1fcb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.170+0000 7f8863400640 1 --2- 172.21.15.16:0/2615346509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1bf22f0 0x5604c1fcb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.170+0000 7f8863400640 1 -- 172.21.15.16:0/2615346509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 msgr2=0x5604c1fcbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.170+0000 7f8863400640 1 --2- 172.21.15.16:0/2615346509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 0x5604c1fcbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.170+0000 7f8863400640 1 -- 172.21.15.16:0/2615346509 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604c1c86790 con 0x5604c1c1b1b0 2024-09-21T03:55:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.170+0000 7f8862a00640 1 --2- 172.21.15.16:0/2615346509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1bf22f0 0x5604c1fcb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:55:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.170+0000 7f8862000640 1 --2- 172.21.15.16:0/2615346509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 0x5604c1fcbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.170+0000 7f8863400640 1 --2- 172.21.15.16:0/2615346509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1c1b1b0 0x5604c1fcc8b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f88500d6db0 tx=0x7f88500ccd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=925945cb7c87b0fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.170+0000 7f884cc00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:55:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.170+0000 7f884cc00640 1 -- 172.21.15.16:0/2615346509 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88500c9440 con 0x5604c1c1b1b0 2024-09-21T03:55:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.170+0000 7f8870a2f740 1 -- 172.21.15.16:0/2615346509 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8850030020 con 0x5604c1c1b1b0 2024-09-21T03:55:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.170+0000 7f884cc00640 1 -- 172.21.15.16:0/2615346509 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f8850003a60 con 0x5604c1c1b1b0 2024-09-21T03:55:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.171+0000 7f884cc00640 1 -- 172.21.15.16:0/2615346509 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f88500c95e0 con 0x5604c1c1b1b0 2024-09-21T03:55:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.171+0000 7f8870a2f740 10 client.26434 fetch_fsmap learned FSMap version 30 2024-09-21T03:55:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.171+0000 7f8870a2f740 1 -- 172.21.15.16:0/2615346509 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5604c1cff620 con 0x5604c1c1b1b0 2024-09-21T03:55:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.171+0000 7f884cc00640 1 -- 172.21.15.16:0/2615346509 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8850030020 con 0x5604c1c1b1b0 2024-09-21T03:55:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.171+0000 7f884cc00640 1 -- 172.21.15.16:0/2615346509 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f88500c8b20 con 0x5604c1c1b1b0 2024-09-21T03:55:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.171+0000 7f8870a2f740 10 client.26434 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:55:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.171+0000 7f8870a2f740 10 client.26434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.171+0000 7f8870a2f740 20 client.26434 populate_metadata read hostname 'smithi016' 2024-09-21T03:55:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.171+0000 7f8870a2f740 1 --2- 172.21.15.16:0/2615346509 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5604c1bd9d80 0x5604c1fd8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.171+0000 7f8870a2f740 4 client.26434 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.171+0000 7f8870a2f740 1 -- 172.21.15.16:0/2615346509 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604c1f37600 con 0x5604c1bd9d80 2024-09-21T03:55:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.171+0000 7f8862a00640 1 --2- 172.21.15.16:0/2615346509 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5604c1bd9d80 0x5604c1fd8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.172+0000 7f8862a00640 1 --2- 172.21.15.16:0/2615346509 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5604c1bd9d80 0x5604c1fd8b30 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e18abedb31e9873 server_cookie=1fec1b308c2a5190 in_seq=0 out_seq=0 2024-09-21T03:55:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.172+0000 7f884cc00640 10 client.26434 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.173+0000 7f884cc00640 1 -- 172.21.15.16:0/2615346509 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5604c1f37600 con 0x5604c1bd9d80 2024-09-21T03:55:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.173+0000 7f884cc00640 10 client.26434 handle_command_reply: tid=0 2024-09-21T03:55:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.173+0000 7f8870a2f740 1 client.26434 shutdown 2024-09-21T03:55:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.173+0000 7f8870a2f740 1 -- 172.21.15.16:0/2615346509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1c1b1b0 msgr2=0x5604c1fcc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.173+0000 7f8870a2f740 1 --2- 172.21.15.16:0/2615346509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1c1b1b0 0x5604c1fcc8b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f88500d6db0 tx=0x7f88500ccd50 comp rx=0 tx=0).stop 2024-09-21T03:55:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.173+0000 7f8870a2f740 1 -- 172.21.15.16:0/2615346509 shutdown_connections 2024-09-21T03:55:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.173+0000 7f8870a2f740 1 --2- 172.21.15.16:0/2615346509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1c1b1b0 0x5604c1fcc8b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.173+0000 7f8870a2f740 1 --2- 172.21.15.16:0/2615346509 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5604c1bd9d80 0x5604c1fd8b30 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.173+0000 7f8870a2f740 1 --2- 172.21.15.16:0/2615346509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 0x5604c1fcbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.173+0000 7f8870a2f740 1 --2- 172.21.15.16:0/2615346509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1bf22f0 0x5604c1fcb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.173+0000 7f884cc00640 0 client.26434 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.173+0000 7f8870a2f740 1 -- 172.21.15.16:0/2615346509 >> 172.21.15.16:0/2615346509 conn(0x5604c1c0f300 msgr2=0x5604c1fbc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.173+0000 7f8870a2f740 1 -- 172.21.15.16:0/2615346509 shutdown_connections 2024-09-21T03:55:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.174+0000 7f8870a2f740 1 -- 172.21.15.16:0/2615346509 wait complete. 2024-09-21T03:55:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.174+0000 7f8870a2f740 20 client.26434 trim_cache size 0 max 16384 2024-09-21T03:55:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.229+0000 7f8870a2f740 1 Processor -- start 2024-09-21T03:55:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.229+0000 7f8870a2f740 1 -- start start 2024-09-21T03:55:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.229+0000 7f8870a2f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1bf22f0 0x5604c1fc2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.229+0000 7f8870a2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 0x5604c1fc25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.230+0000 7f8870a2f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1c1b1b0 0x5604c1fbe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.230+0000 7f8870a2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88641bb3c0 con 0x5604c1bfa4d0 2024-09-21T03:55:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.230+0000 7f8870a2f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f88641bcf00 con 0x5604c1c1b1b0 2024-09-21T03:55:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.230+0000 7f8870a2f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f88641bba90 con 0x5604c1bf22f0 2024-09-21T03:55:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.230+0000 7f8862000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 0x5604c1fc25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.230+0000 7f8862a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1bf22f0 0x5604c1fc2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.230+0000 7f8862a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1bf22f0 0x5604c1fc2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:45016/0 (socket says 172.21.15.16:45016) 2024-09-21T03:55:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.230+0000 7f8862000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 0x5604c1fc25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33150/0 (socket says 172.21.15.16:33150) 2024-09-21T03:55:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.230+0000 7f8862a00640 1 -- 172.21.15.16:0/2733621756 learned_addr learned my addr 172.21.15.16:0/2733621756 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.230+0000 7f8863400640 1 --2- 172.21.15.16:0/2733621756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1c1b1b0 0x5604c1fbe7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.230+0000 7f8862a00640 1 -- 172.21.15.16:0/2733621756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1c1b1b0 msgr2=0x5604c1fbe7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.231+0000 7f8862a00640 1 --2- 172.21.15.16:0/2733621756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1c1b1b0 0x5604c1fbe7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.231+0000 7f8862a00640 1 -- 172.21.15.16:0/2733621756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 msgr2=0x5604c1fc25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.231+0000 7f8862a00640 1 --2- 172.21.15.16:0/2733621756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 0x5604c1fc25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.231+0000 7f8862a00640 1 -- 172.21.15.16:0/2733621756 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604c1fc9890 con 0x5604c1bf22f0 2024-09-21T03:55:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.231+0000 7f8863400640 1 --2- 172.21.15.16:0/2733621756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1c1b1b0 0x5604c1fbe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.231+0000 7f8862000640 1 --2- 172.21.15.16:0/2733621756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 0x5604c1fc25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:55:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.231+0000 7f8862a00640 1 --2- 172.21.15.16:0/2733621756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1bf22f0 0x5604c1fc2060 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5604c1fd9210 tx=0x7f885c00f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf91e8da0a57f05b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.232+0000 7f884f400640 1 -- 172.21.15.16:0/2733621756 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c028070 con 0x5604c1bf22f0 2024-09-21T03:55:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.232+0000 7f884f400640 1 -- 172.21.15.16:0/2733621756 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f885c0198d0 con 0x5604c1bf22f0 2024-09-21T03:55:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.233+0000 7f884f400640 1 -- 172.21.15.16:0/2733621756 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c018870 con 0x5604c1bf22f0 2024-09-21T03:55:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.235+0000 7f8870a2f740 1 -- 172.21.15.16:0/2733621756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1bf22f0 msgr2=0x5604c1fc2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.235+0000 7f8870a2f740 1 --2- 172.21.15.16:0/2733621756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1bf22f0 0x5604c1fc2060 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5604c1fd9210 tx=0x7f885c00f000 comp rx=0 tx=0).stop 2024-09-21T03:55:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.236+0000 7f8870a2f740 1 -- 172.21.15.16:0/2733621756 shutdown_connections 2024-09-21T03:55:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.236+0000 7f8870a2f740 1 --2- 172.21.15.16:0/2733621756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1c1b1b0 0x5604c1fbe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.236+0000 7f8870a2f740 1 --2- 172.21.15.16:0/2733621756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bfa4d0 0x5604c1fc25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.236+0000 7f8870a2f740 1 --2- 172.21.15.16:0/2733621756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1bf22f0 0x5604c1fc2060 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.236+0000 7f8870a2f740 1 -- 172.21.15.16:0/2733621756 >> 172.21.15.16:0/2733621756 conn(0x5604c1c0f300 msgr2=0x5604c1fc9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.236+0000 7f8870a2f740 1 -- 172.21.15.16:0/2733621756 shutdown_connections 2024-09-21T03:55:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.236+0000 7f8870a2f740 1 -- 172.21.15.16:0/2733621756 wait complete. 2024-09-21T03:55:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.237+0000 7f8870a2f740 1 -- start start 2024-09-21T03:55:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.238+0000 7f8870a2f740 1 Processor -- start 2024-09-21T03:55:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.238+0000 7f8870a2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bf22f0 0x5604c1fd24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.239+0000 7f8870a2f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1bfa4d0 0x5604c1fd2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.239+0000 7f8870a2f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1c1b1b0 0x5604c1fd2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.239+0000 7f8862a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bf22f0 0x5604c1fd24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.239+0000 7f8862a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bf22f0 0x5604c1fd24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33160/0 (socket says 172.21.15.16:33160) 2024-09-21T03:55:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.239+0000 7f8870a2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88641bc830 con 0x5604c1bf22f0 2024-09-21T03:55:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.239+0000 7f8870a2f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f88641bc160 con 0x5604c1c1b1b0 2024-09-21T03:55:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.239+0000 7f8870a2f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f88641b93a0 con 0x5604c1bfa4d0 2024-09-21T03:55:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.239+0000 7f8862a00640 1 -- 172.21.15.16:0/360993234 learned_addr learned my addr 172.21.15.16:0/360993234 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.239+0000 7f8870a2f740 10 client.0 _ll_register_callbacks cb 0x7f885400b290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.240+0000 7f8862000640 1 --2- 172.21.15.16:0/360993234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1bfa4d0 0x5604c1fd2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.240+0000 7f8863400640 1 --2- 172.21.15.16:0/360993234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1c1b1b0 0x5604c1fd2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.240+0000 7f8862a00640 1 -- 172.21.15.16:0/360993234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1bfa4d0 msgr2=0x5604c1fd2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.240+0000 7f8862a00640 1 --2- 172.21.15.16:0/360993234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1bfa4d0 0x5604c1fd2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.240+0000 7f8862a00640 1 -- 172.21.15.16:0/360993234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1c1b1b0 msgr2=0x5604c1fd2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.240+0000 7f8862a00640 1 --2- 172.21.15.16:0/360993234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1c1b1b0 0x5604c1fd2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.240+0000 7f8862a00640 1 -- 172.21.15.16:0/360993234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604c1fd11b0 con 0x5604c1bf22f0 2024-09-21T03:55:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.240+0000 7f8862a00640 1 --2- 172.21.15.16:0/360993234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bf22f0 0x5604c1fd24e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f885c019ce0 tx=0x7f885c019d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df749404bf74ce25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.240+0000 7f884cc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:55:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.241+0000 7f884cc00640 1 -- 172.21.15.16:0/360993234 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c028070 con 0x5604c1bf22f0 2024-09-21T03:55:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.241+0000 7f884cc00640 1 -- 172.21.15.16:0/360993234 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f885c018800 con 0x5604c1bf22f0 2024-09-21T03:55:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.241+0000 7f8870a2f740 1 -- 172.21.15.16:0/360993234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f885c0198d0 con 0x5604c1bf22f0 2024-09-21T03:55:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.241+0000 7f884cc00640 1 -- 172.21.15.16:0/360993234 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f885c01e050 con 0x5604c1bf22f0 2024-09-21T03:55:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.241+0000 7f8870a2f740 10 client.16275 fetch_fsmap learned FSMap version 30 2024-09-21T03:55:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.241+0000 7f8870a2f740 1 -- 172.21.15.16:0/360993234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5604c2043770 con 0x5604c1bf22f0 2024-09-21T03:55:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.242+0000 7f884cc00640 1 -- 172.21.15.16:0/360993234 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f885c0198d0 con 0x5604c1bf22f0 2024-09-21T03:55:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.242+0000 7f884cc00640 1 -- 172.21.15.16:0/360993234 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f885c037020 con 0x5604c1bf22f0 2024-09-21T03:55:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.242+0000 7f8870a2f740 10 client.16275 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:55:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.242+0000 7f8870a2f740 10 client.16275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.242+0000 7f8870a2f740 20 client.16275 populate_metadata read hostname 'smithi016' 2024-09-21T03:55:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.242+0000 7f8870a2f740 1 --2- 172.21.15.16:0/360993234 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5604c1baf1c0 0x5604c2046170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.242+0000 7f8870a2f740 4 client.16275 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:55:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.242+0000 7f8870a2f740 1 -- 172.21.15.16:0/360993234 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5604c1f330b0 con 0x5604c1baf1c0 2024-09-21T03:55:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.242+0000 7f8862000640 1 --2- 172.21.15.16:0/360993234 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5604c1baf1c0 0x5604c2046170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.243+0000 7f8862000640 1 --2- 172.21.15.16:0/360993234 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5604c1baf1c0 0x5604c2046170 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5e32b2452c0edd0 server_cookie=89e9b20073a60ea7 in_seq=0 out_seq=0 2024-09-21T03:55:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.243+0000 7f884cc00640 10 client.16275 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.256+0000 7f884cc00640 1 -- 172.21.15.16:0/360993234 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5604c1f330b0 con 0x5604c1baf1c0 2024-09-21T03:55:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.256+0000 7f884cc00640 10 client.16275 handle_command_reply: tid=0 2024-09-21T03:55:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.257+0000 7f8870a2f740 1 client.16275 shutdown 2024-09-21T03:55:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.257+0000 7f8870a2f740 1 -- 172.21.15.16:0/360993234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bf22f0 msgr2=0x5604c1fd24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.257+0000 7f8870a2f740 1 --2- 172.21.15.16:0/360993234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bf22f0 0x5604c1fd24e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f885c019ce0 tx=0x7f885c019d10 comp rx=0 tx=0).stop 2024-09-21T03:55:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.257+0000 7f8870a2f740 1 -- 172.21.15.16:0/360993234 shutdown_connections 2024-09-21T03:55:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.257+0000 7f8870a2f740 1 --2- 172.21.15.16:0/360993234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c1c1b1b0 0x5604c1fd2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.257+0000 7f8870a2f740 1 --2- 172.21.15.16:0/360993234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c1bfa4d0 0x5604c1fd2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.257+0000 7f8870a2f740 1 --2- 172.21.15.16:0/360993234 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5604c1baf1c0 0x5604c2046170 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.257+0000 7f8870a2f740 1 --2- 172.21.15.16:0/360993234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604c1bf22f0 0x5604c1fd24e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.257+0000 7f884cc00640 0 client.16275 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.257+0000 7f8870a2f740 1 -- 172.21.15.16:0/360993234 >> 172.21.15.16:0/360993234 conn(0x5604c1c0f300 msgr2=0x5604c1fc1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.257+0000 7f8870a2f740 1 -- 172.21.15.16:0/360993234 shutdown_connections 2024-09-21T03:55:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.257+0000 7f8870a2f740 1 -- 172.21.15.16:0/360993234 wait complete. 2024-09-21T03:55:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.257+0000 7f8870a2f740 20 client.16275 trim_cache size 0 max 16384 2024-09-21T03:55:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:55:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:55:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:55:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:55:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.258+0000 7f8863e00640 1 -- 172.21.15.16:0/3017402806 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f88480b65d0 msgr2=0x7f88480b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.258+0000 7f8863e00640 1 --2- 172.21.15.16:0/3017402806 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f88480b65d0 0x7f88480b8ae0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f885c00ab90 tx=0x7f885c009250 comp rx=0 tx=0).stop 2024-09-21T03:55:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.258+0000 7f8863e00640 1 -- 172.21.15.16:0/3017402806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8864103e60 msgr2=0x7f8864198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.258+0000 7f8863e00640 1 --2- 172.21.15.16:0/3017402806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8864103e60 0x7f8864198df0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8850004850 tx=0x7f8850034a30 comp rx=0 tx=0).stop 2024-09-21T03:55:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.258+0000 7f8863e00640 1 -- 172.21.15.16:0/3017402806 shutdown_connections 2024-09-21T03:55:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.258+0000 7f8863e00640 1 --2- 172.21.15.16:0/3017402806 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f88480b65d0 0x7f88480b8ae0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.258+0000 7f8863e00640 1 --2- 172.21.15.16:0/3017402806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8864103e60 0x7f8864198df0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.258+0000 7f8863e00640 1 --2- 172.21.15.16:0/3017402806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88641034c0 0x7f8864198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.258+0000 7f8863e00640 1 --2- 172.21.15.16:0/3017402806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8864102440 0x7f8864197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.258+0000 7f8863e00640 1 -- 172.21.15.16:0/3017402806 >> 172.21.15.16:0/3017402806 conn(0x7f88640fdd80 msgr2=0x7f88641067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.259+0000 7f8863e00640 1 -- 172.21.15.16:0/3017402806 shutdown_connections 2024-09-21T03:55:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:10.259+0000 7f8863e00640 1 -- 172.21.15.16:0/3017402806 wait complete. 2024-09-21T03:55:10.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:10 smithi134 ceph-mon[35031]: pgmap v684: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 198 MiB/s wr, 1.59k op/s 2024-09-21T03:55:10.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:10 smithi123 ceph-mon[34673]: pgmap v684: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 198 MiB/s wr, 1.59k op/s 2024-09-21T03:55:10.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:10 smithi016 ceph-mon[30375]: pgmap v684: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 198 MiB/s wr, 1.59k op/s 2024-09-21T03:55:11.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.764+0000 7efc26000640 1 -- 172.21.15.16:0/2758598936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc20073460 msgr2=0x7efc20073c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:11.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.764+0000 7efc26000640 1 --2- 172.21.15.16:0/2758598936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc20073460 0x7efc20073c80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efc18009ca0 tx=0x7efc1802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.771+0000 7efc26000640 1 -- 172.21.15.16:0/2758598936 shutdown_connections 2024-09-21T03:55:11.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.771+0000 7efc26000640 1 --2- 172.21.15.16:0/2758598936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc20073460 0x7efc20073c80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.771+0000 7efc26000640 1 --2- 172.21.15.16:0/2758598936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc20072b10 0x7efc20072f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.771+0000 7efc26000640 1 --2- 172.21.15.16:0/2758598936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc2006d1d0 0x7efc2006d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.771+0000 7efc26000640 1 -- 172.21.15.16:0/2758598936 >> 172.21.15.16:0/2758598936 conn(0x7efc20079be0 msgr2=0x7efc2007a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:11.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.774+0000 7efc26000640 1 -- 172.21.15.16:0/2758598936 shutdown_connections 2024-09-21T03:55:11.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.778+0000 7efc26000640 1 -- 172.21.15.16:0/2758598936 wait complete. 2024-09-21T03:55:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.780+0000 7efc26000640 1 Processor -- start 2024-09-21T03:55:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.781+0000 7efc26000640 1 -- start start 2024-09-21T03:55:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.782+0000 7efc26000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2006d1d0 0x7efc20120e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.782+0000 7efc26000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc20072b10 0x7efc2011b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.782+0000 7efc26000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc20073460 0x7efc2011b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.782+0000 7efc26000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc20119590 con 0x7efc2006d1d0 2024-09-21T03:55:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.782+0000 7efc26000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc20119410 con 0x7efc20072b10 2024-09-21T03:55:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.782+0000 7efc26000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc20119710 con 0x7efc20073460 2024-09-21T03:55:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.783+0000 7efc1fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc20072b10 0x7efc2011b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.783+0000 7efc1fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc20072b10 0x7efc2011b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:46032/0 (socket says 172.21.15.16:46032) 2024-09-21T03:55:11.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.783+0000 7efc1fe00640 1 -- 172.21.15.16:0/3240716074 learned_addr learned my addr 172.21.15.16:0/3240716074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:11.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.784+0000 7efc25600640 1 --2- 172.21.15.16:0/3240716074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc20073460 0x7efc2011b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.784+0000 7efc24c00640 1 --2- 172.21.15.16:0/3240716074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2006d1d0 0x7efc20120e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.785+0000 7efc25600640 1 -- 172.21.15.16:0/3240716074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc20072b10 msgr2=0x7efc2011b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.785+0000 7efc25600640 1 --2- 172.21.15.16:0/3240716074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc20072b10 0x7efc2011b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.785+0000 7efc25600640 1 -- 172.21.15.16:0/3240716074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2006d1d0 msgr2=0x7efc20120e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.785+0000 7efc25600640 1 --2- 172.21.15.16:0/3240716074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2006d1d0 0x7efc20120e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.785+0000 7efc25600640 1 -- 172.21.15.16:0/3240716074 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2011bf00 con 0x7efc20073460 2024-09-21T03:55:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.786+0000 7efc25600640 1 --2- 172.21.15.16:0/3240716074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc20073460 0x7efc2011b6e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efc1802ade0 tx=0x7efc180319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.791+0000 7efc1d600640 1 -- 172.21.15.16:0/3240716074 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc18003b70 con 0x7efc20073460 2024-09-21T03:55:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.791+0000 7efc1d600640 1 -- 172.21.15.16:0/3240716074 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc18003d10 con 0x7efc20073460 2024-09-21T03:55:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.791+0000 7efc1d600640 1 -- 172.21.15.16:0/3240716074 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1802fa60 con 0x7efc20073460 2024-09-21T03:55:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.791+0000 7efc26000640 1 -- 172.21.15.16:0/3240716074 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc201bdbe0 con 0x7efc20073460 2024-09-21T03:55:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.791+0000 7efc26000640 1 -- 172.21.15.16:0/3240716074 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc201be150 con 0x7efc20073460 2024-09-21T03:55:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.802+0000 7efc1d600640 1 -- 172.21.15.16:0/3240716074 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7efc1802fc00 con 0x7efc20073460 2024-09-21T03:55:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.804+0000 7efc1d600640 1 --2- 172.21.15.16:0/3240716074 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efc040b6600 0x7efc040b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.804+0000 7efc1d600640 1 -- 172.21.15.16:0/3240716074 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7efc18108a50 con 0x7efc20073460 2024-09-21T03:55:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.804+0000 7efc24c00640 1 --2- 172.21.15.16:0/3240716074 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efc040b6600 0x7efc040b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.805+0000 7efc24c00640 1 --2- 172.21.15.16:0/3240716074 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efc040b6600 0x7efc040b8b10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7efc20120b60 tx=0x7efc0c005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:11.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.867+0000 7efc2e266740 1 Processor -- start 2024-09-21T03:55:11.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.870+0000 7efc2e266740 1 -- start start 2024-09-21T03:55:11.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.870+0000 7efc2e266740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f888850 0x56190fc2fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.870+0000 7efc2e266740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f85f990 0x56190fc300d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.870+0000 7efc2e266740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f867b70 0x56190fc30610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.870+0000 7efc2e266740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56190fba5a90 con 0x56190f85f990 2024-09-21T03:55:11.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.870+0000 7efc2e266740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56190fb94ec0 con 0x56190f888850 2024-09-21T03:55:11.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.870+0000 7efc2e266740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56190fb98cd0 con 0x56190f867b70 2024-09-21T03:55:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.871+0000 7efc25600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f867b70 0x56190fc30610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.872+0000 7efc1fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f85f990 0x56190fc300d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.872+0000 7efc1fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f85f990 0x56190fc300d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40214/0 (socket says 172.21.15.16:40214) 2024-09-21T03:55:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.872+0000 7efc1fe00640 1 -- 172.21.15.16:0/2622647291 learned_addr learned my addr 172.21.15.16:0/2622647291 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.873+0000 7efc25600640 1 -- 172.21.15.16:0/2622647291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f888850 msgr2=0x56190fc2fb90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:55:11.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.876+0000 7efc24c00640 1 --2- 172.21.15.16:0/2622647291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f888850 0x56190fc2fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.876+0000 7efc25600640 1 --2- 172.21.15.16:0/2622647291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f888850 0x56190fc2fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.876+0000 7efc25600640 1 -- 172.21.15.16:0/2622647291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f85f990 msgr2=0x56190fc300d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.876+0000 7efc25600640 1 --2- 172.21.15.16:0/2622647291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f85f990 0x56190fc300d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.876+0000 7efc25600640 1 -- 172.21.15.16:0/2622647291 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56190fb3a9d0 con 0x56190f867b70 2024-09-21T03:55:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.877+0000 7efc1fe00640 1 --2- 172.21.15.16:0/2622647291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f85f990 0x56190fc300d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:55:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.877+0000 7efc24c00640 1 --2- 172.21.15.16:0/2622647291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f888850 0x56190fc2fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.879+0000 7efc25600640 1 --2- 172.21.15.16:0/2622647291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f867b70 0x56190fc30610 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efc18009ca0 tx=0x7efc18004900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2714d904f74c59f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.879+0000 7efc15c00640 1 -- 172.21.15.16:0/2622647291 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc180cdf40 con 0x56190f867b70 2024-09-21T03:55:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.879+0000 7efc15c00640 1 -- 172.21.15.16:0/2622647291 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc18036030 con 0x56190f867b70 2024-09-21T03:55:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.879+0000 7efc15c00640 1 -- 172.21.15.16:0/2622647291 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1800a2f0 con 0x56190f867b70 2024-09-21T03:55:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.881+0000 7efc2e266740 1 -- 172.21.15.16:0/2622647291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f867b70 msgr2=0x56190fc30610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.881+0000 7efc2e266740 1 --2- 172.21.15.16:0/2622647291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f867b70 0x56190fc30610 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efc18009ca0 tx=0x7efc18004900 comp rx=0 tx=0).stop 2024-09-21T03:55:11.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.884+0000 7efc2e266740 1 -- 172.21.15.16:0/2622647291 shutdown_connections 2024-09-21T03:55:11.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.884+0000 7efc2e266740 1 --2- 172.21.15.16:0/2622647291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f867b70 0x56190fc30610 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.884+0000 7efc2e266740 1 --2- 172.21.15.16:0/2622647291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f85f990 0x56190fc300d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.884+0000 7efc2e266740 1 --2- 172.21.15.16:0/2622647291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f888850 0x56190fc2fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.884+0000 7efc2e266740 1 -- 172.21.15.16:0/2622647291 >> 172.21.15.16:0/2622647291 conn(0x56190f87cc40 msgr2=0x56190fc2da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:11.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.888+0000 7efc2e266740 1 -- 172.21.15.16:0/2622647291 shutdown_connections 2024-09-21T03:55:11.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.890+0000 7efc2e266740 1 -- 172.21.15.16:0/2622647291 wait complete. 2024-09-21T03:55:11.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.890+0000 7efc2e266740 1 -- start start 2024-09-21T03:55:11.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.891+0000 7efc2e266740 1 Processor -- start 2024-09-21T03:55:11.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.892+0000 7efc2e266740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f85f990 0x56190fc3b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.893+0000 7efc2e266740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f867b70 0x56190fc3bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.893+0000 7efc2e266740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f888850 0x56190fc47210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.894+0000 7efc24c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f85f990 0x56190fc3b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.896+0000 7efc2e266740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56190f9f67b0 con 0x56190f867b70 2024-09-21T03:55:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.896+0000 7efc25600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f888850 0x56190fc47210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.894+0000 7efc24c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f85f990 0x56190fc3b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:42040/0 (socket says 172.21.15.16:42040) 2024-09-21T03:55:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.896+0000 7efc24c00640 1 -- 172.21.15.16:0/1640533295 learned_addr learned my addr 172.21.15.16:0/1640533295 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.896+0000 7efc2e266740 1 -- 172.21.15.16:0/1640533295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56190fb39650 con 0x56190f888850 2024-09-21T03:55:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.896+0000 7efc2e266740 1 -- 172.21.15.16:0/1640533295 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56190fb98cd0 con 0x56190f85f990 2024-09-21T03:55:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.896+0000 7efc1fe00640 1 --2- 172.21.15.16:0/1640533295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f867b70 0x56190fc3bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.897+0000 7efc2e266740 10 client.0 _ll_register_callbacks cb 0x56190f8a00a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.897+0000 7efc24c00640 1 -- 172.21.15.16:0/1640533295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f888850 msgr2=0x56190fc47210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:11.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.897+0000 7efc24c00640 1 --2- 172.21.15.16:0/1640533295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f888850 0x56190fc47210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.897+0000 7efc24c00640 1 -- 172.21.15.16:0/1640533295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f867b70 msgr2=0x56190fc3bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:11.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.897+0000 7efc24c00640 1 --2- 172.21.15.16:0/1640533295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f867b70 0x56190fc3bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.897+0000 7efc24c00640 1 -- 172.21.15.16:0/1640533295 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56190f902ba0 con 0x56190f85f990 2024-09-21T03:55:11.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.898+0000 7efc24c00640 1 --2- 172.21.15.16:0/1640533295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f85f990 0x56190fc3b770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efc0c012920 tx=0x7efc0c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b30c75092188ec8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:11.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.898+0000 7efc0a000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:55:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.899+0000 7efc0a000640 1 -- 172.21.15.16:0/1640533295 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c01ecc0 con 0x56190f85f990 2024-09-21T03:55:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.899+0000 7efc0a000640 1 -- 172.21.15.16:0/1640533295 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7efc0c01f420 con 0x56190f85f990 2024-09-21T03:55:11.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.901+0000 7efc2e266740 1 -- 172.21.15.16:0/1640533295 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc18036030 con 0x56190f85f990 2024-09-21T03:55:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.901+0000 7efc1fe00640 1 --2- 172.21.15.16:0/1640533295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f867b70 0x56190fc3bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.901+0000 7efc0a000640 1 -- 172.21.15.16:0/1640533295 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efc0c032050 con 0x56190f85f990 2024-09-21T03:55:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.902+0000 7efc2e266740 10 client.26452 fetch_fsmap learned FSMap version 30 2024-09-21T03:55:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.902+0000 7efc2e266740 1 -- 172.21.15.16:0/1640533295 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56190fc49f30 con 0x56190f85f990 2024-09-21T03:55:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.902+0000 7efc0a000640 1 -- 172.21.15.16:0/1640533295 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc0c01ee60 con 0x56190f85f990 2024-09-21T03:55:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.902+0000 7efc0a000640 1 -- 172.21.15.16:0/1640533295 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efc0c01c700 con 0x56190f85f990 2024-09-21T03:55:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.903+0000 7efc2e266740 10 client.26452 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:55:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.903+0000 7efc2e266740 10 client.26452 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:55:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.903+0000 7efc2e266740 20 client.26452 populate_metadata read hostname 'smithi016' 2024-09-21T03:55:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.903+0000 7efc2e266740 1 --2- 172.21.15.16:0/1640533295 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56190f8471d0 0x56190fc4d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.903+0000 7efc2e266740 4 client.26452 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.903+0000 7efc2e266740 1 -- 172.21.15.16:0/1640533295 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efc18036030 con 0x56190f8471d0 2024-09-21T03:55:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.904+0000 7efc1fe00640 1 --2- 172.21.15.16:0/1640533295 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56190f8471d0 0x56190fc4d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.908+0000 7efc1fe00640 1 --2- 172.21.15.16:0/1640533295 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56190f8471d0 0x56190fc4d4d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12845b699509bda4 server_cookie=d3c3c1396279b0d4 in_seq=0 out_seq=0 2024-09-21T03:55:11.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.908+0000 7efc0a000640 10 client.26452 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.909+0000 7efc0a000640 1 -- 172.21.15.16:0/1640533295 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7efc18036030 con 0x56190f8471d0 2024-09-21T03:55:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.909+0000 7efc0a000640 10 client.26452 handle_command_reply: tid=0 2024-09-21T03:55:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.910+0000 7efc2e266740 1 client.26452 shutdown 2024-09-21T03:55:11.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.914+0000 7efc2e266740 1 -- 172.21.15.16:0/1640533295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f85f990 msgr2=0x56190fc3b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:11.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.914+0000 7efc2e266740 1 --2- 172.21.15.16:0/1640533295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f85f990 0x56190fc3b770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efc0c012920 tx=0x7efc0c004730 comp rx=0 tx=0).stop 2024-09-21T03:55:11.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.914+0000 7efc2e266740 1 -- 172.21.15.16:0/1640533295 shutdown_connections 2024-09-21T03:55:11.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.914+0000 7efc2e266740 1 --2- 172.21.15.16:0/1640533295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f888850 0x56190fc47210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.914+0000 7efc2e266740 1 --2- 172.21.15.16:0/1640533295 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56190f8471d0 0x56190fc4d4d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.914+0000 7efc2e266740 1 --2- 172.21.15.16:0/1640533295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f867b70 0x56190fc3bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.914+0000 7efc2e266740 1 --2- 172.21.15.16:0/1640533295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f85f990 0x56190fc3b770 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.914+0000 7efc2e266740 1 -- 172.21.15.16:0/1640533295 >> 172.21.15.16:0/1640533295 conn(0x56190f87cc40 msgr2=0x56190fc32d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:11.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.914+0000 7efc0a000640 0 client.26452 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:11.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.915+0000 7efc2e266740 1 -- 172.21.15.16:0/1640533295 shutdown_connections 2024-09-21T03:55:11.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.918+0000 7efc2e266740 1 -- 172.21.15.16:0/1640533295 wait complete. 2024-09-21T03:55:11.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.918+0000 7efc2e266740 20 client.26452 trim_cache size 0 max 16384 2024-09-21T03:55:11.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.965+0000 7efc2e266740 1 Processor -- start 2024-09-21T03:55:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.965+0000 7efc2e266740 1 -- start start 2024-09-21T03:55:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.965+0000 7efc2e266740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f85f990 0x56190fc2d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.965+0000 7efc2e266740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f867b70 0x56190fc2d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.965+0000 7efc2e266740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f888850 0x56190fc38760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.965+0000 7efc2e266740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc201c81f0 con 0x56190f888850 2024-09-21T03:55:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.965+0000 7efc2e266740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc201c9d30 con 0x56190f867b70 2024-09-21T03:55:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.965+0000 7efc2e266740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc201c88c0 con 0x56190f85f990 2024-09-21T03:55:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.966+0000 7efc24c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f85f990 0x56190fc2d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.966+0000 7efc24c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f85f990 0x56190fc2d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:42052/0 (socket says 172.21.15.16:42052) 2024-09-21T03:55:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.966+0000 7efc24c00640 1 -- 172.21.15.16:0/1445983485 learned_addr learned my addr 172.21.15.16:0/1445983485 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.967+0000 7efc25600640 1 --2- 172.21.15.16:0/1445983485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f888850 0x56190fc38760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.967+0000 7efc24c00640 1 -- 172.21.15.16:0/1445983485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f867b70 msgr2=0x56190fc2d5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:55:11.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.967+0000 7efc24c00640 1 --2- 172.21.15.16:0/1445983485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f867b70 0x56190fc2d5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.967+0000 7efc24c00640 1 -- 172.21.15.16:0/1445983485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f888850 msgr2=0x56190fc38760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:11.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.967+0000 7efc24c00640 1 --2- 172.21.15.16:0/1445983485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f888850 0x56190fc38760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.967+0000 7efc24c00640 1 -- 172.21.15.16:0/1445983485 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56190fc3a4d0 con 0x56190f85f990 2024-09-21T03:55:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.968+0000 7efc24c00640 1 --2- 172.21.15.16:0/1445983485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f85f990 0x56190fc2d070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efc0c01cb50 tx=0x7efc0c0273f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ab5f5b77557a33f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.968+0000 7efc15200640 1 -- 172.21.15.16:0/1445983485 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c03c020 con 0x56190f85f990 2024-09-21T03:55:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.968+0000 7efc15200640 1 -- 172.21.15.16:0/1445983485 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc0c01c3e0 con 0x56190f85f990 2024-09-21T03:55:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.968+0000 7efc15200640 1 -- 172.21.15.16:0/1445983485 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c0276f0 con 0x56190f85f990 2024-09-21T03:55:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.970+0000 7efc2e266740 1 -- 172.21.15.16:0/1445983485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f85f990 msgr2=0x56190fc2d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.970+0000 7efc2e266740 1 --2- 172.21.15.16:0/1445983485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f85f990 0x56190fc2d070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efc0c01cb50 tx=0x7efc0c0273f0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.970+0000 7efc2e266740 1 -- 172.21.15.16:0/1445983485 shutdown_connections 2024-09-21T03:55:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.970+0000 7efc2e266740 1 --2- 172.21.15.16:0/1445983485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f888850 0x56190fc38760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.970+0000 7efc2e266740 1 --2- 172.21.15.16:0/1445983485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f867b70 0x56190fc2d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.970+0000 7efc2e266740 1 --2- 172.21.15.16:0/1445983485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f85f990 0x56190fc2d070 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.970+0000 7efc2e266740 1 -- 172.21.15.16:0/1445983485 >> 172.21.15.16:0/1445983485 conn(0x56190f87cc40 msgr2=0x56190fc36720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.971+0000 7efc2e266740 1 -- 172.21.15.16:0/1445983485 shutdown_connections 2024-09-21T03:55:11.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.971+0000 7efc2e266740 1 -- 172.21.15.16:0/1445983485 wait complete. 2024-09-21T03:55:11.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.972+0000 7efc2e266740 1 -- start start 2024-09-21T03:55:11.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.973+0000 7efc2e266740 1 Processor -- start 2024-09-21T03:55:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.974+0000 7efc2e266740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f85f990 0x56190fc43a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.974+0000 7efc2e266740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f867b70 0x56190fc3fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.975+0000 7efc2e266740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f888850 0x56190fc404d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.975+0000 7efc2e266740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc201c9660 con 0x56190f85f990 2024-09-21T03:55:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.975+0000 7efc2e266740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc201c8f90 con 0x56190f888850 2024-09-21T03:55:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.975+0000 7efc1fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f867b70 0x56190fc3fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.976+0000 7efc2e266740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc201c61d0 con 0x56190f867b70 2024-09-21T03:55:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.976+0000 7efc2e266740 10 client.0 _ll_register_callbacks cb 0x7efc0c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.976+0000 7efc1fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f867b70 0x56190fc3fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:42068/0 (socket says 172.21.15.16:42068) 2024-09-21T03:55:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.977+0000 7efc1fe00640 1 -- 172.21.15.16:0/1633082832 learned_addr learned my addr 172.21.15.16:0/1633082832 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.977+0000 7efc24c00640 1 --2- 172.21.15.16:0/1633082832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f85f990 0x56190fc43a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.978+0000 7efc25600640 1 --2- 172.21.15.16:0/1633082832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f888850 0x56190fc404d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.978+0000 7efc1fe00640 1 -- 172.21.15.16:0/1633082832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f888850 msgr2=0x56190fc404d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.978+0000 7efc1fe00640 1 --2- 172.21.15.16:0/1633082832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f888850 0x56190fc404d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.978+0000 7efc1fe00640 1 -- 172.21.15.16:0/1633082832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f85f990 msgr2=0x56190fc43a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.978+0000 7efc1fe00640 1 --2- 172.21.15.16:0/1633082832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f85f990 0x56190fc43a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.979+0000 7efc1fe00640 1 -- 172.21.15.16:0/1633082832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56190fc40cb0 con 0x56190f867b70 2024-09-21T03:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.979+0000 7efc1fe00640 1 --2- 172.21.15.16:0/1633082832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f867b70 0x56190fc3fed0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efc10000c00 tx=0x7efc100178d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74ed7cd15387093e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.979+0000 7efc0a000640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.980+0000 7efc0a000640 1 -- 172.21.15.16:0/1633082832 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10015690 con 0x56190f867b70 2024-09-21T03:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.980+0000 7efc0a000640 1 -- 172.21.15.16:0/1633082832 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7efc1000f040 con 0x56190f867b70 2024-09-21T03:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.980+0000 7efc24c00640 1 --2- 172.21.15.16:0/1633082832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f85f990 0x56190fc43a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.981+0000 7efc2e266740 1 -- 172.21.15.16:0/1633082832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc0c01c3e0 con 0x56190f867b70 2024-09-21T03:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.981+0000 7efc0a000640 1 -- 172.21.15.16:0/1633082832 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efc10015830 con 0x56190f867b70 2024-09-21T03:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.982+0000 7efc2e266740 10 client.26458 fetch_fsmap learned FSMap version 30 2024-09-21T03:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.982+0000 7efc2e266740 1 -- 172.21.15.16:0/1633082832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56190fcb8f00 con 0x56190f867b70 2024-09-21T03:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.983+0000 7efc0a000640 1 -- 172.21.15.16:0/1633082832 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc0c01c3e0 con 0x56190f867b70 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.983+0000 7efc0a000640 1 -- 172.21.15.16:0/1633082832 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efc10030a30 con 0x56190f867b70 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.985+0000 7efc2e266740 10 client.26458 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.985+0000 7efc2e266740 10 client.26458 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.985+0000 7efc2e266740 20 client.26458 populate_metadata read hostname 'smithi016' 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.985+0000 7efc2e266740 1 --2- 172.21.15.16:0/1633082832 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56190f81c580 0x56190fcbb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.985+0000 7efc2e266740 4 client.26458 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}] 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.985+0000 7efc2e266740 1 -- 172.21.15.16:0/1633082832 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}) -- 0x56190fba79e0 con 0x56190f81c580 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.986+0000 7efc24c00640 1 --2- 172.21.15.16:0/1633082832 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56190f81c580 0x56190fcbb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.987+0000 7efc24c00640 1 --2- 172.21.15.16:0/1633082832 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56190f81c580 0x56190fcbb6a0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2deddb337ded69cc server_cookie=3a74fcbd4326c68a in_seq=0 out_seq=0 2024-09-21T03:55:11.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:11.987+0000 7efc0a000640 10 client.26458 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.130+0000 7efc0a000640 1 -- 172.21.15.16:0/1633082832 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56190fba79e0 con 0x56190f81c580 2024-09-21T03:55:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.130+0000 7efc0a000640 10 client.26458 handle_command_reply: tid=0 2024-09-21T03:55:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.130+0000 7efc2e266740 1 client.26458 shutdown 2024-09-21T03:55:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.130+0000 7efc2e266740 1 -- 172.21.15.16:0/1633082832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f867b70 msgr2=0x56190fc3fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.130+0000 7efc2e266740 1 --2- 172.21.15.16:0/1633082832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f867b70 0x56190fc3fed0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efc10000c00 tx=0x7efc100178d0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.130+0000 7efc2e266740 1 -- 172.21.15.16:0/1633082832 shutdown_connections 2024-09-21T03:55:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.130+0000 7efc2e266740 1 --2- 172.21.15.16:0/1633082832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56190f888850 0x56190fc404d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.130+0000 7efc2e266740 1 --2- 172.21.15.16:0/1633082832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56190f867b70 0x56190fc3fed0 secure :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efc10000c00 tx=0x7efc100178d0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.130+0000 7efc2e266740 1 --2- 172.21.15.16:0/1633082832 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56190f81c580 0x56190fcbb6a0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.130+0000 7efc2e266740 1 --2- 172.21.15.16:0/1633082832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56190f85f990 0x56190fc43a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.130+0000 7efc2e266740 1 -- 172.21.15.16:0/1633082832 >> 172.21.15.16:0/1633082832 conn(0x56190f87cc40 msgr2=0x56190fc30bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.130+0000 7efc0a000640 0 client.26458 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.131+0000 7efc2e266740 1 -- 172.21.15.16:0/1633082832 shutdown_connections 2024-09-21T03:55:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.131+0000 7efc2e266740 1 -- 172.21.15.16:0/1633082832 wait complete. 2024-09-21T03:55:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.131+0000 7efc2e266740 20 client.26458 trim_cache size 0 max 16384 2024-09-21T03:55:12.138 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:55:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:55:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:55:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 34, 2024-09-21T03:55:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:55:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "74ddd6a7": { 2024-09-21T03:55:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 34, 2024-09-21T03:55:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:55:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:55:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:55:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:55:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:55:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:55:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 141.1, 2024-09-21T03:55:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:55:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:55:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:55:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:55:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:55:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:55:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:55:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:55:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:55:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:55:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:55:12.140 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:55:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.133+0000 7efc26000640 1 -- 172.21.15.16:0/3240716074 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efc040b6600 msgr2=0x7efc040b8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.133+0000 7efc26000640 1 --2- 172.21.15.16:0/3240716074 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efc040b6600 0x7efc040b8b10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7efc20120b60 tx=0x7efc0c005f90 comp rx=0 tx=0).stop 2024-09-21T03:55:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.133+0000 7efc26000640 1 -- 172.21.15.16:0/3240716074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc20073460 msgr2=0x7efc2011b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.133+0000 7efc26000640 1 --2- 172.21.15.16:0/3240716074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc20073460 0x7efc2011b6e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efc1802ade0 tx=0x7efc180319f0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.133+0000 7efc26000640 1 -- 172.21.15.16:0/3240716074 shutdown_connections 2024-09-21T03:55:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.133+0000 7efc26000640 1 --2- 172.21.15.16:0/3240716074 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efc040b6600 0x7efc040b8b10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.133+0000 7efc26000640 1 --2- 172.21.15.16:0/3240716074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc20073460 0x7efc2011b6e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.133+0000 7efc26000640 1 --2- 172.21.15.16:0/3240716074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc20072b10 0x7efc2011b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.133+0000 7efc26000640 1 --2- 172.21.15.16:0/3240716074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2006d1d0 0x7efc20120e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.133+0000 7efc26000640 1 -- 172.21.15.16:0/3240716074 >> 172.21.15.16:0/3240716074 conn(0x7efc20079be0 msgr2=0x7efc20100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.136+0000 7efc26000640 1 -- 172.21.15.16:0/3240716074 shutdown_connections 2024-09-21T03:55:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:12.136+0000 7efc26000640 1 -- 172.21.15.16:0/3240716074 wait complete. 2024-09-21T03:55:13.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:12 smithi134 ceph-mon[35031]: pgmap v685: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 213 MiB/s wr, 1.60k op/s 2024-09-21T03:55:13.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:12 smithi123 ceph-mon[34673]: pgmap v685: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 213 MiB/s wr, 1.60k op/s 2024-09-21T03:55:13.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:12 smithi016 ceph-mon[30375]: pgmap v685: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 213 MiB/s wr, 1.60k op/s 2024-09-21T03:55:14.591 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '74ddd6a7', quiesce took 20.2 seconds. Will release after: 0.8500000000000014 2024-09-21T03:55:15.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:14 smithi134 ceph-mon[35031]: pgmap v686: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 218 MiB/s wr, 1.53k op/s 2024-09-21T03:55:15.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:14 smithi123 ceph-mon[34673]: pgmap v686: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 218 MiB/s wr, 1.53k op/s 2024-09-21T03:55:15.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:14 smithi016 ceph-mon[30375]: pgmap v686: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 218 MiB/s wr, 1.53k op/s 2024-09-21T03:55:15.442 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:55:15.999 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:16.813 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:16 smithi016 ceph-mon[30375]: pgmap v687: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 224 MiB/s wr, 1.44k op/s 2024-09-21T03:55:17.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:16 smithi134 ceph-mon[35031]: pgmap v687: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 224 MiB/s wr, 1.44k op/s 2024-09-21T03:55:17.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:16 smithi123 ceph-mon[34673]: pgmap v687: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 224 MiB/s wr, 1.44k op/s 2024-09-21T03:55:19.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:18 smithi134 ceph-mon[35031]: pgmap v688: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 230 MiB/s wr, 1.39k op/s 2024-09-21T03:55:19.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:18 smithi123 ceph-mon[34673]: pgmap v688: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 230 MiB/s wr, 1.39k op/s 2024-09-21T03:55:19.157 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:18 smithi016 ceph-mon[30375]: pgmap v688: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 230 MiB/s wr, 1.39k op/s 2024-09-21T03:55:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.650+0000 7f6d00600640 1 -- 172.21.15.16:0/1643118049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf8116640 msgr2=0x7f6cf8118b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.650+0000 7f6d00600640 1 --2- 172.21.15.16:0/1643118049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf8116640 0x7f6cf8118b50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009be0 tx=0x7f6cf002a6c0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.650+0000 7f6d00600640 1 -- 172.21.15.16:0/1643118049 shutdown_connections 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.650+0000 7f6d00600640 1 --2- 172.21.15.16:0/1643118049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf8116640 0x7f6cf8118b50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.650+0000 7f6d00600640 1 --2- 172.21.15.16:0/1643118049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cf810aee0 0x7f6cf8115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.650+0000 7f6d00600640 1 --2- 172.21.15.16:0/1643118049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf810a590 0x7f6cf810a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.650+0000 7f6d00600640 1 -- 172.21.15.16:0/1643118049 >> 172.21.15.16:0/1643118049 conn(0x7f6cf8070ec0 msgr2=0x7f6cf80712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.650+0000 7f6d00600640 1 -- 172.21.15.16:0/1643118049 shutdown_connections 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.650+0000 7f6d00600640 1 -- 172.21.15.16:0/1643118049 wait complete. 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.652+0000 7f6d00600640 1 Processor -- start 2024-09-21T03:55:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.652+0000 7f6d00600640 1 -- start start 2024-09-21T03:55:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.653+0000 7f6d00600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cf810a590 0x7f6cf8115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.653+0000 7f6d00600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf810aee0 0x7f6cf810fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.653+0000 7f6d00600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf8116640 0x7f6cf8110390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.653+0000 7f6d00600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf811a0b0 con 0x7f6cf810aee0 2024-09-21T03:55:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.653+0000 7f6d00600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf8119f30 con 0x7f6cf810a590 2024-09-21T03:55:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.653+0000 7f6d00600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf811a230 con 0x7f6cf8116640 2024-09-21T03:55:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.653+0000 7f6cfdc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cf810a590 0x7f6cf8115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.653+0000 7f6cfd200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf810aee0 0x7f6cf810fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.653+0000 7f6cfd200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf810aee0 0x7f6cf810fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40258/0 (socket says 172.21.15.16:40258) 2024-09-21T03:55:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.653+0000 7f6cfdc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cf810a590 0x7f6cf8115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:46112/0 (socket says 172.21.15.16:46112) 2024-09-21T03:55:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.653+0000 7f6cfd200640 1 -- 172.21.15.16:0/1379261810 learned_addr learned my addr 172.21.15.16:0/1379261810 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.654+0000 7f6cfd200640 1 -- 172.21.15.16:0/1379261810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf8116640 msgr2=0x7f6cf8110390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:55:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.654+0000 7f6cfd200640 1 --2- 172.21.15.16:0/1379261810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf8116640 0x7f6cf8110390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.654+0000 7f6cfd200640 1 -- 172.21.15.16:0/1379261810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cf810a590 msgr2=0x7f6cf8115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.654+0000 7f6cfd200640 1 --2- 172.21.15.16:0/1379261810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cf810a590 0x7f6cf8115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.654+0000 7f6cfd200640 1 -- 172.21.15.16:0/1379261810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf8110900 con 0x7f6cf810aee0 2024-09-21T03:55:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.654+0000 7f6cfdc00640 1 --2- 172.21.15.16:0/1379261810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cf810a590 0x7f6cf8115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T03:55:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.655+0000 7f6cfd200640 1 --2- 172.21.15.16:0/1379261810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf810aee0 0x7f6cf810fe50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400d740 tx=0x7f6ce400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.655+0000 7f6cf6000640 1 -- 172.21.15.16:0/1379261810 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4017070 con 0x7f6cf810aee0 2024-09-21T03:55:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.655+0000 7f6cf6000640 1 -- 172.21.15.16:0/1379261810 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ce400de50 con 0x7f6cf810aee0 2024-09-21T03:55:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.655+0000 7f6cf6000640 1 -- 172.21.15.16:0/1379261810 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce400f620 con 0x7f6cf810aee0 2024-09-21T03:55:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.655+0000 7f6d00600640 1 -- 172.21.15.16:0/1379261810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf8110bf0 con 0x7f6cf810aee0 2024-09-21T03:55:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.655+0000 7f6d00600640 1 -- 172.21.15.16:0/1379261810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf81be3c0 con 0x7f6cf810aee0 2024-09-21T03:55:19.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.658+0000 7f6d00600640 1 -- 172.21.15.16:0/1379261810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf810a9a0 con 0x7f6cf810aee0 2024-09-21T03:55:19.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.663+0000 7f6cf6000640 1 -- 172.21.15.16:0/1379261810 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6ce401a070 con 0x7f6cf810aee0 2024-09-21T03:55:19.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.665+0000 7f6cf6000640 1 --2- 172.21.15.16:0/1379261810 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6ce00b65d0 0x7f6ce00b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:19.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.666+0000 7f6cf6000640 1 -- 172.21.15.16:0/1379261810 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f6ce40e7110 con 0x7f6cf810aee0 2024-09-21T03:55:19.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.667+0000 7f6cfdc00640 1 --2- 172.21.15.16:0/1379261810 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6ce00b65d0 0x7f6ce00b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:19.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.668+0000 7f6cfdc00640 1 --2- 172.21.15.16:0/1379261810 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6ce00b65d0 0x7f6ce00b8ae0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6cf81158e0 tx=0x7f6cec005db0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:19.678+0000 7f6cf6000640 1 -- 172.21.15.16:0/1379261810 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6ce4010e20 con 0x7f6cf810aee0 2024-09-21T03:55:20.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:20.164+0000 7f6d00600640 1 -- 172.21.15.16:0/1379261810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cf81be7c0 con 0x7f6cf810aee0 2024-09-21T03:55:20.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:20.165+0000 7f6cf6000640 1 -- 172.21.15.16:0/1379261810 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f6ce4009c30 con 0x7f6cf810aee0 2024-09-21T03:55:20.171 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:55:20.171 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:55:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:55:20.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:20.173+0000 7f6d00600640 1 -- 172.21.15.16:0/1379261810 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6ce00b65d0 msgr2=0x7f6ce00b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:20.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:20.173+0000 7f6d00600640 1 --2- 172.21.15.16:0/1379261810 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6ce00b65d0 0x7f6ce00b8ae0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6cf81158e0 tx=0x7f6cec005db0 comp rx=0 tx=0).stop 2024-09-21T03:55:20.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:20.173+0000 7f6d00600640 1 -- 172.21.15.16:0/1379261810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf810aee0 msgr2=0x7f6cf810fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:20.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:20.173+0000 7f6d00600640 1 --2- 172.21.15.16:0/1379261810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf810aee0 0x7f6cf810fe50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400d740 tx=0x7f6ce400dc00 comp rx=0 tx=0).stop 2024-09-21T03:55:20.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:20.173+0000 7f6d00600640 1 -- 172.21.15.16:0/1379261810 shutdown_connections 2024-09-21T03:55:20.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:20.173+0000 7f6d00600640 1 --2- 172.21.15.16:0/1379261810 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f6ce00b65d0 0x7f6ce00b8ae0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:20.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:20.173+0000 7f6d00600640 1 --2- 172.21.15.16:0/1379261810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf8116640 0x7f6cf8110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:20.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:20.173+0000 7f6d00600640 1 --2- 172.21.15.16:0/1379261810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf810aee0 0x7f6cf810fe50 secure :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400d740 tx=0x7f6ce400dc00 comp rx=0 tx=0).stop 2024-09-21T03:55:20.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:20.173+0000 7f6d00600640 1 --2- 172.21.15.16:0/1379261810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cf810a590 0x7f6cf8115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:20.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:20.173+0000 7f6d00600640 1 -- 172.21.15.16:0/1379261810 >> 172.21.15.16:0/1379261810 conn(0x7f6cf8070ec0 msgr2=0x7f6cf811ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:20.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:20.173+0000 7f6d00600640 1 -- 172.21.15.16:0/1379261810 shutdown_connections 2024-09-21T03:55:20.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:20.174+0000 7f6d00600640 1 -- 172.21.15.16:0/1379261810 wait complete. 2024-09-21T03:55:21.122 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:20 smithi016 ceph-mon[30375]: pgmap v689: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 219 MiB/s wr, 1.27k op/s 2024-09-21T03:55:21.122 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:20 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1379261810' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:55:21.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:20 smithi123 ceph-mon[34673]: pgmap v689: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 219 MiB/s wr, 1.27k op/s 2024-09-21T03:55:21.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:20 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1379261810' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:55:21.208 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db --set-id 74ddd6a7 --release --await' 2024-09-21T03:55:21.208 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db --set-id 74ddd6a7 --release --await 2024-09-21T03:55:21.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:20 smithi134 ceph-mon[35031]: pgmap v689: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 219 MiB/s wr, 1.27k op/s 2024-09-21T03:55:21.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:20 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1379261810' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:55:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:23.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:22 smithi123 ceph-mon[34673]: pgmap v690: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 231 MiB/s wr, 1.30k op/s 2024-09-21T03:55:23.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:22 smithi016 ceph-mon[30375]: pgmap v690: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 231 MiB/s wr, 1.30k op/s 2024-09-21T03:55:23.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:22 smithi134 ceph-mon[35031]: pgmap v690: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 231 MiB/s wr, 1.30k op/s 2024-09-21T03:55:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.563+0000 7f5d9e400640 1 -- 172.21.15.16:0/2219092951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d9806b710 msgr2=0x7f5d98106180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.563+0000 7f5d9e400640 1 --2- 172.21.15.16:0/2219092951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d9806b710 0x7f5d98106180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009900 tx=0x7f5d8c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.563+0000 7f5d9e400640 1 -- 172.21.15.16:0/2219092951 shutdown_connections 2024-09-21T03:55:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.563+0000 7f5d9e400640 1 --2- 172.21.15.16:0/2219092951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d98109280 0x7f5d9810b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.563+0000 7f5d9e400640 1 --2- 172.21.15.16:0/2219092951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d981066c0 0x7f5d98108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.563+0000 7f5d9e400640 1 --2- 172.21.15.16:0/2219092951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d9806b710 0x7f5d98106180 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.563+0000 7f5d9e400640 1 -- 172.21.15.16:0/2219092951 >> 172.21.15.16:0/2219092951 conn(0x7f5d980fdd80 msgr2=0x7f5d98100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.564+0000 7f5d9e400640 1 -- 172.21.15.16:0/2219092951 shutdown_connections 2024-09-21T03:55:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.564+0000 7f5d9e400640 1 -- 172.21.15.16:0/2219092951 wait complete. 2024-09-21T03:55:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.565+0000 7f5d9e400640 1 Processor -- start 2024-09-21T03:55:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.565+0000 7f5d9e400640 1 -- start start 2024-09-21T03:55:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.566+0000 7f5d9e400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d9806b710 0x7f5d98199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.566+0000 7f5d9e400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d981066c0 0x7f5d9819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.566+0000 7f5d9e400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d98109280 0x7f5d9819b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.566+0000 7f5d9e400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9810f190 con 0x7f5d9806b710 2024-09-21T03:55:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.566+0000 7f5d9e400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9810f010 con 0x7f5d981066c0 2024-09-21T03:55:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.566+0000 7f5d9e400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9810f310 con 0x7f5d98109280 2024-09-21T03:55:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.567+0000 7f5d9d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d9806b710 0x7f5d98199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.567+0000 7f5d9d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d9806b710 0x7f5d98199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45012/0 (socket says 172.21.15.16:45012) 2024-09-21T03:55:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.567+0000 7f5d9d000640 1 -- 172.21.15.16:0/1002416357 learned_addr learned my addr 172.21.15.16:0/1002416357 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.567+0000 7f5d9da00640 1 --2- 172.21.15.16:0/1002416357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d98109280 0x7f5d9819b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.567+0000 7f5d9d000640 1 -- 172.21.15.16:0/1002416357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d98109280 msgr2=0x7f5d9819b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.567+0000 7f5d9d000640 1 --2- 172.21.15.16:0/1002416357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d98109280 0x7f5d9819b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.567+0000 7f5d9d000640 1 -- 172.21.15.16:0/1002416357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d981066c0 msgr2=0x7f5d9819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.567+0000 7f5d9d000640 1 --2- 172.21.15.16:0/1002416357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d981066c0 0x7f5d9819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.567+0000 7f5d9d000640 1 -- 172.21.15.16:0/1002416357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9819b850 con 0x7f5d9806b710 2024-09-21T03:55:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.568+0000 7f5d9d000640 1 --2- 172.21.15.16:0/1002416357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d9806b710 0x7f5d98199e10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c031440 tx=0x7f5d8c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.569+0000 7f5d95600640 1 -- 172.21.15.16:0/1002416357 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c03f070 con 0x7f5d9806b710 2024-09-21T03:55:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.569+0000 7f5d95600640 1 -- 172.21.15.16:0/1002416357 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d8c03b460 con 0x7f5d9806b710 2024-09-21T03:55:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.569+0000 7f5d9e400640 1 -- 172.21.15.16:0/1002416357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9819bae0 con 0x7f5d9806b710 2024-09-21T03:55:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.569+0000 7f5d9e400640 1 -- 172.21.15.16:0/1002416357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9819bdf0 con 0x7f5d9806b710 2024-09-21T03:55:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.570+0000 7f5d95600640 1 -- 172.21.15.16:0/1002416357 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c035030 con 0x7f5d9806b710 2024-09-21T03:55:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.573+0000 7f5da6c4a740 1 Processor -- start 2024-09-21T03:55:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.573+0000 7f5da6c4a740 1 -- start start 2024-09-21T03:55:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.574+0000 7f5da6c4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8dc71b0 0x55cba916e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.574+0000 7f5da6c4a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8d9e2f0 0x55cba916e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.574+0000 7f5da6c4a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba916f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.574+0000 7f5da6c4a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cba90e16b0 con 0x55cba8d9e2f0 2024-09-21T03:55:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.574+0000 7f5da6c4a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cba90d0ae0 con 0x55cba8da64d0 2024-09-21T03:55:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.574+0000 7f5da6c4a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cba90d48f0 con 0x55cba8dc71b0 2024-09-21T03:55:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.575+0000 7f5d9da00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8dc71b0 0x55cba916e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.579+0000 7f5d97e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba916f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.580+0000 7f5d97e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba916f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:51200/0 (socket says 172.21.15.16:51200) 2024-09-21T03:55:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.580+0000 7f5d97e00640 1 -- 172.21.15.16:0/1744653639 learned_addr learned my addr 172.21.15.16:0/1744653639 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.580+0000 7f5d95600640 1 -- 172.21.15.16:0/1002416357 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5d8c042070 con 0x7f5d9806b710 2024-09-21T03:55:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.583+0000 7f5d9da00640 1 -- 172.21.15.16:0/1744653639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 msgr2=0x55cba916f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.584+0000 7f5d9da00640 1 --2- 172.21.15.16:0/1744653639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba916f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.584+0000 7f5d9da00640 1 -- 172.21.15.16:0/1744653639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8d9e2f0 msgr2=0x55cba916e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.583+0000 7f5d9d000640 1 --2- 172.21.15.16:0/1744653639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8d9e2f0 0x55cba916e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.584+0000 7f5d9da00640 1 --2- 172.21.15.16:0/1744653639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8d9e2f0 0x55cba916e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.584+0000 7f5d9da00640 1 -- 172.21.15.16:0/1744653639 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cba909d780 con 0x55cba8dc71b0 2024-09-21T03:55:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.585+0000 7f5d95600640 1 --2- 172.21.15.16:0/1002416357 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d800b6510 0x7f5d800b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.585+0000 7f5d97e00640 1 --2- 172.21.15.16:0/1744653639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba916f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:55:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.585+0000 7f5d95600640 1 -- 172.21.15.16:0/1002416357 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f5d8c10a240 con 0x7f5d9806b710 2024-09-21T03:55:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.585+0000 7f5d9d000640 1 --2- 172.21.15.16:0/1744653639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8d9e2f0 0x55cba916e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.586+0000 7f5d9da00640 1 --2- 172.21.15.16:0/1744653639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8dc71b0 0x55cba916e2b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009ec0 tx=0x7f5d9000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7e01af954994904 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.586+0000 7f5d9d000640 1 --2- 172.21.15.16:0/1002416357 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d800b6510 0x7f5d800b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.587+0000 7f5d8a000640 1 -- 172.21.15.16:0/1744653639 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90015070 con 0x55cba8dc71b0 2024-09-21T03:55:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.587+0000 7f5d8a000640 1 -- 172.21.15.16:0/1744653639 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d90004d20 con 0x55cba8dc71b0 2024-09-21T03:55:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.587+0000 7f5d8a000640 1 -- 172.21.15.16:0/1744653639 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90011770 con 0x55cba8dc71b0 2024-09-21T03:55:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.589+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1744653639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8dc71b0 msgr2=0x55cba916e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.589+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/1744653639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8dc71b0 0x55cba916e2b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009ec0 tx=0x7f5d9000f730 comp rx=0 tx=0).stop 2024-09-21T03:55:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.589+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1744653639 shutdown_connections 2024-09-21T03:55:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.589+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/1744653639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba916f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.589+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/1744653639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8d9e2f0 0x55cba916e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.589+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/1744653639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8dc71b0 0x55cba916e2b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.589+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1744653639 >> 172.21.15.16:0/1744653639 conn(0x55cba8dbb300 msgr2=0x55cba9169d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.590+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1744653639 shutdown_connections 2024-09-21T03:55:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.591+0000 7f5d9d000640 1 --2- 172.21.15.16:0/1002416357 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d800b6510 0x7f5d800b8a20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c03d040 tx=0x7f5d8c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.591+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1744653639 wait complete. 2024-09-21T03:55:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.592+0000 7f5da6c4a740 1 -- start start 2024-09-21T03:55:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.592+0000 7f5da6c4a740 1 Processor -- start 2024-09-21T03:55:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.597+0000 7f5da6c4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8d9e2f0 0x55cba91779f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.597+0000 7f5da6c4a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba91780b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.597+0000 7f5da6c4a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8dc71b0 0x55cba9178c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.597+0000 7f5da6c4a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cba9008ab0 con 0x55cba8dc71b0 2024-09-21T03:55:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.597+0000 7f5da6c4a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cba9077570 con 0x55cba8da64d0 2024-09-21T03:55:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.597+0000 7f5da6c4a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cba916ff90 con 0x55cba8d9e2f0 2024-09-21T03:55:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.598+0000 7f5d97e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba91780b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.598+0000 7f5d9da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8dc71b0 0x55cba9178c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.598+0000 7f5d97e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba91780b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:51208/0 (socket says 172.21.15.16:51208) 2024-09-21T03:55:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.598+0000 7f5d97e00640 1 -- 172.21.15.16:0/3706760655 learned_addr learned my addr 172.21.15.16:0/3706760655 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.598+0000 7f5da6c4a740 10 client.0 _ll_register_callbacks cb 0x55cba9086100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.598+0000 7f5d9da00640 1 --2- 172.21.15.16:0/3706760655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8d9e2f0 0x55cba91779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.599+0000 7f5d97e00640 1 -- 172.21.15.16:0/3706760655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8d9e2f0 msgr2=0x55cba91779f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.599+0000 7f5d97e00640 1 --2- 172.21.15.16:0/3706760655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8d9e2f0 0x55cba91779f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.599+0000 7f5d97e00640 1 -- 172.21.15.16:0/3706760655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8dc71b0 msgr2=0x55cba9178c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.599+0000 7f5d97e00640 1 --2- 172.21.15.16:0/3706760655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8dc71b0 0x55cba9178c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.599+0000 7f5d97e00640 1 -- 172.21.15.16:0/3706760655 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cba8e32790 con 0x55cba8da64d0 2024-09-21T03:55:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.599+0000 7f5d9da00640 1 --2- 172.21.15.16:0/3706760655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8dc71b0 0x55cba9178c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:55:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.600+0000 7f5d97e00640 1 --2- 172.21.15.16:0/3706760655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba91780b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400b8f0 tx=0x7f5d8400bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c34f46f4863cf901 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.600+0000 7f5d7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:55:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.601+0000 7f5d7ea00640 1 -- 172.21.15.16:0/3706760655 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8401a470 con 0x55cba8da64d0 2024-09-21T03:55:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.601+0000 7f5d7ea00640 1 -- 172.21.15.16:0/3706760655 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f5d8401a610 con 0x55cba8da64d0 2024-09-21T03:55:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.601+0000 7f5da6c4a740 1 -- 172.21.15.16:0/3706760655 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d90004d20 con 0x55cba8da64d0 2024-09-21T03:55:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.601+0000 7f5d7ea00640 1 -- 172.21.15.16:0/3706760655 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d8401a850 con 0x55cba8da64d0 2024-09-21T03:55:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.601+0000 7f5da6c4a740 10 client.26531 fetch_fsmap learned FSMap version 30 2024-09-21T03:55:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.601+0000 7f5da6c4a740 1 -- 172.21.15.16:0/3706760655 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cba8eab620 con 0x55cba8da64d0 2024-09-21T03:55:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.602+0000 7f5d7ea00640 1 -- 172.21.15.16:0/3706760655 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d90004d20 con 0x55cba8da64d0 2024-09-21T03:55:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.603+0000 7f5d7ea00640 1 -- 172.21.15.16:0/3706760655 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5d84011b80 con 0x55cba8da64d0 2024-09-21T03:55:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.603+0000 7f5da6c4a740 10 client.26531 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:55:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.603+0000 7f5da6c4a740 10 client.26531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.604+0000 7f5da6c4a740 20 client.26531 populate_metadata read hostname 'smithi016' 2024-09-21T03:55:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.604+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/3706760655 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cba8d85d80 0x55cba9185010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.604+0000 7f5da6c4a740 4 client.26531 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.604+0000 7f5da6c4a740 1 -- 172.21.15.16:0/3706760655 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cba90e3600 con 0x55cba8d85d80 2024-09-21T03:55:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.604+0000 7f5d9da00640 1 --2- 172.21.15.16:0/3706760655 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cba8d85d80 0x55cba9185010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.605+0000 7f5d9da00640 1 --2- 172.21.15.16:0/3706760655 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cba8d85d80 0x55cba9185010 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aedc70475953f5 server_cookie=2f535b39d5c7c09c in_seq=0 out_seq=0 2024-09-21T03:55:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.605+0000 7f5d7ea00640 10 client.26531 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.607+0000 7f5d7ea00640 1 -- 172.21.15.16:0/3706760655 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cba90e3600 con 0x55cba8d85d80 2024-09-21T03:55:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.607+0000 7f5d7ea00640 10 client.26531 handle_command_reply: tid=0 2024-09-21T03:55:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.608+0000 7f5da6c4a740 1 client.26531 shutdown 2024-09-21T03:55:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.608+0000 7f5da6c4a740 1 -- 172.21.15.16:0/3706760655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 msgr2=0x55cba91780b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.608+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/3706760655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba91780b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400b8f0 tx=0x7f5d8400bcf0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.609+0000 7f5da6c4a740 1 -- 172.21.15.16:0/3706760655 shutdown_connections 2024-09-21T03:55:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.609+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/3706760655 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cba8d85d80 0x55cba9185010 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.609+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/3706760655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8dc71b0 0x55cba9178c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.609+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/3706760655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba91780b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.609+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/3706760655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8d9e2f0 0x55cba91779f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.609+0000 7f5da6c4a740 1 -- 172.21.15.16:0/3706760655 >> 172.21.15.16:0/3706760655 conn(0x55cba8dbb300 msgr2=0x55cba9168360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.609+0000 7f5d7ea00640 0 client.26531 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.609+0000 7f5da6c4a740 1 -- 172.21.15.16:0/3706760655 shutdown_connections 2024-09-21T03:55:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.609+0000 7f5da6c4a740 1 -- 172.21.15.16:0/3706760655 wait complete. 2024-09-21T03:55:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.610+0000 7f5da6c4a740 20 client.26531 trim_cache size 0 max 16384 2024-09-21T03:55:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.642+0000 7f5da6c4a740 1 Processor -- start 2024-09-21T03:55:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.642+0000 7f5da6c4a740 1 -- start start 2024-09-21T03:55:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.642+0000 7f5da6c4a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8d9e2f0 0x55cba9178100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.642+0000 7f5da6c4a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba917fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.642+0000 7f5da6c4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8dc71b0 0x55cba916dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.642+0000 7f5da6c4a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d981b2c80 con 0x55cba8d9e2f0 2024-09-21T03:55:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.642+0000 7f5da6c4a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d981b47c0 con 0x55cba8da64d0 2024-09-21T03:55:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.642+0000 7f5da6c4a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d981b3350 con 0x55cba8dc71b0 2024-09-21T03:55:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.643+0000 7f5d97e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba917fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.643+0000 7f5d97e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba917fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:51210/0 (socket says 172.21.15.16:51210) 2024-09-21T03:55:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.643+0000 7f5d97e00640 1 -- 172.21.15.16:0/1519909413 learned_addr learned my addr 172.21.15.16:0/1519909413 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.644+0000 7f5d97e00640 1 -- 172.21.15.16:0/1519909413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8dc71b0 msgr2=0x55cba916dea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:55:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.644+0000 7f5d97e00640 1 --2- 172.21.15.16:0/1519909413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8dc71b0 0x55cba916dea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.644+0000 7f5d97e00640 1 -- 172.21.15.16:0/1519909413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8d9e2f0 msgr2=0x55cba9178100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:55:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.646+0000 7f5d97e00640 1 --2- 172.21.15.16:0/1519909413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8d9e2f0 0x55cba9178100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.646+0000 7f5d97e00640 1 -- 172.21.15.16:0/1519909413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cba916f210 con 0x55cba8da64d0 2024-09-21T03:55:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.647+0000 7f5d97e00640 1 --2- 172.21.15.16:0/1519909413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba917fb00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400b8c0 tx=0x7f5d840044e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6fcb6f8b6b117de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.648+0000 7f5d89600640 1 -- 172.21.15.16:0/1519909413 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84003de0 con 0x55cba8da64d0 2024-09-21T03:55:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.648+0000 7f5d89600640 1 -- 172.21.15.16:0/1519909413 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d84015070 con 0x55cba8da64d0 2024-09-21T03:55:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.648+0000 7f5d89600640 1 -- 172.21.15.16:0/1519909413 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8401a9f0 con 0x55cba8da64d0 2024-09-21T03:55:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.652+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1519909413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 msgr2=0x55cba917fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.652+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/1519909413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba917fb00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400b8c0 tx=0x7f5d840044e0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.653+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1519909413 shutdown_connections 2024-09-21T03:55:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.653+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/1519909413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8dc71b0 0x55cba916dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.654+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/1519909413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8da64d0 0x55cba917fb00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.654+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/1519909413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8d9e2f0 0x55cba9178100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.654+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1519909413 >> 172.21.15.16:0/1519909413 conn(0x55cba8dbb300 msgr2=0x55cba91779f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.654+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1519909413 shutdown_connections 2024-09-21T03:55:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.654+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1519909413 wait complete. 2024-09-21T03:55:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.655+0000 7f5da6c4a740 1 -- start start 2024-09-21T03:55:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.655+0000 7f5da6c4a740 1 Processor -- start 2024-09-21T03:55:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.656+0000 7f5da6c4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8d9e2f0 0x55cba917f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.657+0000 7f5da6c4a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8da64d0 0x55cba917e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.657+0000 7f5da6c4a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8dc71b0 0x55cba917eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.657+0000 7f5d9da00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8d9e2f0 0x55cba917f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.658+0000 7f5d9da00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8d9e2f0 0x55cba917f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39328/0 (socket says 172.21.15.16:39328) 2024-09-21T03:55:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.658+0000 7f5d9da00640 1 -- 172.21.15.16:0/1170214796 learned_addr learned my addr 172.21.15.16:0/1170214796 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.658+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1170214796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d981b40f0 con 0x55cba8da64d0 2024-09-21T03:55:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.658+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1170214796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d981b3a20 con 0x55cba8dc71b0 2024-09-21T03:55:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.658+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1170214796 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d981b0c60 con 0x55cba8d9e2f0 2024-09-21T03:55:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.658+0000 7f5da6c4a740 10 client.0 _ll_register_callbacks cb 0x7f5d84004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.658+0000 7f5d97e00640 1 --2- 172.21.15.16:0/1170214796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8da64d0 0x55cba917e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.658+0000 7f5d9da00640 1 --2- 172.21.15.16:0/1170214796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8dc71b0 0x55cba917eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.659+0000 7f5d9da00640 1 -- 172.21.15.16:0/1170214796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8d9e2f0 msgr2=0x55cba917f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.659+0000 7f5d9da00640 1 --2- 172.21.15.16:0/1170214796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8d9e2f0 0x55cba917f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.659+0000 7f5d9da00640 1 -- 172.21.15.16:0/1170214796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8da64d0 msgr2=0x55cba917e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.660+0000 7f5d9da00640 1 --2- 172.21.15.16:0/1170214796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8da64d0 0x55cba917e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.660+0000 7f5d9da00640 1 -- 172.21.15.16:0/1170214796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cba917f260 con 0x55cba8dc71b0 2024-09-21T03:55:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.661+0000 7f5d9da00640 1 --2- 172.21.15.16:0/1170214796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8dc71b0 0x55cba917eb00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009e10 tx=0x7f5d90009a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3399ba9594eb27f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.661+0000 7f5d7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:55:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.662+0000 7f5d7ea00640 1 -- 172.21.15.16:0/1170214796 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90015040 con 0x55cba8dc71b0 2024-09-21T03:55:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.662+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1170214796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d84015070 con 0x55cba8dc71b0 2024-09-21T03:55:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.662+0000 7f5d7ea00640 1 -- 172.21.15.16:0/1170214796 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f5d90003970 con 0x55cba8dc71b0 2024-09-21T03:55:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.663+0000 7f5d7ea00640 1 -- 172.21.15.16:0/1170214796 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d90003b70 con 0x55cba8dc71b0 2024-09-21T03:55:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.663+0000 7f5da6c4a740 10 client.26537 fetch_fsmap learned FSMap version 30 2024-09-21T03:55:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.663+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1170214796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cba91efa70 con 0x55cba8dc71b0 2024-09-21T03:55:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.663+0000 7f5d7ea00640 1 -- 172.21.15.16:0/1170214796 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d84015070 con 0x55cba8dc71b0 2024-09-21T03:55:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.664+0000 7f5d7ea00640 1 -- 172.21.15.16:0/1170214796 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5d90027040 con 0x55cba8dc71b0 2024-09-21T03:55:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.664+0000 7f5da6c4a740 10 client.26537 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:55:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.664+0000 7f5da6c4a740 10 client.26537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.664+0000 7f5da6c4a740 20 client.26537 populate_metadata read hostname 'smithi016' 2024-09-21T03:55:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.664+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/1170214796 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cba8d5b1c0 0x55cba91f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.665+0000 7f5da6c4a740 4 client.26537 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:55:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.665+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1170214796 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cba90df0b0 con 0x55cba8d5b1c0 2024-09-21T03:55:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.665+0000 7f5d97e00640 1 --2- 172.21.15.16:0/1170214796 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cba8d5b1c0 0x55cba91f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.666+0000 7f5d7ea00640 10 client.26537 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.666+0000 7f5d97e00640 1 --2- 172.21.15.16:0/1170214796 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cba8d5b1c0 0x55cba91f22d0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0483616488f95f5 server_cookie=d9e3667d593148c4 in_seq=0 out_seq=0 2024-09-21T03:55:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.718+0000 7f5d7ea00640 1 -- 172.21.15.16:0/1170214796 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cba90df0b0 con 0x55cba8d5b1c0 2024-09-21T03:55:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.718+0000 7f5d7ea00640 10 client.26537 handle_command_reply: tid=0 2024-09-21T03:55:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.718+0000 7f5da6c4a740 1 client.26537 shutdown 2024-09-21T03:55:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.721+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1170214796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8dc71b0 msgr2=0x55cba917eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.721+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/1170214796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8dc71b0 0x55cba917eb00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009e10 tx=0x7f5d90009a70 comp rx=0 tx=0).stop 2024-09-21T03:55:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.721+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1170214796 shutdown_connections 2024-09-21T03:55:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.721+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/1170214796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cba8dc71b0 0x55cba917eb00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.721+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/1170214796 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55cba8d5b1c0 0x55cba91f22d0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.721+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/1170214796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba8da64d0 0x55cba917e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.721+0000 7f5d7ea00640 0 client.26537 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.721+0000 7f5da6c4a740 1 --2- 172.21.15.16:0/1170214796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cba8d9e2f0 0x55cba917f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.722+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1170214796 >> 172.21.15.16:0/1170214796 conn(0x55cba8dbb300 msgr2=0x55cba916dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.723+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1170214796 shutdown_connections 2024-09-21T03:55:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.723+0000 7f5da6c4a740 1 -- 172.21.15.16:0/1170214796 wait complete. 2024-09-21T03:55:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.725+0000 7f5da6c4a740 20 client.26537 trim_cache size 0 max 16384 2024-09-21T03:55:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:55:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:55:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:55:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:55:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.725+0000 7f5d9e400640 1 -- 172.21.15.16:0/1002416357 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d800b6510 msgr2=0x7f5d800b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.725+0000 7f5d9e400640 1 --2- 172.21.15.16:0/1002416357 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d800b6510 0x7f5d800b8a20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c03d040 tx=0x7f5d8c009260 comp rx=0 tx=0).stop 2024-09-21T03:55:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.725+0000 7f5d9e400640 1 -- 172.21.15.16:0/1002416357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d9806b710 msgr2=0x7f5d98199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.725+0000 7f5d9e400640 1 --2- 172.21.15.16:0/1002416357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d9806b710 0x7f5d98199e10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c031440 tx=0x7f5d8c003c10 comp rx=0 tx=0).stop 2024-09-21T03:55:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.726+0000 7f5d9e400640 1 -- 172.21.15.16:0/1002416357 shutdown_connections 2024-09-21T03:55:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.726+0000 7f5d9e400640 1 --2- 172.21.15.16:0/1002416357 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f5d800b6510 0x7f5d800b8a20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.726+0000 7f5d9e400640 1 --2- 172.21.15.16:0/1002416357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d98109280 0x7f5d9819b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.726+0000 7f5d9e400640 1 --2- 172.21.15.16:0/1002416357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d981066c0 0x7f5d9819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.726+0000 7f5d9e400640 1 --2- 172.21.15.16:0/1002416357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d9806b710 0x7f5d98199e10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.726+0000 7f5d9e400640 1 -- 172.21.15.16:0/1002416357 >> 172.21.15.16:0/1002416357 conn(0x7f5d980fdd80 msgr2=0x7f5d98104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.726+0000 7f5d9e400640 1 -- 172.21.15.16:0/1002416357 shutdown_connections 2024-09-21T03:55:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:23.727+0000 7f5d9e400640 1 -- 172.21.15.16:0/1002416357 wait complete. 2024-09-21T03:55:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:24 smithi016 ceph-mon[30375]: pgmap v691: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 235 MiB/s wr, 1.32k op/s 2024-09-21T03:55:25.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:24 smithi134 ceph-mon[35031]: pgmap v691: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 235 MiB/s wr, 1.32k op/s 2024-09-21T03:55:25.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:24 smithi123 ceph-mon[34673]: pgmap v691: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 235 MiB/s wr, 1.32k op/s 2024-09-21T03:55:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.439+0000 7f7865600640 1 -- 172.21.15.16:0/3694760804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f786010a900 msgr2=0x7f786010ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.439+0000 7f7865600640 1 --2- 172.21.15.16:0/3694760804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f786010a900 0x7f786010ad10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f784c009900 tx=0x7f784c02b640 comp rx=0 tx=0).stop 2024-09-21T03:55:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.439+0000 7f7865600640 1 -- 172.21.15.16:0/3694760804 shutdown_connections 2024-09-21T03:55:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.440+0000 7f7865600640 1 --2- 172.21.15.16:0/3694760804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860114fb0 0x7f78601173d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.440+0000 7f7865600640 1 --2- 172.21.15.16:0/3694760804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f786010a900 0x7f786010ad10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.440+0000 7f7865600640 1 --2- 172.21.15.16:0/3694760804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f786010f6f0 0x7f786010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.440+0000 7f7865600640 1 -- 172.21.15.16:0/3694760804 >> 172.21.15.16:0/3694760804 conn(0x7f78600712f0 msgr2=0x7f7860073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.440+0000 7f7865600640 1 -- 172.21.15.16:0/3694760804 shutdown_connections 2024-09-21T03:55:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.440+0000 7f7865600640 1 -- 172.21.15.16:0/3694760804 wait complete. 2024-09-21T03:55:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.443+0000 7f7865600640 1 Processor -- start 2024-09-21T03:55:25.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.443+0000 7f7865600640 1 -- start start 2024-09-21T03:55:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.445+0000 7f7865600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f786010a900 0x7f786011cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.445+0000 7f7865600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f786010f6f0 0x7f786011d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.445+0000 7f7865600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7860114fb0 0x7f7860123a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.446+0000 7f7865600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7860119d50 con 0x7f786010a900 2024-09-21T03:55:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.446+0000 7f7865600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7860119bd0 con 0x7f7860114fb0 2024-09-21T03:55:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.446+0000 7f785e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f786010f6f0 0x7f786011d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.446+0000 7f7865600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7860119ed0 con 0x7f786010f6f0 2024-09-21T03:55:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.446+0000 7f785e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f786010f6f0 0x7f786011d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39342/0 (socket says 172.21.15.16:39342) 2024-09-21T03:55:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.446+0000 7f785e000640 1 -- 172.21.15.16:0/968005653 learned_addr learned my addr 172.21.15.16:0/968005653 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:25.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.446+0000 7f785f400640 1 --2- 172.21.15.16:0/968005653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7860114fb0 0x7f7860123a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.447+0000 7f785e000640 1 -- 172.21.15.16:0/968005653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7860114fb0 msgr2=0x7f7860123a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.447+0000 7f785e000640 1 --2- 172.21.15.16:0/968005653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7860114fb0 0x7f7860123a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.447+0000 7f785e000640 1 -- 172.21.15.16:0/968005653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f786010a900 msgr2=0x7f786011cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.447+0000 7f785e000640 1 --2- 172.21.15.16:0/968005653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f786010a900 0x7f786011cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.447+0000 7f785e000640 1 -- 172.21.15.16:0/968005653 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7860123ff0 con 0x7f786010f6f0 2024-09-21T03:55:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.447+0000 7f785e000640 1 --2- 172.21.15.16:0/968005653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f786010f6f0 0x7f786011d0e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f784c02bc30 tx=0x7f784c0315d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.448+0000 7f7857400640 1 -- 172.21.15.16:0/968005653 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c03f070 con 0x7f786010f6f0 2024-09-21T03:55:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.448+0000 7f7865600640 1 -- 172.21.15.16:0/968005653 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7860124280 con 0x7f786010f6f0 2024-09-21T03:55:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.449+0000 7f7865600640 1 -- 172.21.15.16:0/968005653 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7860124830 con 0x7f786010f6f0 2024-09-21T03:55:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.452+0000 7f7857400640 1 -- 172.21.15.16:0/968005653 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f784c02be60 con 0x7f786010f6f0 2024-09-21T03:55:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.452+0000 7f7857400640 1 -- 172.21.15.16:0/968005653 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c03f070 con 0x7f786010f6f0 2024-09-21T03:55:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.459+0000 7f7857400640 1 -- 172.21.15.16:0/968005653 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f784c003850 con 0x7f786010f6f0 2024-09-21T03:55:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.467+0000 7f7857400640 1 --2- 172.21.15.16:0/968005653 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f78440b67e0 0x7f78440b8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.467+0000 7f7857400640 1 -- 172.21.15.16:0/968005653 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f784c108910 con 0x7f786010f6f0 2024-09-21T03:55:25.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.467+0000 7f785ea00640 1 --2- 172.21.15.16:0/968005653 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f78440b67e0 0x7f78440b8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:25.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.484+0000 7f785ea00640 1 --2- 172.21.15.16:0/968005653 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f78440b67e0 0x7f78440b8cf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7860114da0 tx=0x7f7850006c40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:25.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.485+0000 7f786c2c4740 1 Processor -- start 2024-09-21T03:55:25.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.485+0000 7f786c2c4740 1 -- start start 2024-09-21T03:55:25.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.486+0000 7f786c2c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a64706520 0x560a64aad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.486+0000 7f786c2c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646dd660 0x560a64aad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.486+0000 7f786c2c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a646e5840 0x560a64aadc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.486+0000 7f786c2c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560a64a24290 con 0x560a646e5840 2024-09-21T03:55:25.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.486+0000 7f786c2c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560a64a136c0 con 0x560a64706520 2024-09-21T03:55:25.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.486+0000 7f786c2c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560a64a174d0 con 0x560a646dd660 2024-09-21T03:55:25.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.486+0000 7f785e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646dd660 0x560a64aad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:25.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.486+0000 7f785e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646dd660 0x560a64aad700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39346/0 (socket says 172.21.15.16:39346) 2024-09-21T03:55:25.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.486+0000 7f785e000640 1 -- 172.21.15.16:0/2862664932 learned_addr learned my addr 172.21.15.16:0/2862664932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:25.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.487+0000 7f785e000640 1 -- 172.21.15.16:0/2862664932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a64706520 msgr2=0x560a64aad1c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:55:25.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.487+0000 7f785e000640 1 --2- 172.21.15.16:0/2862664932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a64706520 0x560a64aad1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.487+0000 7f785e000640 1 -- 172.21.15.16:0/2862664932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a646e5840 msgr2=0x560a64aadc40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:55:25.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.487+0000 7f785e000640 1 --2- 172.21.15.16:0/2862664932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a646e5840 0x560a64aadc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.487+0000 7f785e000640 1 -- 172.21.15.16:0/2862664932 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a649a0460 con 0x560a646dd660 2024-09-21T03:55:25.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.487+0000 7f785e000640 1 --2- 172.21.15.16:0/2862664932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646dd660 0x560a64aad700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f784c0ce3f0 tx=0x7f784c0ca5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab0092b839414dc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:25.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.489+0000 7f784b400640 1 -- 172.21.15.16:0/2862664932 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c0ce200 con 0x560a646dd660 2024-09-21T03:55:25.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.489+0000 7f784b400640 1 -- 172.21.15.16:0/2862664932 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f784c0cedf0 con 0x560a646dd660 2024-09-21T03:55:25.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.489+0000 7f784b400640 1 -- 172.21.15.16:0/2862664932 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c0cc640 con 0x560a646dd660 2024-09-21T03:55:25.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.491+0000 7f786c2c4740 1 -- 172.21.15.16:0/2862664932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646dd660 msgr2=0x560a64aad700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:25.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.491+0000 7f786c2c4740 1 --2- 172.21.15.16:0/2862664932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646dd660 0x560a64aad700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f784c0ce3f0 tx=0x7f784c0ca5f0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.493+0000 7f786c2c4740 1 -- 172.21.15.16:0/2862664932 shutdown_connections 2024-09-21T03:55:25.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.493+0000 7f786c2c4740 1 --2- 172.21.15.16:0/2862664932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a646e5840 0x560a64aadc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.494+0000 7f786c2c4740 1 --2- 172.21.15.16:0/2862664932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646dd660 0x560a64aad700 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.494+0000 7f786c2c4740 1 --2- 172.21.15.16:0/2862664932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a64706520 0x560a64aad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.494+0000 7f786c2c4740 1 -- 172.21.15.16:0/2862664932 >> 172.21.15.16:0/2862664932 conn(0x560a646fa910 msgr2=0x560a64aab0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:25.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.496+0000 7f786c2c4740 1 -- 172.21.15.16:0/2862664932 shutdown_connections 2024-09-21T03:55:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.497+0000 7f786c2c4740 1 -- 172.21.15.16:0/2862664932 wait complete. 2024-09-21T03:55:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.497+0000 7f786c2c4740 1 -- start start 2024-09-21T03:55:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.498+0000 7f786c2c4740 1 Processor -- start 2024-09-21T03:55:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.499+0000 7f786c2c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a646dd660 0x560a64ab8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.499+0000 7f786c2c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a646e5840 0x560a64ab92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.499+0000 7f786c2c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a64706520 0x560a64ac49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.500+0000 7f785ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a646dd660 0x560a64ab8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.500+0000 7f785e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a646e5840 0x560a64ab92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.500+0000 7f786c2c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560a64874bb0 con 0x560a646dd660 2024-09-21T03:55:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.500+0000 7f786c2c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560a649ae6c0 con 0x560a646e5840 2024-09-21T03:55:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.500+0000 7f786c2c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560a64abb760 con 0x560a64706520 2024-09-21T03:55:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.500+0000 7f785f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a64706520 0x560a64ac49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.500+0000 7f785f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a64706520 0x560a64ac49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39348/0 (socket says 172.21.15.16:39348) 2024-09-21T03:55:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.500+0000 7f785f400640 1 -- 172.21.15.16:0/4227588584 learned_addr learned my addr 172.21.15.16:0/4227588584 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.500+0000 7f786c2c4740 10 client.0 _ll_register_callbacks cb 0x560a649c9490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.500+0000 7f785e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a646e5840 0x560a64ab92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:51254/0 (socket says 172.21.15.16:51254) 2024-09-21T03:55:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.501+0000 7f785f400640 1 -- 172.21.15.16:0/4227588584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a646e5840 msgr2=0x560a64ab92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.501+0000 7f785f400640 1 --2- 172.21.15.16:0/4227588584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a646e5840 0x560a64ab92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.501+0000 7f785f400640 1 -- 172.21.15.16:0/4227588584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a646dd660 msgr2=0x560a64ab8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.501+0000 7f785f400640 1 --2- 172.21.15.16:0/4227588584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a646dd660 0x560a64ab8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.501+0000 7f785f400640 1 -- 172.21.15.16:0/4227588584 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a64771b00 con 0x560a64706520 2024-09-21T03:55:25.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.501+0000 7f785f400640 1 --2- 172.21.15.16:0/4227588584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a64706520 0x560a64ac49c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f785800c9d0 tx=0x7f785800ce90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8c9b51541fb7584 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:25.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.501+0000 7f783fe00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:55:25.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.502+0000 7f783fe00640 1 -- 172.21.15.16:0/4227588584 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785801a060 con 0x560a64706520 2024-09-21T03:55:25.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.502+0000 7f783fe00640 1 -- 172.21.15.16:0/4227588584 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f78580079b0 con 0x560a64706520 2024-09-21T03:55:25.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.502+0000 7f786c2c4740 1 -- 172.21.15.16:0/4227588584 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f784c0cedf0 con 0x560a64706520 2024-09-21T03:55:25.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.502+0000 7f783fe00640 1 -- 172.21.15.16:0/4227588584 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7858021050 con 0x560a64706520 2024-09-21T03:55:25.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.502+0000 7f786c2c4740 10 client.26485 fetch_fsmap learned FSMap version 30 2024-09-21T03:55:25.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.502+0000 7f786c2c4740 1 -- 172.21.15.16:0/4227588584 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560a647e8960 con 0x560a64706520 2024-09-21T03:55:25.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.503+0000 7f783fe00640 1 -- 172.21.15.16:0/4227588584 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f784c0cedf0 con 0x560a64706520 2024-09-21T03:55:25.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.504+0000 7f783fe00640 1 -- 172.21.15.16:0/4227588584 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f785800abe0 con 0x560a64706520 2024-09-21T03:55:25.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.505+0000 7f786c2c4740 10 client.26485 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:55:25.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.505+0000 7f786c2c4740 10 client.26485 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:55:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.505+0000 7f786c2c4740 20 client.26485 populate_metadata read hostname 'smithi016' 2024-09-21T03:55:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.505+0000 7f786c2c4740 1 --2- 172.21.15.16:0/4227588584 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x560a646c4ea0 0x560a64acaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.505+0000 7f786c2c4740 4 client.26485 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.505+0000 7f786c2c4740 1 -- 172.21.15.16:0/4227588584 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a64a261e0 con 0x560a646c4ea0 2024-09-21T03:55:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.506+0000 7f785ea00640 1 --2- 172.21.15.16:0/4227588584 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x560a646c4ea0 0x560a64acaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.507+0000 7f783fe00640 10 client.26485 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.507+0000 7f785ea00640 1 --2- 172.21.15.16:0/4227588584 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x560a646c4ea0 0x560a64acaac0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32468e03a3118e61 server_cookie=968e48d4546830e7 in_seq=0 out_seq=0 2024-09-21T03:55:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.508+0000 7f783fe00640 1 -- 172.21.15.16:0/4227588584 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560a64a261e0 con 0x560a646c4ea0 2024-09-21T03:55:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.508+0000 7f783fe00640 10 client.26485 handle_command_reply: tid=0 2024-09-21T03:55:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.509+0000 7f786c2c4740 1 client.26485 shutdown 2024-09-21T03:55:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.509+0000 7f786c2c4740 1 -- 172.21.15.16:0/4227588584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a64706520 msgr2=0x560a64ac49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.509+0000 7f786c2c4740 1 --2- 172.21.15.16:0/4227588584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a64706520 0x560a64ac49c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f785800c9d0 tx=0x7f785800ce90 comp rx=0 tx=0).stop 2024-09-21T03:55:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.509+0000 7f786c2c4740 1 -- 172.21.15.16:0/4227588584 shutdown_connections 2024-09-21T03:55:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.509+0000 7f786c2c4740 1 --2- 172.21.15.16:0/4227588584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a64706520 0x560a64ac49c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.509+0000 7f786c2c4740 1 --2- 172.21.15.16:0/4227588584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a646e5840 0x560a64ab92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.509+0000 7f786c2c4740 1 --2- 172.21.15.16:0/4227588584 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x560a646c4ea0 0x560a64acaac0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.509+0000 7f786c2c4740 1 --2- 172.21.15.16:0/4227588584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a646dd660 0x560a64ab8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.509+0000 7f786c2c4740 1 -- 172.21.15.16:0/4227588584 >> 172.21.15.16:0/4227588584 conn(0x560a646fa910 msgr2=0x560a64ab0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.509+0000 7f783fe00640 0 client.26485 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.510+0000 7f786c2c4740 1 -- 172.21.15.16:0/4227588584 shutdown_connections 2024-09-21T03:55:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.510+0000 7f786c2c4740 1 -- 172.21.15.16:0/4227588584 wait complete. 2024-09-21T03:55:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.511+0000 7f786c2c4740 20 client.26485 trim_cache size 0 max 16384 2024-09-21T03:55:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.550+0000 7f786c2c4740 1 Processor -- start 2024-09-21T03:55:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.550+0000 7f786c2c4740 1 -- start start 2024-09-21T03:55:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.551+0000 7f786c2c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646dd660 0x560a64ac48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.551+0000 7f786c2c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a646e5840 0x560a64ac4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.551+0000 7f786c2c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a64706520 0x560a64ab5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.551+0000 7f786c2c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f784000fd10 con 0x560a64706520 2024-09-21T03:55:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.551+0000 7f786c2c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7840011850 con 0x560a646e5840 2024-09-21T03:55:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.551+0000 7f786c2c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78400103e0 con 0x560a646dd660 2024-09-21T03:55:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.551+0000 7f785ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646dd660 0x560a64ac48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.552+0000 7f785ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646dd660 0x560a64ac48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39358/0 (socket says 172.21.15.16:39358) 2024-09-21T03:55:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.552+0000 7f785ea00640 1 -- 172.21.15.16:0/3934462864 learned_addr learned my addr 172.21.15.16:0/3934462864 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.552+0000 7f785e000640 1 --2- 172.21.15.16:0/3934462864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a646e5840 0x560a64ac4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.552+0000 7f785e000640 1 -- 172.21.15.16:0/3934462864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646dd660 msgr2=0x560a64ac48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.552+0000 7f785e000640 1 --2- 172.21.15.16:0/3934462864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646dd660 0x560a64ac48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.552+0000 7f785e000640 1 -- 172.21.15.16:0/3934462864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a64706520 msgr2=0x560a64ab5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.552+0000 7f785e000640 1 --2- 172.21.15.16:0/3934462864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a64706520 0x560a64ab5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.553+0000 7f785e000640 1 -- 172.21.15.16:0/3934462864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a64ab4430 con 0x560a646e5840 2024-09-21T03:55:25.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.553+0000 7f785ea00640 1 --2- 172.21.15.16:0/3934462864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646dd660 0x560a64ac48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:55:25.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.554+0000 7f785e000640 1 --2- 172.21.15.16:0/3934462864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a646e5840 0x560a64ac4e00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f784c0ce3a0 tx=0x7f784c0ce140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40b4300368a8b2d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:25.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.554+0000 7f784aa00640 1 -- 172.21.15.16:0/3934462864 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c0d31d0 con 0x560a646e5840 2024-09-21T03:55:25.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.555+0000 7f784aa00640 1 -- 172.21.15.16:0/3934462864 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f784c0d8110 con 0x560a646e5840 2024-09-21T03:55:25.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.555+0000 7f784aa00640 1 -- 172.21.15.16:0/3934462864 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c0d31d0 con 0x560a646e5840 2024-09-21T03:55:25.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.559+0000 7f786c2c4740 1 -- 172.21.15.16:0/3934462864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a646e5840 msgr2=0x560a64ac4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:25.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.559+0000 7f786c2c4740 1 --2- 172.21.15.16:0/3934462864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a646e5840 0x560a64ac4e00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f784c0ce3a0 tx=0x7f784c0ce140 comp rx=0 tx=0).stop 2024-09-21T03:55:25.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.560+0000 7f786c2c4740 1 -- 172.21.15.16:0/3934462864 shutdown_connections 2024-09-21T03:55:25.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.560+0000 7f786c2c4740 1 --2- 172.21.15.16:0/3934462864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a64706520 0x560a64ab5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.560+0000 7f786c2c4740 1 --2- 172.21.15.16:0/3934462864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a646e5840 0x560a64ac4e00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.560+0000 7f786c2c4740 1 --2- 172.21.15.16:0/3934462864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646dd660 0x560a64ac48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.560+0000 7f786c2c4740 1 -- 172.21.15.16:0/3934462864 >> 172.21.15.16:0/3934462864 conn(0x560a646fa910 msgr2=0x560a64ab22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:25.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.561+0000 7f786c2c4740 1 -- 172.21.15.16:0/3934462864 shutdown_connections 2024-09-21T03:55:25.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.561+0000 7f786c2c4740 1 -- 172.21.15.16:0/3934462864 wait complete. 2024-09-21T03:55:25.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.562+0000 7f786c2c4740 1 -- start start 2024-09-21T03:55:25.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.563+0000 7f786c2c4740 1 Processor -- start 2024-09-21T03:55:25.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.564+0000 7f786c2c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a646dd660 0x560a64abd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.564+0000 7f786c2c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646e5840 0x560a64abdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.565+0000 7f786c2c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a64706520 0x560a64abe440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.565+0000 7f785e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646e5840 0x560a64abdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:25.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.565+0000 7f785e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646e5840 0x560a64abdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39370/0 (socket says 172.21.15.16:39370) 2024-09-21T03:55:25.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.566+0000 7f785e000640 1 -- 172.21.15.16:0/4187269569 learned_addr learned my addr 172.21.15.16:0/4187269569 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:25.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.566+0000 7f786c2c4740 1 -- 172.21.15.16:0/4187269569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7840011180 con 0x560a646dd660 2024-09-21T03:55:25.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.566+0000 7f786c2c4740 1 -- 172.21.15.16:0/4187269569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7840010ab0 con 0x560a64706520 2024-09-21T03:55:25.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.566+0000 7f786c2c4740 1 -- 172.21.15.16:0/4187269569 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f784000dcf0 con 0x560a646e5840 2024-09-21T03:55:25.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.566+0000 7f786c2c4740 10 client.0 _ll_register_callbacks cb 0x7f7858009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:25.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.567+0000 7f785f400640 1 --2- 172.21.15.16:0/4187269569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a64706520 0x560a64abe440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:25.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.568+0000 7f785e000640 1 -- 172.21.15.16:0/4187269569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a64706520 msgr2=0x560a64abe440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:25.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.568+0000 7f785e000640 1 --2- 172.21.15.16:0/4187269569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a64706520 0x560a64abe440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.568+0000 7f785e000640 1 -- 172.21.15.16:0/4187269569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a646dd660 msgr2=0x560a64abd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:25.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.568+0000 7f785e000640 1 --2- 172.21.15.16:0/4187269569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a646dd660 0x560a64abd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.568+0000 7f785e000640 1 -- 172.21.15.16:0/4187269569 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a64b35560 con 0x560a646e5840 2024-09-21T03:55:25.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.570+0000 7f785f400640 1 --2- 172.21.15.16:0/4187269569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a64706520 0x560a64abe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:55:25.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.571+0000 7f785e000640 1 --2- 172.21.15.16:0/4187269569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646e5840 0x560a64abdf00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f784c0cefa0 tx=0x7f784c1060b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96dda9a009e24e8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:25.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.571+0000 7f783fe00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:55:25.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.572+0000 7f783fe00640 1 -- 172.21.15.16:0/4187269569 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c07d050 con 0x560a646e5840 2024-09-21T03:55:25.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.572+0000 7f783fe00640 1 -- 172.21.15.16:0/4187269569 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f784c0cc640 con 0x560a646e5840 2024-09-21T03:55:25.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.574+0000 7f786c2c4740 1 -- 172.21.15.16:0/4187269569 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f784c0d8110 con 0x560a646e5840 2024-09-21T03:55:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.576+0000 7f783fe00640 1 -- 172.21.15.16:0/4187269569 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f784c004420 con 0x560a646e5840 2024-09-21T03:55:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.576+0000 7f786c2c4740 10 client.26491 fetch_fsmap learned FSMap version 30 2024-09-21T03:55:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.576+0000 7f786c2c4740 1 -- 172.21.15.16:0/4187269569 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560a64b36820 con 0x560a646e5840 2024-09-21T03:55:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.577+0000 7f783fe00640 1 -- 172.21.15.16:0/4187269569 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f784c0d8110 con 0x560a646e5840 2024-09-21T03:55:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.577+0000 7f783fe00640 1 -- 172.21.15.16:0/4187269569 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f784c0f0310 con 0x560a646e5840 2024-09-21T03:55:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.578+0000 7f786c2c4740 10 client.26491 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:55:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.578+0000 7f786c2c4740 10 client.26491 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:55:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.578+0000 7f786c2c4740 20 client.26491 populate_metadata read hostname 'smithi016' 2024-09-21T03:55:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.578+0000 7f786c2c4740 1 --2- 172.21.15.16:0/4187269569 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x560a6469a250 0x560a64b38ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.579+0000 7f786c2c4740 4 client.26491 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "74ddd6a7", "release": true, "await": true}] 2024-09-21T03:55:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.579+0000 7f786c2c4740 1 -- 172.21.15.16:0/4187269569 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "set_id": "74ddd6a7", "release": true, "await": true}) -- 0x560a64a21c90 con 0x560a6469a250 2024-09-21T03:55:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.579+0000 7f785ea00640 1 --2- 172.21.15.16:0/4187269569 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x560a6469a250 0x560a64b38ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.581+0000 7f785ea00640 1 --2- 172.21.15.16:0/4187269569 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x560a6469a250 0x560a64b38ba0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f92be8cafed4fd67 server_cookie=fbb7a06dba307e22 in_seq=0 out_seq=0 2024-09-21T03:55:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.581+0000 7f783fe00640 10 client.26491 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.586+0000 7f783fe00640 1 -- 172.21.15.16:0/4187269569 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560a64a21c90 con 0x560a6469a250 2024-09-21T03:55:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.587+0000 7f783fe00640 10 client.26491 handle_command_reply: tid=0 2024-09-21T03:55:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.587+0000 7f786c2c4740 1 client.26491 shutdown 2024-09-21T03:55:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.588+0000 7f786c2c4740 1 -- 172.21.15.16:0/4187269569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646e5840 msgr2=0x560a64abdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.588+0000 7f786c2c4740 1 --2- 172.21.15.16:0/4187269569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646e5840 0x560a64abdf00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f784c0cefa0 tx=0x7f784c1060b0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.588+0000 7f786c2c4740 1 -- 172.21.15.16:0/4187269569 shutdown_connections 2024-09-21T03:55:25.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.588+0000 7f786c2c4740 1 --2- 172.21.15.16:0/4187269569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a64706520 0x560a64abe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.588+0000 7f786c2c4740 1 --2- 172.21.15.16:0/4187269569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a646e5840 0x560a64abdf00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.588+0000 7f786c2c4740 1 --2- 172.21.15.16:0/4187269569 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x560a6469a250 0x560a64b38ba0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.588+0000 7f786c2c4740 1 --2- 172.21.15.16:0/4187269569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a646dd660 0x560a64abd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.588+0000 7f786c2c4740 1 -- 172.21.15.16:0/4187269569 >> 172.21.15.16:0/4187269569 conn(0x560a646fa910 msgr2=0x560a64aae1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:25.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.588+0000 7f783fe00640 0 client.26491 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:25.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.590+0000 7f786c2c4740 1 -- 172.21.15.16:0/4187269569 shutdown_connections 2024-09-21T03:55:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.590+0000 7f786c2c4740 1 -- 172.21.15.16:0/4187269569 wait complete. 2024-09-21T03:55:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.590+0000 7f786c2c4740 20 client.26491 trim_cache size 0 max 16384 2024-09-21T03:55:25.600 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:55:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:55:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:55:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 36, 2024-09-21T03:55:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:55:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "74ddd6a7": { 2024-09-21T03:55:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 36, 2024-09-21T03:55:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:55:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:55:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:55:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:55:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:55:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:55:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 141.1, 2024-09-21T03:55:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:55:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:55:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:55:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:55:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:55:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:55:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:55:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:55:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:55:25.602 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:55:25.602 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:55:25.602 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:55:25.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.593+0000 7f7865600640 1 -- 172.21.15.16:0/968005653 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f78440b67e0 msgr2=0x7f78440b8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:25.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.593+0000 7f7865600640 1 --2- 172.21.15.16:0/968005653 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f78440b67e0 0x7f78440b8cf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7860114da0 tx=0x7f7850006c40 comp rx=0 tx=0).stop 2024-09-21T03:55:25.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.593+0000 7f7865600640 1 -- 172.21.15.16:0/968005653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f786010f6f0 msgr2=0x7f786011d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:25.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.593+0000 7f7865600640 1 --2- 172.21.15.16:0/968005653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f786010f6f0 0x7f786011d0e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f784c02bc30 tx=0x7f784c0315d0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.594+0000 7f7865600640 1 -- 172.21.15.16:0/968005653 shutdown_connections 2024-09-21T03:55:25.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.594+0000 7f7865600640 1 --2- 172.21.15.16:0/968005653 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f78440b67e0 0x7f78440b8cf0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.594+0000 7f7865600640 1 --2- 172.21.15.16:0/968005653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7860114fb0 0x7f7860123a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.594+0000 7f7865600640 1 --2- 172.21.15.16:0/968005653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f786010f6f0 0x7f786011d0e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.594+0000 7f7865600640 1 --2- 172.21.15.16:0/968005653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f786010a900 0x7f786011cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.594+0000 7f7865600640 1 -- 172.21.15.16:0/968005653 >> 172.21.15.16:0/968005653 conn(0x7f78600712f0 msgr2=0x7f786010d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.595+0000 7f7865600640 1 -- 172.21.15.16:0/968005653 shutdown_connections 2024-09-21T03:55:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:55:25.597+0000 7f7865600640 1 -- 172.21.15.16:0/968005653 wait complete. 2024-09-21T03:55:27.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:26 smithi134 ceph-mon[35031]: pgmap v692: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 245 MiB/s wr, 1.42k op/s 2024-09-21T03:55:27.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:26 smithi123 ceph-mon[34673]: pgmap v692: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 245 MiB/s wr, 1.42k op/s 2024-09-21T03:55:27.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:26 smithi016 ceph-mon[30375]: pgmap v692: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 245 MiB/s wr, 1.42k op/s 2024-09-21T03:55:28.670 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '74ddd6a7', total seconds elapsed: 34.3 2024-09-21T03:55:29.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:28 smithi134 ceph-mon[35031]: pgmap v693: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 233 MiB/s wr, 1.47k op/s 2024-09-21T03:55:29.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:28 smithi123 ceph-mon[34673]: pgmap v693: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 233 MiB/s wr, 1.47k op/s 2024-09-21T03:55:29.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:28 smithi016 ceph-mon[30375]: pgmap v693: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 233 MiB/s wr, 1.47k op/s 2024-09-21T03:55:31.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:30 smithi134 ceph-mon[35031]: pgmap v694: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 215 MiB/s wr, 1.49k op/s 2024-09-21T03:55:31.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:30 smithi123 ceph-mon[34673]: pgmap v694: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 215 MiB/s wr, 1.49k op/s 2024-09-21T03:55:31.417 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:30 smithi016 ceph-mon[30375]: pgmap v694: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 215 MiB/s wr, 1.49k op/s 2024-09-21T03:55:33.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:32 smithi134 ceph-mon[35031]: pgmap v695: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 215 MiB/s wr, 1.56k op/s 2024-09-21T03:55:33.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:32 smithi123 ceph-mon[34673]: pgmap v695: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 215 MiB/s wr, 1.56k op/s 2024-09-21T03:55:33.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:32 smithi016 ceph-mon[30375]: pgmap v695: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 215 MiB/s wr, 1.56k op/s 2024-09-21T03:55:35.215 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:34 smithi016 ceph-mon[30375]: pgmap v696: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 214 MiB/s wr, 1.62k op/s 2024-09-21T03:55:35.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:34 smithi134 ceph-mon[35031]: pgmap v696: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 214 MiB/s wr, 1.62k op/s 2024-09-21T03:55:35.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:34 smithi123 ceph-mon[34673]: pgmap v696: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 214 MiB/s wr, 1.62k op/s 2024-09-21T03:55:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.204+0000 7f8131600640 1 -- 172.21.15.16:0/525704681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f812c06b710 msgr2=0x7f812c106180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.204+0000 7f8131600640 1 --2- 172.21.15.16:0/525704681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f812c06b710 0x7f812c106180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f811c009900 tx=0x7f811c02a420 comp rx=0 tx=0).stop 2024-09-21T03:55:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.209+0000 7f8131600640 1 -- 172.21.15.16:0/525704681 shutdown_connections 2024-09-21T03:55:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.209+0000 7f8131600640 1 --2- 172.21.15.16:0/525704681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f812c109280 0x7f812c10b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.209+0000 7f8131600640 1 --2- 172.21.15.16:0/525704681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f812c1066c0 0x7f812c108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.209+0000 7f8131600640 1 --2- 172.21.15.16:0/525704681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f812c06b710 0x7f812c106180 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.209+0000 7f8131600640 1 -- 172.21.15.16:0/525704681 >> 172.21.15.16:0/525704681 conn(0x7f812c0fdd80 msgr2=0x7f812c100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.213+0000 7f8131600640 1 -- 172.21.15.16:0/525704681 shutdown_connections 2024-09-21T03:55:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.213+0000 7f8131600640 1 -- 172.21.15.16:0/525704681 wait complete. 2024-09-21T03:55:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.215+0000 7f8131600640 1 Processor -- start 2024-09-21T03:55:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.216+0000 7f8131600640 1 -- start start 2024-09-21T03:55:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.216+0000 7f8131600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f812c06b710 0x7f812c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.217+0000 7f8131600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f812c1066c0 0x7f812c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.217+0000 7f8131600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f812c109280 0x7f812c198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.217+0000 7f8131600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f812c10f200 con 0x7f812c1066c0 2024-09-21T03:55:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.217+0000 7f8131600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f812c10f080 con 0x7f812c109280 2024-09-21T03:55:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.217+0000 7f8131600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f812c10f380 con 0x7f812c06b710 2024-09-21T03:55:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.217+0000 7f812be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f812c06b710 0x7f812c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.217+0000 7f812be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f812c06b710 0x7f812c197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39306/0 (socket says 172.21.15.16:39306) 2024-09-21T03:55:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.217+0000 7f812be00640 1 -- 172.21.15.16:0/3703509028 learned_addr learned my addr 172.21.15.16:0/3703509028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.218+0000 7f8130c00640 1 --2- 172.21.15.16:0/3703509028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f812c109280 0x7f812c198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.218+0000 7f8130c00640 1 -- 172.21.15.16:0/3703509028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f812c06b710 msgr2=0x7f812c197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.218+0000 7f8130c00640 1 --2- 172.21.15.16:0/3703509028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f812c06b710 0x7f812c197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.218+0000 7f8130c00640 1 -- 172.21.15.16:0/3703509028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f812c1066c0 msgr2=0x7f812c1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.218+0000 7f8130c00640 1 --2- 172.21.15.16:0/3703509028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f812c1066c0 0x7f812c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.218+0000 7f8130c00640 1 -- 172.21.15.16:0/3703509028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812c199690 con 0x7f812c109280 2024-09-21T03:55:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.218+0000 7f812be00640 1 --2- 172.21.15.16:0/3703509028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f812c06b710 0x7f812c197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:55:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.219+0000 7f8130c00640 1 --2- 172.21.15.16:0/3703509028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f812c109280 0x7f812c198e70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f812400be20 tx=0x7f812400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.221+0000 7f8128c00640 1 -- 172.21.15.16:0/3703509028 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124019070 con 0x7f812c109280 2024-09-21T03:55:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.221+0000 7f8128c00640 1 -- 172.21.15.16:0/3703509028 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81240148f0 con 0x7f812c109280 2024-09-21T03:55:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.221+0000 7f8128c00640 1 -- 172.21.15.16:0/3703509028 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124012860 con 0x7f812c109280 2024-09-21T03:55:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.221+0000 7f8131600640 1 -- 172.21.15.16:0/3703509028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812c199980 con 0x7f812c109280 2024-09-21T03:55:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.221+0000 7f8131600640 1 -- 172.21.15.16:0/3703509028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f812c1a00f0 con 0x7f812c109280 2024-09-21T03:55:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.233+0000 7f8139c03740 1 Processor -- start 2024-09-21T03:55:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.235+0000 7f8128c00640 1 -- 172.21.15.16:0/3703509028 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f812401c070 con 0x7f812c109280 2024-09-21T03:55:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.238+0000 7f8128c00640 1 --2- 172.21.15.16:0/3703509028 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f81100b66e0 0x7f81100b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.238+0000 7f8139c03740 1 -- start start 2024-09-21T03:55:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.238+0000 7f8128c00640 1 -- 172.21.15.16:0/3703509028 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f81240e6e40 con 0x7f812c109280 2024-09-21T03:55:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.240+0000 7f8139c03740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfd131b0 0x5563d00ba2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.240+0000 7f8139c03740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcea2f0 0x5563d00ba920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.240+0000 7f8139c03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563cfcf24d0 0x5563d00bb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.240+0000 7f8139c03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5563d002d6b0 con 0x5563cfcf24d0 2024-09-21T03:55:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.240+0000 7f8139c03740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5563d001cae0 con 0x5563cfd131b0 2024-09-21T03:55:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.240+0000 7f8139c03740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5563d00208f0 con 0x5563cfcea2f0 2024-09-21T03:55:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.241+0000 7f812be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfd131b0 0x5563d00ba2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.241+0000 7f812be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfd131b0 0x5563d00ba2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:33994/0 (socket says 172.21.15.16:33994) 2024-09-21T03:55:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.241+0000 7f812be00640 1 -- 172.21.15.16:0/2707141052 learned_addr learned my addr 172.21.15.16:0/2707141052 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.242+0000 7f812b400640 1 --2- 172.21.15.16:0/2707141052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcea2f0 0x5563d00ba920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.243+0000 7f812b400640 1 --2- 172.21.15.16:0/3703509028 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f81100b66e0 0x7f81100b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.243+0000 7f812be00640 1 -- 172.21.15.16:0/2707141052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcea2f0 msgr2=0x5563d00ba920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.243+0000 7f812be00640 1 --2- 172.21.15.16:0/2707141052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcea2f0 0x5563d00ba920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.243+0000 7f812be00640 1 -- 172.21.15.16:0/2707141052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563cfcf24d0 msgr2=0x5563d00bb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.243+0000 7f812be00640 1 --2- 172.21.15.16:0/2707141052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563cfcf24d0 0x5563d00bb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.243+0000 7f812be00640 1 -- 172.21.15.16:0/2707141052 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563cffe9780 con 0x5563cfd131b0 2024-09-21T03:55:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.243+0000 7f812be00640 1 --2- 172.21.15.16:0/2707141052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfd131b0 0x5563d00ba2b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f811c0036a0 tx=0x7f811c0036d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cc6609d14ca4e43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.244+0000 7f8121200640 1 -- 172.21.15.16:0/2707141052 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c035030 con 0x5563cfd131b0 2024-09-21T03:55:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.244+0000 7f8121200640 1 -- 172.21.15.16:0/2707141052 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f811c040070 con 0x5563cfd131b0 2024-09-21T03:55:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.244+0000 7f8121200640 1 -- 172.21.15.16:0/2707141052 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c0456d0 con 0x5563cfd131b0 2024-09-21T03:55:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.245+0000 7f8139c03740 1 -- 172.21.15.16:0/2707141052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfd131b0 msgr2=0x5563d00ba2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.245+0000 7f8139c03740 1 --2- 172.21.15.16:0/2707141052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfd131b0 0x5563d00ba2b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f811c0036a0 tx=0x7f811c0036d0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.245+0000 7f8139c03740 1 -- 172.21.15.16:0/2707141052 shutdown_connections 2024-09-21T03:55:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.245+0000 7f8139c03740 1 --2- 172.21.15.16:0/2707141052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563cfcf24d0 0x5563d00bb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.245+0000 7f8139c03740 1 --2- 172.21.15.16:0/2707141052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcea2f0 0x5563d00ba920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.245+0000 7f8139c03740 1 --2- 172.21.15.16:0/2707141052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfd131b0 0x5563d00ba2b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.245+0000 7f8139c03740 1 -- 172.21.15.16:0/2707141052 >> 172.21.15.16:0/2707141052 conn(0x5563cfd07300 msgr2=0x5563d00b5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.250+0000 7f8139c03740 1 -- 172.21.15.16:0/2707141052 shutdown_connections 2024-09-21T03:55:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.251+0000 7f8139c03740 1 -- 172.21.15.16:0/2707141052 wait complete. 2024-09-21T03:55:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.253+0000 7f8139c03740 1 -- start start 2024-09-21T03:55:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.253+0000 7f8139c03740 1 Processor -- start 2024-09-21T03:55:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.254+0000 7f8139c03740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 0x5563d00c3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.254+0000 7f8139c03740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfd131b0 0x5563d00c4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.254+0000 7f8139c03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563cfcd1d80 0x5563d00c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.255+0000 7f812be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 0x5563d00c3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.255+0000 7f812be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 0x5563d00c3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39328/0 (socket says 172.21.15.16:39328) 2024-09-21T03:55:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.255+0000 7f812be00640 1 -- 172.21.15.16:0/2722596914 learned_addr learned my addr 172.21.15.16:0/2722596914 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.255+0000 7f8139c03740 1 -- 172.21.15.16:0/2722596914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5563cff54ab0 con 0x5563cfcd1d80 2024-09-21T03:55:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.255+0000 7f8139c03740 1 -- 172.21.15.16:0/2722596914 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5563cffc3570 con 0x5563cfd131b0 2024-09-21T03:55:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.255+0000 7f8139c03740 1 -- 172.21.15.16:0/2722596914 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5563d00bbf30 con 0x5563cfcf24d0 2024-09-21T03:55:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.256+0000 7f8139c03740 10 client.0 _ll_register_callbacks cb 0x5563cffd2100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.256+0000 7f812be00640 1 -- 172.21.15.16:0/2722596914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfd131b0 msgr2=0x5563d00c4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.256+0000 7f812be00640 1 --2- 172.21.15.16:0/2722596914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfd131b0 0x5563d00c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.256+0000 7f812be00640 1 -- 172.21.15.16:0/2722596914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563cfcd1d80 msgr2=0x5563d00c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.256+0000 7f812be00640 1 --2- 172.21.15.16:0/2722596914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563cfcd1d80 0x5563d00c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.256+0000 7f812b400640 1 --2- 172.21.15.16:0/3703509028 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f81100b66e0 0x7f81100b8bf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8118005fd0 tx=0x7f8118005ac0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.256+0000 7f812be00640 1 -- 172.21.15.16:0/2722596914 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563cfd7e790 con 0x5563cfcf24d0 2024-09-21T03:55:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.256+0000 7f8139c03740 1 -- 172.21.15.16:0/2722596914 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f811c040070 con 0x5563cfcf24d0 2024-09-21T03:55:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.257+0000 7f812be00640 1 --2- 172.21.15.16:0/2722596914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 0x5563d00c3980 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f811c0092a0 tx=0x7f811c003d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c4de04e084d2c6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.257+0000 7f8115600640 10 client.26500 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:55:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.257+0000 7f8115600640 1 -- 172.21.15.16:0/2722596914 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c079050 con 0x5563cfcf24d0 2024-09-21T03:55:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.257+0000 7f8115600640 1 -- 172.21.15.16:0/2722596914 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f811c0317b0 con 0x5563cfcf24d0 2024-09-21T03:55:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.258+0000 7f8115600640 1 -- 172.21.15.16:0/2722596914 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f811c079050 con 0x5563cfcf24d0 2024-09-21T03:55:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.258+0000 7f8139c03740 10 client.26500 fetch_fsmap learned FSMap version 30 2024-09-21T03:55:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.258+0000 7f8139c03740 1 -- 172.21.15.16:0/2722596914 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5563cfdf7620 con 0x5563cfcf24d0 2024-09-21T03:55:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.260+0000 7f8115600640 1 -- 172.21.15.16:0/2722596914 <== mon.1 v2:172.21.15.134:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f811c045470 con 0x5563cfcf24d0 2024-09-21T03:55:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.260+0000 7f8139c03740 10 client.26500 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:55:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.260+0000 7f8139c03740 10 client.26500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.260+0000 7f8139c03740 20 client.26500 populate_metadata read hostname 'smithi016' 2024-09-21T03:55:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.261+0000 7f8139c03740 1 --2- 172.21.15.16:0/2722596914 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5563cfca71c0 0x5563d00d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.261+0000 7f8139c03740 4 client.26500 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.261+0000 7f8139c03740 1 -- 172.21.15.16:0/2722596914 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563d002f600 con 0x5563cfca71c0 2024-09-21T03:55:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.261+0000 7f812b400640 1 --2- 172.21.15.16:0/2722596914 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5563cfca71c0 0x5563d00d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.267+0000 7f8115600640 10 client.26500 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.267+0000 7f812b400640 1 --2- 172.21.15.16:0/2722596914 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5563cfca71c0 0x5563d00d0f50 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1917bf4ae7c10812 server_cookie=3f3808aef2a4d3bb in_seq=0 out_seq=0 2024-09-21T03:55:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.268+0000 7f8115600640 1 -- 172.21.15.16:0/2722596914 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5563d002f600 con 0x5563cfca71c0 2024-09-21T03:55:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.268+0000 7f8115600640 10 client.26500 handle_command_reply: tid=0 2024-09-21T03:55:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.268+0000 7f8139c03740 1 client.26500 shutdown 2024-09-21T03:55:37.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.269+0000 7f8139c03740 1 -- 172.21.15.16:0/2722596914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 msgr2=0x5563d00c3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:37.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.269+0000 7f8139c03740 1 --2- 172.21.15.16:0/2722596914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 0x5563d00c3980 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f811c0092a0 tx=0x7f811c003d20 comp rx=0 tx=0).stop 2024-09-21T03:55:37.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.269+0000 7f8139c03740 1 -- 172.21.15.16:0/2722596914 shutdown_connections 2024-09-21T03:55:37.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.269+0000 7f8139c03740 1 --2- 172.21.15.16:0/2722596914 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5563cfca71c0 0x5563d00d0f50 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.269+0000 7f8115600640 0 client.26500 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.269+0000 7f8139c03740 1 --2- 172.21.15.16:0/2722596914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563cfcd1d80 0x5563d00c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.270+0000 7f8139c03740 1 --2- 172.21.15.16:0/2722596914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfd131b0 0x5563d00c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.270+0000 7f8139c03740 1 --2- 172.21.15.16:0/2722596914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 0x5563d00c3980 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.270+0000 7f8139c03740 1 -- 172.21.15.16:0/2722596914 >> 172.21.15.16:0/2722596914 conn(0x5563cfd07300 msgr2=0x5563d00b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.270+0000 7f8139c03740 1 -- 172.21.15.16:0/2722596914 shutdown_connections 2024-09-21T03:55:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.270+0000 7f8139c03740 1 -- 172.21.15.16:0/2722596914 wait complete. 2024-09-21T03:55:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.271+0000 7f8139c03740 20 client.26500 trim_cache size 0 max 16384 2024-09-21T03:55:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.329+0000 7f8139c03740 1 Processor -- start 2024-09-21T03:55:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.330+0000 7f8139c03740 1 -- start start 2024-09-21T03:55:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.330+0000 7f8139c03740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfca71c0 0x5563d00c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.330+0000 7f8139c03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563cfcd1d80 0x5563d00cba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.331+0000 7f8139c03740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 0x5563d00c1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.331+0000 7f8139c03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f812c1aa3e0 con 0x5563cfcd1d80 2024-09-21T03:55:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.331+0000 7f8139c03740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f812c1abf20 con 0x5563cfca71c0 2024-09-21T03:55:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.331+0000 7f8139c03740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f812c1aaab0 con 0x5563cfcf24d0 2024-09-21T03:55:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.332+0000 7f8130c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 0x5563d00c1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.332+0000 7f8130c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 0x5563d00c1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:39332/0 (socket says 172.21.15.16:39332) 2024-09-21T03:55:37.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.332+0000 7f8130c00640 1 -- 172.21.15.16:0/3933542926 learned_addr learned my addr 172.21.15.16:0/3933542926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.332+0000 7f8130c00640 1 -- 172.21.15.16:0/3933542926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfca71c0 msgr2=0x5563d00c36b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:55:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.332+0000 7f8130c00640 1 --2- 172.21.15.16:0/3933542926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfca71c0 0x5563d00c36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.332+0000 7f8130c00640 1 -- 172.21.15.16:0/3933542926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563cfcd1d80 msgr2=0x5563d00cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.332+0000 7f8130c00640 1 --2- 172.21.15.16:0/3933542926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563cfcd1d80 0x5563d00cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.333+0000 7f8130c00640 1 -- 172.21.15.16:0/3933542926 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563d00c4f20 con 0x5563cfcf24d0 2024-09-21T03:55:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.334+0000 7f8130c00640 1 --2- 172.21.15.16:0/3933542926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 0x5563d00c1f10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f81240ac970 tx=0x7f81240ac3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26ebc6dc66eafc76 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.334+0000 7f8117e00640 1 -- 172.21.15.16:0/3933542926 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81240b16a0 con 0x5563cfcf24d0 2024-09-21T03:55:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.336+0000 7f8117e00640 1 -- 172.21.15.16:0/3933542926 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81240b65e0 con 0x5563cfcf24d0 2024-09-21T03:55:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.336+0000 7f8117e00640 1 -- 172.21.15.16:0/3933542926 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81240b63b0 con 0x5563cfcf24d0 2024-09-21T03:55:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.341+0000 7f8139c03740 1 -- 172.21.15.16:0/3933542926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 msgr2=0x5563d00c1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.341+0000 7f8139c03740 1 --2- 172.21.15.16:0/3933542926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 0x5563d00c1f10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f81240ac970 tx=0x7f81240ac3e0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.344+0000 7f8139c03740 1 -- 172.21.15.16:0/3933542926 shutdown_connections 2024-09-21T03:55:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.344+0000 7f8139c03740 1 --2- 172.21.15.16:0/3933542926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 0x5563d00c1f10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.344+0000 7f8139c03740 1 --2- 172.21.15.16:0/3933542926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563cfcd1d80 0x5563d00cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.344+0000 7f8139c03740 1 --2- 172.21.15.16:0/3933542926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfca71c0 0x5563d00c36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.344+0000 7f8139c03740 1 -- 172.21.15.16:0/3933542926 >> 172.21.15.16:0/3933542926 conn(0x5563cfd07300 msgr2=0x5563d00b9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.345+0000 7f8139c03740 1 -- 172.21.15.16:0/3933542926 shutdown_connections 2024-09-21T03:55:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.345+0000 7f8139c03740 1 -- 172.21.15.16:0/3933542926 wait complete. 2024-09-21T03:55:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.346+0000 7f8139c03740 1 -- start start 2024-09-21T03:55:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.346+0000 7f8139c03740 1 Processor -- start 2024-09-21T03:55:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.347+0000 7f8139c03740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfca71c0 0x5563d00cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.348+0000 7f812be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfca71c0 0x5563d00cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.348+0000 7f812be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfca71c0 0x5563d00cb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34018/0 (socket says 172.21.15.16:34018) 2024-09-21T03:55:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.348+0000 7f812be00640 1 -- 172.21.15.16:0/4130429603 learned_addr learned my addr 172.21.15.16:0/4130429603 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.348+0000 7f8139c03740 1 --2- 172.21.15.16:0/4130429603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563cfcd1d80 0x5563d00cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.348+0000 7f8139c03740 1 --2- 172.21.15.16:0/4130429603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 0x5563d00ca190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.349+0000 7f8139c03740 1 -- 172.21.15.16:0/4130429603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f812c1ab850 con 0x5563cfcd1d80 2024-09-21T03:55:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.349+0000 7f8139c03740 1 -- 172.21.15.16:0/4130429603 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f812c1ab180 con 0x5563cfca71c0 2024-09-21T03:55:37.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:36 smithi134 ceph-mon[35031]: pgmap v697: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 216 MiB/s wr, 1.68k op/s 2024-09-21T03:55:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.349+0000 7f8139c03740 1 -- 172.21.15.16:0/4130429603 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f812c1a8360 con 0x5563cfcf24d0 2024-09-21T03:55:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.349+0000 7f8139c03740 10 client.0 _ll_register_callbacks cb 0x7f811c0035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.350+0000 7f8130c00640 1 --2- 172.21.15.16:0/4130429603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 0x5563d00ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.350+0000 7f8130c00640 1 -- 172.21.15.16:0/4130429603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfca71c0 msgr2=0x5563d00cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.351+0000 7f8130c00640 1 --2- 172.21.15.16:0/4130429603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfca71c0 0x5563d00cb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.351+0000 7f8130c00640 1 -- 172.21.15.16:0/4130429603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563cfcd1d80 msgr2=0x5563d00cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.351+0000 7f8130c00640 1 --2- 172.21.15.16:0/4130429603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563cfcd1d80 0x5563d00cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.351+0000 7f8130c00640 1 -- 172.21.15.16:0/4130429603 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563d00ca920 con 0x5563cfcf24d0 2024-09-21T03:55:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.351+0000 7f8130c00640 1 --2- 172.21.15.16:0/4130429603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 0x5563d00ca190 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f81240ac970 tx=0x7f81240a9170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b42a51f7bc33738 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.351+0000 7f8115600640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:55:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.351+0000 7f8139c03740 1 -- 172.21.15.16:0/4130429603 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81240b65e0 con 0x5563cfcf24d0 2024-09-21T03:55:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.352+0000 7f8115600640 1 -- 172.21.15.16:0/4130429603 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81240b16a0 con 0x5563cfcf24d0 2024-09-21T03:55:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.357+0000 7f8115600640 1 -- 172.21.15.16:0/4130429603 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f81240b11c0 con 0x5563cfcf24d0 2024-09-21T03:55:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.357+0000 7f8115600640 1 -- 172.21.15.16:0/4130429603 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f81240b13c0 con 0x5563cfcf24d0 2024-09-21T03:55:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.357+0000 7f8139c03740 10 client.26506 fetch_fsmap learned FSMap version 30 2024-09-21T03:55:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.357+0000 7f8139c03740 1 -- 172.21.15.16:0/4130429603 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5563d00c96e0 con 0x5563cfcf24d0 2024-09-21T03:55:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.358+0000 7f8115600640 1 -- 172.21.15.16:0/4130429603 <== mon.1 v2:172.21.15.134:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8124014440 con 0x5563cfcf24d0 2024-09-21T03:55:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.358+0000 7f8139c03740 10 client.26506 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:55:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.358+0000 7f8139c03740 10 client.26506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.358+0000 7f8139c03740 20 client.26506 populate_metadata read hostname 'smithi016' 2024-09-21T03:55:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.358+0000 7f8139c03740 1 --2- 172.21.15.16:0/4130429603 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5563cfd131b0 0x5563d013dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.359+0000 7f8139c03740 4 client.26506 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:55:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.359+0000 7f8139c03740 1 -- 172.21.15.16:0/4130429603 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563d002b0b0 con 0x5563cfd131b0 2024-09-21T03:55:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.359+0000 7f812be00640 1 --2- 172.21.15.16:0/4130429603 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5563cfd131b0 0x5563d013dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.363+0000 7f8115600640 10 client.26506 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.363+0000 7f812be00640 1 --2- 172.21.15.16:0/4130429603 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5563cfd131b0 0x5563d013dbe0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3c5c0efa8e71952 server_cookie=21ea5b2e3a678b62 in_seq=0 out_seq=0 2024-09-21T03:55:37.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:36 smithi123 ceph-mon[34673]: pgmap v697: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 216 MiB/s wr, 1.68k op/s 2024-09-21T03:55:37.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:36 smithi016 ceph-mon[30375]: pgmap v697: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 216 MiB/s wr, 1.68k op/s 2024-09-21T03:55:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.436+0000 7f8115600640 1 -- 172.21.15.16:0/4130429603 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563d002b0b0 con 0x5563cfd131b0 2024-09-21T03:55:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.436+0000 7f8115600640 10 client.26506 handle_command_reply: tid=0 2024-09-21T03:55:37.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.436+0000 7f8139c03740 1 client.26506 shutdown 2024-09-21T03:55:37.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.441+0000 7f8139c03740 1 -- 172.21.15.16:0/4130429603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 msgr2=0x5563d00ca190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:37.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.441+0000 7f8139c03740 1 --2- 172.21.15.16:0/4130429603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 0x5563d00ca190 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f81240ac970 tx=0x7f81240a9170 comp rx=0 tx=0).stop 2024-09-21T03:55:37.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.443+0000 7f8139c03740 1 -- 172.21.15.16:0/4130429603 shutdown_connections 2024-09-21T03:55:37.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.443+0000 7f8139c03740 1 --2- 172.21.15.16:0/4130429603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cfcf24d0 0x5563d00ca190 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.443+0000 7f8139c03740 1 --2- 172.21.15.16:0/4130429603 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5563cfd131b0 0x5563d013dbe0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.443+0000 7f8139c03740 1 --2- 172.21.15.16:0/4130429603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563cfcd1d80 0x5563d00cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.443+0000 7f8139c03740 1 --2- 172.21.15.16:0/4130429603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563cfca71c0 0x5563d00cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.443+0000 7f8115600640 0 client.26506 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:37.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.443+0000 7f8139c03740 1 -- 172.21.15.16:0/4130429603 >> 172.21.15.16:0/4130429603 conn(0x5563cfd07300 msgr2=0x5563d00c1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:37.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.445+0000 7f8139c03740 1 -- 172.21.15.16:0/4130429603 shutdown_connections 2024-09-21T03:55:37.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.446+0000 7f8139c03740 1 -- 172.21.15.16:0/4130429603 wait complete. 2024-09-21T03:55:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:55:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:55:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:55:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:55:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.447+0000 7f8139c03740 20 client.26506 trim_cache size 0 max 16384 2024-09-21T03:55:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.455+0000 7f8131600640 1 -- 172.21.15.16:0/3703509028 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f81100b66e0 msgr2=0x7f81100b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.455+0000 7f8131600640 1 --2- 172.21.15.16:0/3703509028 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f81100b66e0 0x7f81100b8bf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8118005fd0 tx=0x7f8118005ac0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.455+0000 7f8131600640 1 -- 172.21.15.16:0/3703509028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f812c109280 msgr2=0x7f812c198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.455+0000 7f8131600640 1 --2- 172.21.15.16:0/3703509028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f812c109280 0x7f812c198e70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f812400be20 tx=0x7f812400bf10 comp rx=0 tx=0).stop 2024-09-21T03:55:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.456+0000 7f8131600640 1 -- 172.21.15.16:0/3703509028 shutdown_connections 2024-09-21T03:55:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.456+0000 7f8131600640 1 --2- 172.21.15.16:0/3703509028 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f81100b66e0 0x7f81100b8bf0 secure :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8118005fd0 tx=0x7f8118005ac0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.456+0000 7f8131600640 1 --2- 172.21.15.16:0/3703509028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f812c109280 0x7f812c198e70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.456+0000 7f8131600640 1 --2- 172.21.15.16:0/3703509028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f812c1066c0 0x7f812c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.456+0000 7f8131600640 1 --2- 172.21.15.16:0/3703509028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f812c06b710 0x7f812c197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.456+0000 7f8131600640 1 -- 172.21.15.16:0/3703509028 >> 172.21.15.16:0/3703509028 conn(0x7f812c0fdd80 msgr2=0x7f812c1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.456+0000 7f8131600640 1 -- 172.21.15.16:0/3703509028 shutdown_connections 2024-09-21T03:55:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:37.456+0000 7f8131600640 1 -- 172.21.15.16:0/3703509028 wait complete. 2024-09-21T03:55:39.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:38 smithi134 ceph-mon[35031]: pgmap v698: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 207 MiB/s wr, 1.60k op/s 2024-09-21T03:55:39.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:38 smithi123 ceph-mon[34673]: pgmap v698: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 207 MiB/s wr, 1.60k op/s 2024-09-21T03:55:39.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:38 smithi016 ceph-mon[30375]: pgmap v698: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 207 MiB/s wr, 1.60k op/s 2024-09-21T03:55:41.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:41 smithi134 ceph-mon[35031]: pgmap v699: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 202 MiB/s wr, 1.50k op/s 2024-09-21T03:55:41.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:41 smithi123 ceph-mon[34673]: pgmap v699: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 202 MiB/s wr, 1.50k op/s 2024-09-21T03:55:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:40 smithi016 ceph-mon[30375]: pgmap v699: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 202 MiB/s wr, 1.50k op/s 2024-09-21T03:55:43.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:43 smithi123 ceph-mon[34673]: pgmap v700: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 212 MiB/s wr, 1.49k op/s 2024-09-21T03:55:43.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:43 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:43.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:43 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:43 smithi134 ceph-mon[35031]: pgmap v700: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 212 MiB/s wr, 1.49k op/s 2024-09-21T03:55:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:43 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:43 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:43.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:43 smithi016 ceph-mon[30375]: pgmap v700: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 212 MiB/s wr, 1.49k op/s 2024-09-21T03:55:43.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:43 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:43.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:43 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:44 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:44.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:44 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:44.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:44 smithi134 ceph-mon[35031]: pgmap v701: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 223 MiB/s wr, 1.52k op/s 2024-09-21T03:55:44.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:44 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:44.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:44 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:44.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:44 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:44.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:44 smithi123 ceph-mon[34673]: pgmap v701: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 223 MiB/s wr, 1.52k op/s 2024-09-21T03:55:44.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:44 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:44.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:44 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:44.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:44 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:44.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:44 smithi016 ceph-mon[30375]: pgmap v701: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 223 MiB/s wr, 1.52k op/s 2024-09-21T03:55:44.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:44 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:46.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:46 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:46.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:46 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:46.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:46 smithi134 ceph-mon[35031]: pgmap v702: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 231 MiB/s wr, 1.54k op/s 2024-09-21T03:55:46.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:46 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:46.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:46 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:46.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:46 smithi123 ceph-mon[34673]: pgmap v702: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 231 MiB/s wr, 1.54k op/s 2024-09-21T03:55:46.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:46 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:46.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:46 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:46.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:46 smithi016 ceph-mon[30375]: pgmap v702: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 231 MiB/s wr, 1.54k op/s 2024-09-21T03:55:49.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:48 smithi123 ceph-mon[34673]: pgmap v703: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 222 MiB/s wr, 1.43k op/s 2024-09-21T03:55:49.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:48 smithi016 ceph-mon[30375]: pgmap v703: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 222 MiB/s wr, 1.43k op/s 2024-09-21T03:55:49.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:48 smithi134 ceph-mon[35031]: pgmap v703: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 222 MiB/s wr, 1.43k op/s 2024-09-21T03:55:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.864+0000 7ff6f1600640 1 -- 172.21.15.16:0/4154071895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6ec10cdb0 msgr2=0x7ff6ec10f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.864+0000 7ff6f1600640 1 --2- 172.21.15.16:0/4154071895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6ec10cdb0 0x7ff6ec10f290 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009900 tx=0x7ff6dc02a410 comp rx=0 tx=0).stop 2024-09-21T03:55:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.865+0000 7ff6f1600640 1 -- 172.21.15.16:0/4154071895 shutdown_connections 2024-09-21T03:55:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.865+0000 7ff6f1600640 1 --2- 172.21.15.16:0/4154071895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6ec10f9d0 0x7ff6ec11ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.865+0000 7ff6f1600640 1 --2- 172.21.15.16:0/4154071895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6ec10cdb0 0x7ff6ec10f290 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.865+0000 7ff6f1600640 1 --2- 172.21.15.16:0/4154071895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec10a3c0 0x7ff6ec10c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.865+0000 7ff6f1600640 1 -- 172.21.15.16:0/4154071895 >> 172.21.15.16:0/4154071895 conn(0x7ff6ec070810 msgr2=0x7ff6ec072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.865+0000 7ff6f1600640 1 -- 172.21.15.16:0/4154071895 shutdown_connections 2024-09-21T03:55:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.866+0000 7ff6f1600640 1 -- 172.21.15.16:0/4154071895 wait complete. 2024-09-21T03:55:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.869+0000 7ff6f1600640 1 Processor -- start 2024-09-21T03:55:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.869+0000 7ff6f1600640 1 -- start start 2024-09-21T03:55:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.873+0000 7ff6f1600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6ec10a3c0 0x7ff6ec1178e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.875+0000 7ff6f1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6ec10cdb0 0x7ff6ec117f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.875+0000 7ff6f1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec10f9d0 0x7ff6ec118b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.875+0000 7ff6f1600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec11e120 con 0x7ff6ec10f9d0 2024-09-21T03:55:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.875+0000 7ff6f1600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec11dfa0 con 0x7ff6ec10cdb0 2024-09-21T03:55:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.875+0000 7ff6f1600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec11e2a0 con 0x7ff6ec10a3c0 2024-09-21T03:55:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.876+0000 7ff6eb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec10f9d0 0x7ff6ec118b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.877+0000 7ff6eb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec10f9d0 0x7ff6ec118b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48002/0 (socket says 172.21.15.16:48002) 2024-09-21T03:55:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.877+0000 7ff6eb400640 1 -- 172.21.15.16:0/2858620067 learned_addr learned my addr 172.21.15.16:0/2858620067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.878+0000 7ff6eb400640 1 -- 172.21.15.16:0/2858620067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6ec10a3c0 msgr2=0x7ff6ec1178e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:55:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.878+0000 7ff6eb400640 1 --2- 172.21.15.16:0/2858620067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6ec10a3c0 0x7ff6ec1178e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.878+0000 7ff6eb400640 1 -- 172.21.15.16:0/2858620067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6ec10cdb0 msgr2=0x7ff6ec117f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.878+0000 7ff6eb400640 1 --2- 172.21.15.16:0/2858620067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6ec10cdb0 0x7ff6ec117f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.878+0000 7ff6eb400640 1 -- 172.21.15.16:0/2858620067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ec119290 con 0x7ff6ec10f9d0 2024-09-21T03:55:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.880+0000 7ff6eb400640 1 --2- 172.21.15.16:0/2858620067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec10f9d0 0x7ff6ec118b00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400eb10 tx=0x7ff6e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.886+0000 7ff6e3e00640 1 -- 172.21.15.16:0/2858620067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4004b20 con 0x7ff6ec10f9d0 2024-09-21T03:55:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.887+0000 7ff6e3e00640 1 -- 172.21.15.16:0/2858620067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6e4004cc0 con 0x7ff6ec10f9d0 2024-09-21T03:55:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.887+0000 7ff6e3e00640 1 -- 172.21.15.16:0/2858620067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4010670 con 0x7ff6ec10f9d0 2024-09-21T03:55:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.887+0000 7ff6f1600640 1 -- 172.21.15.16:0/2858620067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ec119580 con 0x7ff6ec10f9d0 2024-09-21T03:55:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.887+0000 7ff6f1600640 1 -- 172.21.15.16:0/2858620067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ec119af0 con 0x7ff6ec10f9d0 2024-09-21T03:55:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.908+0000 7ff6e3e00640 1 -- 172.21.15.16:0/2858620067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff6e4010830 con 0x7ff6ec10f9d0 2024-09-21T03:55:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.909+0000 7ff6f85e7740 1 Processor -- start 2024-09-21T03:55:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.914+0000 7ff6f85e7740 1 -- start start 2024-09-21T03:55:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.924+0000 7ff6f85e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210083c1b0 0x562100be30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.924+0000 7ff6f85e7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621008132f0 0x562100be3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.924+0000 7ff6f85e7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56210081b4d0 0x562100be4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.924+0000 7ff6f85e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562100b566b0 con 0x56210083c1b0 2024-09-21T03:55:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.924+0000 7ff6f85e7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562100b45ae0 con 0x56210081b4d0 2024-09-21T03:55:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.924+0000 7ff6f85e7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562100b498f0 con 0x5621008132f0 2024-09-21T03:55:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.925+0000 7ff6e3e00640 1 --2- 172.21.15.16:0/2858620067 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff6d00b6720 0x7ff6d00b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.925+0000 7ff6e2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56210081b4d0 0x562100be4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.925+0000 7ff6e2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56210081b4d0 0x562100be4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:36230/0 (socket says 172.21.15.16:36230) 2024-09-21T03:55:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.925+0000 7ff6e2000640 1 -- 172.21.15.16:0/3744734867 learned_addr learned my addr 172.21.15.16:0/3744734867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.925+0000 7ff6e3e00640 1 -- 172.21.15.16:0/2858620067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7ff6e40e7cd0 con 0x7ff6ec10f9d0 2024-09-21T03:55:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.926+0000 7ff6e2000640 1 -- 172.21.15.16:0/3744734867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621008132f0 msgr2=0x562100be3740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:55:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.926+0000 7ff6e2000640 1 --2- 172.21.15.16:0/3744734867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621008132f0 0x562100be3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.926+0000 7ff6e2000640 1 -- 172.21.15.16:0/3744734867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210083c1b0 msgr2=0x562100be30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.926+0000 7ff6e2000640 1 --2- 172.21.15.16:0/3744734867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210083c1b0 0x562100be30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.926+0000 7ff6e2000640 1 -- 172.21.15.16:0/3744734867 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562100b12780 con 0x56210081b4d0 2024-09-21T03:55:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.929+0000 7ff6eb400640 1 --2- 172.21.15.16:0/2858620067 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff6d00b6720 0x7ff6d00b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.930+0000 7ff6e2000640 1 --2- 172.21.15.16:0/3744734867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56210081b4d0 0x562100be4290 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0309b0 tx=0x7ff6dc005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=273e3dc5319a557f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.930+0000 7ff6db400640 1 -- 172.21.15.16:0/3744734867 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc030620 con 0x56210081b4d0 2024-09-21T03:55:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.930+0000 7ff6db400640 1 -- 172.21.15.16:0/3744734867 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6dc004030 con 0x56210081b4d0 2024-09-21T03:55:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.930+0000 7ff6db400640 1 -- 172.21.15.16:0/3744734867 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc005530 con 0x56210081b4d0 2024-09-21T03:55:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.932+0000 7ff6f85e7740 1 -- 172.21.15.16:0/3744734867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56210081b4d0 msgr2=0x562100be4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.932+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/3744734867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56210081b4d0 0x562100be4290 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0309b0 tx=0x7ff6dc005ec0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.932+0000 7ff6eb400640 1 --2- 172.21.15.16:0/2858620067 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff6d00b6720 0x7ff6d00b8c30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40ad9e0 tx=0x7ff6e400cf70 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.934+0000 7ff6f85e7740 1 -- 172.21.15.16:0/3744734867 shutdown_connections 2024-09-21T03:55:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.934+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/3744734867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56210081b4d0 0x562100be4290 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.934+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/3744734867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621008132f0 0x562100be3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.934+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/3744734867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210083c1b0 0x562100be30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.934+0000 7ff6f85e7740 1 -- 172.21.15.16:0/3744734867 >> 172.21.15.16:0/3744734867 conn(0x562100830300 msgr2=0x562100bdeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.937+0000 7ff6f85e7740 1 -- 172.21.15.16:0/3744734867 shutdown_connections 2024-09-21T03:55:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.937+0000 7ff6f85e7740 1 -- 172.21.15.16:0/3744734867 wait complete. 2024-09-21T03:55:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.938+0000 7ff6f85e7740 1 -- start start 2024-09-21T03:55:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.938+0000 7ff6f85e7740 1 Processor -- start 2024-09-21T03:55:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.945+0000 7ff6f85e7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621008132f0 0x562100bec830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.945+0000 7ff6f85e7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56210081b4d0 0x562100becef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.945+0000 7ff6f85e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210083c1b0 0x562100beda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.946+0000 7ff6f85e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562100a7dab0 con 0x56210083c1b0 2024-09-21T03:55:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.946+0000 7ff6f85e7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562100aec570 con 0x5621008132f0 2024-09-21T03:55:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.946+0000 7ff6f85e7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562100be4d50 con 0x56210081b4d0 2024-09-21T03:55:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.946+0000 7ff6f85e7740 10 client.0 _ll_register_callbacks cb 0x562100afb100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.947+0000 7ff6e2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210083c1b0 0x562100beda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.947+0000 7ff6e2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621008132f0 0x562100bec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.947+0000 7ff6eaa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56210081b4d0 0x562100becef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.947+0000 7ff6eaa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56210081b4d0 0x562100becef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:43734/0 (socket says 172.21.15.16:43734) 2024-09-21T03:55:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.947+0000 7ff6eaa00640 1 -- 172.21.15.16:0/3060749704 learned_addr learned my addr 172.21.15.16:0/3060749704 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.948+0000 7ff6eaa00640 1 -- 172.21.15.16:0/3060749704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621008132f0 msgr2=0x562100bec830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.948+0000 7ff6eaa00640 1 --2- 172.21.15.16:0/3060749704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621008132f0 0x562100bec830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.948+0000 7ff6eaa00640 1 -- 172.21.15.16:0/3060749704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210083c1b0 msgr2=0x562100beda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.948+0000 7ff6eaa00640 1 --2- 172.21.15.16:0/3060749704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210083c1b0 0x562100beda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.948+0000 7ff6eaa00640 1 -- 172.21.15.16:0/3060749704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621008a7790 con 0x56210081b4d0 2024-09-21T03:55:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.957+0000 7ff6eaa00640 1 --2- 172.21.15.16:0/3060749704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56210081b4d0 0x562100becef0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400c380 tx=0x7ff6d400c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b530202800608a9b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.958+0000 7ff6cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:55:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.958+0000 7ff6cfe00640 1 -- 172.21.15.16:0/3060749704 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d400e880 con 0x56210081b4d0 2024-09-21T03:55:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.958+0000 7ff6cfe00640 1 -- 172.21.15.16:0/3060749704 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7ff6d400ea20 con 0x56210081b4d0 2024-09-21T03:55:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.960+0000 7ff6f85e7740 1 -- 172.21.15.16:0/3060749704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6dc004030 con 0x56210081b4d0 2024-09-21T03:55:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.961+0000 7ff6cfe00640 1 -- 172.21.15.16:0/3060749704 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff6d400ec60 con 0x56210081b4d0 2024-09-21T03:55:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.961+0000 7ff6f85e7740 10 client.26512 fetch_fsmap learned FSMap version 30 2024-09-21T03:55:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.961+0000 7ff6f85e7740 1 -- 172.21.15.16:0/3060749704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562100920620 con 0x56210081b4d0 2024-09-21T03:55:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.965+0000 7ff6cfe00640 1 -- 172.21.15.16:0/3060749704 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6dc004030 con 0x56210081b4d0 2024-09-21T03:55:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.966+0000 7ff6cfe00640 1 -- 172.21.15.16:0/3060749704 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff6d40224b0 con 0x56210081b4d0 2024-09-21T03:55:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.967+0000 7ff6f85e7740 10 client.26512 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:55:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.967+0000 7ff6f85e7740 10 client.26512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.967+0000 7ff6f85e7740 20 client.26512 populate_metadata read hostname 'smithi016' 2024-09-21T03:55:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.967+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/3060749704 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5621007fad80 0x562100bf9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.967+0000 7ff6f85e7740 4 client.26512 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:55:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.967+0000 7ff6f85e7740 1 -- 172.21.15.16:0/3060749704 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562100b58600 con 0x5621007fad80 2024-09-21T03:55:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.969+0000 7ff6e2000640 1 --2- 172.21.15.16:0/3060749704 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5621007fad80 0x562100bf9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.972+0000 7ff6cfe00640 10 client.26512 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.973+0000 7ff6e2000640 1 --2- 172.21.15.16:0/3060749704 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5621007fad80 0x562100bf9fd0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b7463227e60b3e1 server_cookie=bfd476e92c46adf5 in_seq=0 out_seq=0 2024-09-21T03:55:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.975+0000 7ff6cfe00640 1 -- 172.21.15.16:0/3060749704 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562100b58600 con 0x5621007fad80 2024-09-21T03:55:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.975+0000 7ff6cfe00640 10 client.26512 handle_command_reply: tid=0 2024-09-21T03:55:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.975+0000 7ff6f85e7740 1 client.26512 shutdown 2024-09-21T03:55:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.977+0000 7ff6f85e7740 1 -- 172.21.15.16:0/3060749704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56210081b4d0 msgr2=0x562100becef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.977+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/3060749704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56210081b4d0 0x562100becef0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400c380 tx=0x7ff6d400c840 comp rx=0 tx=0).stop 2024-09-21T03:55:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.978+0000 7ff6f85e7740 1 -- 172.21.15.16:0/3060749704 shutdown_connections 2024-09-21T03:55:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.978+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/3060749704 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5621007fad80 0x562100bf9fd0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.978+0000 7ff6cfe00640 0 client.26512 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.980+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/3060749704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210083c1b0 0x562100beda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.980+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/3060749704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56210081b4d0 0x562100becef0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.980+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/3060749704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621008132f0 0x562100bec830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.980+0000 7ff6f85e7740 1 -- 172.21.15.16:0/3060749704 >> 172.21.15.16:0/3060749704 conn(0x562100830300 msgr2=0x562100bdd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.981+0000 7ff6f85e7740 1 -- 172.21.15.16:0/3060749704 shutdown_connections 2024-09-21T03:55:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.982+0000 7ff6f85e7740 1 -- 172.21.15.16:0/3060749704 wait complete. 2024-09-21T03:55:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:50.991+0000 7ff6f85e7740 20 client.26512 trim_cache size 0 max 16384 2024-09-21T03:55:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.043+0000 7ff6f85e7740 1 Processor -- start 2024-09-21T03:55:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.044+0000 7ff6f85e7740 1 -- start start 2024-09-21T03:55:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.044+0000 7ff6f85e7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621008132f0 0x562100bead40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.044+0000 7ff6f85e7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56210081b4d0 0x562100bf4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.044+0000 7ff6f85e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210083c1b0 0x562100beb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.044+0000 7ff6f85e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c800fd20 con 0x56210083c1b0 2024-09-21T03:55:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.044+0000 7ff6f85e7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c8011860 con 0x5621008132f0 2024-09-21T03:55:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.044+0000 7ff6f85e7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c80103f0 con 0x56210081b4d0 2024-09-21T03:55:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.044+0000 7ff6e2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621008132f0 0x562100bead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.046+0000 7ff6eaa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56210081b4d0 0x562100bf4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.047+0000 7ff6eaa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56210081b4d0 0x562100bf4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:43744/0 (socket says 172.21.15.16:43744) 2024-09-21T03:55:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.047+0000 7ff6e2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621008132f0 0x562100bead40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:36252/0 (socket says 172.21.15.16:36252) 2024-09-21T03:55:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.047+0000 7ff6e2000640 1 -- 172.21.15.16:0/1032561692 learned_addr learned my addr 172.21.15.16:0/1032561692 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.048+0000 7ff6e2000640 1 --2- 172.21.15.16:0/1032561692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210083c1b0 0x562100beb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.050+0000 7ff6e2000640 1 -- 172.21.15.16:0/1032561692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56210081b4d0 msgr2=0x562100bf4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.050+0000 7ff6e2000640 1 --2- 172.21.15.16:0/1032561692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56210081b4d0 0x562100bf4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.050+0000 7ff6e2000640 1 -- 172.21.15.16:0/1032561692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210083c1b0 msgr2=0x562100beb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.050+0000 7ff6e2000640 1 --2- 172.21.15.16:0/1032561692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210083c1b0 0x562100beb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.051+0000 7ff6e2000640 1 -- 172.21.15.16:0/1032561692 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562100b05c90 con 0x5621008132f0 2024-09-21T03:55:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.052+0000 7ff6eaa00640 1 --2- 172.21.15.16:0/1032561692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56210081b4d0 0x562100bf4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:55:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.054+0000 7ff6e2000640 1 --2- 172.21.15.16:0/1032561692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621008132f0 0x562100bead40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009230 tx=0x7ff6dc049040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a6f19621e1b2b27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.054+0000 7ff6daa00640 1 -- 172.21.15.16:0/1032561692 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc055040 con 0x5621008132f0 2024-09-21T03:55:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.055+0000 7ff6daa00640 1 -- 172.21.15.16:0/1032561692 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562100b58600 con 0x5621008132f0 2024-09-21T03:55:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.055+0000 7ff6daa00640 1 -- 172.21.15.16:0/1032561692 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc003db0 con 0x5621008132f0 2024-09-21T03:55:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.057+0000 7ff6f85e7740 1 -- 172.21.15.16:0/1032561692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621008132f0 msgr2=0x562100bead40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.057+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/1032561692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621008132f0 0x562100bead40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009230 tx=0x7ff6dc049040 comp rx=0 tx=0).stop 2024-09-21T03:55:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.059+0000 7ff6f85e7740 1 -- 172.21.15.16:0/1032561692 shutdown_connections 2024-09-21T03:55:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.060+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/1032561692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210083c1b0 0x562100beb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.060+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/1032561692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56210081b4d0 0x562100bf4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.060+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/1032561692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621008132f0 0x562100bead40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.060+0000 7ff6f85e7740 1 -- 172.21.15.16:0/1032561692 >> 172.21.15.16:0/1032561692 conn(0x562100830300 msgr2=0x562100bdd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.063+0000 7ff6f85e7740 1 -- 172.21.15.16:0/1032561692 shutdown_connections 2024-09-21T03:55:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.063+0000 7ff6f85e7740 1 -- 172.21.15.16:0/1032561692 wait complete. 2024-09-21T03:55:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.064+0000 7ff6f85e7740 1 -- start start 2024-09-21T03:55:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.064+0000 7ff6f85e7740 1 Processor -- start 2024-09-21T03:55:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.066+0000 7ff6f85e7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621008132f0 0x562100bf48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.066+0000 7ff6e2000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621008132f0 0x562100bf48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.067+0000 7ff6e2000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621008132f0 0x562100bf48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:43748/0 (socket says 172.21.15.16:43748) 2024-09-21T03:55:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.067+0000 7ff6e2000640 1 -- 172.21.15.16:0/985046508 learned_addr learned my addr 172.21.15.16:0/985046508 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:55:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.068+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/985046508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210081b4d0 0x562100bf3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.068+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/985046508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56210083c1b0 0x562100bf3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.068+0000 7ff6f85e7740 1 -- 172.21.15.16:0/985046508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c8011190 con 0x56210081b4d0 2024-09-21T03:55:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.068+0000 7ff6f85e7740 1 -- 172.21.15.16:0/985046508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c8010ac0 con 0x56210083c1b0 2024-09-21T03:55:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.068+0000 7ff6f85e7740 1 -- 172.21.15.16:0/985046508 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c800dd00 con 0x5621008132f0 2024-09-21T03:55:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.068+0000 7ff6f85e7740 10 client.0 _ll_register_callbacks cb 0x7ff6d4005ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:55:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.069+0000 7ff6eaa00640 1 --2- 172.21.15.16:0/985046508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210081b4d0 0x562100bf3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.069+0000 7ff6e2000640 1 --2- 172.21.15.16:0/985046508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56210083c1b0 0x562100bf3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.069+0000 7ff6eaa00640 1 -- 172.21.15.16:0/985046508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621008132f0 msgr2=0x562100bf48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.069+0000 7ff6eaa00640 1 --2- 172.21.15.16:0/985046508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621008132f0 0x562100bf48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.069+0000 7ff6eaa00640 1 -- 172.21.15.16:0/985046508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56210083c1b0 msgr2=0x562100bf3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:55:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.069+0000 7ff6eaa00640 1 --2- 172.21.15.16:0/985046508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56210083c1b0 0x562100bf3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.070+0000 7ff6eaa00640 1 -- 172.21.15.16:0/985046508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562100bf2170 con 0x56210081b4d0 2024-09-21T03:55:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.070+0000 7ff6e2000640 1 --2- 172.21.15.16:0/985046508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56210083c1b0 0x562100bf3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.070+0000 7ff6cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:55:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.070+0000 7ff6eaa00640 1 --2- 172.21.15.16:0/985046508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210081b4d0 0x562100bf3710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4015e80 tx=0x7ff6d4016530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1eeb42a75354c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.072+0000 7ff6cfe00640 1 -- 172.21.15.16:0/985046508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d400cd40 con 0x56210081b4d0 2024-09-21T03:55:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.072+0000 7ff6cfe00640 1 -- 172.21.15.16:0/985046508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7ff6d4015880 con 0x56210081b4d0 2024-09-21T03:55:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.072+0000 7ff6f85e7740 1 -- 172.21.15.16:0/985046508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562100b58600 con 0x56210081b4d0 2024-09-21T03:55:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.073+0000 7ff6cfe00640 1 -- 172.21.15.16:0/985046508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff6d4017360 con 0x56210081b4d0 2024-09-21T03:55:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.073+0000 7ff6f85e7740 10 client.16311 fetch_fsmap learned FSMap version 30 2024-09-21T03:55:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.073+0000 7ff6f85e7740 1 -- 172.21.15.16:0/985046508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562100c64cd0 con 0x56210081b4d0 2024-09-21T03:55:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.075+0000 7ff6cfe00640 1 -- 172.21.15.16:0/985046508 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562100b58600 con 0x56210081b4d0 2024-09-21T03:55:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.077+0000 7ff6cfe00640 1 -- 172.21.15.16:0/985046508 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff6d40224b0 con 0x56210081b4d0 2024-09-21T03:55:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.077+0000 7ff6f85e7740 10 client.16311 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:55:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.077+0000 7ff6f85e7740 10 client.16311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:55:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.077+0000 7ff6f85e7740 20 client.16311 populate_metadata read hostname 'smithi016' 2024-09-21T03:55:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.077+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/985046508 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5621007d01c0 0x562100c676c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.077+0000 7ff6f85e7740 4 client.16311 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:55:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.077+0000 7ff6f85e7740 1 -- 172.21.15.16:0/985046508 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562100b540b0 con 0x5621007d01c0 2024-09-21T03:55:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.078+0000 7ff6e2000640 1 --2- 172.21.15.16:0/985046508 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5621007d01c0 0x562100c676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.079+0000 7ff6e2000640 1 --2- 172.21.15.16:0/985046508 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5621007d01c0 0x562100c676c0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=167d3e5b971d31ab server_cookie=7fb97f4dd7c581ce in_seq=0 out_seq=0 2024-09-21T03:55:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.079+0000 7ff6cfe00640 10 client.16311 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.094+0000 7ff6cfe00640 1 -- 172.21.15.16:0/985046508 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562100b540b0 con 0x5621007d01c0 2024-09-21T03:55:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.094+0000 7ff6cfe00640 10 client.16311 handle_command_reply: tid=0 2024-09-21T03:55:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.095+0000 7ff6f85e7740 1 client.16311 shutdown 2024-09-21T03:55:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.095+0000 7ff6f85e7740 1 -- 172.21.15.16:0/985046508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210081b4d0 msgr2=0x562100bf3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.095+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/985046508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210081b4d0 0x562100bf3710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4015e80 tx=0x7ff6d4016530 comp rx=0 tx=0).stop 2024-09-21T03:55:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.095+0000 7ff6f85e7740 1 -- 172.21.15.16:0/985046508 shutdown_connections 2024-09-21T03:55:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.095+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/985046508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56210083c1b0 0x562100bf3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.095+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/985046508 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5621007d01c0 0x562100c676c0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.095+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/985046508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210081b4d0 0x562100bf3710 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.095+0000 7ff6f85e7740 1 --2- 172.21.15.16:0/985046508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621008132f0 0x562100bf48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.095+0000 7ff6f85e7740 1 -- 172.21.15.16:0/985046508 >> 172.21.15.16:0/985046508 conn(0x562100830300 msgr2=0x562100bead00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.095+0000 7ff6cfe00640 0 client.16311 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:55:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.096+0000 7ff6f85e7740 1 -- 172.21.15.16:0/985046508 shutdown_connections 2024-09-21T03:55:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.096+0000 7ff6f85e7740 1 -- 172.21.15.16:0/985046508 wait complete. 2024-09-21T03:55:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.096+0000 7ff6f85e7740 20 client.16311 trim_cache size 0 max 16384 2024-09-21T03:55:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:55:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:55:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:55:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:55:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.098+0000 7ff6f1600640 1 -- 172.21.15.16:0/2858620067 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff6d00b6720 msgr2=0x7ff6d00b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.098+0000 7ff6f1600640 1 --2- 172.21.15.16:0/2858620067 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff6d00b6720 0x7ff6d00b8c30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40ad9e0 tx=0x7ff6e400cf70 comp rx=0 tx=0).stop 2024-09-21T03:55:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.098+0000 7ff6f1600640 1 -- 172.21.15.16:0/2858620067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec10f9d0 msgr2=0x7ff6ec118b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.098+0000 7ff6f1600640 1 --2- 172.21.15.16:0/2858620067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec10f9d0 0x7ff6ec118b00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400eb10 tx=0x7ff6e400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.099+0000 7ff6f1600640 1 -- 172.21.15.16:0/2858620067 shutdown_connections 2024-09-21T03:55:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.099+0000 7ff6f1600640 1 --2- 172.21.15.16:0/2858620067 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff6d00b6720 0x7ff6d00b8c30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.099+0000 7ff6f1600640 1 --2- 172.21.15.16:0/2858620067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec10f9d0 0x7ff6ec118b00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.099+0000 7ff6f1600640 1 --2- 172.21.15.16:0/2858620067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6ec10cdb0 0x7ff6ec117f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.099+0000 7ff6f1600640 1 --2- 172.21.15.16:0/2858620067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6ec10a3c0 0x7ff6ec1178e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.099+0000 7ff6f1600640 1 -- 172.21.15.16:0/2858620067 >> 172.21.15.16:0/2858620067 conn(0x7ff6ec070810 msgr2=0x7ff6ec10b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.100+0000 7ff6f1600640 1 -- 172.21.15.16:0/2858620067 shutdown_connections 2024-09-21T03:55:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:55:51.101+0000 7ff6f1600640 1 -- 172.21.15.16:0/2858620067 wait complete. 2024-09-21T03:55:51.154 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:50 smithi134 ceph-mon[35031]: pgmap v704: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 220 MiB/s wr, 1.39k op/s 2024-09-21T03:55:51.154 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:51.154 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:50 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:51.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:50 smithi123 ceph-mon[34673]: pgmap v704: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 220 MiB/s wr, 1.39k op/s 2024-09-21T03:55:51.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:50 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:51.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:50 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:51.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:50 smithi016 ceph-mon[30375]: pgmap v704: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 220 MiB/s wr, 1.39k op/s 2024-09-21T03:55:51.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:51.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:50 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:52.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:51 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:52.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:51 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:52.157 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:51 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:52.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:51 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:52.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:51 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:52.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:51 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:52.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:51 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:52.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:51 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:52.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:51 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:55:53.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:52 smithi123 ceph-mon[34673]: pgmap v705: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 231 MiB/s wr, 1.42k op/s 2024-09-21T03:55:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:52 smithi016 ceph-mon[30375]: pgmap v705: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 231 MiB/s wr, 1.42k op/s 2024-09-21T03:55:53.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:52 smithi134 ceph-mon[35031]: pgmap v705: 129 pgs: 129 active+clean; 10 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 231 MiB/s wr, 1.42k op/s 2024-09-21T03:55:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:54 smithi016 ceph-mon[30375]: pgmap v706: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 235 MiB/s wr, 1.37k op/s 2024-09-21T03:55:55.268 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:54 smithi123 ceph-mon[34673]: pgmap v706: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 235 MiB/s wr, 1.37k op/s 2024-09-21T03:55:55.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:54 smithi134 ceph-mon[35031]: pgmap v706: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 235 MiB/s wr, 1.37k op/s 2024-09-21T03:55:57.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:56 smithi134 ceph-mon[35031]: pgmap v707: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 241 MiB/s wr, 1.33k op/s 2024-09-21T03:55:57.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:56 smithi123 ceph-mon[34673]: pgmap v707: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 241 MiB/s wr, 1.33k op/s 2024-09-21T03:55:57.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:56 smithi016 ceph-mon[30375]: pgmap v707: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 241 MiB/s wr, 1.33k op/s 2024-09-21T03:55:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:55:58 smithi134 ceph-mon[35031]: pgmap v708: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 231 MiB/s wr, 1.20k op/s 2024-09-21T03:55:59.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:55:58 smithi123 ceph-mon[34673]: pgmap v708: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 231 MiB/s wr, 1.20k op/s 2024-09-21T03:55:59.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:55:58 smithi016 ceph-mon[30375]: pgmap v708: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 231 MiB/s wr, 1.20k op/s 2024-09-21T03:56:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:00 smithi134 ceph-mon[35031]: pgmap v709: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 233 MiB/s wr, 1.18k op/s 2024-09-21T03:56:01.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:00 smithi123 ceph-mon[34673]: pgmap v709: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 233 MiB/s wr, 1.18k op/s 2024-09-21T03:56:01.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:00 smithi016 ceph-mon[30375]: pgmap v709: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 233 MiB/s wr, 1.18k op/s 2024-09-21T03:56:03.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:02 smithi134 ceph-mon[35031]: pgmap v710: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 236 MiB/s wr, 1.15k op/s 2024-09-21T03:56:03.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:02 smithi123 ceph-mon[34673]: pgmap v710: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 236 MiB/s wr, 1.15k op/s 2024-09-21T03:56:03.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:02 smithi016 ceph-mon[30375]: pgmap v710: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 236 MiB/s wr, 1.15k op/s 2024-09-21T03:56:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.426+0000 7f0d19a00640 1 -- 172.21.15.16:0/2007050311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d14104750 msgr2=0x7f0d14104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.426+0000 7f0d19a00640 1 --2- 172.21.15.16:0/2007050311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d14104750 0x7f0d14104b60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00dd10 tx=0x7f0cfc032960 comp rx=0 tx=0).stop 2024-09-21T03:56:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.426+0000 7f0d19a00640 1 -- 172.21.15.16:0/2007050311 shutdown_connections 2024-09-21T03:56:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.426+0000 7f0d19a00640 1 --2- 172.21.15.16:0/2007050311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d141050f0 0x7f0d1410ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.426+0000 7f0d19a00640 1 --2- 172.21.15.16:0/2007050311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d14104750 0x7f0d14104b60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.426+0000 7f0d19a00640 1 --2- 172.21.15.16:0/2007050311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d141036d0 0x7f0d14103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.426+0000 7f0d19a00640 1 -- 172.21.15.16:0/2007050311 >> 172.21.15.16:0/2007050311 conn(0x7f0d140ff030 msgr2=0x7f0d141014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.426+0000 7f0d19a00640 1 -- 172.21.15.16:0/2007050311 shutdown_connections 2024-09-21T03:56:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.426+0000 7f0d19a00640 1 -- 172.21.15.16:0/2007050311 wait complete. 2024-09-21T03:56:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.428+0000 7f0d19a00640 1 Processor -- start 2024-09-21T03:56:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.428+0000 7f0d19a00640 1 -- start start 2024-09-21T03:56:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.429+0000 7f0d19a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d141036d0 0x7f0d14197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.429+0000 7f0d19a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d14104750 0x7f0d14198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.429+0000 7f0d19a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d141050f0 0x7f0d14198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.429+0000 7f0d19a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1410d130 con 0x7f0d141050f0 2024-09-21T03:56:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.429+0000 7f0d19a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1410cfb0 con 0x7f0d141036d0 2024-09-21T03:56:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.429+0000 7f0d19a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1410d2b0 con 0x7f0d14104750 2024-09-21T03:56:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.429+0000 7f0d12a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d141036d0 0x7f0d14197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.429+0000 7f0d12a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d141036d0 0x7f0d14197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:53212/0 (socket says 172.21.15.16:53212) 2024-09-21T03:56:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.429+0000 7f0d12a00640 1 -- 172.21.15.16:0/1225605018 learned_addr learned my addr 172.21.15.16:0/1225605018 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.429+0000 7f0d0a000640 1 --2- 172.21.15.16:0/1225605018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d14104750 0x7f0d14198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.429+0000 7f0d13400640 1 --2- 172.21.15.16:0/1225605018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d141050f0 0x7f0d14198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.429+0000 7f0d12a00640 1 -- 172.21.15.16:0/1225605018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d14104750 msgr2=0x7f0d14198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.429+0000 7f0d12a00640 1 --2- 172.21.15.16:0/1225605018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d14104750 0x7f0d14198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.429+0000 7f0d12a00640 1 -- 172.21.15.16:0/1225605018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d141050f0 msgr2=0x7f0d14198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.429+0000 7f0d12a00640 1 --2- 172.21.15.16:0/1225605018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d141050f0 0x7f0d14198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.429+0000 7f0d12a00640 1 -- 172.21.15.16:0/1225605018 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d14199740 con 0x7f0d141036d0 2024-09-21T03:56:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.430+0000 7f0d13400640 1 --2- 172.21.15.16:0/1225605018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d141050f0 0x7f0d14198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.430+0000 7f0d12a00640 1 --2- 172.21.15.16:0/1225605018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d141036d0 0x7f0d14197d00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009d50 tx=0x7f0d0400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.430+0000 7f0d0be00640 1 -- 172.21.15.16:0/1225605018 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d04003b70 con 0x7f0d141036d0 2024-09-21T03:56:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.430+0000 7f0d0be00640 1 -- 172.21.15.16:0/1225605018 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d04003d10 con 0x7f0d141036d0 2024-09-21T03:56:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.430+0000 7f0d0be00640 1 -- 172.21.15.16:0/1225605018 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d040108a0 con 0x7f0d141036d0 2024-09-21T03:56:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.431+0000 7f0d19a00640 1 -- 172.21.15.16:0/1225605018 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d14199a30 con 0x7f0d141036d0 2024-09-21T03:56:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.431+0000 7f0d19a00640 1 -- 172.21.15.16:0/1225605018 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d141a0120 con 0x7f0d141036d0 2024-09-21T03:56:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.439+0000 7f0d0be00640 1 -- 172.21.15.16:0/1225605018 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0d04004000 con 0x7f0d141036d0 2024-09-21T03:56:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.442+0000 7f0d0be00640 1 --2- 172.21.15.16:0/1225605018 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0cf80b6610 0x7f0cf80b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.443+0000 7f0d0a000640 1 --2- 172.21.15.16:0/1225605018 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0cf80b6610 0x7f0cf80b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.443+0000 7f0d0be00640 1 -- 172.21.15.16:0/1225605018 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0d040e7e10 con 0x7f0d141036d0 2024-09-21T03:56:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.445+0000 7f0d0a000640 1 --2- 172.21.15.16:0/1225605018 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0cf80b6610 0x7f0cf80b8b20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc005f70 tx=0x7f0cfc000bc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.445+0000 7f0d20a96740 1 Processor -- start 2024-09-21T03:56:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.446+0000 7f0d20a96740 1 -- start start 2024-09-21T03:56:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.447+0000 7f0d20a96740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cdbb1b0 0x56442d1620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.447+0000 7f0d20a96740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cd922f0 0x56442d162740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.447+0000 7f0d20a96740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cd9a4d0 0x56442d163290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.447+0000 7f0d20a96740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56442d0d56b0 con 0x56442cdbb1b0 2024-09-21T03:56:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.447+0000 7f0d20a96740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56442d0c4ae0 con 0x56442cd922f0 2024-09-21T03:56:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.447+0000 7f0d20a96740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56442d0c88f0 con 0x56442cd9a4d0 2024-09-21T03:56:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.447+0000 7f0d13400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cd9a4d0 0x56442d163290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.448+0000 7f0d13400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cd9a4d0 0x56442d163290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:55234/0 (socket says 172.21.15.16:55234) 2024-09-21T03:56:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.448+0000 7f0d13400640 1 -- 172.21.15.16:0/2506118348 learned_addr learned my addr 172.21.15.16:0/2506118348 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.448+0000 7f0d13400640 1 -- 172.21.15.16:0/2506118348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cd922f0 msgr2=0x56442d162740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:56:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.448+0000 7f0d13400640 1 --2- 172.21.15.16:0/2506118348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cd922f0 0x56442d162740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.448+0000 7f0d13400640 1 -- 172.21.15.16:0/2506118348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cdbb1b0 msgr2=0x56442d1620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.448+0000 7f0d13400640 1 --2- 172.21.15.16:0/2506118348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cdbb1b0 0x56442d1620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.448+0000 7f0d13400640 1 -- 172.21.15.16:0/2506118348 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56442d091780 con 0x56442cd9a4d0 2024-09-21T03:56:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.448+0000 7f0d13400640 1 --2- 172.21.15.16:0/2506118348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cd9a4d0 0x56442d163290 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009bd0 tx=0x7f0d0c00e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1482466e97ea1ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.449+0000 7f0d03e00640 1 -- 172.21.15.16:0/2506118348 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c016070 con 0x56442cd9a4d0 2024-09-21T03:56:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.449+0000 7f0d03e00640 1 -- 172.21.15.16:0/2506118348 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d0c011920 con 0x56442cd9a4d0 2024-09-21T03:56:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.449+0000 7f0d03e00640 1 -- 172.21.15.16:0/2506118348 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c010790 con 0x56442cd9a4d0 2024-09-21T03:56:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.451+0000 7f0d20a96740 1 -- 172.21.15.16:0/2506118348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cd9a4d0 msgr2=0x56442d163290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.451+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2506118348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cd9a4d0 0x56442d163290 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009bd0 tx=0x7f0d0c00e6c0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.451+0000 7f0d20a96740 1 -- 172.21.15.16:0/2506118348 shutdown_connections 2024-09-21T03:56:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.451+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2506118348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cd9a4d0 0x56442d163290 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.451+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2506118348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cd922f0 0x56442d162740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.451+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2506118348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cdbb1b0 0x56442d1620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.451+0000 7f0d20a96740 1 -- 172.21.15.16:0/2506118348 >> 172.21.15.16:0/2506118348 conn(0x56442cdaf300 msgr2=0x56442d15db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.451+0000 7f0d20a96740 1 -- 172.21.15.16:0/2506118348 shutdown_connections 2024-09-21T03:56:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.451+0000 7f0d20a96740 1 -- 172.21.15.16:0/2506118348 wait complete. 2024-09-21T03:56:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.452+0000 7f0d20a96740 1 -- start start 2024-09-21T03:56:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.452+0000 7f0d20a96740 1 Processor -- start 2024-09-21T03:56:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.453+0000 7f0d20a96740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cd922f0 0x56442d17c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.453+0000 7f0d20a96740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cd9a4d0 0x56442d17ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.453+0000 7f0d20a96740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cdbb1b0 0x56442d17da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.453+0000 7f0d20a96740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56442cffcab0 con 0x56442cd922f0 2024-09-21T03:56:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.453+0000 7f0d20a96740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56442d06b570 con 0x56442cdbb1b0 2024-09-21T03:56:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.453+0000 7f0d20a96740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56442d163db0 con 0x56442cd9a4d0 2024-09-21T03:56:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.453+0000 7f0d20a96740 10 client.0 _ll_register_callbacks cb 0x56442d07a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.454+0000 7f0d13400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cdbb1b0 0x56442d17da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.454+0000 7f0d13400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cdbb1b0 0x56442d17da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:53218/0 (socket says 172.21.15.16:53218) 2024-09-21T03:56:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.454+0000 7f0d13400640 1 -- 172.21.15.16:0/2670119297 learned_addr learned my addr 172.21.15.16:0/2670119297 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.456+0000 7f0d0a000640 1 --2- 172.21.15.16:0/2670119297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cd9a4d0 0x56442d17ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.456+0000 7f0d12a00640 1 --2- 172.21.15.16:0/2670119297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cd922f0 0x56442d17c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.460+0000 7f0d13400640 1 -- 172.21.15.16:0/2670119297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cd9a4d0 msgr2=0x56442d17ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.460+0000 7f0d13400640 1 --2- 172.21.15.16:0/2670119297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cd9a4d0 0x56442d17ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.460+0000 7f0d13400640 1 -- 172.21.15.16:0/2670119297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cd922f0 msgr2=0x56442d17c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.460+0000 7f0d13400640 1 --2- 172.21.15.16:0/2670119297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cd922f0 0x56442d17c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.460+0000 7f0d13400640 1 -- 172.21.15.16:0/2670119297 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56442ce26790 con 0x56442cdbb1b0 2024-09-21T03:56:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.460+0000 7f0d12a00640 1 --2- 172.21.15.16:0/2670119297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cd922f0 0x56442d17c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.460+0000 7f0d13400640 1 --2- 172.21.15.16:0/2670119297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cdbb1b0 0x56442d17da40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c011ad0 tx=0x7f0d0c011b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7bb6969232090b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.460+0000 7f0d00c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:56:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.461+0000 7f0d00c00640 1 -- 172.21.15.16:0/2670119297 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c028060 con 0x56442cdbb1b0 2024-09-21T03:56:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.461+0000 7f0d00c00640 1 -- 172.21.15.16:0/2670119297 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0d0c005240 con 0x56442cdbb1b0 2024-09-21T03:56:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.462+0000 7f0d20a96740 1 -- 172.21.15.16:0/2670119297 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d0c011920 con 0x56442cdbb1b0 2024-09-21T03:56:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.462+0000 7f0d00c00640 1 -- 172.21.15.16:0/2670119297 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0d0c016070 con 0x56442cdbb1b0 2024-09-21T03:56:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.463+0000 7f0d20a96740 10 client.26579 fetch_fsmap learned FSMap version 30 2024-09-21T03:56:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.464+0000 7f0d20a96740 1 -- 172.21.15.16:0/2670119297 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56442ce9f620 con 0x56442cdbb1b0 2024-09-21T03:56:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.465+0000 7f0d00c00640 1 -- 172.21.15.16:0/2670119297 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d0c011920 con 0x56442cdbb1b0 2024-09-21T03:56:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.467+0000 7f0d00c00640 1 -- 172.21.15.16:0/2670119297 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0d0c00f040 con 0x56442cdbb1b0 2024-09-21T03:56:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.468+0000 7f0d20a96740 10 client.26579 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:56:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.468+0000 7f0d20a96740 10 client.26579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.468+0000 7f0d20a96740 20 client.26579 populate_metadata read hostname 'smithi016' 2024-09-21T03:56:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.468+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2670119297 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56442cd79d80 0x56442d189cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.468+0000 7f0d20a96740 4 client.26579 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.468+0000 7f0d20a96740 1 -- 172.21.15.16:0/2670119297 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56442d0d7600 con 0x56442cd79d80 2024-09-21T03:56:04.472 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.470+0000 7f0d12a00640 1 --2- 172.21.15.16:0/2670119297 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56442cd79d80 0x56442d189cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.472+0000 7f0d00c00640 10 client.26579 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.472+0000 7f0d12a00640 1 --2- 172.21.15.16:0/2670119297 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56442cd79d80 0x56442d189cf0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2efa2ebd1a3ced1e server_cookie=38d4e865e0781d14 in_seq=0 out_seq=0 2024-09-21T03:56:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.476+0000 7f0d00c00640 1 -- 172.21.15.16:0/2670119297 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56442d0d7600 con 0x56442cd79d80 2024-09-21T03:56:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.476+0000 7f0d00c00640 10 client.26579 handle_command_reply: tid=0 2024-09-21T03:56:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.476+0000 7f0d20a96740 1 client.26579 shutdown 2024-09-21T03:56:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.481+0000 7f0d20a96740 1 -- 172.21.15.16:0/2670119297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cdbb1b0 msgr2=0x56442d17da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.481+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2670119297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cdbb1b0 0x56442d17da40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c011ad0 tx=0x7f0d0c011b00 comp rx=0 tx=0).stop 2024-09-21T03:56:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.484+0000 7f0d20a96740 1 -- 172.21.15.16:0/2670119297 shutdown_connections 2024-09-21T03:56:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.484+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2670119297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cdbb1b0 0x56442d17da40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.484+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2670119297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cd9a4d0 0x56442d17ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.484+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2670119297 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56442cd79d80 0x56442d189cf0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.484+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2670119297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cd922f0 0x56442d17c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.484+0000 7f0d20a96740 1 -- 172.21.15.16:0/2670119297 >> 172.21.15.16:0/2670119297 conn(0x56442cdaf300 msgr2=0x56442d15c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.484+0000 7f0d00c00640 0 client.26579 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.485+0000 7f0d20a96740 1 -- 172.21.15.16:0/2670119297 shutdown_connections 2024-09-21T03:56:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.485+0000 7f0d20a96740 1 -- 172.21.15.16:0/2670119297 wait complete. 2024-09-21T03:56:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.485+0000 7f0d20a96740 20 client.26579 trim_cache size 0 max 16384 2024-09-21T03:56:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.535+0000 7f0d20a96740 1 Processor -- start 2024-09-21T03:56:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.536+0000 7f0d20a96740 1 -- start start 2024-09-21T03:56:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.536+0000 7f0d20a96740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cd922f0 0x56442d166b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.536+0000 7f0d20a96740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cd9a4d0 0x56442d184920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.536+0000 7f0d20a96740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cdbb1b0 0x56442d15d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.536+0000 7f0d20a96740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141a1e40 con 0x56442cd922f0 2024-09-21T03:56:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.536+0000 7f0d20a96740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141a3980 con 0x56442cd9a4d0 2024-09-21T03:56:04.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.536+0000 7f0d20a96740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141a2510 con 0x56442cdbb1b0 2024-09-21T03:56:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.538+0000 7f0d13400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cdbb1b0 0x56442d15d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.538+0000 7f0d0a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cd9a4d0 0x56442d184920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.538+0000 7f0d0a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cd9a4d0 0x56442d184920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:53220/0 (socket says 172.21.15.16:53220) 2024-09-21T03:56:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.538+0000 7f0d0a000640 1 -- 172.21.15.16:0/2072046966 learned_addr learned my addr 172.21.15.16:0/2072046966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.539+0000 7f0d13400640 1 -- 172.21.15.16:0/2072046966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cd9a4d0 msgr2=0x56442d184920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.539+0000 7f0d13400640 1 --2- 172.21.15.16:0/2072046966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cd9a4d0 0x56442d184920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.539+0000 7f0d13400640 1 -- 172.21.15.16:0/2072046966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cd922f0 msgr2=0x56442d166b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.539+0000 7f0d13400640 1 --2- 172.21.15.16:0/2072046966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cd922f0 0x56442d166b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.539+0000 7f0d13400640 1 -- 172.21.15.16:0/2072046966 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56442d159330 con 0x56442cdbb1b0 2024-09-21T03:56:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.540+0000 7f0d13400640 1 --2- 172.21.15.16:0/2072046966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cdbb1b0 0x56442d15d4a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c005e90 tx=0x7f0d0c00ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cda4c8d6c46b34ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.540+0000 7f0d03400640 1 -- 172.21.15.16:0/2072046966 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c016070 con 0x56442cdbb1b0 2024-09-21T03:56:04.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.541+0000 7f0d03400640 1 -- 172.21.15.16:0/2072046966 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d0c0103d0 con 0x56442cdbb1b0 2024-09-21T03:56:04.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.541+0000 7f0d03400640 1 -- 172.21.15.16:0/2072046966 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c01b900 con 0x56442cdbb1b0 2024-09-21T03:56:04.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.543+0000 7f0d20a96740 1 -- 172.21.15.16:0/2072046966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cdbb1b0 msgr2=0x56442d15d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.543+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2072046966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cdbb1b0 0x56442d15d4a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c005e90 tx=0x7f0d0c00ecf0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.543+0000 7f0d20a96740 1 -- 172.21.15.16:0/2072046966 shutdown_connections 2024-09-21T03:56:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.543+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2072046966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cdbb1b0 0x56442d15d4a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.543+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2072046966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cd9a4d0 0x56442d184920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.543+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2072046966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cd922f0 0x56442d166b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.544+0000 7f0d20a96740 1 -- 172.21.15.16:0/2072046966 >> 172.21.15.16:0/2072046966 conn(0x56442cdaf300 msgr2=0x56442d161cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:04.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.544+0000 7f0d20a96740 1 -- 172.21.15.16:0/2072046966 shutdown_connections 2024-09-21T03:56:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.544+0000 7f0d20a96740 1 -- 172.21.15.16:0/2072046966 wait complete. 2024-09-21T03:56:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.545+0000 7f0d20a96740 1 -- start start 2024-09-21T03:56:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.545+0000 7f0d20a96740 1 Processor -- start 2024-09-21T03:56:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.546+0000 7f0d20a96740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cd922f0 0x56442d184910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.547+0000 7f0d20a96740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cd9a4d0 0x56442d183750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.547+0000 7f0d20a96740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cdbb1b0 0x56442d183e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.548+0000 7f0d20a96740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141a32b0 con 0x56442cdbb1b0 2024-09-21T03:56:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.548+0000 7f0d20a96740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141a2be0 con 0x56442cd922f0 2024-09-21T03:56:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.548+0000 7f0d20a96740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d140750e0 con 0x56442cd9a4d0 2024-09-21T03:56:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.548+0000 7f0d20a96740 10 client.0 _ll_register_callbacks cb 0x7f0d0c009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.548+0000 7f0d13400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cdbb1b0 0x56442d183e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.548+0000 7f0d12a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cd922f0 0x56442d184910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.549+0000 7f0d12a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cd922f0 0x56442d184910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:53236/0 (socket says 172.21.15.16:53236) 2024-09-21T03:56:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.549+0000 7f0d13400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cdbb1b0 0x56442d183e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35604/0 (socket says 172.21.15.16:35604) 2024-09-21T03:56:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.549+0000 7f0d13400640 1 -- 172.21.15.16:0/2436784629 learned_addr learned my addr 172.21.15.16:0/2436784629 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.549+0000 7f0d0a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cd9a4d0 0x56442d183750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.550+0000 7f0d13400640 1 -- 172.21.15.16:0/2436784629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cd9a4d0 msgr2=0x56442d183750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.550+0000 7f0d13400640 1 --2- 172.21.15.16:0/2436784629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cd9a4d0 0x56442d183750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.550+0000 7f0d13400640 1 -- 172.21.15.16:0/2436784629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cd922f0 msgr2=0x56442d184910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.550+0000 7f0d13400640 1 --2- 172.21.15.16:0/2436784629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cd922f0 0x56442d184910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.550+0000 7f0d13400640 1 -- 172.21.15.16:0/2436784629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56442d1821c0 con 0x56442cdbb1b0 2024-09-21T03:56:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.551+0000 7f0d12a00640 1 --2- 172.21.15.16:0/2436784629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cd922f0 0x56442d184910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.552+0000 7f0d13400640 1 --2- 172.21.15.16:0/2436784629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cdbb1b0 0x56442d183e60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0105e0 tx=0x7f0d0c011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff3e59c8b032c98 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.552+0000 7f0d00c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:56:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.553+0000 7f0d00c00640 1 -- 172.21.15.16:0/2436784629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c03a020 con 0x56442cdbb1b0 2024-09-21T03:56:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.554+0000 7f0d00c00640 1 -- 172.21.15.16:0/2436784629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0d0c005040 con 0x56442cdbb1b0 2024-09-21T03:56:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.554+0000 7f0d20a96740 1 -- 172.21.15.16:0/2436784629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d0c0103d0 con 0x56442cdbb1b0 2024-09-21T03:56:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.555+0000 7f0d00c00640 1 -- 172.21.15.16:0/2436784629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0d0c016070 con 0x56442cdbb1b0 2024-09-21T03:56:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.555+0000 7f0d20a96740 10 client.16320 fetch_fsmap learned FSMap version 30 2024-09-21T03:56:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.555+0000 7f0d20a96740 1 -- 172.21.15.16:0/2436784629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56442d1f4880 con 0x56442cdbb1b0 2024-09-21T03:56:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.557+0000 7f0d00c00640 1 -- 172.21.15.16:0/2436784629 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d0c0103d0 con 0x56442cdbb1b0 2024-09-21T03:56:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.557+0000 7f0d00c00640 1 -- 172.21.15.16:0/2436784629 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0d0c004b30 con 0x56442cdbb1b0 2024-09-21T03:56:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.559+0000 7f0d20a96740 10 client.16320 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:56:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.560+0000 7f0d20a96740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.560+0000 7f0d20a96740 20 client.16320 populate_metadata read hostname 'smithi016' 2024-09-21T03:56:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.560+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2436784629 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56442cd4f1c0 0x56442d1f7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.560+0000 7f0d20a96740 4 client.16320 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:56:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.560+0000 7f0d20a96740 1 -- 172.21.15.16:0/2436784629 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56442d0d30b0 con 0x56442cd4f1c0 2024-09-21T03:56:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.561+0000 7f0d12a00640 1 --2- 172.21.15.16:0/2436784629 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56442cd4f1c0 0x56442d1f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.561+0000 7f0d12a00640 1 --2- 172.21.15.16:0/2436784629 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56442cd4f1c0 0x56442d1f7210 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6ca5335df13784c server_cookie=443dd3bddd3a79df in_seq=0 out_seq=0 2024-09-21T03:56:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.565+0000 7f0d00c00640 10 client.16320 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.754+0000 7f0d00c00640 1 -- 172.21.15.16:0/2436784629 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56442d0d30b0 con 0x56442cd4f1c0 2024-09-21T03:56:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.754+0000 7f0d00c00640 10 client.16320 handle_command_reply: tid=0 2024-09-21T03:56:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.754+0000 7f0d20a96740 1 client.16320 shutdown 2024-09-21T03:56:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.756+0000 7f0d20a96740 1 -- 172.21.15.16:0/2436784629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cdbb1b0 msgr2=0x56442d183e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.756+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2436784629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cdbb1b0 0x56442d183e60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0105e0 tx=0x7f0d0c011c20 comp rx=0 tx=0).stop 2024-09-21T03:56:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.757+0000 7f0d20a96740 1 -- 172.21.15.16:0/2436784629 shutdown_connections 2024-09-21T03:56:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.757+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2436784629 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56442cd4f1c0 0x56442d1f7210 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.757+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2436784629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56442cdbb1b0 0x56442d183e60 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.757+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2436784629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56442cd9a4d0 0x56442d183750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.757+0000 7f0d20a96740 1 --2- 172.21.15.16:0/2436784629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56442cd922f0 0x56442d184910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.757+0000 7f0d20a96740 1 -- 172.21.15.16:0/2436784629 >> 172.21.15.16:0/2436784629 conn(0x56442cdaf300 msgr2=0x56442d161cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.757+0000 7f0d00c00640 0 client.16320 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.758+0000 7f0d20a96740 1 -- 172.21.15.16:0/2436784629 shutdown_connections 2024-09-21T03:56:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.759+0000 7f0d20a96740 1 -- 172.21.15.16:0/2436784629 wait complete. 2024-09-21T03:56:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.760+0000 7f0d20a96740 20 client.16320 trim_cache size 0 max 16384 2024-09-21T03:56:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:56:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:56:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:56:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:56:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.768+0000 7f0d19a00640 1 -- 172.21.15.16:0/1225605018 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0cf80b6610 msgr2=0x7f0cf80b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.768+0000 7f0d19a00640 1 --2- 172.21.15.16:0/1225605018 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0cf80b6610 0x7f0cf80b8b20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc005f70 tx=0x7f0cfc000bc0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.768+0000 7f0d19a00640 1 -- 172.21.15.16:0/1225605018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d141036d0 msgr2=0x7f0d14197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.768+0000 7f0d19a00640 1 --2- 172.21.15.16:0/1225605018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d141036d0 0x7f0d14197d00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009d50 tx=0x7f0d0400c6a0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.769+0000 7f0d19a00640 1 -- 172.21.15.16:0/1225605018 shutdown_connections 2024-09-21T03:56:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.769+0000 7f0d19a00640 1 --2- 172.21.15.16:0/1225605018 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0cf80b6610 0x7f0cf80b8b20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.769+0000 7f0d19a00640 1 --2- 172.21.15.16:0/1225605018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d141050f0 0x7f0d14198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.769+0000 7f0d19a00640 1 --2- 172.21.15.16:0/1225605018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d14104750 0x7f0d14198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.769+0000 7f0d19a00640 1 --2- 172.21.15.16:0/1225605018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d141036d0 0x7f0d14197d00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.769+0000 7f0d19a00640 1 -- 172.21.15.16:0/1225605018 >> 172.21.15.16:0/1225605018 conn(0x7f0d140ff030 msgr2=0x7f0d14107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.771+0000 7f0d19a00640 1 -- 172.21.15.16:0/1225605018 shutdown_connections 2024-09-21T03:56:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:04.771+0000 7f0d19a00640 1 -- 172.21.15.16:0/1225605018 wait complete. 2024-09-21T03:56:05.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:04 smithi134 ceph-mon[35031]: pgmap v711: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 235 MiB/s wr, 1.12k op/s 2024-09-21T03:56:05.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:04 smithi123 ceph-mon[34673]: pgmap v711: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 235 MiB/s wr, 1.12k op/s 2024-09-21T03:56:05.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:04 smithi016 ceph-mon[30375]: pgmap v711: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 235 MiB/s wr, 1.12k op/s 2024-09-21T03:56:07.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:07 smithi134 ceph-mon[35031]: pgmap v712: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 240 MiB/s wr, 1.16k op/s 2024-09-21T03:56:07.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:07 smithi123 ceph-mon[34673]: pgmap v712: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 240 MiB/s wr, 1.16k op/s 2024-09-21T03:56:07.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:07 smithi016 ceph-mon[30375]: pgmap v712: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 240 MiB/s wr, 1.16k op/s 2024-09-21T03:56:09.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:09 smithi134 ceph-mon[35031]: pgmap v713: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 232 MiB/s wr, 1.12k op/s 2024-09-21T03:56:09.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:09 smithi123 ceph-mon[34673]: pgmap v713: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 232 MiB/s wr, 1.12k op/s 2024-09-21T03:56:09.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:09 smithi016 ceph-mon[30375]: pgmap v713: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 232 MiB/s wr, 1.12k op/s 2024-09-21T03:56:10.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:10 smithi123 ceph-mon[34673]: pgmap v714: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 257 MiB/s wr, 1.28k op/s 2024-09-21T03:56:10.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:10 smithi016 ceph-mon[30375]: pgmap v714: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 257 MiB/s wr, 1.28k op/s 2024-09-21T03:56:10.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:10 smithi134 ceph-mon[35031]: pgmap v714: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 257 MiB/s wr, 1.28k op/s 2024-09-21T03:56:13.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:12 smithi123 ceph-mon[34673]: pgmap v715: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 241 MiB/s wr, 1.21k op/s 2024-09-21T03:56:13.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:12 smithi016 ceph-mon[30375]: pgmap v715: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 241 MiB/s wr, 1.21k op/s 2024-09-21T03:56:13.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:12 smithi134 ceph-mon[35031]: pgmap v715: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 241 MiB/s wr, 1.21k op/s 2024-09-21T03:56:15.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:14 smithi123 ceph-mon[34673]: pgmap v716: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 246 MiB/s wr, 1.27k op/s 2024-09-21T03:56:15.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:14 smithi016 ceph-mon[30375]: pgmap v716: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 246 MiB/s wr, 1.27k op/s 2024-09-21T03:56:15.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:14 smithi134 ceph-mon[35031]: pgmap v716: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 246 MiB/s wr, 1.27k op/s 2024-09-21T03:56:17.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:16 smithi123 ceph-mon[34673]: pgmap v717: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 261 MiB/s wr, 1.36k op/s 2024-09-21T03:56:17.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:16 smithi016 ceph-mon[30375]: pgmap v717: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 261 MiB/s wr, 1.36k op/s 2024-09-21T03:56:17.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:16 smithi134 ceph-mon[35031]: pgmap v717: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 261 MiB/s wr, 1.36k op/s 2024-09-21T03:56:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.153+0000 7fb877e00640 1 -- 172.21.15.16:0/400711067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb878109af0 msgr2=0x7fb878109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.153+0000 7fb877e00640 1 --2- 172.21.15.16:0/400711067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb878109af0 0x7fb878109f60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb870009900 tx=0x7fb87002a410 comp rx=0 tx=0).stop 2024-09-21T03:56:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.156+0000 7fb877e00640 1 -- 172.21.15.16:0/400711067 shutdown_connections 2024-09-21T03:56:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.156+0000 7fb877e00640 1 --2- 172.21.15.16:0/400711067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb87810b510 0x7fb878111eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.156+0000 7fb877e00640 1 --2- 172.21.15.16:0/400711067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87810ab70 0x7fb87810af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.156+0000 7fb877e00640 1 --2- 172.21.15.16:0/400711067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb878109af0 0x7fb878109f60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.156+0000 7fb877e00640 1 -- 172.21.15.16:0/400711067 >> 172.21.15.16:0/400711067 conn(0x7fb878078e30 msgr2=0x7fb87807b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.161+0000 7fb877e00640 1 -- 172.21.15.16:0/400711067 shutdown_connections 2024-09-21T03:56:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.161+0000 7fb877e00640 1 -- 172.21.15.16:0/400711067 wait complete. 2024-09-21T03:56:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.163+0000 7fb877e00640 1 Processor -- start 2024-09-21T03:56:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.163+0000 7fb877e00640 1 -- start start 2024-09-21T03:56:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.164+0000 7fb877e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb878109af0 0x7fb8781a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.164+0000 7fb877e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb87810ab70 0x7fb8781a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.164+0000 7fb877e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87810b510 0x7fb8781a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.164+0000 7fb877e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781157e0 con 0x7fb878109af0 2024-09-21T03:56:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.164+0000 7fb877e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb878115660 con 0x7fb87810ab70 2024-09-21T03:56:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.164+0000 7fb877e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb878115960 con 0x7fb87810b510 2024-09-21T03:56:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.164+0000 7fb876000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb87810ab70 0x7fb8781a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.164+0000 7fb876000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb87810ab70 0x7fb8781a0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:41026/0 (socket says 172.21.15.16:41026) 2024-09-21T03:56:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.164+0000 7fb876000640 1 -- 172.21.15.16:0/712099833 learned_addr learned my addr 172.21.15.16:0/712099833 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.164+0000 7fb876a00640 1 --2- 172.21.15.16:0/712099833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb878109af0 0x7fb8781a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.164+0000 7fb877400640 1 --2- 172.21.15.16:0/712099833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87810b510 0x7fb8781a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.165+0000 7fb877400640 1 -- 172.21.15.16:0/712099833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb87810ab70 msgr2=0x7fb8781a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.165+0000 7fb877400640 1 --2- 172.21.15.16:0/712099833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb87810ab70 0x7fb8781a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.165+0000 7fb877400640 1 -- 172.21.15.16:0/712099833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb878109af0 msgr2=0x7fb8781a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.165+0000 7fb877400640 1 --2- 172.21.15.16:0/712099833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb878109af0 0x7fb8781a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.165+0000 7fb877400640 1 -- 172.21.15.16:0/712099833 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8781a1e50 con 0x7fb87810b510 2024-09-21T03:56:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.165+0000 7fb877400640 1 --2- 172.21.15.16:0/712099833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87810b510 0x7fb8781a1630 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb86400d6b0 tx=0x7fb86400db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.166+0000 7fb86f400640 1 -- 172.21.15.16:0/712099833 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864004230 con 0x7fb87810b510 2024-09-21T03:56:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.166+0000 7fb86f400640 1 -- 172.21.15.16:0/712099833 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8640043d0 con 0x7fb87810b510 2024-09-21T03:56:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.166+0000 7fb86f400640 1 -- 172.21.15.16:0/712099833 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864003de0 con 0x7fb87810b510 2024-09-21T03:56:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.166+0000 7fb877e00640 1 -- 172.21.15.16:0/712099833 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8781a2140 con 0x7fb87810b510 2024-09-21T03:56:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.166+0000 7fb877e00640 1 -- 172.21.15.16:0/712099833 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8781a8830 con 0x7fb87810b510 2024-09-21T03:56:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.172+0000 7fb884ccf740 1 Processor -- start 2024-09-21T03:56:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.176+0000 7fb86f400640 1 -- 172.21.15.16:0/712099833 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb864009820 con 0x7fb87810b510 2024-09-21T03:56:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.178+0000 7fb884ccf740 1 -- start start 2024-09-21T03:56:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.179+0000 7fb884ccf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cf01b0 0x559cf60972b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.179+0000 7fb884ccf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5cc72f0 0x559cf6097920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.179+0000 7fb884ccf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5ccf4d0 0x559cf6098470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.179+0000 7fb884ccf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559cf600a6b0 con 0x559cf5cc72f0 2024-09-21T03:56:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.179+0000 7fb884ccf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559cf5ff9ae0 con 0x559cf5cf01b0 2024-09-21T03:56:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.179+0000 7fb884ccf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559cf5ffd8f0 con 0x559cf5ccf4d0 2024-09-21T03:56:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.181+0000 7fb86f400640 1 --2- 172.21.15.16:0/712099833 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb85c0b6650 0x7fb85c0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.181+0000 7fb86f400640 1 -- 172.21.15.16:0/712099833 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb8640e70c0 con 0x7fb87810b510 2024-09-21T03:56:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.182+0000 7fb876000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5ccf4d0 0x559cf6098470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.182+0000 7fb876000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cf01b0 0x559cf60972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.182+0000 7fb876000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cf01b0 0x559cf60972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:41034/0 (socket says 172.21.15.16:41034) 2024-09-21T03:56:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.182+0000 7fb876000640 1 -- 172.21.15.16:0/4219094909 learned_addr learned my addr 172.21.15.16:0/4219094909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.183+0000 7fb876a00640 1 --2- 172.21.15.16:0/4219094909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5cc72f0 0x559cf6097920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.183+0000 7fb876000640 1 -- 172.21.15.16:0/4219094909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5ccf4d0 msgr2=0x559cf6098470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.183+0000 7fb876000640 1 --2- 172.21.15.16:0/4219094909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5ccf4d0 0x559cf6098470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.184+0000 7fb876000640 1 -- 172.21.15.16:0/4219094909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5cc72f0 msgr2=0x559cf6097920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.184+0000 7fb876000640 1 --2- 172.21.15.16:0/4219094909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5cc72f0 0x559cf6097920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.184+0000 7fb876000640 1 -- 172.21.15.16:0/4219094909 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cf5fc6780 con 0x559cf5cf01b0 2024-09-21T03:56:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.184+0000 7fb877400640 1 --2- 172.21.15.16:0/712099833 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb85c0b6650 0x7fb85c0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.185+0000 7fb876000640 1 --2- 172.21.15.16:0/4219094909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cf01b0 0x559cf60972b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb868008c70 tx=0x7fb868009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=648ba17283fbf1fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.186+0000 7fb863e00640 1 -- 172.21.15.16:0/4219094909 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86801f070 con 0x559cf5cf01b0 2024-09-21T03:56:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.186+0000 7fb863e00640 1 -- 172.21.15.16:0/4219094909 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb86801a460 con 0x559cf5cf01b0 2024-09-21T03:56:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.186+0000 7fb863e00640 1 -- 172.21.15.16:0/4219094909 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8680185a0 con 0x559cf5cf01b0 2024-09-21T03:56:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.187+0000 7fb884ccf740 1 -- 172.21.15.16:0/4219094909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cf01b0 msgr2=0x559cf60972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.187+0000 7fb884ccf740 1 --2- 172.21.15.16:0/4219094909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cf01b0 0x559cf60972b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb868008c70 tx=0x7fb868009790 comp rx=0 tx=0).stop 2024-09-21T03:56:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.188+0000 7fb884ccf740 1 -- 172.21.15.16:0/4219094909 shutdown_connections 2024-09-21T03:56:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.188+0000 7fb884ccf740 1 --2- 172.21.15.16:0/4219094909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5ccf4d0 0x559cf6098470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.188+0000 7fb884ccf740 1 --2- 172.21.15.16:0/4219094909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5cc72f0 0x559cf6097920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.188+0000 7fb884ccf740 1 --2- 172.21.15.16:0/4219094909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cf01b0 0x559cf60972b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.188+0000 7fb884ccf740 1 -- 172.21.15.16:0/4219094909 >> 172.21.15.16:0/4219094909 conn(0x559cf5ce4300 msgr2=0x559cf6092d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.189+0000 7fb877400640 1 --2- 172.21.15.16:0/712099833 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb85c0b6650 0x7fb85c0b8b60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb8640acbf0 tx=0x7fb864003f80 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.191+0000 7fb884ccf740 1 -- 172.21.15.16:0/4219094909 shutdown_connections 2024-09-21T03:56:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.191+0000 7fb884ccf740 1 -- 172.21.15.16:0/4219094909 wait complete. 2024-09-21T03:56:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.191+0000 7fb884ccf740 1 -- start start 2024-09-21T03:56:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.193+0000 7fb884ccf740 1 Processor -- start 2024-09-21T03:56:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.194+0000 7fb884ccf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5cc72f0 0x559cf60b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.194+0000 7fb884ccf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5ccf4d0 0x559cf60b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.194+0000 7fb884ccf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5cf01b0 0x559cf60b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.194+0000 7fb884ccf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559cf5f31ab0 con 0x559cf5cc72f0 2024-09-21T03:56:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.194+0000 7fb884ccf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559cf5fa0570 con 0x559cf5ccf4d0 2024-09-21T03:56:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.194+0000 7fb884ccf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559cf6098f90 con 0x559cf5cf01b0 2024-09-21T03:56:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.195+0000 7fb884ccf740 10 client.0 _ll_register_callbacks cb 0x559cf5faf100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.196+0000 7fb876000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5cc72f0 0x559cf60b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.196+0000 7fb876000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5cc72f0 0x559cf60b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52618/0 (socket says 172.21.15.16:52618) 2024-09-21T03:56:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.196+0000 7fb876000640 1 -- 172.21.15.16:0/1663678892 learned_addr learned my addr 172.21.15.16:0/1663678892 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.196+0000 7fb876000640 1 --2- 172.21.15.16:0/1663678892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5cf01b0 0x559cf60b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.197+0000 7fb876000640 1 -- 172.21.15.16:0/1663678892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5ccf4d0 msgr2=0x559cf60b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.197+0000 7fb876000640 1 --2- 172.21.15.16:0/1663678892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5ccf4d0 0x559cf60b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.197+0000 7fb876000640 1 -- 172.21.15.16:0/1663678892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5cc72f0 msgr2=0x559cf60b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.197+0000 7fb876000640 1 --2- 172.21.15.16:0/1663678892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5cc72f0 0x559cf60b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.197+0000 7fb876000640 1 -- 172.21.15.16:0/1663678892 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cf5d5b790 con 0x559cf5cf01b0 2024-09-21T03:56:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.198+0000 7fb876000640 1 --2- 172.21.15.16:0/1663678892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5cf01b0 0x559cf60b2c50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb868004a80 tx=0x7fb868008430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f55b31d696f476a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.198+0000 7fb860c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:56:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.199+0000 7fb860c00640 1 -- 172.21.15.16:0/1663678892 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86802c060 con 0x559cf5cf01b0 2024-09-21T03:56:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.199+0000 7fb884ccf740 1 -- 172.21.15.16:0/1663678892 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb86801a460 con 0x559cf5cf01b0 2024-09-21T03:56:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.199+0000 7fb860c00640 1 -- 172.21.15.16:0/1663678892 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb86801a610 con 0x559cf5cf01b0 2024-09-21T03:56:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.200+0000 7fb860c00640 1 -- 172.21.15.16:0/1663678892 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb86801f070 con 0x559cf5cf01b0 2024-09-21T03:56:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.201+0000 7fb884ccf740 10 client.26545 fetch_fsmap learned FSMap version 30 2024-09-21T03:56:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.201+0000 7fb884ccf740 1 -- 172.21.15.16:0/1663678892 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559cf5dd4620 con 0x559cf5cf01b0 2024-09-21T03:56:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.202+0000 7fb860c00640 1 -- 172.21.15.16:0/1663678892 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb86801a460 con 0x559cf5cf01b0 2024-09-21T03:56:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.202+0000 7fb860c00640 1 -- 172.21.15.16:0/1663678892 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb868018040 con 0x559cf5cf01b0 2024-09-21T03:56:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.204+0000 7fb884ccf740 10 client.26545 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:56:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.204+0000 7fb884ccf740 10 client.26545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.204+0000 7fb884ccf740 20 client.26545 populate_metadata read hostname 'smithi016' 2024-09-21T03:56:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.204+0000 7fb884ccf740 1 --2- 172.21.15.16:0/1663678892 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x559cf5caed80 0x559cf60bedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.204+0000 7fb884ccf740 4 client.26545 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.204+0000 7fb884ccf740 1 -- 172.21.15.16:0/1663678892 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cf600c600 con 0x559cf5caed80 2024-09-21T03:56:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.215+0000 7fb876a00640 1 --2- 172.21.15.16:0/1663678892 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x559cf5caed80 0x559cf60bedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.216+0000 7fb876a00640 1 --2- 172.21.15.16:0/1663678892 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x559cf5caed80 0x559cf60bedf0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=910698ff4c3558a9 server_cookie=421f2ae90ffb7d5 in_seq=0 out_seq=0 2024-09-21T03:56:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.216+0000 7fb860c00640 10 client.26545 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.217+0000 7fb860c00640 1 -- 172.21.15.16:0/1663678892 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559cf600c600 con 0x559cf5caed80 2024-09-21T03:56:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.217+0000 7fb860c00640 10 client.26545 handle_command_reply: tid=0 2024-09-21T03:56:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.218+0000 7fb884ccf740 1 client.26545 shutdown 2024-09-21T03:56:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.221+0000 7fb884ccf740 1 -- 172.21.15.16:0/1663678892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5cf01b0 msgr2=0x559cf60b2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.221+0000 7fb884ccf740 1 --2- 172.21.15.16:0/1663678892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5cf01b0 0x559cf60b2c50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb868004a80 tx=0x7fb868008430 comp rx=0 tx=0).stop 2024-09-21T03:56:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.223+0000 7fb884ccf740 1 -- 172.21.15.16:0/1663678892 shutdown_connections 2024-09-21T03:56:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.223+0000 7fb884ccf740 1 --2- 172.21.15.16:0/1663678892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5cf01b0 0x559cf60b2c50 secure :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb868004a80 tx=0x7fb868008430 comp rx=0 tx=0).stop 2024-09-21T03:56:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.223+0000 7fb884ccf740 1 --2- 172.21.15.16:0/1663678892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5ccf4d0 0x559cf60b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.223+0000 7fb884ccf740 1 --2- 172.21.15.16:0/1663678892 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x559cf5caed80 0x559cf60bedf0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.223+0000 7fb884ccf740 1 --2- 172.21.15.16:0/1663678892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5cc72f0 0x559cf60b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.223+0000 7fb884ccf740 1 -- 172.21.15.16:0/1663678892 >> 172.21.15.16:0/1663678892 conn(0x559cf5ce4300 msgr2=0x559cf6091360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.224+0000 7fb860c00640 0 client.26545 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.228+0000 7fb884ccf740 1 -- 172.21.15.16:0/1663678892 shutdown_connections 2024-09-21T03:56:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.230+0000 7fb884ccf740 1 -- 172.21.15.16:0/1663678892 wait complete. 2024-09-21T03:56:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.235+0000 7fb884ccf740 20 client.26545 trim_cache size 0 max 16384 2024-09-21T03:56:18.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.307+0000 7fb884ccf740 1 Processor -- start 2024-09-21T03:56:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.308+0000 7fb884ccf740 1 -- start start 2024-09-21T03:56:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.310+0000 7fb884ccf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cc72f0 0x559cf6091360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.310+0000 7fb884ccf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5ccf4d0 0x559cf60918a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.310+0000 7fb884ccf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5c841c0 0x559cf6091de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.310+0000 7fb884ccf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781aa480 con 0x559cf5c841c0 2024-09-21T03:56:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.310+0000 7fb884ccf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781abfc0 con 0x559cf5cc72f0 2024-09-21T03:56:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.310+0000 7fb884ccf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781aab50 con 0x559cf5ccf4d0 2024-09-21T03:56:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.314+0000 7fb876a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5ccf4d0 0x559cf60918a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.314+0000 7fb876a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5ccf4d0 0x559cf60918a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:35776/0 (socket says 172.21.15.16:35776) 2024-09-21T03:56:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.314+0000 7fb876a00640 1 -- 172.21.15.16:0/1947933772 learned_addr learned my addr 172.21.15.16:0/1947933772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.317+0000 7fb876000640 1 --2- 172.21.15.16:0/1947933772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5c841c0 0x559cf6091de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.319+0000 7fb876000640 1 --2- 172.21.15.16:0/1947933772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cc72f0 0x559cf6091360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.320+0000 7fb876a00640 1 -- 172.21.15.16:0/1947933772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cc72f0 msgr2=0x559cf6091360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.321+0000 7fb876a00640 1 --2- 172.21.15.16:0/1947933772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cc72f0 0x559cf6091360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.321+0000 7fb876a00640 1 -- 172.21.15.16:0/1947933772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5c841c0 msgr2=0x559cf6091de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.321+0000 7fb876a00640 1 --2- 172.21.15.16:0/1947933772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5c841c0 0x559cf6091de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.322+0000 7fb876a00640 1 -- 172.21.15.16:0/1947933772 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cf60b1d90 con 0x559cf5ccf4d0 2024-09-21T03:56:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.322+0000 7fb876000640 1 --2- 172.21.15.16:0/1947933772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cc72f0 0x559cf6091360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.323+0000 7fb876a00640 1 --2- 172.21.15.16:0/1947933772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5ccf4d0 0x559cf60918a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb870005ab0 tx=0x7fb870005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a40fdbe38a1dafbd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.324+0000 7fb863400640 1 -- 172.21.15.16:0/1947933772 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870035040 con 0x559cf5ccf4d0 2024-09-21T03:56:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.324+0000 7fb863400640 1 -- 172.21.15.16:0/1947933772 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb870003de0 con 0x559cf5ccf4d0 2024-09-21T03:56:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.324+0000 7fb863400640 1 -- 172.21.15.16:0/1947933772 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87003c440 con 0x559cf5ccf4d0 2024-09-21T03:56:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.325+0000 7fb884ccf740 1 -- 172.21.15.16:0/1947933772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5ccf4d0 msgr2=0x559cf60918a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.325+0000 7fb884ccf740 1 --2- 172.21.15.16:0/1947933772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5ccf4d0 0x559cf60918a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb870005ab0 tx=0x7fb870005fd0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.325+0000 7fb884ccf740 1 -- 172.21.15.16:0/1947933772 shutdown_connections 2024-09-21T03:56:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.325+0000 7fb884ccf740 1 --2- 172.21.15.16:0/1947933772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5c841c0 0x559cf6091de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.325+0000 7fb884ccf740 1 --2- 172.21.15.16:0/1947933772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5ccf4d0 0x559cf60918a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.325+0000 7fb884ccf740 1 --2- 172.21.15.16:0/1947933772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cc72f0 0x559cf6091360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.325+0000 7fb884ccf740 1 -- 172.21.15.16:0/1947933772 >> 172.21.15.16:0/1947933772 conn(0x559cf5ce4300 msgr2=0x559cf6098210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.328+0000 7fb884ccf740 1 -- 172.21.15.16:0/1947933772 shutdown_connections 2024-09-21T03:56:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.329+0000 7fb884ccf740 1 -- 172.21.15.16:0/1947933772 wait complete. 2024-09-21T03:56:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.329+0000 7fb884ccf740 1 -- start start 2024-09-21T03:56:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.329+0000 7fb884ccf740 1 Processor -- start 2024-09-21T03:56:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.334+0000 7fb884ccf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5c841c0 0x559cf60b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.334+0000 7fb884ccf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cc72f0 0x559cf60b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.334+0000 7fb884ccf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5ccf4d0 0x559cf60b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.334+0000 7fb884ccf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781ab8f0 con 0x559cf5ccf4d0 2024-09-21T03:56:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.334+0000 7fb884ccf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781ab220 con 0x559cf5cc72f0 2024-09-21T03:56:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.334+0000 7fb884ccf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb878075040 con 0x559cf5c841c0 2024-09-21T03:56:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.334+0000 7fb884ccf740 10 client.0 _ll_register_callbacks cb 0x7fb86801d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.335+0000 7fb876a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cc72f0 0x559cf60b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.335+0000 7fb876a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cc72f0 0x559cf60b6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:41072/0 (socket says 172.21.15.16:41072) 2024-09-21T03:56:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.335+0000 7fb876a00640 1 -- 172.21.15.16:0/203984855 learned_addr learned my addr 172.21.15.16:0/203984855 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.336+0000 7fb876a00640 1 -- 172.21.15.16:0/203984855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5c841c0 msgr2=0x559cf60b68c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:56:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.336+0000 7fb876a00640 1 --2- 172.21.15.16:0/203984855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5c841c0 0x559cf60b68c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.336+0000 7fb876a00640 1 -- 172.21.15.16:0/203984855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5ccf4d0 msgr2=0x559cf60b7af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:56:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.340+0000 7fb876000640 1 --2- 172.21.15.16:0/203984855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5ccf4d0 0x559cf60b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.340+0000 7fb876a00640 1 --2- 172.21.15.16:0/203984855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5ccf4d0 0x559cf60b7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.340+0000 7fb876a00640 1 -- 172.21.15.16:0/203984855 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cf60b8250 con 0x559cf5cc72f0 2024-09-21T03:56:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.340+0000 7fb876000640 1 --2- 172.21.15.16:0/203984855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5ccf4d0 0x559cf60b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.340+0000 7fb876a00640 1 --2- 172.21.15.16:0/203984855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cc72f0 0x559cf60b6f80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb8700044a0 tx=0x7fb870000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b00311e4731e033 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.340+0000 7fb860c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:56:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.344+0000 7fb860c00640 1 -- 172.21.15.16:0/203984855 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870055070 con 0x559cf5cc72f0 2024-09-21T03:56:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.344+0000 7fb860c00640 1 -- 172.21.15.16:0/203984855 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb870003930 con 0x559cf5cc72f0 2024-09-21T03:56:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.344+0000 7fb884ccf740 1 -- 172.21.15.16:0/203984855 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb870003de0 con 0x559cf5cc72f0 2024-09-21T03:56:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.356+0000 7fb860c00640 1 -- 172.21.15.16:0/203984855 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb87005f050 con 0x559cf5cc72f0 2024-09-21T03:56:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.357+0000 7fb884ccf740 10 client.26594 fetch_fsmap learned FSMap version 30 2024-09-21T03:56:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.357+0000 7fb884ccf740 1 -- 172.21.15.16:0/203984855 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559cf60b94e0 con 0x559cf5cc72f0 2024-09-21T03:56:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.358+0000 7fb860c00640 1 -- 172.21.15.16:0/203984855 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb870003de0 con 0x559cf5cc72f0 2024-09-21T03:56:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.359+0000 7fb860c00640 1 -- 172.21.15.16:0/203984855 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb870031ce0 con 0x559cf5cc72f0 2024-09-21T03:56:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.359+0000 7fb884ccf740 10 client.26594 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:56:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.359+0000 7fb884ccf740 10 client.26594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.359+0000 7fb884ccf740 20 client.26594 populate_metadata read hostname 'smithi016' 2024-09-21T03:56:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.359+0000 7fb884ccf740 1 --2- 172.21.15.16:0/203984855 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x559cf613a9c0 0x559cf613ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.359+0000 7fb884ccf740 4 client.26594 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:56:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.359+0000 7fb884ccf740 1 -- 172.21.15.16:0/203984855 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559cf60080b0 con 0x559cf613a9c0 2024-09-21T03:56:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.361+0000 7fb876000640 1 --2- 172.21.15.16:0/203984855 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x559cf613a9c0 0x559cf613ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.362+0000 7fb860c00640 10 client.26594 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.362+0000 7fb876000640 1 --2- 172.21.15.16:0/203984855 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x559cf613a9c0 0x559cf613ce10 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c78b93481ab9c7fc server_cookie=ae0cae724cf1c452 in_seq=0 out_seq=0 2024-09-21T03:56:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.483+0000 7fb860c00640 1 -- 172.21.15.16:0/203984855 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559cf60080b0 con 0x559cf613a9c0 2024-09-21T03:56:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.483+0000 7fb860c00640 10 client.26594 handle_command_reply: tid=0 2024-09-21T03:56:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.485+0000 7fb884ccf740 1 client.26594 shutdown 2024-09-21T03:56:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.490+0000 7fb884ccf740 1 -- 172.21.15.16:0/203984855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cc72f0 msgr2=0x559cf60b6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.490+0000 7fb884ccf740 1 --2- 172.21.15.16:0/203984855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cc72f0 0x559cf60b6f80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb8700044a0 tx=0x7fb870000bc0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.491+0000 7fb884ccf740 1 -- 172.21.15.16:0/203984855 shutdown_connections 2024-09-21T03:56:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.491+0000 7fb884ccf740 1 --2- 172.21.15.16:0/203984855 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x559cf613a9c0 0x559cf613ce10 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.491+0000 7fb884ccf740 1 --2- 172.21.15.16:0/203984855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cf5ccf4d0 0x559cf60b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.491+0000 7fb884ccf740 1 --2- 172.21.15.16:0/203984855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559cf5cc72f0 0x559cf60b6f80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.491+0000 7fb884ccf740 1 --2- 172.21.15.16:0/203984855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cf5c841c0 0x559cf60b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.491+0000 7fb884ccf740 1 -- 172.21.15.16:0/203984855 >> 172.21.15.16:0/203984855 conn(0x559cf5ce4300 msgr2=0x559cf609b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.491+0000 7fb860c00640 0 client.26594 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.492+0000 7fb884ccf740 1 -- 172.21.15.16:0/203984855 shutdown_connections 2024-09-21T03:56:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.492+0000 7fb884ccf740 1 -- 172.21.15.16:0/203984855 wait complete. 2024-09-21T03:56:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.492+0000 7fb884ccf740 20 client.26594 trim_cache size 0 max 16384 2024-09-21T03:56:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:56:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:56:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:56:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:56:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.496+0000 7fb877e00640 1 -- 172.21.15.16:0/712099833 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb85c0b6650 msgr2=0x7fb85c0b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.496+0000 7fb877e00640 1 --2- 172.21.15.16:0/712099833 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb85c0b6650 0x7fb85c0b8b60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb8640acbf0 tx=0x7fb864003f80 comp rx=0 tx=0).stop 2024-09-21T03:56:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.496+0000 7fb877e00640 1 -- 172.21.15.16:0/712099833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87810b510 msgr2=0x7fb8781a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.496+0000 7fb877e00640 1 --2- 172.21.15.16:0/712099833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87810b510 0x7fb8781a1630 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb86400d6b0 tx=0x7fb86400db70 comp rx=0 tx=0).stop 2024-09-21T03:56:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.496+0000 7fb877e00640 1 -- 172.21.15.16:0/712099833 shutdown_connections 2024-09-21T03:56:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.496+0000 7fb877e00640 1 --2- 172.21.15.16:0/712099833 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb85c0b6650 0x7fb85c0b8b60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.496+0000 7fb877e00640 1 --2- 172.21.15.16:0/712099833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87810b510 0x7fb8781a1630 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.496+0000 7fb877e00640 1 --2- 172.21.15.16:0/712099833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb87810ab70 0x7fb8781a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.496+0000 7fb877e00640 1 --2- 172.21.15.16:0/712099833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb878109af0 0x7fb8781a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.496+0000 7fb877e00640 1 -- 172.21.15.16:0/712099833 >> 172.21.15.16:0/712099833 conn(0x7fb878078e30 msgr2=0x7fb87810de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.497+0000 7fb877e00640 1 -- 172.21.15.16:0/712099833 shutdown_connections 2024-09-21T03:56:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:18.497+0000 7fb877e00640 1 -- 172.21.15.16:0/712099833 wait complete. 2024-09-21T03:56:19.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:18 smithi134 ceph-mon[35031]: pgmap v718: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 243 MiB/s wr, 1.25k op/s 2024-09-21T03:56:19.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:18 smithi123 ceph-mon[34673]: pgmap v718: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 243 MiB/s wr, 1.25k op/s 2024-09-21T03:56:19.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:18 smithi016 ceph-mon[30375]: pgmap v718: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 243 MiB/s wr, 1.25k op/s 2024-09-21T03:56:21.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:20 smithi134 ceph-mon[35031]: pgmap v719: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 259 MiB/s wr, 1.34k op/s 2024-09-21T03:56:21.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:20 smithi123 ceph-mon[34673]: pgmap v719: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 259 MiB/s wr, 1.34k op/s 2024-09-21T03:56:21.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:20 smithi016 ceph-mon[30375]: pgmap v719: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 259 MiB/s wr, 1.34k op/s 2024-09-21T03:56:23.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:22 smithi134 ceph-mon[35031]: pgmap v720: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 226 MiB/s wr, 1.16k op/s 2024-09-21T03:56:23.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:22 smithi123 ceph-mon[34673]: pgmap v720: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 226 MiB/s wr, 1.16k op/s 2024-09-21T03:56:23.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:22 smithi016 ceph-mon[30375]: pgmap v720: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 226 MiB/s wr, 1.16k op/s 2024-09-21T03:56:25.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:24 smithi134 ceph-mon[35031]: pgmap v721: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 237 MiB/s wr, 1.23k op/s 2024-09-21T03:56:25.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:24 smithi123 ceph-mon[34673]: pgmap v721: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 237 MiB/s wr, 1.23k op/s 2024-09-21T03:56:25.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:24 smithi016 ceph-mon[30375]: pgmap v721: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 237 MiB/s wr, 1.23k op/s 2024-09-21T03:56:27.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:26 smithi134 ceph-mon[35031]: pgmap v722: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 256 MiB/s wr, 1.32k op/s 2024-09-21T03:56:27.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:26 smithi123 ceph-mon[34673]: pgmap v722: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 256 MiB/s wr, 1.32k op/s 2024-09-21T03:56:27.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:26 smithi016 ceph-mon[30375]: pgmap v722: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 256 MiB/s wr, 1.32k op/s 2024-09-21T03:56:29.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:28 smithi134 ceph-mon[35031]: pgmap v723: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 222 MiB/s wr, 1.14k op/s 2024-09-21T03:56:29.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:28 smithi123 ceph-mon[34673]: pgmap v723: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 222 MiB/s wr, 1.14k op/s 2024-09-21T03:56:29.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:28 smithi016 ceph-mon[30375]: pgmap v723: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 222 MiB/s wr, 1.14k op/s 2024-09-21T03:56:31.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:30 smithi134 ceph-mon[35031]: pgmap v724: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 253 MiB/s wr, 1.31k op/s 2024-09-21T03:56:31.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:30 smithi123 ceph-mon[34673]: pgmap v724: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 253 MiB/s wr, 1.31k op/s 2024-09-21T03:56:31.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:30 smithi016 ceph-mon[30375]: pgmap v724: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 253 MiB/s wr, 1.31k op/s 2024-09-21T03:56:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.911+0000 7f265fe00640 1 -- 172.21.15.16:0/736663694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2660104560 msgr2=0x7f2660104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.911+0000 7f265fe00640 1 --2- 172.21.15.16:0/736663694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2660104560 0x7f2660104970 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f264c009900 tx=0x7f264c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.913+0000 7f265fe00640 1 -- 172.21.15.16:0/736663694 shutdown_connections 2024-09-21T03:56:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.913+0000 7f265fe00640 1 --2- 172.21.15.16:0/736663694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2660104f00 0x7f266010b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.913+0000 7f265fe00640 1 --2- 172.21.15.16:0/736663694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2660104560 0x7f2660104970 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.913+0000 7f265fe00640 1 --2- 172.21.15.16:0/736663694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26601034e0 0x7f2660103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.913+0000 7f265fe00640 1 -- 172.21.15.16:0/736663694 >> 172.21.15.16:0/736663694 conn(0x7f26600fee20 msgr2=0x7f26601012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:31.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.917+0000 7f265fe00640 1 -- 172.21.15.16:0/736663694 shutdown_connections 2024-09-21T03:56:31.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.917+0000 7f265fe00640 1 -- 172.21.15.16:0/736663694 wait complete. 2024-09-21T03:56:31.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.918+0000 7f265fe00640 1 Processor -- start 2024-09-21T03:56:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.919+0000 7f265fe00640 1 -- start start 2024-09-21T03:56:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.919+0000 7f265fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26601034e0 0x7f26601a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.920+0000 7f265fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2660104560 0x7f26601a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.920+0000 7f265ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26601034e0 0x7f26601a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.920+0000 7f265ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26601034e0 0x7f26601a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52438/0 (socket says 172.21.15.16:52438) 2024-09-21T03:56:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.920+0000 7f265ea00640 1 -- 172.21.15.16:0/3408995871 learned_addr learned my addr 172.21.15.16:0/3408995871 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.920+0000 7f265fe00640 1 --2- 172.21.15.16:0/3408995871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2660104f00 0x7f26601a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.920+0000 7f265fe00640 1 -- 172.21.15.16:0/3408995871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2660115780 con 0x7f2660104f00 2024-09-21T03:56:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.921+0000 7f265e000640 1 --2- 172.21.15.16:0/3408995871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2660104560 0x7f26601a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.921+0000 7f265fe00640 1 -- 172.21.15.16:0/3408995871 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2660115600 con 0x7f2660104560 2024-09-21T03:56:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.921+0000 7f265fe00640 1 -- 172.21.15.16:0/3408995871 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2660115900 con 0x7f26601034e0 2024-09-21T03:56:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.921+0000 7f265ea00640 1 -- 172.21.15.16:0/3408995871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2660104560 msgr2=0x7f26601a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.921+0000 7f265ea00640 1 --2- 172.21.15.16:0/3408995871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2660104560 0x7f26601a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.921+0000 7f265ea00640 1 -- 172.21.15.16:0/3408995871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2660104f00 msgr2=0x7f26601a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.921+0000 7f265ea00640 1 --2- 172.21.15.16:0/3408995871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2660104f00 0x7f26601a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.921+0000 7f265ea00640 1 -- 172.21.15.16:0/3408995871 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26601a1da0 con 0x7f26601034e0 2024-09-21T03:56:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.922+0000 7f265ea00640 1 --2- 172.21.15.16:0/3408995871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26601034e0 0x7f26601a03d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f265000d950 tx=0x7f265000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.922+0000 7f2657400640 1 -- 172.21.15.16:0/3408995871 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650017070 con 0x7f26601034e0 2024-09-21T03:56:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.922+0000 7f265fe00640 1 -- 172.21.15.16:0/3408995871 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26601a2090 con 0x7f26601034e0 2024-09-21T03:56:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.922+0000 7f265fe00640 1 -- 172.21.15.16:0/3408995871 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26601a8780 con 0x7f26601034e0 2024-09-21T03:56:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.922+0000 7f2657400640 1 -- 172.21.15.16:0/3408995871 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2650009c30 con 0x7f26601034e0 2024-09-21T03:56:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.922+0000 7f2657400640 1 -- 172.21.15.16:0/3408995871 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265000f620 con 0x7f26601034e0 2024-09-21T03:56:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.928+0000 7f2657400640 1 -- 172.21.15.16:0/3408995871 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2650010460 con 0x7f26601034e0 2024-09-21T03:56:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.929+0000 7f266cc45740 1 Processor -- start 2024-09-21T03:56:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.929+0000 7f266cc45740 1 -- start start 2024-09-21T03:56:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.929+0000 7f266cc45740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150881b0 0x56441542f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.929+0000 7f266cc45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56441505f2f0 0x56441542f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.929+0000 7f266cc45740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644150674d0 0x564415430470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.929+0000 7f266cc45740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5644153a26b0 con 0x5644150881b0 2024-09-21T03:56:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.929+0000 7f266cc45740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564415391ae0 con 0x5644150674d0 2024-09-21T03:56:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.929+0000 7f266cc45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5644153958f0 con 0x56441505f2f0 2024-09-21T03:56:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.930+0000 7f265f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150881b0 0x56441542f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.930+0000 7f265f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150881b0 0x56441542f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50114/0 (socket says 172.21.15.16:50114) 2024-09-21T03:56:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.930+0000 7f265f400640 1 -- 172.21.15.16:0/1597120117 learned_addr learned my addr 172.21.15.16:0/1597120117 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.930+0000 7f265e000640 1 --2- 172.21.15.16:0/1597120117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644150674d0 0x564415430470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.931+0000 7f265f400640 1 -- 172.21.15.16:0/1597120117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56441505f2f0 msgr2=0x56441542f920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:56:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.931+0000 7f265f400640 1 --2- 172.21.15.16:0/1597120117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56441505f2f0 0x56441542f920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.931+0000 7f265f400640 1 -- 172.21.15.16:0/1597120117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644150674d0 msgr2=0x564415430470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.931+0000 7f265f400640 1 --2- 172.21.15.16:0/1597120117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644150674d0 0x564415430470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.931+0000 7f265f400640 1 -- 172.21.15.16:0/1597120117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56441535e780 con 0x5644150881b0 2024-09-21T03:56:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.932+0000 7f265f400640 1 --2- 172.21.15.16:0/1597120117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150881b0 0x56441542f2b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2658009ea0 tx=0x7f2658009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7aca6c8717dcf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.932+0000 7f264be00640 1 -- 172.21.15.16:0/1597120117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658004d20 con 0x5644150881b0 2024-09-21T03:56:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.932+0000 7f264be00640 1 -- 172.21.15.16:0/1597120117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f265800ee60 con 0x5644150881b0 2024-09-21T03:56:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.933+0000 7f266cc45740 1 -- 172.21.15.16:0/1597120117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150881b0 msgr2=0x56441542f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.933+0000 7f264be00640 1 -- 172.21.15.16:0/1597120117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658004d20 con 0x5644150881b0 2024-09-21T03:56:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.934+0000 7f266cc45740 1 --2- 172.21.15.16:0/1597120117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150881b0 0x56441542f2b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2658009ea0 tx=0x7f2658009f90 comp rx=0 tx=0).stop 2024-09-21T03:56:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.934+0000 7f266cc45740 1 -- 172.21.15.16:0/1597120117 shutdown_connections 2024-09-21T03:56:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.934+0000 7f266cc45740 1 --2- 172.21.15.16:0/1597120117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644150674d0 0x564415430470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.934+0000 7f266cc45740 1 --2- 172.21.15.16:0/1597120117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56441505f2f0 0x56441542f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.934+0000 7f266cc45740 1 --2- 172.21.15.16:0/1597120117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150881b0 0x56441542f2b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.934+0000 7f266cc45740 1 -- 172.21.15.16:0/1597120117 >> 172.21.15.16:0/1597120117 conn(0x56441507c300 msgr2=0x56441542ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.934+0000 7f266cc45740 1 -- 172.21.15.16:0/1597120117 shutdown_connections 2024-09-21T03:56:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.934+0000 7f266cc45740 1 -- 172.21.15.16:0/1597120117 wait complete. 2024-09-21T03:56:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.936+0000 7f2657400640 1 --2- 172.21.15.16:0/3408995871 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f26440b64c0 0x7f26440b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.936+0000 7f2657400640 1 -- 172.21.15.16:0/3408995871 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f26500e71e0 con 0x7f26601034e0 2024-09-21T03:56:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.937+0000 7f265f400640 1 --2- 172.21.15.16:0/3408995871 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f26440b64c0 0x7f26440b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.938+0000 7f266cc45740 1 -- start start 2024-09-21T03:56:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.938+0000 7f266cc45740 1 Processor -- start 2024-09-21T03:56:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.939+0000 7f265f400640 1 --2- 172.21.15.16:0/3408995871 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f26440b64c0 0x7f26440b89d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f265800f010 tx=0x7f2658005a60 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.940+0000 7f266cc45740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56441505f2f0 0x564415438990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.940+0000 7f266cc45740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150674d0 0x564415439050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.940+0000 7f266cc45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644150881b0 0x564415439bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.941+0000 7f265ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56441505f2f0 0x564415438990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.941+0000 7f266cc45740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5644152c9ab0 con 0x5644150674d0 2024-09-21T03:56:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.941+0000 7f266cc45740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564415338570 con 0x56441505f2f0 2024-09-21T03:56:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.941+0000 7f266cc45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564415430fe0 con 0x5644150881b0 2024-09-21T03:56:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.941+0000 7f265e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150674d0 0x564415439050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.941+0000 7f266cc45740 10 client.0 _ll_register_callbacks cb 0x564415347100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.941+0000 7f265e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150674d0 0x564415439050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50118/0 (socket says 172.21.15.16:50118) 2024-09-21T03:56:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.941+0000 7f265e000640 1 -- 172.21.15.16:0/693933997 learned_addr learned my addr 172.21.15.16:0/693933997 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.941+0000 7f265f400640 1 --2- 172.21.15.16:0/693933997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644150881b0 0x564415439bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.942+0000 7f265e000640 1 -- 172.21.15.16:0/693933997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644150881b0 msgr2=0x564415439bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.942+0000 7f265e000640 1 --2- 172.21.15.16:0/693933997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644150881b0 0x564415439bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.942+0000 7f265e000640 1 -- 172.21.15.16:0/693933997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56441505f2f0 msgr2=0x564415438990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.943+0000 7f265e000640 1 --2- 172.21.15.16:0/693933997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56441505f2f0 0x564415438990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.943+0000 7f265e000640 1 -- 172.21.15.16:0/693933997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644150f3790 con 0x5644150674d0 2024-09-21T03:56:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.944+0000 7f265ea00640 1 --2- 172.21.15.16:0/693933997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56441505f2f0 0x564415438990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.962+0000 7f2648c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:56:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.962+0000 7f265e000640 1 --2- 172.21.15.16:0/693933997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150674d0 0x564415439050 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f264c02a910 tx=0x7f264c031910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df596cdb52a99d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.965+0000 7f2648c00640 1 -- 172.21.15.16:0/693933997 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c035040 con 0x5644150674d0 2024-09-21T03:56:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.965+0000 7f2648c00640 1 -- 172.21.15.16:0/693933997 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f264c004040 con 0x5644150674d0 2024-09-21T03:56:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.969+0000 7f266cc45740 1 -- 172.21.15.16:0/693933997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f265800ee60 con 0x5644150674d0 2024-09-21T03:56:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.969+0000 7f2648c00640 1 -- 172.21.15.16:0/693933997 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f264c035040 con 0x5644150674d0 2024-09-21T03:56:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.971+0000 7f266cc45740 10 client.16341 fetch_fsmap learned FSMap version 30 2024-09-21T03:56:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.971+0000 7f266cc45740 1 -- 172.21.15.16:0/693933997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56441516c620 con 0x5644150674d0 2024-09-21T03:56:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.972+0000 7f2648c00640 1 -- 172.21.15.16:0/693933997 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f265800ee60 con 0x5644150674d0 2024-09-21T03:56:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.972+0000 7f2648c00640 1 -- 172.21.15.16:0/693933997 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f264c03c440 con 0x5644150674d0 2024-09-21T03:56:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.972+0000 7f266cc45740 10 client.16341 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:56:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.972+0000 7f266cc45740 10 client.16341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.973+0000 7f266cc45740 20 client.16341 populate_metadata read hostname 'smithi016' 2024-09-21T03:56:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.973+0000 7f266cc45740 1 --2- 172.21.15.16:0/693933997 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x564415046d80 0x564415446190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.974+0000 7f266cc45740 4 client.16341 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.974+0000 7f266cc45740 1 -- 172.21.15.16:0/693933997 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644153a4600 con 0x564415046d80 2024-09-21T03:56:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.974+0000 7f265ea00640 1 --2- 172.21.15.16:0/693933997 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x564415046d80 0x564415446190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.976+0000 7f265ea00640 1 --2- 172.21.15.16:0/693933997 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x564415046d80 0x564415446190 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43480e60d230d501 server_cookie=7830447270fc2420 in_seq=0 out_seq=0 2024-09-21T03:56:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.976+0000 7f2648c00640 10 client.16341 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.977+0000 7f2648c00640 1 -- 172.21.15.16:0/693933997 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5644153a4600 con 0x564415046d80 2024-09-21T03:56:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.977+0000 7f2648c00640 10 client.16341 handle_command_reply: tid=0 2024-09-21T03:56:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.977+0000 7f266cc45740 1 client.16341 shutdown 2024-09-21T03:56:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.978+0000 7f266cc45740 1 -- 172.21.15.16:0/693933997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150674d0 msgr2=0x564415439050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.978+0000 7f266cc45740 1 --2- 172.21.15.16:0/693933997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150674d0 0x564415439050 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f264c02a910 tx=0x7f264c031910 comp rx=0 tx=0).stop 2024-09-21T03:56:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.978+0000 7f266cc45740 1 -- 172.21.15.16:0/693933997 shutdown_connections 2024-09-21T03:56:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.978+0000 7f266cc45740 1 --2- 172.21.15.16:0/693933997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644150881b0 0x564415439bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.978+0000 7f266cc45740 1 --2- 172.21.15.16:0/693933997 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x564415046d80 0x564415446190 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.978+0000 7f266cc45740 1 --2- 172.21.15.16:0/693933997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150674d0 0x564415439050 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.978+0000 7f266cc45740 1 --2- 172.21.15.16:0/693933997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56441505f2f0 0x564415438990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.978+0000 7f266cc45740 1 -- 172.21.15.16:0/693933997 >> 172.21.15.16:0/693933997 conn(0x56441507c300 msgr2=0x564415430320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.978+0000 7f2648c00640 0 client.16341 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.979+0000 7f266cc45740 1 -- 172.21.15.16:0/693933997 shutdown_connections 2024-09-21T03:56:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.979+0000 7f266cc45740 1 -- 172.21.15.16:0/693933997 wait complete. 2024-09-21T03:56:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:31.982+0000 7f266cc45740 20 client.16341 trim_cache size 0 max 16384 2024-09-21T03:56:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.043+0000 7f266cc45740 1 Processor -- start 2024-09-21T03:56:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.043+0000 7f266cc45740 1 -- start start 2024-09-21T03:56:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.044+0000 7f266cc45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56441505f2f0 0x5644154382e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.044+0000 7f266cc45740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644150674d0 0x564415433400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.044+0000 7f266cc45740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150881b0 0x564415433940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.044+0000 7f266cc45740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f264000ff80 con 0x5644150881b0 2024-09-21T03:56:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.044+0000 7f266cc45740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2640011ac0 con 0x5644150674d0 2024-09-21T03:56:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.044+0000 7f266cc45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2640010650 con 0x56441505f2f0 2024-09-21T03:56:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.044+0000 7f265ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56441505f2f0 0x5644154382e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.044+0000 7f265ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56441505f2f0 0x5644154382e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52456/0 (socket says 172.21.15.16:52456) 2024-09-21T03:56:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.044+0000 7f265ea00640 1 -- 172.21.15.16:0/1116701709 learned_addr learned my addr 172.21.15.16:0/1116701709 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.044+0000 7f265e000640 1 --2- 172.21.15.16:0/1116701709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644150674d0 0x564415433400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.045+0000 7f265ea00640 1 -- 172.21.15.16:0/1116701709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644150674d0 msgr2=0x564415433400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.045+0000 7f265ea00640 1 --2- 172.21.15.16:0/1116701709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644150674d0 0x564415433400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.049+0000 7f265ea00640 1 -- 172.21.15.16:0/1116701709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150881b0 msgr2=0x564415433940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.049+0000 7f265ea00640 1 --2- 172.21.15.16:0/1116701709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150881b0 0x564415433940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.049+0000 7f265ea00640 1 -- 172.21.15.16:0/1116701709 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56441542fd40 con 0x56441505f2f0 2024-09-21T03:56:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.049+0000 7f265ea00640 1 --2- 172.21.15.16:0/1116701709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56441505f2f0 0x5644154382e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f26500acc60 tx=0x7f26500d0040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cc90a0c524304f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.049+0000 7f264b400640 1 -- 172.21.15.16:0/1116701709 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650020040 con 0x56441505f2f0 2024-09-21T03:56:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.049+0000 7f264b400640 1 -- 172.21.15.16:0/1116701709 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2650009de0 con 0x56441505f2f0 2024-09-21T03:56:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.049+0000 7f264b400640 1 -- 172.21.15.16:0/1116701709 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26500a8b10 con 0x56441505f2f0 2024-09-21T03:56:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.051+0000 7f266cc45740 1 -- 172.21.15.16:0/1116701709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56441505f2f0 msgr2=0x5644154382e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.051+0000 7f266cc45740 1 --2- 172.21.15.16:0/1116701709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56441505f2f0 0x5644154382e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f26500acc60 tx=0x7f26500d0040 comp rx=0 tx=0).stop 2024-09-21T03:56:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.051+0000 7f266cc45740 1 -- 172.21.15.16:0/1116701709 shutdown_connections 2024-09-21T03:56:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.051+0000 7f266cc45740 1 --2- 172.21.15.16:0/1116701709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644150881b0 0x564415433940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.051+0000 7f266cc45740 1 --2- 172.21.15.16:0/1116701709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644150674d0 0x564415433400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.051+0000 7f266cc45740 1 --2- 172.21.15.16:0/1116701709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56441505f2f0 0x5644154382e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.051+0000 7f266cc45740 1 -- 172.21.15.16:0/1116701709 >> 172.21.15.16:0/1116701709 conn(0x56441507c300 msgr2=0x564415437bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.051+0000 7f266cc45740 1 -- 172.21.15.16:0/1116701709 shutdown_connections 2024-09-21T03:56:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.051+0000 7f266cc45740 1 -- 172.21.15.16:0/1116701709 wait complete. 2024-09-21T03:56:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.052+0000 7f266cc45740 1 -- start start 2024-09-21T03:56:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.052+0000 7f266cc45740 1 Processor -- start 2024-09-21T03:56:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.055+0000 7f266cc45740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56441505f2f0 0x56441543dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.055+0000 7f266cc45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644150674d0 0x56441543e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.055+0000 7f266cc45740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644150881b0 0x56441543ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.057+0000 7f265e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644150674d0 0x56441543e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.057+0000 7f265e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644150674d0 0x56441543e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:52460/0 (socket says 172.21.15.16:52460) 2024-09-21T03:56:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.058+0000 7f266cc45740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26400113f0 con 0x56441505f2f0 2024-09-21T03:56:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.058+0000 7f266cc45740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2640010d20 con 0x5644150881b0 2024-09-21T03:56:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.058+0000 7f266cc45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f264000df00 con 0x5644150674d0 2024-09-21T03:56:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.058+0000 7f266cc45740 10 client.0 _ll_register_callbacks cb 0x7f264c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.064+0000 7f265f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644150881b0 0x56441543ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.064+0000 7f265f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644150881b0 0x56441543ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:46848/0 (socket says 172.21.15.16:46848) 2024-09-21T03:56:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.064+0000 7f265f400640 1 -- 172.21.15.16:0/672228306 learned_addr learned my addr 172.21.15.16:0/672228306 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.064+0000 7f265ea00640 1 --2- 172.21.15.16:0/672228306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56441505f2f0 0x56441543dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.064+0000 7f265e000640 1 -- 172.21.15.16:0/672228306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644150881b0 msgr2=0x56441543ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.064+0000 7f265e000640 1 --2- 172.21.15.16:0/672228306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644150881b0 0x56441543ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.064+0000 7f265e000640 1 -- 172.21.15.16:0/672228306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56441505f2f0 msgr2=0x56441543dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.064+0000 7f265e000640 1 --2- 172.21.15.16:0/672228306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56441505f2f0 0x56441543dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.064+0000 7f265e000640 1 -- 172.21.15.16:0/672228306 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56441543f6d0 con 0x5644150674d0 2024-09-21T03:56:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.065+0000 7f265e000640 1 --2- 172.21.15.16:0/672228306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644150674d0 0x56441543e450 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f264c03ca00 tx=0x7f264c03c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=272404593ea3782c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.065+0000 7f2648c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:56:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.066+0000 7f2648c00640 1 -- 172.21.15.16:0/672228306 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c000e60 con 0x5644150674d0 2024-09-21T03:56:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.066+0000 7f2648c00640 1 -- 172.21.15.16:0/672228306 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f264c03cc90 con 0x5644150674d0 2024-09-21T03:56:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.067+0000 7f266cc45740 1 -- 172.21.15.16:0/672228306 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2650009de0 con 0x5644150674d0 2024-09-21T03:56:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.071+0000 7f2648c00640 1 -- 172.21.15.16:0/672228306 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f264c035040 con 0x5644150674d0 2024-09-21T03:56:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.071+0000 7f266cc45740 10 client.26563 fetch_fsmap learned FSMap version 30 2024-09-21T03:56:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.071+0000 7f266cc45740 1 -- 172.21.15.16:0/672228306 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564415440a30 con 0x5644150674d0 2024-09-21T03:56:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.072+0000 7f2648c00640 1 -- 172.21.15.16:0/672228306 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2650009de0 con 0x5644150674d0 2024-09-21T03:56:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.072+0000 7f2648c00640 1 -- 172.21.15.16:0/672228306 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f264c003aa0 con 0x5644150674d0 2024-09-21T03:56:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.072+0000 7f266cc45740 10 client.26563 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:56:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.073+0000 7f266cc45740 10 client.26563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.073+0000 7f266cc45740 20 client.26563 populate_metadata read hostname 'smithi016' 2024-09-21T03:56:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.073+0000 7f266cc45740 1 --2- 172.21.15.16:0/672228306 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56441501c1c0 0x5644154b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.073+0000 7f266cc45740 4 client.26563 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:56:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.073+0000 7f266cc45740 1 -- 172.21.15.16:0/672228306 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644153a00b0 con 0x56441501c1c0 2024-09-21T03:56:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.076+0000 7f265ea00640 1 --2- 172.21.15.16:0/672228306 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56441501c1c0 0x5644154b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.078+0000 7f2648c00640 10 client.26563 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.078+0000 7f265ea00640 1 --2- 172.21.15.16:0/672228306 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56441501c1c0 0x5644154b2f80 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67cc6f451e0a2ba9 server_cookie=f95c4dcc5081f488 in_seq=0 out_seq=0 2024-09-21T03:56:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.106+0000 7f2648c00640 1 -- 172.21.15.16:0/672228306 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644153a00b0 con 0x56441501c1c0 2024-09-21T03:56:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.106+0000 7f2648c00640 10 client.26563 handle_command_reply: tid=0 2024-09-21T03:56:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.107+0000 7f266cc45740 1 client.26563 shutdown 2024-09-21T03:56:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.108+0000 7f266cc45740 1 -- 172.21.15.16:0/672228306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644150674d0 msgr2=0x56441543e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.108+0000 7f266cc45740 1 --2- 172.21.15.16:0/672228306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644150674d0 0x56441543e450 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f264c03ca00 tx=0x7f264c03c6f0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.109+0000 7f266cc45740 1 -- 172.21.15.16:0/672228306 shutdown_connections 2024-09-21T03:56:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.109+0000 7f266cc45740 1 --2- 172.21.15.16:0/672228306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644150881b0 0x56441543ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.109+0000 7f266cc45740 1 --2- 172.21.15.16:0/672228306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644150674d0 0x56441543e450 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.109+0000 7f266cc45740 1 --2- 172.21.15.16:0/672228306 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x56441501c1c0 0x5644154b2f80 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.109+0000 7f266cc45740 1 --2- 172.21.15.16:0/672228306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56441505f2f0 0x56441543dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.109+0000 7f266cc45740 1 -- 172.21.15.16:0/672228306 >> 172.21.15.16:0/672228306 conn(0x56441507c300 msgr2=0x564415433400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.109+0000 7f2648c00640 0 client.26563 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.110+0000 7f266cc45740 1 -- 172.21.15.16:0/672228306 shutdown_connections 2024-09-21T03:56:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.110+0000 7f266cc45740 1 -- 172.21.15.16:0/672228306 wait complete. 2024-09-21T03:56:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.111+0000 7f266cc45740 20 client.26563 trim_cache size 0 max 16384 2024-09-21T03:56:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:56:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:56:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:56:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:56:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.114+0000 7f265fe00640 1 -- 172.21.15.16:0/3408995871 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f26440b64c0 msgr2=0x7f26440b89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.114+0000 7f265fe00640 1 --2- 172.21.15.16:0/3408995871 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f26440b64c0 0x7f26440b89d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f265800f010 tx=0x7f2658005a60 comp rx=0 tx=0).stop 2024-09-21T03:56:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.114+0000 7f265fe00640 1 -- 172.21.15.16:0/3408995871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26601034e0 msgr2=0x7f26601a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.114+0000 7f265fe00640 1 --2- 172.21.15.16:0/3408995871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26601034e0 0x7f26601a03d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f265000d950 tx=0x7f265000de10 comp rx=0 tx=0).stop 2024-09-21T03:56:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.115+0000 7f265fe00640 1 -- 172.21.15.16:0/3408995871 shutdown_connections 2024-09-21T03:56:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.115+0000 7f265fe00640 1 --2- 172.21.15.16:0/3408995871 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f26440b64c0 0x7f26440b89d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.115+0000 7f265fe00640 1 --2- 172.21.15.16:0/3408995871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2660104f00 0x7f26601a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.115+0000 7f265fe00640 1 --2- 172.21.15.16:0/3408995871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2660104560 0x7f26601a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.115+0000 7f265fe00640 1 --2- 172.21.15.16:0/3408995871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26601034e0 0x7f26601a03d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.115+0000 7f265fe00640 1 -- 172.21.15.16:0/3408995871 >> 172.21.15.16:0/3408995871 conn(0x7f26600fee20 msgr2=0x7f2660107880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.115+0000 7f265fe00640 1 -- 172.21.15.16:0/3408995871 shutdown_connections 2024-09-21T03:56:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:32.115+0000 7f265fe00640 1 -- 172.21.15.16:0/3408995871 wait complete. 2024-09-21T03:56:33.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:33 smithi134 ceph-mon[35031]: pgmap v725: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 225 MiB/s wr, 1.17k op/s 2024-09-21T03:56:33.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:33 smithi123 ceph-mon[34673]: pgmap v725: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 225 MiB/s wr, 1.17k op/s 2024-09-21T03:56:33.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:33 smithi016 ceph-mon[30375]: pgmap v725: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 225 MiB/s wr, 1.17k op/s 2024-09-21T03:56:35.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:35 smithi134 ceph-mon[35031]: pgmap v726: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 227 MiB/s wr, 1.19k op/s 2024-09-21T03:56:35.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:35 smithi123 ceph-mon[34673]: pgmap v726: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 227 MiB/s wr, 1.19k op/s 2024-09-21T03:56:35.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:35 smithi016 ceph-mon[30375]: pgmap v726: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 227 MiB/s wr, 1.19k op/s 2024-09-21T03:56:37.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:37 smithi134 ceph-mon[35031]: pgmap v727: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 250 MiB/s wr, 1.30k op/s 2024-09-21T03:56:37.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:37 smithi123 ceph-mon[34673]: pgmap v727: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 250 MiB/s wr, 1.30k op/s 2024-09-21T03:56:37.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:37 smithi016 ceph-mon[30375]: pgmap v727: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 250 MiB/s wr, 1.30k op/s 2024-09-21T03:56:39.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:39 smithi134 ceph-mon[35031]: pgmap v728: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 213 MiB/s wr, 1.11k op/s 2024-09-21T03:56:39.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:39 smithi123 ceph-mon[34673]: pgmap v728: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 213 MiB/s wr, 1.11k op/s 2024-09-21T03:56:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:39 smithi016 ceph-mon[30375]: pgmap v728: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 213 MiB/s wr, 1.11k op/s 2024-09-21T03:56:41.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:41 smithi134 ceph-mon[35031]: pgmap v729: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 248 MiB/s wr, 1.30k op/s 2024-09-21T03:56:41.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:41 smithi123 ceph-mon[34673]: pgmap v729: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 248 MiB/s wr, 1.30k op/s 2024-09-21T03:56:41.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:41 smithi016 ceph-mon[30375]: pgmap v729: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 248 MiB/s wr, 1.30k op/s 2024-09-21T03:56:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:43 smithi134 ceph-mon[35031]: pgmap v730: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 221 MiB/s wr, 1.15k op/s 2024-09-21T03:56:43.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:43 smithi123 ceph-mon[34673]: pgmap v730: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 221 MiB/s wr, 1.15k op/s 2024-09-21T03:56:43.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:43 smithi016 ceph-mon[30375]: pgmap v730: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 221 MiB/s wr, 1.15k op/s 2024-09-21T03:56:44.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:44 smithi134 ceph-mon[35031]: pgmap v731: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 218 MiB/s wr, 1.14k op/s 2024-09-21T03:56:44.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:44 smithi123 ceph-mon[34673]: pgmap v731: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 218 MiB/s wr, 1.14k op/s 2024-09-21T03:56:44.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:44 smithi016 ceph-mon[30375]: pgmap v731: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 218 MiB/s wr, 1.14k op/s 2024-09-21T03:56:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.436+0000 7fac27a00640 1 -- 172.21.15.16:0/620560173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac2010f9d0 msgr2=0x7fac2011ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.436+0000 7fac27a00640 1 --2- 172.21.15.16:0/620560173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac2010f9d0 0x7fac2011ab90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fac10009900 tx=0x7fac1002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.437+0000 7fac27a00640 1 -- 172.21.15.16:0/620560173 shutdown_connections 2024-09-21T03:56:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.437+0000 7fac27a00640 1 --2- 172.21.15.16:0/620560173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac2010f9d0 0x7fac2011ab90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.437+0000 7fac27a00640 1 --2- 172.21.15.16:0/620560173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2010cdb0 0x7fac2010f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.437+0000 7fac27a00640 1 --2- 172.21.15.16:0/620560173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2010a3c0 0x7fac2010c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.437+0000 7fac27a00640 1 -- 172.21.15.16:0/620560173 >> 172.21.15.16:0/620560173 conn(0x7fac20070810 msgr2=0x7fac20072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.437+0000 7fac27a00640 1 -- 172.21.15.16:0/620560173 shutdown_connections 2024-09-21T03:56:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.437+0000 7fac27a00640 1 -- 172.21.15.16:0/620560173 wait complete. 2024-09-21T03:56:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.438+0000 7fac27a00640 1 Processor -- start 2024-09-21T03:56:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.438+0000 7fac27a00640 1 -- start start 2024-09-21T03:56:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.438+0000 7fac27a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2010a3c0 0x7fac201a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.438+0000 7fac27a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2010cdb0 0x7fac201a9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.438+0000 7fac27a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac2010f9d0 0x7fac201a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.438+0000 7fac27a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac2011e0d0 con 0x7fac2010cdb0 2024-09-21T03:56:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.438+0000 7fac27a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac2011df50 con 0x7fac2010a3c0 2024-09-21T03:56:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.438+0000 7fac27a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac2011e250 con 0x7fac2010f9d0 2024-09-21T03:56:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.439+0000 7fac25a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac2010f9d0 0x7fac201a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.439+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2010cdb0 0x7fac201a9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.439+0000 7fac25000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2010a3c0 0x7fac201a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.439+0000 7fac25a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac2010f9d0 0x7fac201a9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:48720/0 (socket says 172.21.15.16:48720) 2024-09-21T03:56:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.439+0000 7fac25a00640 1 -- 172.21.15.16:0/2468343513 learned_addr learned my addr 172.21.15.16:0/2468343513 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.439+0000 7fac25a00640 1 -- 172.21.15.16:0/2468343513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2010a3c0 msgr2=0x7fac201a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.439+0000 7fac25a00640 1 --2- 172.21.15.16:0/2468343513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2010a3c0 0x7fac201a8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.439+0000 7fac25a00640 1 -- 172.21.15.16:0/2468343513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2010cdb0 msgr2=0x7fac201a9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.439+0000 7fac25a00640 1 --2- 172.21.15.16:0/2468343513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2010cdb0 0x7fac201a9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.439+0000 7fac25a00640 1 -- 172.21.15.16:0/2468343513 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac201aa760 con 0x7fac2010f9d0 2024-09-21T03:56:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.440+0000 7fac25a00640 1 --2- 172.21.15.16:0/2468343513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac2010f9d0 0x7fac201a9f40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fac10004650 tx=0x7fac10003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.440+0000 7fac1d600640 1 -- 172.21.15.16:0/2468343513 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1003f070 con 0x7fac2010f9d0 2024-09-21T03:56:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.440+0000 7fac1d600640 1 -- 172.21.15.16:0/2468343513 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac10031c50 con 0x7fac2010f9d0 2024-09-21T03:56:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.440+0000 7fac27a00640 1 -- 172.21.15.16:0/2468343513 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac201aa9f0 con 0x7fac2010f9d0 2024-09-21T03:56:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.440+0000 7fac1d600640 1 -- 172.21.15.16:0/2468343513 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10035030 con 0x7fac2010f9d0 2024-09-21T03:56:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.440+0000 7fac27a00640 1 -- 172.21.15.16:0/2468343513 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac201b1140 con 0x7fac2010f9d0 2024-09-21T03:56:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.443+0000 7fac2ea44740 1 Processor -- start 2024-09-21T03:56:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.446+0000 7fac1d600640 1 -- 172.21.15.16:0/2468343513 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fac10042070 con 0x7fac2010f9d0 2024-09-21T03:56:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.446+0000 7fac2ea44740 1 -- start start 2024-09-21T03:56:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.446+0000 7fac2ea44740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d51171b0 0x55a2d54be0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.446+0000 7fac2ea44740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d50ee2f0 0x55a2d54be740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.446+0000 7fac2ea44740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 0x55a2d54bf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.446+0000 7fac2ea44740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a2d54316b0 con 0x55a2d50ee2f0 2024-09-21T03:56:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.446+0000 7fac2ea44740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a2d5420ae0 con 0x55a2d51171b0 2024-09-21T03:56:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.446+0000 7fac2ea44740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a2d54248f0 con 0x55a2d50f64d0 2024-09-21T03:56:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.446+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d51171b0 0x55a2d54be0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.446+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d51171b0 0x55a2d54be0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:46316/0 (socket says 172.21.15.16:46316) 2024-09-21T03:56:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.446+0000 7fac1fe00640 1 -- 172.21.15.16:0/813091191 learned_addr learned my addr 172.21.15.16:0/813091191 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.447+0000 7fac1fe00640 1 --2- 172.21.15.16:0/813091191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 0x55a2d54bf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.447+0000 7fac25000640 1 --2- 172.21.15.16:0/813091191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d50ee2f0 0x55a2d54be740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.447+0000 7fac1fe00640 1 -- 172.21.15.16:0/813091191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 msgr2=0x55a2d54bf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.447+0000 7fac1fe00640 1 --2- 172.21.15.16:0/813091191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 0x55a2d54bf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.447+0000 7fac1fe00640 1 -- 172.21.15.16:0/813091191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d50ee2f0 msgr2=0x55a2d54be740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.447+0000 7fac1fe00640 1 --2- 172.21.15.16:0/813091191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d50ee2f0 0x55a2d54be740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.447+0000 7fac1fe00640 1 -- 172.21.15.16:0/813091191 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d53ed780 con 0x55a2d51171b0 2024-09-21T03:56:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.447+0000 7fac1fe00640 1 --2- 172.21.15.16:0/813091191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d51171b0 0x55a2d54be0d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fac0c012820 tx=0x7fac0c00b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d36db83ec302e893 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.447+0000 7fac16000640 1 -- 172.21.15.16:0/813091191 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c012da0 con 0x55a2d51171b0 2024-09-21T03:56:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.448+0000 7fac16000640 1 -- 172.21.15.16:0/813091191 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac0c010c20 con 0x55a2d51171b0 2024-09-21T03:56:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.448+0000 7fac16000640 1 -- 172.21.15.16:0/813091191 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c01eb80 con 0x55a2d51171b0 2024-09-21T03:56:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.448+0000 7fac1d600640 1 --2- 172.21.15.16:0/2468343513 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fac040b65d0 0x7fac040b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.448+0000 7fac1d600640 1 -- 172.21.15.16:0/2468343513 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fac10107f70 con 0x7fac2010f9d0 2024-09-21T03:56:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.448+0000 7fac25000640 1 --2- 172.21.15.16:0/2468343513 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fac040b65d0 0x7fac040b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.449+0000 7fac2ea44740 1 -- 172.21.15.16:0/813091191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d51171b0 msgr2=0x55a2d54be0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.449+0000 7fac2ea44740 1 --2- 172.21.15.16:0/813091191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d51171b0 0x55a2d54be0d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fac0c012820 tx=0x7fac0c00b7e0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.449+0000 7fac25000640 1 --2- 172.21.15.16:0/2468343513 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fac040b65d0 0x7fac040b8ae0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fac18005fd0 tx=0x7fac18009500 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.449+0000 7fac2ea44740 1 -- 172.21.15.16:0/813091191 shutdown_connections 2024-09-21T03:56:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.449+0000 7fac2ea44740 1 --2- 172.21.15.16:0/813091191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 0x55a2d54bf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.449+0000 7fac2ea44740 1 --2- 172.21.15.16:0/813091191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d50ee2f0 0x55a2d54be740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.449+0000 7fac2ea44740 1 --2- 172.21.15.16:0/813091191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d51171b0 0x55a2d54be0d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.449+0000 7fac2ea44740 1 -- 172.21.15.16:0/813091191 >> 172.21.15.16:0/813091191 conn(0x55a2d510b300 msgr2=0x55a2d54b9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.449+0000 7fac2ea44740 1 -- 172.21.15.16:0/813091191 shutdown_connections 2024-09-21T03:56:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.449+0000 7fac2ea44740 1 -- 172.21.15.16:0/813091191 wait complete. 2024-09-21T03:56:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.450+0000 7fac2ea44740 1 -- start start 2024-09-21T03:56:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.450+0000 7fac2ea44740 1 Processor -- start 2024-09-21T03:56:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.451+0000 7fac2ea44740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d50ee2f0 0x55a2d54c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.451+0000 7fac2ea44740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 0x55a2d54c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.451+0000 7fac2ea44740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d51171b0 0x55a2d54c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.451+0000 7fac2ea44740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a2d5358ab0 con 0x55a2d51171b0 2024-09-21T03:56:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.451+0000 7fac2ea44740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a2d53c7570 con 0x55a2d50ee2f0 2024-09-21T03:56:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.451+0000 7fac2ea44740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a2d54bfdb0 con 0x55a2d50f64d0 2024-09-21T03:56:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.451+0000 7fac2ea44740 10 client.0 _ll_register_callbacks cb 0x55a2d53d6100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.452+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 0x55a2d54c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.452+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 0x55a2d54c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:48750/0 (socket says 172.21.15.16:48750) 2024-09-21T03:56:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.452+0000 7fac1fe00640 1 -- 172.21.15.16:0/1752059694 learned_addr learned my addr 172.21.15.16:0/1752059694 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.452+0000 7fac25a00640 1 --2- 172.21.15.16:0/1752059694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d51171b0 0x55a2d54c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.452+0000 7fac25000640 1 --2- 172.21.15.16:0/1752059694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d50ee2f0 0x55a2d54c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.452+0000 7fac1fe00640 1 -- 172.21.15.16:0/1752059694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d50ee2f0 msgr2=0x55a2d54c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.452+0000 7fac1fe00640 1 --2- 172.21.15.16:0/1752059694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d50ee2f0 0x55a2d54c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.452+0000 7fac1fe00640 1 -- 172.21.15.16:0/1752059694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d51171b0 msgr2=0x55a2d54c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.452+0000 7fac1fe00640 1 --2- 172.21.15.16:0/1752059694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d51171b0 0x55a2d54c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.453+0000 7fac1fe00640 1 -- 172.21.15.16:0/1752059694 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d5182790 con 0x55a2d50f64d0 2024-09-21T03:56:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.453+0000 7fac1fe00640 1 --2- 172.21.15.16:0/1752059694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 0x55a2d54c7e80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0058c0 tx=0x7fac0c00b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c852ff2c60a48af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.453+0000 7fac0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:56:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.453+0000 7fac0aa00640 1 -- 172.21.15.16:0/1752059694 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c004030 con 0x55a2d50f64d0 2024-09-21T03:56:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.454+0000 7fac0aa00640 1 -- 172.21.15.16:0/1752059694 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fac0c0041d0 con 0x55a2d50f64d0 2024-09-21T03:56:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.454+0000 7fac2ea44740 1 -- 172.21.15.16:0/1752059694 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac0c010c20 con 0x55a2d50f64d0 2024-09-21T03:56:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.454+0000 7fac0aa00640 1 -- 172.21.15.16:0/1752059694 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fac0c0043d0 con 0x55a2d50f64d0 2024-09-21T03:56:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.454+0000 7fac2ea44740 10 client.26575 fetch_fsmap learned FSMap version 30 2024-09-21T03:56:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.454+0000 7fac2ea44740 1 -- 172.21.15.16:0/1752059694 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a2d51fb620 con 0x55a2d50f64d0 2024-09-21T03:56:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.455+0000 7fac0aa00640 1 -- 172.21.15.16:0/1752059694 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac0c010c20 con 0x55a2d50f64d0 2024-09-21T03:56:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.455+0000 7fac0aa00640 1 -- 172.21.15.16:0/1752059694 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fac0c032020 con 0x55a2d50f64d0 2024-09-21T03:56:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.455+0000 7fac2ea44740 10 client.26575 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:56:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.455+0000 7fac2ea44740 10 client.26575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.455+0000 7fac2ea44740 20 client.26575 populate_metadata read hostname 'smithi016' 2024-09-21T03:56:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.455+0000 7fac2ea44740 1 --2- 172.21.15.16:0/1752059694 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a2d50d5d80 0x55a2d54d4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.455+0000 7fac2ea44740 4 client.26575 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.455+0000 7fac2ea44740 1 -- 172.21.15.16:0/1752059694 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2d5433600 con 0x55a2d50d5d80 2024-09-21T03:56:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.456+0000 7fac25000640 1 --2- 172.21.15.16:0/1752059694 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a2d50d5d80 0x55a2d54d4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.457+0000 7fac25000640 1 --2- 172.21.15.16:0/1752059694 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a2d50d5d80 0x55a2d54d4d40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ed2ecb8181b9a93 server_cookie=e8c86d11d9c3bfd in_seq=0 out_seq=0 2024-09-21T03:56:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.457+0000 7fac0aa00640 10 client.26575 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.457+0000 7fac0aa00640 1 -- 172.21.15.16:0/1752059694 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a2d5433600 con 0x55a2d50d5d80 2024-09-21T03:56:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.457+0000 7fac0aa00640 10 client.26575 handle_command_reply: tid=0 2024-09-21T03:56:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.458+0000 7fac2ea44740 1 client.26575 shutdown 2024-09-21T03:56:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.458+0000 7fac2ea44740 1 -- 172.21.15.16:0/1752059694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 msgr2=0x55a2d54c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.458+0000 7fac2ea44740 1 --2- 172.21.15.16:0/1752059694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 0x55a2d54c7e80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0058c0 tx=0x7fac0c00b410 comp rx=0 tx=0).stop 2024-09-21T03:56:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.458+0000 7fac2ea44740 1 -- 172.21.15.16:0/1752059694 shutdown_connections 2024-09-21T03:56:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.458+0000 7fac2ea44740 1 --2- 172.21.15.16:0/1752059694 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a2d50d5d80 0x55a2d54d4d40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.458+0000 7fac2ea44740 1 --2- 172.21.15.16:0/1752059694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d51171b0 0x55a2d54c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.458+0000 7fac2ea44740 1 --2- 172.21.15.16:0/1752059694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 0x55a2d54c7e80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.458+0000 7fac2ea44740 1 --2- 172.21.15.16:0/1752059694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d50ee2f0 0x55a2d54c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.458+0000 7fac2ea44740 1 -- 172.21.15.16:0/1752059694 >> 172.21.15.16:0/1752059694 conn(0x55a2d510b300 msgr2=0x55a2d54be630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.458+0000 7fac0aa00640 0 client.26575 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.458+0000 7fac2ea44740 1 -- 172.21.15.16:0/1752059694 shutdown_connections 2024-09-21T03:56:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.459+0000 7fac2ea44740 1 -- 172.21.15.16:0/1752059694 wait complete. 2024-09-21T03:56:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.459+0000 7fac2ea44740 20 client.26575 trim_cache size 0 max 16384 2024-09-21T03:56:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.479+0000 7fac2ea44740 1 Processor -- start 2024-09-21T03:56:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.480+0000 7fac2ea44740 1 -- start start 2024-09-21T03:56:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.480+0000 7fac2ea44740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d50ee2f0 0x55a2d54be590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.481+0000 7fac2ea44740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 0x55a2d54cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.481+0000 7fac2ea44740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d51171b0 0x55a2d54b9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.481+0000 7fac2ea44740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac201bb4f0 con 0x55a2d51171b0 2024-09-21T03:56:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.481+0000 7fac2ea44740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac201bd030 con 0x55a2d50ee2f0 2024-09-21T03:56:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.481+0000 7fac2ea44740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac201bbbc0 con 0x55a2d50f64d0 2024-09-21T03:56:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.481+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 0x55a2d54cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.482+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 0x55a2d54cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:48760/0 (socket says 172.21.15.16:48760) 2024-09-21T03:56:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.482+0000 7fac1fe00640 1 -- 172.21.15.16:0/2744790734 learned_addr learned my addr 172.21.15.16:0/2744790734 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.482+0000 7fac25000640 1 --2- 172.21.15.16:0/2744790734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d50ee2f0 0x55a2d54be590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.482+0000 7fac25000640 1 -- 172.21.15.16:0/2744790734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 msgr2=0x55a2d54cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.482+0000 7fac25000640 1 --2- 172.21.15.16:0/2744790734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 0x55a2d54cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.482+0000 7fac25000640 1 -- 172.21.15.16:0/2744790734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d51171b0 msgr2=0x55a2d54b9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.482+0000 7fac25000640 1 --2- 172.21.15.16:0/2744790734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d51171b0 0x55a2d54b9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.482+0000 7fac25000640 1 -- 172.21.15.16:0/2744790734 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d54c7540 con 0x55a2d50ee2f0 2024-09-21T03:56:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.483+0000 7fac25000640 1 --2- 172.21.15.16:0/2744790734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d50ee2f0 0x55a2d54be590 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fac0c012630 tx=0x7fac1801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=967968f49fcfda11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.485+0000 7fac15600640 1 -- 172.21.15.16:0/2744790734 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18004b20 con 0x55a2d50ee2f0 2024-09-21T03:56:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.485+0000 7fac15600640 1 -- 172.21.15.16:0/2744790734 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac18003e20 con 0x55a2d50ee2f0 2024-09-21T03:56:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.485+0000 7fac15600640 1 -- 172.21.15.16:0/2744790734 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac180047b0 con 0x55a2d50ee2f0 2024-09-21T03:56:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.486+0000 7fac2ea44740 1 -- 172.21.15.16:0/2744790734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d50ee2f0 msgr2=0x55a2d54be590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.487+0000 7fac2ea44740 1 --2- 172.21.15.16:0/2744790734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d50ee2f0 0x55a2d54be590 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fac0c012630 tx=0x7fac1801a000 comp rx=0 tx=0).stop 2024-09-21T03:56:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.487+0000 7fac2ea44740 1 -- 172.21.15.16:0/2744790734 shutdown_connections 2024-09-21T03:56:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.487+0000 7fac2ea44740 1 --2- 172.21.15.16:0/2744790734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d51171b0 0x55a2d54b9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.487+0000 7fac2ea44740 1 --2- 172.21.15.16:0/2744790734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50f64d0 0x55a2d54cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.487+0000 7fac2ea44740 1 --2- 172.21.15.16:0/2744790734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d50ee2f0 0x55a2d54be590 secure :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fac0c012630 tx=0x7fac1801a000 comp rx=0 tx=0).stop 2024-09-21T03:56:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.487+0000 7fac2ea44740 1 -- 172.21.15.16:0/2744790734 >> 172.21.15.16:0/2744790734 conn(0x55a2d510b300 msgr2=0x55a2d54bde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.487+0000 7fac2ea44740 1 -- 172.21.15.16:0/2744790734 shutdown_connections 2024-09-21T03:56:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.488+0000 7fac2ea44740 1 -- 172.21.15.16:0/2744790734 wait complete. 2024-09-21T03:56:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.488+0000 7fac2ea44740 1 -- start start 2024-09-21T03:56:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.489+0000 7fac2ea44740 1 Processor -- start 2024-09-21T03:56:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.493+0000 7fac2ea44740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d50f64d0 0x55a2d54cad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.493+0000 7fac2ea44740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d51171b0 0x55a2d54cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.493+0000 7fac2ea44740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50ab1c0 0x55a2d54cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.493+0000 7fac2ea44740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac201bc960 con 0x55a2d50f64d0 2024-09-21T03:56:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.493+0000 7fac2ea44740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac201bc290 con 0x55a2d51171b0 2024-09-21T03:56:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.493+0000 7fac2ea44740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac201b94d0 con 0x55a2d50ab1c0 2024-09-21T03:56:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.493+0000 7fac2ea44740 10 client.0 _ll_register_callbacks cb 0x7fac0c00bef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.495+0000 7fac25000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d50f64d0 0x55a2d54cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.495+0000 7fac25000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d50f64d0 0x55a2d54cad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60140/0 (socket says 172.21.15.16:60140) 2024-09-21T03:56:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.495+0000 7fac25000640 1 -- 172.21.15.16:0/3513694824 learned_addr learned my addr 172.21.15.16:0/3513694824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.496+0000 7fac1fe00640 1 --2- 172.21.15.16:0/3513694824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d51171b0 0x55a2d54cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.496+0000 7fac25a00640 1 --2- 172.21.15.16:0/3513694824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50ab1c0 0x55a2d54cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.497+0000 7fac25a00640 1 -- 172.21.15.16:0/3513694824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d51171b0 msgr2=0x55a2d54cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.497+0000 7fac25a00640 1 --2- 172.21.15.16:0/3513694824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d51171b0 0x55a2d54cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.497+0000 7fac25a00640 1 -- 172.21.15.16:0/3513694824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d50f64d0 msgr2=0x55a2d54cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.497+0000 7fac25a00640 1 --2- 172.21.15.16:0/3513694824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d50f64d0 0x55a2d54cad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.498+0000 7fac25a00640 1 -- 172.21.15.16:0/3513694824 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d54cf720 con 0x55a2d50ab1c0 2024-09-21T03:56:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.498+0000 7fac25a00640 1 --2- 172.21.15.16:0/3513694824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50ab1c0 0x55a2d54cefc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fac100cdd30 tx=0x7fac100092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=420f0f1f4d188651 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.498+0000 7fac0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:56:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.499+0000 7fac0aa00640 1 -- 172.21.15.16:0/3513694824 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10049030 con 0x55a2d50ab1c0 2024-09-21T03:56:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.499+0000 7fac0aa00640 1 -- 172.21.15.16:0/3513694824 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fac100d7280 con 0x55a2d50ab1c0 2024-09-21T03:56:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.499+0000 7fac2ea44740 1 -- 172.21.15.16:0/3513694824 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac18003e20 con 0x55a2d50ab1c0 2024-09-21T03:56:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.501+0000 7fac0aa00640 1 -- 172.21.15.16:0/3513694824 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fac100cd8b0 con 0x55a2d50ab1c0 2024-09-21T03:56:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.501+0000 7fac2ea44740 10 client.26581 fetch_fsmap learned FSMap version 30 2024-09-21T03:56:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.501+0000 7fac2ea44740 1 -- 172.21.15.16:0/3513694824 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a2d54cca40 con 0x55a2d50ab1c0 2024-09-21T03:56:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.502+0000 7fac0aa00640 1 -- 172.21.15.16:0/3513694824 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac18003e20 con 0x55a2d50ab1c0 2024-09-21T03:56:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.502+0000 7fac0aa00640 1 -- 172.21.15.16:0/3513694824 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fac100f5080 con 0x55a2d50ab1c0 2024-09-21T03:56:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.503+0000 7fac2ea44740 10 client.26581 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:56:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.503+0000 7fac2ea44740 10 client.26581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.503+0000 7fac2ea44740 20 client.26581 populate_metadata read hostname 'smithi016' 2024-09-21T03:56:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.503+0000 7fac2ea44740 1 --2- 172.21.15.16:0/3513694824 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a2d553fbe0 0x55a2d5542000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.503+0000 7fac2ea44740 4 client.26581 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:56:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.503+0000 7fac2ea44740 1 -- 172.21.15.16:0/3513694824 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a2d542f0b0 con 0x55a2d553fbe0 2024-09-21T03:56:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.504+0000 7fac25000640 1 --2- 172.21.15.16:0/3513694824 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a2d553fbe0 0x55a2d5542000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.505+0000 7fac25000640 1 --2- 172.21.15.16:0/3513694824 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a2d553fbe0 0x55a2d5542000 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed25b184d6c024d6 server_cookie=7a73cf3d28882621 in_seq=0 out_seq=0 2024-09-21T03:56:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.505+0000 7fac0aa00640 10 client.26581 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.521+0000 7fac0aa00640 1 -- 172.21.15.16:0/3513694824 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a2d542f0b0 con 0x55a2d553fbe0 2024-09-21T03:56:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.521+0000 7fac0aa00640 10 client.26581 handle_command_reply: tid=0 2024-09-21T03:56:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.521+0000 7fac2ea44740 1 client.26581 shutdown 2024-09-21T03:56:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.524+0000 7fac2ea44740 1 -- 172.21.15.16:0/3513694824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50ab1c0 msgr2=0x55a2d54cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.524+0000 7fac2ea44740 1 --2- 172.21.15.16:0/3513694824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50ab1c0 0x55a2d54cefc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fac100cdd30 tx=0x7fac100092d0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.525+0000 7fac2ea44740 1 -- 172.21.15.16:0/3513694824 shutdown_connections 2024-09-21T03:56:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.525+0000 7fac2ea44740 1 --2- 172.21.15.16:0/3513694824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2d50ab1c0 0x55a2d54cefc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.525+0000 7fac2ea44740 1 --2- 172.21.15.16:0/3513694824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2d51171b0 0x55a2d54cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.525+0000 7fac2ea44740 1 --2- 172.21.15.16:0/3513694824 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a2d553fbe0 0x55a2d5542000 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.525+0000 7fac2ea44740 1 --2- 172.21.15.16:0/3513694824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2d50f64d0 0x55a2d54cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.525+0000 7fac2ea44740 1 -- 172.21.15.16:0/3513694824 >> 172.21.15.16:0/3513694824 conn(0x55a2d510b300 msgr2=0x55a2d54b80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.525+0000 7fac0aa00640 0 client.26581 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.525+0000 7fac2ea44740 1 -- 172.21.15.16:0/3513694824 shutdown_connections 2024-09-21T03:56:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.525+0000 7fac2ea44740 1 -- 172.21.15.16:0/3513694824 wait complete. 2024-09-21T03:56:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.526+0000 7fac2ea44740 20 client.26581 trim_cache size 0 max 16384 2024-09-21T03:56:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:56:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:56:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:56:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:56:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.527+0000 7fac27a00640 1 -- 172.21.15.16:0/2468343513 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fac040b65d0 msgr2=0x7fac040b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.527+0000 7fac27a00640 1 --2- 172.21.15.16:0/2468343513 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fac040b65d0 0x7fac040b8ae0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fac18005fd0 tx=0x7fac18009500 comp rx=0 tx=0).stop 2024-09-21T03:56:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.527+0000 7fac27a00640 1 -- 172.21.15.16:0/2468343513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac2010f9d0 msgr2=0x7fac201a9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.527+0000 7fac27a00640 1 --2- 172.21.15.16:0/2468343513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac2010f9d0 0x7fac201a9f40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fac10004650 tx=0x7fac10003b90 comp rx=0 tx=0).stop 2024-09-21T03:56:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.530+0000 7fac27a00640 1 -- 172.21.15.16:0/2468343513 shutdown_connections 2024-09-21T03:56:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.531+0000 7fac27a00640 1 --2- 172.21.15.16:0/2468343513 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fac040b65d0 0x7fac040b8ae0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.532+0000 7fac27a00640 1 --2- 172.21.15.16:0/2468343513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac2010f9d0 0x7fac201a9f40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.532+0000 7fac27a00640 1 --2- 172.21.15.16:0/2468343513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2010cdb0 0x7fac201a9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.532+0000 7fac27a00640 1 --2- 172.21.15.16:0/2468343513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac2010a3c0 0x7fac201a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.532+0000 7fac27a00640 1 -- 172.21.15.16:0/2468343513 >> 172.21.15.16:0/2468343513 conn(0x7fac20070810 msgr2=0x7fac2010b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.532+0000 7fac27a00640 1 -- 172.21.15.16:0/2468343513 shutdown_connections 2024-09-21T03:56:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:45.532+0000 7fac27a00640 1 -- 172.21.15.16:0/2468343513 wait complete. 2024-09-21T03:56:46.857 INFO:tasks.quiescer.fs.[cephfs]:Total duration (49.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T03:56:46.857 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-21T03:56:46.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:56:47.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:46 smithi123 ceph-mon[34673]: pgmap v732: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 249 MiB/s wr, 1.30k op/s 2024-09-21T03:56:47.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:46 smithi016 ceph-mon[30375]: pgmap v732: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 249 MiB/s wr, 1.30k op/s 2024-09-21T03:56:47.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:46 smithi134 ceph-mon[35031]: pgmap v732: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 249 MiB/s wr, 1.30k op/s 2024-09-21T03:56:47.530 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:49.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:48 smithi123 ceph-mon[34673]: pgmap v733: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 221 MiB/s wr, 1.15k op/s 2024-09-21T03:56:49.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:48 smithi016 ceph-mon[30375]: pgmap v733: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 221 MiB/s wr, 1.15k op/s 2024-09-21T03:56:49.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:48 smithi134 ceph-mon[35031]: pgmap v733: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 221 MiB/s wr, 1.15k op/s 2024-09-21T03:56:49.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.585+0000 7f8586600640 1 -- 172.21.15.16:0/1506184539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8580103b80 msgr2=0x7f8580103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:49.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.585+0000 7f8586600640 1 --2- 172.21.15.16:0/1506184539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8580103b80 0x7f8580103f90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8570009900 tx=0x7f857002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:56:49.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.586+0000 7f8586600640 1 -- 172.21.15.16:0/1506184539 shutdown_connections 2024-09-21T03:56:49.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.586+0000 7f8586600640 1 --2- 172.21.15.16:0/1506184539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8580109e10 0x7f8580110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:49.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.586+0000 7f8586600640 1 --2- 172.21.15.16:0/1506184539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85801094c0 0x7f85801098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:49.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.586+0000 7f8586600640 1 --2- 172.21.15.16:0/1506184539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8580103b80 0x7f8580103f90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:49.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.586+0000 7f8586600640 1 -- 172.21.15.16:0/1506184539 >> 172.21.15.16:0/1506184539 conn(0x7f85800ff590 msgr2=0x7f85801019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:49.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.586+0000 7f8586600640 1 -- 172.21.15.16:0/1506184539 shutdown_connections 2024-09-21T03:56:49.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.586+0000 7f8586600640 1 -- 172.21.15.16:0/1506184539 wait complete. 2024-09-21T03:56:49.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.587+0000 7f8586600640 1 Processor -- start 2024-09-21T03:56:49.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.587+0000 7f8586600640 1 -- start start 2024-09-21T03:56:49.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.588+0000 7f8586600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8580103b80 0x7f858019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:49.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.588+0000 7f8586600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85801094c0 0x7f858019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:49.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.588+0000 7f8586600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8580109e10 0x7f85801a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:49.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.588+0000 7f8586600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8580115d20 con 0x7f8580109e10 2024-09-21T03:56:49.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.588+0000 7f8586600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8580115ba0 con 0x7f8580103b80 2024-09-21T03:56:49.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.588+0000 7f8586600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8580115ea0 con 0x7f85801094c0 2024-09-21T03:56:49.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.589+0000 7f857ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85801094c0 0x7f858019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:49.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.589+0000 7f857fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8580109e10 0x7f85801a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:49.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.589+0000 7f857ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85801094c0 0x7f858019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:48790/0 (socket says 172.21.15.16:48790) 2024-09-21T03:56:49.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.589+0000 7f857ea00640 1 -- 172.21.15.16:0/1212499439 learned_addr learned my addr 172.21.15.16:0/1212499439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:49.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.589+0000 7f857f400640 1 --2- 172.21.15.16:0/1212499439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8580103b80 0x7f858019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:49.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.589+0000 7f857ea00640 1 -- 172.21.15.16:0/1212499439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8580103b80 msgr2=0x7f858019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:49.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.589+0000 7f857ea00640 1 --2- 172.21.15.16:0/1212499439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8580103b80 0x7f858019c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:49.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.589+0000 7f857ea00640 1 -- 172.21.15.16:0/1212499439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8580109e10 msgr2=0x7f85801a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:49.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.589+0000 7f857ea00640 1 --2- 172.21.15.16:0/1212499439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8580109e10 0x7f85801a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:49.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.589+0000 7f857ea00640 1 -- 172.21.15.16:0/1212499439 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85801a2600 con 0x7f85801094c0 2024-09-21T03:56:49.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.589+0000 7f857f400640 1 --2- 172.21.15.16:0/1212499439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8580103b80 0x7f858019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:49.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.590+0000 7f857ea00640 1 --2- 172.21.15.16:0/1212499439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85801094c0 0x7f858019ccf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f856c00c8c0 tx=0x7f856c00cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:49.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.590+0000 7f8577e00640 1 -- 172.21.15.16:0/1212499439 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c007bb0 con 0x7f85801094c0 2024-09-21T03:56:49.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.590+0000 7f8586600640 1 -- 172.21.15.16:0/1212499439 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85801a28f0 con 0x7f85801094c0 2024-09-21T03:56:49.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.590+0000 7f8577e00640 1 -- 172.21.15.16:0/1212499439 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f856c007d50 con 0x7f85801094c0 2024-09-21T03:56:49.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.590+0000 7f8577e00640 1 -- 172.21.15.16:0/1212499439 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c0187c0 con 0x7f85801094c0 2024-09-21T03:56:49.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.590+0000 7f8586600640 1 -- 172.21.15.16:0/1212499439 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85801a9110 con 0x7f85801094c0 2024-09-21T03:56:49.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.591+0000 7f8586600640 1 -- 172.21.15.16:0/1212499439 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85600052c0 con 0x7f85801094c0 2024-09-21T03:56:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.597+0000 7f8577e00640 1 -- 172.21.15.16:0/1212499439 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f856c018960 con 0x7f85801094c0 2024-09-21T03:56:49.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.599+0000 7f8577e00640 1 --2- 172.21.15.16:0/1212499439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f85640b6640 0x7f85640b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:49.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.600+0000 7f8577e00640 1 -- 172.21.15.16:0/1212499439 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f856c0e7b40 con 0x7f85801094c0 2024-09-21T03:56:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.601+0000 7f8577e00640 1 -- 172.21.15.16:0/1212499439 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f856c014030 con 0x7f85801094c0 2024-09-21T03:56:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.602+0000 7f857f400640 1 --2- 172.21.15.16:0/1212499439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f85640b6640 0x7f85640b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:49.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:49.603+0000 7f857f400640 1 --2- 172.21.15.16:0/1212499439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f85640b6640 0x7f85640b8b50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8570000c00 tx=0x7f8570031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:50.075+0000 7f8586600640 1 -- 172.21.15.16:0/1212499439 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85600055b0 con 0x7f85801094c0 2024-09-21T03:56:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:50.082+0000 7f8577e00640 1 -- 172.21.15.16:0/1212499439 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f856c0ae040 con 0x7f85801094c0 2024-09-21T03:56:50.087 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:56:50.088 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:56:50.088 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:56:50.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:50.091+0000 7f8586600640 1 -- 172.21.15.16:0/1212499439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f85640b6640 msgr2=0x7f85640b8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:50.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:50.091+0000 7f8586600640 1 --2- 172.21.15.16:0/1212499439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f85640b6640 0x7f85640b8b50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8570000c00 tx=0x7f8570031400 comp rx=0 tx=0).stop 2024-09-21T03:56:50.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:50.091+0000 7f8586600640 1 -- 172.21.15.16:0/1212499439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85801094c0 msgr2=0x7f858019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:50.091+0000 7f8586600640 1 --2- 172.21.15.16:0/1212499439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85801094c0 0x7f858019ccf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f856c00c8c0 tx=0x7f856c00cd80 comp rx=0 tx=0).stop 2024-09-21T03:56:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:50.091+0000 7f8586600640 1 -- 172.21.15.16:0/1212499439 shutdown_connections 2024-09-21T03:56:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:50.091+0000 7f8586600640 1 --2- 172.21.15.16:0/1212499439 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f85640b6640 0x7f85640b8b50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:50.091+0000 7f8586600640 1 --2- 172.21.15.16:0/1212499439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8580109e10 0x7f85801a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:50.091+0000 7f8586600640 1 --2- 172.21.15.16:0/1212499439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85801094c0 0x7f858019ccf0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:50.091+0000 7f8586600640 1 --2- 172.21.15.16:0/1212499439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8580103b80 0x7f858019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:50.091+0000 7f8586600640 1 -- 172.21.15.16:0/1212499439 >> 172.21.15.16:0/1212499439 conn(0x7f85800ff590 msgr2=0x7f8580107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:50.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:50.094+0000 7f8586600640 1 -- 172.21.15.16:0/1212499439 shutdown_connections 2024-09-21T03:56:50.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:50.094+0000 7f8586600640 1 -- 172.21.15.16:0/1212499439 wait complete. 2024-09-21T03:56:51.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:50 smithi016 ceph-mon[30375]: pgmap v734: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 244 MiB/s wr, 1.27k op/s 2024-09-21T03:56:51.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:50 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1212499439' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:56:51.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:50 smithi134 ceph-mon[35031]: pgmap v734: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 244 MiB/s wr, 1.27k op/s 2024-09-21T03:56:51.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:50 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1212499439' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:56:51.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:50 smithi123 ceph-mon[34673]: pgmap v734: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 244 MiB/s wr, 1.27k op/s 2024-09-21T03:56:51.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:50 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1212499439' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:56:52.746 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-21T03:56:52.747 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-21T03:56:52.948 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:52 smithi016 ceph-mon[30375]: pgmap v735: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 218 MiB/s wr, 1.14k op/s 2024-09-21T03:56:52.948 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:52.948 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:56:52.948 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:56:52.948 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:56:52.948 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:52 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:56:53.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:52 smithi134 ceph-mon[35031]: pgmap v735: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 218 MiB/s wr, 1.14k op/s 2024-09-21T03:56:53.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:53.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:56:53.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:56:53.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:56:53.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:52 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:56:53.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:52 smithi123 ceph-mon[34673]: pgmap v735: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 218 MiB/s wr, 1.14k op/s 2024-09-21T03:56:53.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:53.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:56:53.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:56:53.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:56:53.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:52 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:56:53.614 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:54.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:54.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:54.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:53 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:56:54.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:54.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:54.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:53 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:56:54.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:54.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:54.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:53 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' 2024-09-21T03:56:55.307 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:54 smithi123 ceph-mon[34673]: pgmap v736: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 215 MiB/s wr, 1.13k op/s 2024-09-21T03:56:55.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:54 smithi134 ceph-mon[35031]: pgmap v736: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 215 MiB/s wr, 1.13k op/s 2024-09-21T03:56:55.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:54 smithi016 ceph-mon[30375]: pgmap v736: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 215 MiB/s wr, 1.13k op/s 2024-09-21T03:56:56.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.139+0000 7f98f7e00640 1 -- 172.21.15.16:0/1589439223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98f8109c50 msgr2=0x7f98f8110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:56.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.139+0000 7f98f7e00640 1 --2- 172.21.15.16:0/1589439223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98f8109c50 0x7f98f8110480 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009ca0 tx=0x7f98f002a8e0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.144+0000 7f98f7e00640 1 -- 172.21.15.16:0/1589439223 shutdown_connections 2024-09-21T03:56:56.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.144+0000 7f98f7e00640 1 --2- 172.21.15.16:0/1589439223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98f8109c50 0x7f98f8110480 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.144+0000 7f98f7e00640 1 --2- 172.21.15.16:0/1589439223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f8109300 0x7f98f8109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.144+0000 7f98f7e00640 1 --2- 172.21.15.16:0/1589439223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98f81039c0 0x7f98f8103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.144+0000 7f98f7e00640 1 -- 172.21.15.16:0/1589439223 >> 172.21.15.16:0/1589439223 conn(0x7f98f80ff430 msgr2=0x7f98f8101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:56.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.144+0000 7f98f7e00640 1 -- 172.21.15.16:0/1589439223 shutdown_connections 2024-09-21T03:56:56.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.145+0000 7f98f7e00640 1 -- 172.21.15.16:0/1589439223 wait complete. 2024-09-21T03:56:56.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.147+0000 7f98f7e00640 1 Processor -- start 2024-09-21T03:56:56.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.147+0000 7f98f7e00640 1 -- start start 2024-09-21T03:56:56.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.154+0000 7f98f7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98f81039c0 0x7f98f810ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.154+0000 7f98f7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98f8109300 0x7f98f810f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.154+0000 7f98f7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f8109c50 0x7f98f81ac660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.155+0000 7f98f6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98f8109300 0x7f98f810f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:56.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.155+0000 7f98f7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f8109c50 0x7f98f81ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:56.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.156+0000 7f98f7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f8109c50 0x7f98f81ac660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47302/0 (socket says 172.21.15.16:47302) 2024-09-21T03:56:56.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.156+0000 7f98f6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98f8109300 0x7f98f810f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:49224/0 (socket says 172.21.15.16:49224) 2024-09-21T03:56:56.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.160+0000 7f98f7400640 1 -- 172.21.15.16:0/3491515836 learned_addr learned my addr 172.21.15.16:0/3491515836 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:56.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.160+0000 7f98f6a00640 1 --2- 172.21.15.16:0/3491515836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98f81039c0 0x7f98f810ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:56.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.160+0000 7f98f7e00640 1 -- 172.21.15.16:0/3491515836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98f8111720 con 0x7f98f8109c50 2024-09-21T03:56:56.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.160+0000 7f98f7e00640 1 -- 172.21.15.16:0/3491515836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98f81115a0 con 0x7f98f81039c0 2024-09-21T03:56:56.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.160+0000 7f98f7e00640 1 -- 172.21.15.16:0/3491515836 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98f81118a0 con 0x7f98f8109300 2024-09-21T03:56:56.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.161+0000 7f98f6000640 1 -- 172.21.15.16:0/3491515836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98f81039c0 msgr2=0x7f98f810ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:56.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.161+0000 7f98f6000640 1 --2- 172.21.15.16:0/3491515836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98f81039c0 0x7f98f810ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.161+0000 7f98f6000640 1 -- 172.21.15.16:0/3491515836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f8109c50 msgr2=0x7f98f81ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:56.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.161+0000 7f98f6000640 1 --2- 172.21.15.16:0/3491515836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f8109c50 0x7f98f81ac660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.161+0000 7f98f6000640 1 -- 172.21.15.16:0/3491515836 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f810f8b0 con 0x7f98f8109300 2024-09-21T03:56:56.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.162+0000 7f98f6000640 1 --2- 172.21.15.16:0/3491515836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98f8109300 0x7f98f810f010 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f98e4005fd0 tx=0x7f98e400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:56.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.162+0000 7f98fca00640 1 -- 172.21.15.16:0/3491515836 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4009c00 con 0x7f98f8109300 2024-09-21T03:56:56.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.164+0000 7f98fca00640 1 -- 172.21.15.16:0/3491515836 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98e4009da0 con 0x7f98f8109300 2024-09-21T03:56:56.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.164+0000 7f98fca00640 1 -- 172.21.15.16:0/3491515836 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4003db0 con 0x7f98f8109300 2024-09-21T03:56:56.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.167+0000 7f98f7e00640 1 -- 172.21.15.16:0/3491515836 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f81acc60 con 0x7f98f8109300 2024-09-21T03:56:56.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.167+0000 7f98f7e00640 1 -- 172.21.15.16:0/3491515836 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f81ad120 con 0x7f98f8109300 2024-09-21T03:56:56.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.178+0000 7f98fca00640 1 -- 172.21.15.16:0/3491515836 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f98e4003f50 con 0x7f98f8109300 2024-09-21T03:56:56.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.181+0000 7f98fca00640 1 --2- 172.21.15.16:0/3491515836 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f98dc0b65d0 0x7f98dc0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.182+0000 7f98f6a00640 1 --2- 172.21.15.16:0/3491515836 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f98dc0b65d0 0x7f98dc0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:56.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.184+0000 7f98fca00640 1 -- 172.21.15.16:0/3491515836 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f98e40e70d0 con 0x7f98f8109300 2024-09-21T03:56:56.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.185+0000 7f98f6a00640 1 --2- 172.21.15.16:0/3491515836 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f98dc0b65d0 0x7f98dc0b8ae0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f98f8110520 tx=0x7f98e800b390 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:56.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.261+0000 7f9904a9e740 1 Processor -- start 2024-09-21T03:56:56.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.261+0000 7f9904a9e740 1 -- start start 2024-09-21T03:56:56.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.261+0000 7f9904a9e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f9f840 0x562847347510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.261+0000 7f9904a9e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f76980 0x562847347a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.261+0000 7f9904a9e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f7eb60 0x562847347f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.262+0000 7f9904a9e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5628472bcca0 con 0x562846f9f840 2024-09-21T03:56:56.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.262+0000 7f9904a9e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5628472ac0d0 con 0x562846f7eb60 2024-09-21T03:56:56.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.262+0000 7f9904a9e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5628472afee0 con 0x562846f76980 2024-09-21T03:56:56.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.263+0000 7f98f7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f7eb60 0x562847347f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:56.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.263+0000 7f98f7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f7eb60 0x562847347f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:60042/0 (socket says 172.21.15.16:60042) 2024-09-21T03:56:56.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.263+0000 7f98f7400640 1 -- 172.21.15.16:0/1256616556 learned_addr learned my addr 172.21.15.16:0/1256616556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:56.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.263+0000 7f98f6000640 1 --2- 172.21.15.16:0/1256616556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f76980 0x562847347a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:56.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.264+0000 7f98f7400640 1 -- 172.21.15.16:0/1256616556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f76980 msgr2=0x562847347a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:56.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.265+0000 7f98f6a00640 1 --2- 172.21.15.16:0/1256616556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f9f840 0x562847347510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:56.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.265+0000 7f98f7400640 1 --2- 172.21.15.16:0/1256616556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f76980 0x562847347a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.265+0000 7f98f7400640 1 -- 172.21.15.16:0/1256616556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f9f840 msgr2=0x562847347510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:56.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.265+0000 7f98f7400640 1 --2- 172.21.15.16:0/1256616556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f9f840 0x562847347510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.265+0000 7f98f7400640 1 -- 172.21.15.16:0/1256616556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56284725e400 con 0x562846f7eb60 2024-09-21T03:56:56.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.265+0000 7f98f6000640 1 --2- 172.21.15.16:0/1256616556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f76980 0x562847347a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:56.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.266+0000 7f98f6a00640 1 --2- 172.21.15.16:0/1256616556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f9f840 0x562847347510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:56.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.267+0000 7f98f7400640 1 --2- 172.21.15.16:0/1256616556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f7eb60 0x562847347f90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009a50 tx=0x7f98f0031470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2397fe1610f55c2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:56.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.270+0000 7f98e3e00640 1 -- 172.21.15.16:0/1256616556 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f0003970 con 0x562846f7eb60 2024-09-21T03:56:56.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.272+0000 7f98e3e00640 1 -- 172.21.15.16:0/1256616556 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98f0036040 con 0x562846f7eb60 2024-09-21T03:56:56.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.272+0000 7f98e3e00640 1 -- 172.21.15.16:0/1256616556 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f0004460 con 0x562846f7eb60 2024-09-21T03:56:56.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.273+0000 7f9904a9e740 1 -- 172.21.15.16:0/1256616556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f7eb60 msgr2=0x562847347f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:56.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.273+0000 7f9904a9e740 1 --2- 172.21.15.16:0/1256616556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f7eb60 0x562847347f90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009a50 tx=0x7f98f0031470 comp rx=0 tx=0).stop 2024-09-21T03:56:56.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.274+0000 7f9904a9e740 1 -- 172.21.15.16:0/1256616556 shutdown_connections 2024-09-21T03:56:56.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.274+0000 7f9904a9e740 1 --2- 172.21.15.16:0/1256616556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f7eb60 0x562847347f90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.274+0000 7f9904a9e740 1 --2- 172.21.15.16:0/1256616556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f76980 0x562847347a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.274+0000 7f9904a9e740 1 --2- 172.21.15.16:0/1256616556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f9f840 0x562847347510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.274+0000 7f9904a9e740 1 -- 172.21.15.16:0/1256616556 >> 172.21.15.16:0/1256616556 conn(0x562846f93990 msgr2=0x562847345400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:56.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.274+0000 7f9904a9e740 1 -- 172.21.15.16:0/1256616556 shutdown_connections 2024-09-21T03:56:56.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.274+0000 7f9904a9e740 1 -- 172.21.15.16:0/1256616556 wait complete. 2024-09-21T03:56:56.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.276+0000 7f9904a9e740 1 -- start start 2024-09-21T03:56:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.276+0000 7f9904a9e740 1 Processor -- start 2024-09-21T03:56:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.277+0000 7f9904a9e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f76980 0x562847352b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.277+0000 7f9904a9e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f7eb60 0x5628473530a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.277+0000 7f9904a9e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f9f840 0x56284735ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.279+0000 7f9904a9e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56284710d7a0 con 0x562846f76980 2024-09-21T03:56:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.279+0000 7f9904a9e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562847251a10 con 0x562846f7eb60 2024-09-21T03:56:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.279+0000 7f9904a9e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562847355c90 con 0x562846f9f840 2024-09-21T03:56:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.279+0000 7f98f7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f9f840 0x56284735ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.279+0000 7f98f7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f9f840 0x56284735ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:49256/0 (socket says 172.21.15.16:49256) 2024-09-21T03:56:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.279+0000 7f98f7400640 1 -- 172.21.15.16:0/200829462 learned_addr learned my addr 172.21.15.16:0/200829462 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.279+0000 7f98f6000640 1 --2- 172.21.15.16:0/200829462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f7eb60 0x5628473530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.280+0000 7f98f7400640 1 -- 172.21.15.16:0/200829462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f7eb60 msgr2=0x5628473530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.280+0000 7f98f7400640 1 --2- 172.21.15.16:0/200829462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f7eb60 0x5628473530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.280+0000 7f98f7400640 1 -- 172.21.15.16:0/200829462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f76980 msgr2=0x562847352b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.280+0000 7f98f7400640 1 --2- 172.21.15.16:0/200829462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f76980 0x562847352b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.279+0000 7f9904a9e740 10 client.0 _ll_register_callbacks cb 0x562847261be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.281+0000 7f98f7400640 1 -- 172.21.15.16:0/200829462 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562847019b90 con 0x562846f9f840 2024-09-21T03:56:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.281+0000 7f9904a9e740 1 -- 172.21.15.16:0/200829462 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98f0036040 con 0x562846f9f840 2024-09-21T03:56:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.281+0000 7f98f7400640 1 --2- 172.21.15.16:0/200829462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f9f840 0x56284735ef80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f98f00094b0 tx=0x7f98f0031610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d733f81bcd0a3e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.282+0000 7f98e0c00640 10 client.26596 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:56:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.286+0000 7f98e0c00640 1 -- 172.21.15.16:0/200829462 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f0049040 con 0x562846f9f840 2024-09-21T03:56:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.286+0000 7f98e0c00640 1 -- 172.21.15.16:0/200829462 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f98f0031b20 con 0x562846f9f840 2024-09-21T03:56:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.286+0000 7f98e0c00640 1 -- 172.21.15.16:0/200829462 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f98f0003970 con 0x562846f9f840 2024-09-21T03:56:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.287+0000 7f9904a9e740 10 client.26596 fetch_fsmap learned FSMap version 30 2024-09-21T03:56:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.287+0000 7f9904a9e740 1 -- 172.21.15.16:0/200829462 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5628473617d0 con 0x562846f9f840 2024-09-21T03:56:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.288+0000 7f98e0c00640 1 -- 172.21.15.16:0/200829462 <== mon.1 v2:172.21.15.134:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f98f004c070 con 0x562846f9f840 2024-09-21T03:56:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.288+0000 7f9904a9e740 10 client.26596 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:56:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.288+0000 7f9904a9e740 10 client.26596 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:56:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.288+0000 7f9904a9e740 20 client.26596 populate_metadata read hostname 'smithi016' 2024-09-21T03:56:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.288+0000 7f9904a9e740 1 --2- 172.21.15.16:0/200829462 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562846f5e1c0 0x562847364b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.288+0000 7f9904a9e740 4 client.26596 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.288+0000 7f9904a9e740 1 -- 172.21.15.16:0/200829462 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628472bebf0 con 0x562846f5e1c0 2024-09-21T03:56:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.289+0000 7f98f6a00640 1 --2- 172.21.15.16:0/200829462 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562846f5e1c0 0x562847364b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.290+0000 7f98f6a00640 1 --2- 172.21.15.16:0/200829462 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562846f5e1c0 0x562847364b30 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cbc6e7b9b027782 server_cookie=5eb03f9b0f805d2d in_seq=0 out_seq=0 2024-09-21T03:56:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.290+0000 7f98e0c00640 10 client.26596 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.291+0000 7f98e0c00640 1 -- 172.21.15.16:0/200829462 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5628472bebf0 con 0x562846f5e1c0 2024-09-21T03:56:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.291+0000 7f98e0c00640 10 client.26596 handle_command_reply: tid=0 2024-09-21T03:56:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.291+0000 7f9904a9e740 1 client.26596 shutdown 2024-09-21T03:56:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.291+0000 7f9904a9e740 1 -- 172.21.15.16:0/200829462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f9f840 msgr2=0x56284735ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.291+0000 7f9904a9e740 1 --2- 172.21.15.16:0/200829462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f9f840 0x56284735ef80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f98f00094b0 tx=0x7f98f0031610 comp rx=0 tx=0).stop 2024-09-21T03:56:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.292+0000 7f9904a9e740 1 -- 172.21.15.16:0/200829462 shutdown_connections 2024-09-21T03:56:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.292+0000 7f9904a9e740 1 --2- 172.21.15.16:0/200829462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f9f840 0x56284735ef80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.292+0000 7f9904a9e740 1 --2- 172.21.15.16:0/200829462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f7eb60 0x5628473530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.292+0000 7f9904a9e740 1 --2- 172.21.15.16:0/200829462 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562846f5e1c0 0x562847364b30 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.292+0000 7f98e0c00640 0 client.26596 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.292+0000 7f9904a9e740 1 --2- 172.21.15.16:0/200829462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f76980 0x562847352b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.292+0000 7f9904a9e740 1 -- 172.21.15.16:0/200829462 >> 172.21.15.16:0/200829462 conn(0x562846f93990 msgr2=0x56284734a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.292+0000 7f9904a9e740 1 -- 172.21.15.16:0/200829462 shutdown_connections 2024-09-21T03:56:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.292+0000 7f9904a9e740 1 -- 172.21.15.16:0/200829462 wait complete. 2024-09-21T03:56:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.293+0000 7f9904a9e740 20 client.26596 trim_cache size 0 max 16384 2024-09-21T03:56:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.356+0000 7f9904a9e740 1 Processor -- start 2024-09-21T03:56:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.356+0000 7f9904a9e740 1 -- start start 2024-09-21T03:56:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.356+0000 7f9904a9e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f5e1c0 0x56284734ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.357+0000 7f9904a9e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f76980 0x562847353060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.357+0000 7f9904a9e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f7eb60 0x562847354a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.357+0000 7f9904a9e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98f81b7100 con 0x562846f5e1c0 2024-09-21T03:56:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.357+0000 7f9904a9e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98f81b8c40 con 0x562846f7eb60 2024-09-21T03:56:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.357+0000 7f9904a9e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98f81b77d0 con 0x562846f76980 2024-09-21T03:56:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.358+0000 7f98f6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f5e1c0 0x56284734ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.358+0000 7f98f6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f5e1c0 0x56284734ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47338/0 (socket says 172.21.15.16:47338) 2024-09-21T03:56:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.358+0000 7f98f6a00640 1 -- 172.21.15.16:0/3355955529 learned_addr learned my addr 172.21.15.16:0/3355955529 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.358+0000 7f98f6000640 1 --2- 172.21.15.16:0/3355955529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f76980 0x562847353060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.359+0000 7f98f6a00640 1 -- 172.21.15.16:0/3355955529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f76980 msgr2=0x562847353060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.359+0000 7f98f6a00640 1 --2- 172.21.15.16:0/3355955529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f76980 0x562847353060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.359+0000 7f98f6a00640 1 -- 172.21.15.16:0/3355955529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f7eb60 msgr2=0x562847354a40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:56:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.359+0000 7f98f6a00640 1 --2- 172.21.15.16:0/3355955529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f7eb60 0x562847354a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.359+0000 7f98f6a00640 1 -- 172.21.15.16:0/3355955529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562847353e20 con 0x562846f5e1c0 2024-09-21T03:56:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.361+0000 7f98f6a00640 1 --2- 172.21.15.16:0/3355955529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f5e1c0 0x56284734ac80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f98e8010d50 tx=0x7f98e8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0b8533167eed51 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:56.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.362+0000 7f98e3400640 1 -- 172.21.15.16:0/3355955529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e801e420 con 0x562846f5e1c0 2024-09-21T03:56:56.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.362+0000 7f98e3400640 1 -- 172.21.15.16:0/3355955529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98e802a070 con 0x562846f5e1c0 2024-09-21T03:56:56.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.362+0000 7f98e3400640 1 -- 172.21.15.16:0/3355955529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e8027840 con 0x562846f5e1c0 2024-09-21T03:56:56.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.363+0000 7f9904a9e740 1 -- 172.21.15.16:0/3355955529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f5e1c0 msgr2=0x56284734ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:56.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.363+0000 7f9904a9e740 1 --2- 172.21.15.16:0/3355955529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f5e1c0 0x56284734ac80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f98e8010d50 tx=0x7f98e8013000 comp rx=0 tx=0).stop 2024-09-21T03:56:56.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.364+0000 7f9904a9e740 1 -- 172.21.15.16:0/3355955529 shutdown_connections 2024-09-21T03:56:56.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.364+0000 7f9904a9e740 1 --2- 172.21.15.16:0/3355955529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f7eb60 0x562847354a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.364+0000 7f9904a9e740 1 --2- 172.21.15.16:0/3355955529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f76980 0x562847353060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.364+0000 7f9904a9e740 1 --2- 172.21.15.16:0/3355955529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f5e1c0 0x56284734ac80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.364+0000 7f9904a9e740 1 -- 172.21.15.16:0/3355955529 >> 172.21.15.16:0/3355955529 conn(0x562846f93990 msgr2=0x562847348540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:56.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.366+0000 7f9904a9e740 1 -- 172.21.15.16:0/3355955529 shutdown_connections 2024-09-21T03:56:56.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.366+0000 7f9904a9e740 1 -- 172.21.15.16:0/3355955529 wait complete. 2024-09-21T03:56:56.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.366+0000 7f9904a9e740 1 -- start start 2024-09-21T03:56:56.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.367+0000 7f9904a9e740 1 Processor -- start 2024-09-21T03:56:56.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.369+0000 7f9904a9e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f5e1c0 0x5628473662a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.370+0000 7f9904a9e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f76980 0x5628473667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.371+0000 7f9904a9e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f7eb60 0x5628473ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.371+0000 7f98f6a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f5e1c0 0x5628473662a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:56.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.371+0000 7f98f6a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f5e1c0 0x5628473662a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:49272/0 (socket says 172.21.15.16:49272) 2024-09-21T03:56:56.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.371+0000 7f98f6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f76980 0x5628473667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:56.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.371+0000 7f98f6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f76980 0x5628473667e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:60058/0 (socket says 172.21.15.16:60058) 2024-09-21T03:56:56.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.372+0000 7f9904a9e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98f81b8570 con 0x562846f7eb60 2024-09-21T03:56:56.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.372+0000 7f9904a9e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98f81b7ea0 con 0x562846f76980 2024-09-21T03:56:56.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.372+0000 7f9904a9e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98f81b50e0 con 0x562846f5e1c0 2024-09-21T03:56:56.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.372+0000 7f9904a9e740 10 client.0 _ll_register_callbacks cb 0x7f98f00099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:56.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.373+0000 7f98f6a00640 1 -- 172.21.15.16:0/1618244721 learned_addr learned my addr 172.21.15.16:0/1618244721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:56.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.373+0000 7f98f7400640 1 --2- 172.21.15.16:0/1618244721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f7eb60 0x5628473ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:56.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.376+0000 7f98f6000640 1 -- 172.21.15.16:0/1618244721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f5e1c0 msgr2=0x5628473662a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.376+0000 7f98f6000640 1 --2- 172.21.15.16:0/1618244721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f5e1c0 0x5628473662a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.376+0000 7f98f6000640 1 -- 172.21.15.16:0/1618244721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f7eb60 msgr2=0x5628473ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.376+0000 7f98f6000640 1 --2- 172.21.15.16:0/1618244721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f7eb60 0x5628473ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.376+0000 7f98f6000640 1 -- 172.21.15.16:0/1618244721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628473ce7f0 con 0x562846f76980 2024-09-21T03:56:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.377+0000 7f98f7400640 1 --2- 172.21.15.16:0/1618244721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f7eb60 0x5628473ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:56:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.378+0000 7f98f6a00640 1 --2- 172.21.15.16:0/1618244721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f5e1c0 0x5628473662a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:56:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.382+0000 7f98f6000640 1 --2- 172.21.15.16:0/1618244721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f76980 0x5628473667e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f98e4005d00 tx=0x7f98e40ac8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=159a9974db3792a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.383+0000 7f98e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:56:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.383+0000 7f98e0c00640 1 -- 172.21.15.16:0/1618244721 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e40ad5b0 con 0x562846f76980 2024-09-21T03:56:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.383+0000 7f98e0c00640 1 -- 172.21.15.16:0/1618244721 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f98e4004240 con 0x562846f76980 2024-09-21T03:56:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.383+0000 7f9904a9e740 1 -- 172.21.15.16:0/1618244721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98e802a070 con 0x562846f76980 2024-09-21T03:56:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.384+0000 7f98e0c00640 1 -- 172.21.15.16:0/1618244721 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f98e40b1990 con 0x562846f76980 2024-09-21T03:56:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.384+0000 7f9904a9e740 10 client.26642 fetch_fsmap learned FSMap version 30 2024-09-21T03:56:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.384+0000 7f9904a9e740 1 -- 172.21.15.16:0/1618244721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5628473cfab0 con 0x562846f76980 2024-09-21T03:56:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.384+0000 7f98e0c00640 1 -- 172.21.15.16:0/1618244721 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98e802a070 con 0x562846f76980 2024-09-21T03:56:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.384+0000 7f98e0c00640 1 -- 172.21.15.16:0/1618244721 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f98e40b1050 con 0x562846f76980 2024-09-21T03:56:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.384+0000 7f9904a9e740 10 client.26642 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:56:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.384+0000 7f9904a9e740 10 client.26642 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:56:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.384+0000 7f9904a9e740 20 client.26642 populate_metadata read hostname 'smithi016' 2024-09-21T03:56:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.384+0000 7f9904a9e740 1 --2- 172.21.15.16:0/1618244721 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562846f9f840 0x5628473d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.384+0000 7f9904a9e740 4 client.26642 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-21T03:56:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.384+0000 7f9904a9e740 1 -- 172.21.15.16:0/1618244721 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x5628472ba6a0 con 0x562846f9f840 2024-09-21T03:56:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.385+0000 7f98f6a00640 1 --2- 172.21.15.16:0/1618244721 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562846f9f840 0x5628473d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.387+0000 7f98f6a00640 1 --2- 172.21.15.16:0/1618244721 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562846f9f840 0x5628473d22e0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46780d8bcda39bb4 server_cookie=bf464e0ac4450192 in_seq=0 out_seq=0 2024-09-21T03:56:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.387+0000 7f98e0c00640 10 client.26642 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.537+0000 7f98e0c00640 1 -- 172.21.15.16:0/1618244721 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5628472ba6a0 con 0x562846f9f840 2024-09-21T03:56:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.537+0000 7f98e0c00640 10 client.26642 handle_command_reply: tid=0 2024-09-21T03:56:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.537+0000 7f9904a9e740 1 client.26642 shutdown 2024-09-21T03:56:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.538+0000 7f9904a9e740 1 -- 172.21.15.16:0/1618244721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f76980 msgr2=0x5628473667e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.538+0000 7f9904a9e740 1 --2- 172.21.15.16:0/1618244721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f76980 0x5628473667e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f98e4005d00 tx=0x7f98e40ac8c0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.539+0000 7f9904a9e740 1 -- 172.21.15.16:0/1618244721 shutdown_connections 2024-09-21T03:56:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.539+0000 7f9904a9e740 1 --2- 172.21.15.16:0/1618244721 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x562846f9f840 0x5628473d22e0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.539+0000 7f9904a9e740 1 --2- 172.21.15.16:0/1618244721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562846f7eb60 0x5628473ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.539+0000 7f9904a9e740 1 --2- 172.21.15.16:0/1618244721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562846f76980 0x5628473667e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.539+0000 7f9904a9e740 1 --2- 172.21.15.16:0/1618244721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562846f5e1c0 0x5628473662a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.539+0000 7f9904a9e740 1 -- 172.21.15.16:0/1618244721 >> 172.21.15.16:0/1618244721 conn(0x562846f93990 msgr2=0x562847354780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.539+0000 7f98e0c00640 0 client.26642 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.540+0000 7f9904a9e740 1 -- 172.21.15.16:0/1618244721 shutdown_connections 2024-09-21T03:56:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.540+0000 7f9904a9e740 1 -- 172.21.15.16:0/1618244721 wait complete. 2024-09-21T03:56:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.541+0000 7f9904a9e740 20 client.26642 trim_cache size 0 max 16384 2024-09-21T03:56:56.606 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:56:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:56:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:56:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 38, 2024-09-21T03:56:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:56:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "fc03fa01": { 2024-09-21T03:56:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 38, 2024-09-21T03:56:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:56:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:56:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:56:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:56:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:56:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:56:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 144.5, 2024-09-21T03:56:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:56:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:56:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:56:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:56:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:56:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:56:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:56:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:56:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:56:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:56:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:56:56.608 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:56:56.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.545+0000 7f98f7e00640 1 -- 172.21.15.16:0/3491515836 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f98dc0b65d0 msgr2=0x7f98dc0b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:56.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.545+0000 7f98f7e00640 1 --2- 172.21.15.16:0/3491515836 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f98dc0b65d0 0x7f98dc0b8ae0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f98f8110520 tx=0x7f98e800b390 comp rx=0 tx=0).stop 2024-09-21T03:56:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.545+0000 7f98f7e00640 1 -- 172.21.15.16:0/3491515836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98f8109300 msgr2=0x7f98f810f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.545+0000 7f98f7e00640 1 --2- 172.21.15.16:0/3491515836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98f8109300 0x7f98f810f010 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f98e4005fd0 tx=0x7f98e400dde0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.546+0000 7f98f7e00640 1 -- 172.21.15.16:0/3491515836 shutdown_connections 2024-09-21T03:56:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.546+0000 7f98f7e00640 1 --2- 172.21.15.16:0/3491515836 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f98dc0b65d0 0x7f98dc0b8ae0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.546+0000 7f98f7e00640 1 --2- 172.21.15.16:0/3491515836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f8109c50 0x7f98f81ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.546+0000 7f98f7e00640 1 --2- 172.21.15.16:0/3491515836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98f8109300 0x7f98f810f010 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.546+0000 7f98f7e00640 1 --2- 172.21.15.16:0/3491515836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98f81039c0 0x7f98f810ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.546+0000 7f98f7e00640 1 -- 172.21.15.16:0/3491515836 >> 172.21.15.16:0/3491515836 conn(0x7f98f80ff430 msgr2=0x7f98f8107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.554+0000 7f98f7e00640 1 -- 172.21.15.16:0/3491515836 shutdown_connections 2024-09-21T03:56:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:56:56.554+0000 7f98f7e00640 1 -- 172.21.15.16:0/3491515836 wait complete. 2024-09-21T03:56:57.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:56 smithi134 ceph-mon[35031]: pgmap v737: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 243 MiB/s wr, 1.27k op/s 2024-09-21T03:56:57.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:56 smithi123 ceph-mon[34673]: pgmap v737: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 243 MiB/s wr, 1.27k op/s 2024-09-21T03:56:57.448 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:56 smithi016 ceph-mon[30375]: pgmap v737: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 243 MiB/s wr, 1.27k op/s 2024-09-21T03:56:58.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.970+0000 7f0b3ca00640 1 -- 172.21.15.16:0/2837589690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b381034e0 msgr2=0x7f0b38103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:58.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.970+0000 7f0b3ca00640 1 --2- 172.21.15.16:0/2837589690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b381034e0 0x7f0b38103950 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009900 tx=0x7f0b2802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.970+0000 7f0b3ca00640 1 -- 172.21.15.16:0/2837589690 shutdown_connections 2024-09-21T03:56:58.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.970+0000 7f0b3ca00640 1 --2- 172.21.15.16:0/2837589690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b38104f00 0x7f0b3810b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.970+0000 7f0b3ca00640 1 --2- 172.21.15.16:0/2837589690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b38104560 0x7f0b38104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.970+0000 7f0b3ca00640 1 --2- 172.21.15.16:0/2837589690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b381034e0 0x7f0b38103950 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.970+0000 7f0b3ca00640 1 -- 172.21.15.16:0/2837589690 >> 172.21.15.16:0/2837589690 conn(0x7f0b380fee20 msgr2=0x7f0b381012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.970+0000 7f0b3ca00640 1 -- 172.21.15.16:0/2837589690 shutdown_connections 2024-09-21T03:56:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.970+0000 7f0b3ca00640 1 -- 172.21.15.16:0/2837589690 wait complete. 2024-09-21T03:56:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.971+0000 7f0b3ca00640 1 Processor -- start 2024-09-21T03:56:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.971+0000 7f0b3ca00640 1 -- start start 2024-09-21T03:56:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.972+0000 7f0b3ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b381034e0 0x7f0b38076de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.972+0000 7f0b3ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b38104560 0x7f0b38075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.972+0000 7f0b3ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b38104f00 0x7f0b38075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.972+0000 7f0b3ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3810f100 con 0x7f0b381034e0 2024-09-21T03:56:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.972+0000 7f0b3ca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3810ef80 con 0x7f0b38104f00 2024-09-21T03:56:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.972+0000 7f0b3ca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3810f280 con 0x7f0b38104560 2024-09-21T03:56:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.972+0000 7f0b35e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b38104560 0x7f0b38075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.972+0000 7f0b35e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b38104560 0x7f0b38075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:49286/0 (socket says 172.21.15.16:49286) 2024-09-21T03:56:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.972+0000 7f0b35e00640 1 -- 172.21.15.16:0/3084049634 learned_addr learned my addr 172.21.15.16:0/3084049634 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.972+0000 7f0b36800640 1 --2- 172.21.15.16:0/3084049634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b381034e0 0x7f0b38076de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.972+0000 7f0b37200640 1 --2- 172.21.15.16:0/3084049634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b38104f00 0x7f0b38075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.972+0000 7f0b35e00640 1 -- 172.21.15.16:0/3084049634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b38104f00 msgr2=0x7f0b38075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.972+0000 7f0b35e00640 1 --2- 172.21.15.16:0/3084049634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b38104f00 0x7f0b38075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.972+0000 7f0b35e00640 1 -- 172.21.15.16:0/3084049634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b381034e0 msgr2=0x7f0b38076de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.972+0000 7f0b35e00640 1 --2- 172.21.15.16:0/3084049634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b381034e0 0x7f0b38076de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.972+0000 7f0b35e00640 1 -- 172.21.15.16:0/3084049634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b380762f0 con 0x7f0b38104560 2024-09-21T03:56:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.973+0000 7f0b35e00640 1 --2- 172.21.15.16:0/3084049634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b38104560 0x7f0b38075710 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400ea60 tx=0x7f0b2400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.973+0000 7f0b2f400640 1 -- 172.21.15.16:0/3084049634 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2400cde0 con 0x7f0b38104560 2024-09-21T03:56:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.973+0000 7f0b2f400640 1 -- 172.21.15.16:0/3084049634 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b240035f0 con 0x7f0b38104560 2024-09-21T03:56:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.973+0000 7f0b3ca00640 1 -- 172.21.15.16:0/3084049634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b381a3e30 con 0x7f0b38104560 2024-09-21T03:56:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.973+0000 7f0b2f400640 1 -- 172.21.15.16:0/3084049634 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24010690 con 0x7f0b38104560 2024-09-21T03:56:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.973+0000 7f0b3ca00640 1 -- 172.21.15.16:0/3084049634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b381a41e0 con 0x7f0b38104560 2024-09-21T03:56:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.977+0000 7f0b443bc740 1 Processor -- start 2024-09-21T03:56:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.978+0000 7f0b443bc740 1 -- start start 2024-09-21T03:56:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.978+0000 7f0b443bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5db22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.978+0000 7f0b443bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c59e22f0 0x55d6c5db2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.978+0000 7f0b443bc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59ea4d0 0x55d6c5db3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.978+0000 7f0b443bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d6c5d256b0 con 0x55d6c59e22f0 2024-09-21T03:56:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.978+0000 7f0b443bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d6c5d14ae0 con 0x55d6c5a0b1b0 2024-09-21T03:56:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.978+0000 7f0b443bc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d6c5d188f0 con 0x55d6c59ea4d0 2024-09-21T03:56:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.978+0000 7f0b36800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c59e22f0 0x55d6c5db2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.978+0000 7f0b36800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c59e22f0 0x55d6c5db2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47380/0 (socket says 172.21.15.16:47380) 2024-09-21T03:56:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.978+0000 7f0b36800640 1 -- 172.21.15.16:0/1060911598 learned_addr learned my addr 172.21.15.16:0/1060911598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.978+0000 7f0b37200640 1 --2- 172.21.15.16:0/1060911598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5db22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.979+0000 7f0b37200640 1 -- 172.21.15.16:0/1060911598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59ea4d0 msgr2=0x55d6c5db3470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:56:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.979+0000 7f0b37200640 1 --2- 172.21.15.16:0/1060911598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59ea4d0 0x55d6c5db3470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.979+0000 7f0b37200640 1 -- 172.21.15.16:0/1060911598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c59e22f0 msgr2=0x55d6c5db2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.979+0000 7f0b37200640 1 --2- 172.21.15.16:0/1060911598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c59e22f0 0x55d6c5db2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.979+0000 7f0b37200640 1 -- 172.21.15.16:0/1060911598 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6c5ce1780 con 0x55d6c5a0b1b0 2024-09-21T03:56:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.979+0000 7f0b2f400640 1 -- 172.21.15.16:0/3084049634 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0b240037a0 con 0x7f0b38104560 2024-09-21T03:56:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.979+0000 7f0b37200640 1 --2- 172.21.15.16:0/1060911598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5db22b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009f00 tx=0x7f0b30011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b017fbfbace34b04 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.979+0000 7f0b23400640 1 -- 172.21.15.16:0/1060911598 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30003c30 con 0x55d6c5a0b1b0 2024-09-21T03:56:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.979+0000 7f0b23400640 1 -- 172.21.15.16:0/1060911598 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b30003dd0 con 0x55d6c5a0b1b0 2024-09-21T03:56:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.979+0000 7f0b23400640 1 -- 172.21.15.16:0/1060911598 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3000f940 con 0x55d6c5a0b1b0 2024-09-21T03:56:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.980+0000 7f0b443bc740 1 -- 172.21.15.16:0/1060911598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c5a0b1b0 msgr2=0x55d6c5db22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.980+0000 7f0b443bc740 1 --2- 172.21.15.16:0/1060911598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5db22b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009f00 tx=0x7f0b30011810 comp rx=0 tx=0).stop 2024-09-21T03:56:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.980+0000 7f0b443bc740 1 -- 172.21.15.16:0/1060911598 shutdown_connections 2024-09-21T03:56:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.980+0000 7f0b443bc740 1 --2- 172.21.15.16:0/1060911598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59ea4d0 0x55d6c5db3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.980+0000 7f0b443bc740 1 --2- 172.21.15.16:0/1060911598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c59e22f0 0x55d6c5db2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.980+0000 7f0b443bc740 1 --2- 172.21.15.16:0/1060911598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5db22b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.980+0000 7f0b443bc740 1 -- 172.21.15.16:0/1060911598 >> 172.21.15.16:0/1060911598 conn(0x55d6c59ff300 msgr2=0x55d6c5dadd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.981+0000 7f0b443bc740 1 -- 172.21.15.16:0/1060911598 shutdown_connections 2024-09-21T03:56:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.981+0000 7f0b443bc740 1 -- 172.21.15.16:0/1060911598 wait complete. 2024-09-21T03:56:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.981+0000 7f0b2f400640 1 --2- 172.21.15.16:0/3084049634 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0b1c0b6420 0x7f0b1c0b8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.981+0000 7f0b2f400640 1 -- 172.21.15.16:0/3084049634 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0b240e7990 con 0x7f0b38104560 2024-09-21T03:56:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.981+0000 7f0b36800640 1 --2- 172.21.15.16:0/3084049634 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0b1c0b6420 0x7f0b1c0b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.981+0000 7f0b443bc740 1 -- start start 2024-09-21T03:56:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.982+0000 7f0b443bc740 1 Processor -- start 2024-09-21T03:56:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.982+0000 7f0b36800640 1 --2- 172.21.15.16:0/3084049634 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0b1c0b6420 0x7f0b1c0b8930 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0b280096c0 tx=0x7f0b28031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.982+0000 7f0b443bc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59e22f0 0x55d6c5dbb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.982+0000 7f0b443bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59ea4d0 0x55d6c5dbc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.982+0000 7f0b443bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5dbcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.982+0000 7f0b443bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d6c5c4cab0 con 0x55d6c5a0b1b0 2024-09-21T03:56:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.982+0000 7f0b443bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d6c5cbb570 con 0x55d6c59ea4d0 2024-09-21T03:56:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.982+0000 7f0b443bc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d6c5db3f90 con 0x55d6c59e22f0 2024-09-21T03:56:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.982+0000 7f0b443bc740 10 client.0 _ll_register_callbacks cb 0x55d6c5cca100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.983+0000 7f0b35e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59ea4d0 0x55d6c5dbc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.983+0000 7f0b35e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59ea4d0 0x55d6c5dbc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:60086/0 (socket says 172.21.15.16:60086) 2024-09-21T03:56:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.983+0000 7f0b35e00640 1 -- 172.21.15.16:0/9381034 learned_addr learned my addr 172.21.15.16:0/9381034 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.983+0000 7f0b36800640 1 --2- 172.21.15.16:0/9381034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59e22f0 0x55d6c5dbb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.983+0000 7f0b35e00640 1 -- 172.21.15.16:0/9381034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59e22f0 msgr2=0x55d6c5dbb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.983+0000 7f0b35e00640 1 --2- 172.21.15.16:0/9381034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59e22f0 0x55d6c5dbb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.983+0000 7f0b35e00640 1 -- 172.21.15.16:0/9381034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c5a0b1b0 msgr2=0x55d6c5dbcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.983+0000 7f0b37200640 1 --2- 172.21.15.16:0/9381034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5dbcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.983+0000 7f0b35e00640 1 --2- 172.21.15.16:0/9381034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5dbcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.983+0000 7f0b35e00640 1 -- 172.21.15.16:0/9381034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6c5a76790 con 0x55d6c59ea4d0 2024-09-21T03:56:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.983+0000 7f0b37200640 1 --2- 172.21.15.16:0/9381034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5dbcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.984+0000 7f0b35e00640 1 --2- 172.21.15.16:0/9381034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59ea4d0 0x55d6c5dbc060 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009230 tx=0x7f0b24003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=756c902eff2b7b47 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.984+0000 7f0b17e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:56:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.984+0000 7f0b17e00640 1 -- 172.21.15.16:0/9381034 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b240ace80 con 0x55d6c59ea4d0 2024-09-21T03:56:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.984+0000 7f0b443bc740 1 -- 172.21.15.16:0/9381034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b30003dd0 con 0x55d6c59ea4d0 2024-09-21T03:56:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.984+0000 7f0b17e00640 1 -- 172.21.15.16:0/9381034 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0b240acb70 con 0x55d6c59ea4d0 2024-09-21T03:56:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.984+0000 7f0b17e00640 1 -- 172.21.15.16:0/9381034 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0b240ad220 con 0x55d6c59ea4d0 2024-09-21T03:56:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.984+0000 7f0b443bc740 10 client.26654 fetch_fsmap learned FSMap version 30 2024-09-21T03:56:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.984+0000 7f0b443bc740 1 -- 172.21.15.16:0/9381034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d6c5aef620 con 0x55d6c59ea4d0 2024-09-21T03:56:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.986+0000 7f0b17e00640 1 -- 172.21.15.16:0/9381034 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b30003dd0 con 0x55d6c59ea4d0 2024-09-21T03:56:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.986+0000 7f0b17e00640 1 -- 172.21.15.16:0/9381034 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0b240ac580 con 0x55d6c59ea4d0 2024-09-21T03:56:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.986+0000 7f0b443bc740 10 client.26654 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:56:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.986+0000 7f0b443bc740 10 client.26654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.986+0000 7f0b443bc740 20 client.26654 populate_metadata read hostname 'smithi016' 2024-09-21T03:56:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.986+0000 7f0b443bc740 1 --2- 172.21.15.16:0/9381034 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d6c59c9d80 0x55d6c5dc90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.986+0000 7f0b443bc740 4 client.26654 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:56:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.986+0000 7f0b443bc740 1 -- 172.21.15.16:0/9381034 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6c5d27600 con 0x55d6c59c9d80 2024-09-21T03:56:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.987+0000 7f0b36800640 1 --2- 172.21.15.16:0/9381034 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d6c59c9d80 0x55d6c5dc90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.988+0000 7f0b36800640 1 --2- 172.21.15.16:0/9381034 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d6c59c9d80 0x55d6c5dc90e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e805afdbfac116d server_cookie=1c4a7210c5e40a96 in_seq=0 out_seq=0 2024-09-21T03:56:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.988+0000 7f0b17e00640 10 client.26654 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.988+0000 7f0b17e00640 1 -- 172.21.15.16:0/9381034 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d6c5d27600 con 0x55d6c59c9d80 2024-09-21T03:56:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.988+0000 7f0b17e00640 10 client.26654 handle_command_reply: tid=0 2024-09-21T03:56:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.989+0000 7f0b443bc740 1 client.26654 shutdown 2024-09-21T03:56:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.989+0000 7f0b443bc740 1 -- 172.21.15.16:0/9381034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59ea4d0 msgr2=0x55d6c5dbc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.989+0000 7f0b443bc740 1 --2- 172.21.15.16:0/9381034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59ea4d0 0x55d6c5dbc060 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009230 tx=0x7f0b24003cf0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.989+0000 7f0b443bc740 1 -- 172.21.15.16:0/9381034 shutdown_connections 2024-09-21T03:56:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.989+0000 7f0b443bc740 1 --2- 172.21.15.16:0/9381034 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d6c59c9d80 0x55d6c5dc90e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.989+0000 7f0b443bc740 1 --2- 172.21.15.16:0/9381034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5dbcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.989+0000 7f0b443bc740 1 --2- 172.21.15.16:0/9381034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59ea4d0 0x55d6c5dbc060 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.989+0000 7f0b443bc740 1 --2- 172.21.15.16:0/9381034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59e22f0 0x55d6c5dbb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.989+0000 7f0b17e00640 0 client.26654 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.989+0000 7f0b443bc740 1 -- 172.21.15.16:0/9381034 >> 172.21.15.16:0/9381034 conn(0x55d6c59ff300 msgr2=0x55d6c5dac360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.989+0000 7f0b443bc740 1 -- 172.21.15.16:0/9381034 shutdown_connections 2024-09-21T03:56:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.989+0000 7f0b443bc740 1 -- 172.21.15.16:0/9381034 wait complete. 2024-09-21T03:56:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:58.990+0000 7f0b443bc740 20 client.26654 trim_cache size 0 max 16384 2024-09-21T03:56:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.015+0000 7f0b443bc740 1 Processor -- start 2024-09-21T03:56:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.015+0000 7f0b443bc740 1 -- start start 2024-09-21T03:56:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.016+0000 7f0b443bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59e22f0 0x55d6c5db6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.016+0000 7f0b443bc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59ea4d0 0x55d6c5dc3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.016+0000 7f0b443bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5dad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.016+0000 7f0b443bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b381ae7b0 con 0x55d6c5a0b1b0 2024-09-21T03:56:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.016+0000 7f0b443bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0b381b02f0 con 0x55d6c59e22f0 2024-09-21T03:56:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.016+0000 7f0b443bc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b381aee80 con 0x55d6c59ea4d0 2024-09-21T03:56:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.018+0000 7f0b36800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59e22f0 0x55d6c5db6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.018+0000 7f0b36800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59e22f0 0x55d6c5db6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:60090/0 (socket says 172.21.15.16:60090) 2024-09-21T03:56:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.018+0000 7f0b35e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59ea4d0 0x55d6c5dc3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.021+0000 7f0b35e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59ea4d0 0x55d6c5dc3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:49304/0 (socket says 172.21.15.16:49304) 2024-09-21T03:56:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.021+0000 7f0b35e00640 1 -- 172.21.15.16:0/3044368743 learned_addr learned my addr 172.21.15.16:0/3044368743 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.021+0000 7f0b37200640 1 --2- 172.21.15.16:0/3044368743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5dad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.022+0000 7f0b35e00640 1 -- 172.21.15.16:0/3044368743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59e22f0 msgr2=0x55d6c5db6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.022+0000 7f0b35e00640 1 --2- 172.21.15.16:0/3044368743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59e22f0 0x55d6c5db6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.022+0000 7f0b35e00640 1 -- 172.21.15.16:0/3044368743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c5a0b1b0 msgr2=0x55d6c5dad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.022+0000 7f0b35e00640 1 --2- 172.21.15.16:0/3044368743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5dad370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.022+0000 7f0b35e00640 1 -- 172.21.15.16:0/3044368743 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6c5dbcf40 con 0x55d6c59ea4d0 2024-09-21T03:56:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.022+0000 7f0b36800640 1 --2- 172.21.15.16:0/3044368743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59e22f0 0x55d6c5db6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.023+0000 7f0b35e00640 1 --2- 172.21.15.16:0/3044368743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59ea4d0 0x55d6c5dc3ab0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400d010 tx=0x7f0b240057e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fc75ce1f429440c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.023+0000 7f0b22a00640 1 -- 172.21.15.16:0/3044368743 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b240cd070 con 0x55d6c59ea4d0 2024-09-21T03:56:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.024+0000 7f0b22a00640 1 -- 172.21.15.16:0/3044368743 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b240ade10 con 0x55d6c59ea4d0 2024-09-21T03:56:59.026 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.024+0000 7f0b22a00640 1 -- 172.21.15.16:0/3044368743 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b240ac970 con 0x55d6c59ea4d0 2024-09-21T03:56:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.025+0000 7f0b443bc740 1 -- 172.21.15.16:0/3044368743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59ea4d0 msgr2=0x55d6c5dc3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.025+0000 7f0b443bc740 1 --2- 172.21.15.16:0/3044368743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59ea4d0 0x55d6c5dc3ab0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400d010 tx=0x7f0b240057e0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.026+0000 7f0b443bc740 1 -- 172.21.15.16:0/3044368743 shutdown_connections 2024-09-21T03:56:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.026+0000 7f0b443bc740 1 --2- 172.21.15.16:0/3044368743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5dad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.026+0000 7f0b443bc740 1 --2- 172.21.15.16:0/3044368743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59ea4d0 0x55d6c5dc3ab0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.026+0000 7f0b443bc740 1 --2- 172.21.15.16:0/3044368743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59e22f0 0x55d6c5db6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.026+0000 7f0b443bc740 1 -- 172.21.15.16:0/3044368743 >> 172.21.15.16:0/3044368743 conn(0x55d6c59ff300 msgr2=0x55d6c5db63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.029+0000 7f0b443bc740 1 -- 172.21.15.16:0/3044368743 shutdown_connections 2024-09-21T03:56:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.029+0000 7f0b443bc740 1 -- 172.21.15.16:0/3044368743 wait complete. 2024-09-21T03:56:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.029+0000 7f0b443bc740 1 -- start start 2024-09-21T03:56:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.030+0000 7f0b443bc740 1 Processor -- start 2024-09-21T03:56:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.030+0000 7f0b443bc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59e22f0 0x55d6c5dc2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.030+0000 7f0b443bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59ea4d0 0x55d6c5dc34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.030+0000 7f0b443bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5dc1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.030+0000 7f0b443bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b381afc20 con 0x55d6c5a0b1b0 2024-09-21T03:56:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.030+0000 7f0b443bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0b381af550 con 0x55d6c59ea4d0 2024-09-21T03:56:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.030+0000 7f0b443bc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b381ac790 con 0x55d6c59e22f0 2024-09-21T03:56:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.032+0000 7f0b443bc740 10 client.0 _ll_register_callbacks cb 0x7f0b240a95b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:56:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.032+0000 7f0b36800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59e22f0 0x55d6c5dc2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.032+0000 7f0b36800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59e22f0 0x55d6c5dc2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:49310/0 (socket says 172.21.15.16:49310) 2024-09-21T03:56:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.032+0000 7f0b36800640 1 -- 172.21.15.16:0/3437956987 learned_addr learned my addr 172.21.15.16:0/3437956987 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:56:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.032+0000 7f0b35e00640 1 --2- 172.21.15.16:0/3437956987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59ea4d0 0x55d6c5dc34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.032+0000 7f0b37200640 1 --2- 172.21.15.16:0/3437956987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5dc1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.033+0000 7f0b36800640 1 -- 172.21.15.16:0/3437956987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59ea4d0 msgr2=0x55d6c5dc34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.033+0000 7f0b36800640 1 --2- 172.21.15.16:0/3437956987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59ea4d0 0x55d6c5dc34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.033+0000 7f0b36800640 1 -- 172.21.15.16:0/3437956987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c5a0b1b0 msgr2=0x55d6c5dc1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:56:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.033+0000 7f0b36800640 1 --2- 172.21.15.16:0/3437956987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5dc1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.033+0000 7f0b36800640 1 -- 172.21.15.16:0/3437956987 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6c5dc1f40 con 0x55d6c59e22f0 2024-09-21T03:56:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.033+0000 7f0b37200640 1 --2- 172.21.15.16:0/3437956987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5dc1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:56:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.034+0000 7f0b36800640 1 --2- 172.21.15.16:0/3437956987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59e22f0 0x55d6c5dc2e30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0b2802a910 tx=0x7f0b28009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d7b5f85c78ca5a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.034+0000 7f0b17e00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:56:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.034+0000 7f0b17e00640 1 -- 172.21.15.16:0/3437956987 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28035040 con 0x55d6c59e22f0 2024-09-21T03:56:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.034+0000 7f0b17e00640 1 -- 172.21.15.16:0/3437956987 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0b28003ac0 con 0x55d6c59e22f0 2024-09-21T03:56:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.034+0000 7f0b443bc740 1 -- 172.21.15.16:0/3437956987 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b240ade10 con 0x55d6c59e22f0 2024-09-21T03:56:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.035+0000 7f0b17e00640 1 -- 172.21.15.16:0/3437956987 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0b28003cc0 con 0x55d6c59e22f0 2024-09-21T03:56:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.035+0000 7f0b443bc740 10 client.26614 fetch_fsmap learned FSMap version 30 2024-09-21T03:56:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.035+0000 7f0b443bc740 1 -- 172.21.15.16:0/3437956987 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d6c5e34450 con 0x55d6c59e22f0 2024-09-21T03:56:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.035+0000 7f0b17e00640 1 -- 172.21.15.16:0/3437956987 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b240ade10 con 0x55d6c59e22f0 2024-09-21T03:56:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.036+0000 7f0b17e00640 1 -- 172.21.15.16:0/3437956987 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0b28044040 con 0x55d6c59e22f0 2024-09-21T03:56:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.037+0000 7f0b443bc740 10 client.26614 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:56:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.037+0000 7f0b443bc740 10 client.26614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:56:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.037+0000 7f0b443bc740 20 client.26614 populate_metadata read hostname 'smithi016' 2024-09-21T03:56:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.037+0000 7f0b443bc740 1 --2- 172.21.15.16:0/3437956987 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d6c599f1c0 0x55d6c5e36e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.037+0000 7f0b443bc740 4 client.26614 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:56:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.037+0000 7f0b443bc740 1 -- 172.21.15.16:0/3437956987 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d6c5d230b0 con 0x55d6c599f1c0 2024-09-21T03:56:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.038+0000 7f0b35e00640 1 --2- 172.21.15.16:0/3437956987 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d6c599f1c0 0x55d6c5e36e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.038+0000 7f0b35e00640 1 --2- 172.21.15.16:0/3437956987 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d6c599f1c0 0x55d6c5e36e70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a822e5fff5cb35a8 server_cookie=1fc4790a38bbbec6 in_seq=0 out_seq=0 2024-09-21T03:56:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.038+0000 7f0b17e00640 10 client.26614 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.052+0000 7f0b17e00640 1 -- 172.21.15.16:0/3437956987 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d6c5d230b0 con 0x55d6c599f1c0 2024-09-21T03:56:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.052+0000 7f0b17e00640 10 client.26614 handle_command_reply: tid=0 2024-09-21T03:56:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.052+0000 7f0b443bc740 1 client.26614 shutdown 2024-09-21T03:56:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.052+0000 7f0b443bc740 1 -- 172.21.15.16:0/3437956987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59e22f0 msgr2=0x55d6c5dc2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.052+0000 7f0b443bc740 1 --2- 172.21.15.16:0/3437956987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59e22f0 0x55d6c5dc2e30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0b2802a910 tx=0x7f0b28009260 comp rx=0 tx=0).stop 2024-09-21T03:56:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.052+0000 7f0b443bc740 1 -- 172.21.15.16:0/3437956987 shutdown_connections 2024-09-21T03:56:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.052+0000 7f0b443bc740 1 --2- 172.21.15.16:0/3437956987 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55d6c599f1c0 0x55d6c5e36e70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.052+0000 7f0b443bc740 1 --2- 172.21.15.16:0/3437956987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6c5a0b1b0 0x55d6c5dc1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.052+0000 7f0b443bc740 1 --2- 172.21.15.16:0/3437956987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6c59ea4d0 0x55d6c5dc34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.052+0000 7f0b17e00640 0 client.26614 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:56:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.052+0000 7f0b443bc740 1 --2- 172.21.15.16:0/3437956987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6c59e22f0 0x55d6c5dc2e30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.052+0000 7f0b443bc740 1 -- 172.21.15.16:0/3437956987 >> 172.21.15.16:0/3437956987 conn(0x55d6c59ff300 msgr2=0x55d6c5dac360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.052+0000 7f0b443bc740 1 -- 172.21.15.16:0/3437956987 shutdown_connections 2024-09-21T03:56:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.052+0000 7f0b443bc740 1 -- 172.21.15.16:0/3437956987 wait complete. 2024-09-21T03:56:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.052+0000 7f0b443bc740 20 client.26614 trim_cache size 0 max 16384 2024-09-21T03:56:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:56:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:56:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:56:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:56:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.054+0000 7f0b3ca00640 1 -- 172.21.15.16:0/3084049634 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0b1c0b6420 msgr2=0x7f0b1c0b8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.054+0000 7f0b3ca00640 1 --2- 172.21.15.16:0/3084049634 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0b1c0b6420 0x7f0b1c0b8930 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0b280096c0 tx=0x7f0b28031400 comp rx=0 tx=0).stop 2024-09-21T03:56:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.054+0000 7f0b3ca00640 1 -- 172.21.15.16:0/3084049634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b38104560 msgr2=0x7f0b38075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.054+0000 7f0b3ca00640 1 --2- 172.21.15.16:0/3084049634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b38104560 0x7f0b38075710 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400ea60 tx=0x7f0b2400ef20 comp rx=0 tx=0).stop 2024-09-21T03:56:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.054+0000 7f0b3ca00640 1 -- 172.21.15.16:0/3084049634 shutdown_connections 2024-09-21T03:56:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.054+0000 7f0b3ca00640 1 --2- 172.21.15.16:0/3084049634 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0b1c0b6420 0x7f0b1c0b8930 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.054+0000 7f0b3ca00640 1 --2- 172.21.15.16:0/3084049634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b38104f00 0x7f0b38075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.054+0000 7f0b3ca00640 1 --2- 172.21.15.16:0/3084049634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b38104560 0x7f0b38075710 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.054+0000 7f0b3ca00640 1 --2- 172.21.15.16:0/3084049634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b381034e0 0x7f0b38076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.054+0000 7f0b3ca00640 1 -- 172.21.15.16:0/3084049634 >> 172.21.15.16:0/3084049634 conn(0x7f0b380fee20 msgr2=0x7f0b38107880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.055+0000 7f0b3ca00640 1 -- 172.21.15.16:0/3084049634 shutdown_connections 2024-09-21T03:56:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:56:59.055+0000 7f0b3ca00640 1 -- 172.21.15.16:0/3084049634 wait complete. 2024-09-21T03:56:59.322 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:56:58 smithi016 ceph-mon[30375]: pgmap v738: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 207 MiB/s wr, 1.06k op/s 2024-09-21T03:56:59.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:56:58 smithi134 ceph-mon[35031]: pgmap v738: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 207 MiB/s wr, 1.06k op/s 2024-09-21T03:56:59.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:56:58 smithi123 ceph-mon[34673]: pgmap v738: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 207 MiB/s wr, 1.06k op/s 2024-09-21T03:56:59.675 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fc03fa01', quiesce took 12.8 seconds. Will release after: 11.7 2024-09-21T03:57:01.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:01 smithi134 ceph-mon[35031]: pgmap v739: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 236 MiB/s wr, 1.18k op/s 2024-09-21T03:57:01.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:01 smithi123 ceph-mon[34673]: pgmap v739: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 236 MiB/s wr, 1.18k op/s 2024-09-21T03:57:01.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:01 smithi016 ceph-mon[30375]: pgmap v739: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 236 MiB/s wr, 1.18k op/s 2024-09-21T03:57:02.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:02 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:57:02.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:02 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:57:02.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:02 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:57:02.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:02 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:57:02.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:02 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:02.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:02 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:57:02.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:02 smithi134 ceph-mon[35031]: from='mgr.14154 ' entity='mgr.x' 2024-09-21T03:57:02.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:02 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:57:02.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:02 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:57:02.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:02 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:57:02.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:02 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:57:02.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:02 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:02.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:02 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:57:02.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:02 smithi123 ceph-mon[34673]: from='mgr.14154 ' entity='mgr.x' 2024-09-21T03:57:02.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:02 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:57:02.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:02 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:57:02.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:02 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:57:02.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:02 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:57:02.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:02 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:02.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:02 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:57:02.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:02 smithi016 ceph-mon[30375]: from='mgr.14154 ' entity='mgr.x' 2024-09-21T03:57:05.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:05 smithi123 ceph-mon[34673]: pgmap v740: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 223 MiB/s wr, 1.09k op/s 2024-09-21T03:57:05.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:05 smithi123 ceph-mon[34673]: pgmap v741: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 216 MiB/s wr, 1.07k op/s 2024-09-21T03:57:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:05 smithi016 ceph-mon[30375]: pgmap v740: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 223 MiB/s wr, 1.09k op/s 2024-09-21T03:57:05.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:05 smithi016 ceph-mon[30375]: pgmap v741: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 216 MiB/s wr, 1.07k op/s 2024-09-21T03:57:06.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:05 smithi134 ceph-mon[35031]: pgmap v740: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 223 MiB/s wr, 1.09k op/s 2024-09-21T03:57:06.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:05 smithi134 ceph-mon[35031]: pgmap v741: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 216 MiB/s wr, 1.07k op/s 2024-09-21T03:57:06.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:06 smithi123 ceph-mon[34673]: pgmap v742: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 245 MiB/s wr, 1.28k op/s 2024-09-21T03:57:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:06 smithi016 ceph-mon[30375]: pgmap v742: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 245 MiB/s wr, 1.28k op/s 2024-09-21T03:57:07.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:06 smithi134 ceph-mon[35031]: pgmap v742: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 245 MiB/s wr, 1.28k op/s 2024-09-21T03:57:09.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:08 smithi134 ceph-mon[35031]: pgmap v743: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 218 MiB/s wr, 1.19k op/s 2024-09-21T03:57:09.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:08 smithi123 ceph-mon[34673]: pgmap v743: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 218 MiB/s wr, 1.19k op/s 2024-09-21T03:57:09.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:08 smithi016 ceph-mon[30375]: pgmap v743: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 218 MiB/s wr, 1.19k op/s 2024-09-21T03:57:11.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:10 smithi134 ceph-mon[35031]: pgmap v744: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 247 MiB/s wr, 1.48k op/s 2024-09-21T03:57:11.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:57:11.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:10 smithi123 ceph-mon[34673]: pgmap v744: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 247 MiB/s wr, 1.48k op/s 2024-09-21T03:57:11.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:10 smithi016 ceph-mon[30375]: pgmap v744: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 247 MiB/s wr, 1.48k op/s 2024-09-21T03:57:12.048 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:57:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.407+0000 7f66e5c00640 1 -- 172.21.15.16:0/1643495151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e01050f0 msgr2=0x7f66e010ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.407+0000 7f66e5c00640 1 --2- 172.21.15.16:0/1643495151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e01050f0 0x7f66e010ba90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009ca0 tx=0x7f66d802a8f0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.408+0000 7f66e5c00640 1 -- 172.21.15.16:0/1643495151 shutdown_connections 2024-09-21T03:57:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.408+0000 7f66e5c00640 1 --2- 172.21.15.16:0/1643495151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e01050f0 0x7f66e010ba90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.408+0000 7f66e5c00640 1 --2- 172.21.15.16:0/1643495151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e0104750 0x7f66e0104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.408+0000 7f66e5c00640 1 --2- 172.21.15.16:0/1643495151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66e01036d0 0x7f66e0103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.408+0000 7f66e5c00640 1 -- 172.21.15.16:0/1643495151 >> 172.21.15.16:0/1643495151 conn(0x7f66e00ff030 msgr2=0x7f66e01014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.408+0000 7f66e5c00640 1 -- 172.21.15.16:0/1643495151 shutdown_connections 2024-09-21T03:57:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.408+0000 7f66e5c00640 1 -- 172.21.15.16:0/1643495151 wait complete. 2024-09-21T03:57:12.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.410+0000 7f66e5c00640 1 Processor -- start 2024-09-21T03:57:12.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.410+0000 7f66e5c00640 1 -- start start 2024-09-21T03:57:12.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.411+0000 7f66e5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e01036d0 0x7f66e01a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.411+0000 7f66e5c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66e0104750 0x7f66e01a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.411+0000 7f66e5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e01050f0 0x7f66e01a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.412+0000 7f66dea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e01036d0 0x7f66e01a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.412+0000 7f66dea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e01036d0 0x7f66e01a0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:52108/0 (socket says 172.21.15.16:52108) 2024-09-21T03:57:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.412+0000 7f66de000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66e0104750 0x7f66e01a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.412+0000 7f66df400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e01050f0 0x7f66e01a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.412+0000 7f66df400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e01050f0 0x7f66e01a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36000/0 (socket says 172.21.15.16:36000) 2024-09-21T03:57:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.412+0000 7f66de000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66e0104750 0x7f66e01a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:36248/0 (socket says 172.21.15.16:36248) 2024-09-21T03:57:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.412+0000 7f66dea00640 1 -- 172.21.15.16:0/1687216145 learned_addr learned my addr 172.21.15.16:0/1687216145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.412+0000 7f66e5c00640 1 -- 172.21.15.16:0/1687216145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66e010f350 con 0x7f66e01050f0 2024-09-21T03:57:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.412+0000 7f66e5c00640 1 -- 172.21.15.16:0/1687216145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66e010f1d0 con 0x7f66e01036d0 2024-09-21T03:57:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.412+0000 7f66e5c00640 1 -- 172.21.15.16:0/1687216145 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66e010f4d0 con 0x7f66e0104750 2024-09-21T03:57:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.413+0000 7f66dea00640 1 -- 172.21.15.16:0/1687216145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66e0104750 msgr2=0x7f66e01a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.413+0000 7f66dea00640 1 --2- 172.21.15.16:0/1687216145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66e0104750 0x7f66e01a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.413+0000 7f66dea00640 1 -- 172.21.15.16:0/1687216145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e01050f0 msgr2=0x7f66e01a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.413+0000 7f66dea00640 1 --2- 172.21.15.16:0/1687216145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e01050f0 0x7f66e01a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.413+0000 7f66dea00640 1 -- 172.21.15.16:0/1687216145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e01a1f60 con 0x7f66e01036d0 2024-09-21T03:57:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.413+0000 7f66de000640 1 --2- 172.21.15.16:0/1687216145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66e0104750 0x7f66e01a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:57:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.413+0000 7f66dea00640 1 --2- 172.21.15.16:0/1687216145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e01036d0 0x7f66e01a0520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f66c800d950 tx=0x7f66c800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.414+0000 7f66d7400640 1 -- 172.21.15.16:0/1687216145 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8017070 con 0x7f66e01036d0 2024-09-21T03:57:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.415+0000 7f66d7400640 1 -- 172.21.15.16:0/1687216145 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66c8009c30 con 0x7f66e01036d0 2024-09-21T03:57:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.415+0000 7f66d7400640 1 -- 172.21.15.16:0/1687216145 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c800f620 con 0x7f66e01036d0 2024-09-21T03:57:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.415+0000 7f66e5c00640 1 -- 172.21.15.16:0/1687216145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e01a2250 con 0x7f66e01036d0 2024-09-21T03:57:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.415+0000 7f66e5c00640 1 -- 172.21.15.16:0/1687216145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e01a8940 con 0x7f66e01036d0 2024-09-21T03:57:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.423+0000 7f66ecc55740 1 Processor -- start 2024-09-21T03:57:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.424+0000 7f66ecc55740 1 -- start start 2024-09-21T03:57:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.424+0000 7f66ecc55740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777451b0 0x555e77aec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.424+0000 7f66ecc55740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e7771c2f0 0x555e77aec740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.424+0000 7f66ecc55740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e777244d0 0x555e77aed290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.424+0000 7f66ecc55740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555e77a5f6b0 con 0x555e777244d0 2024-09-21T03:57:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.424+0000 7f66ecc55740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555e77a4eae0 con 0x555e7771c2f0 2024-09-21T03:57:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.424+0000 7f66ecc55740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555e77a528f0 con 0x555e777451b0 2024-09-21T03:57:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.425+0000 7f66de000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e777244d0 0x555e77aed290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.425+0000 7f66de000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e777244d0 0x555e77aed290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36008/0 (socket says 172.21.15.16:36008) 2024-09-21T03:57:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.425+0000 7f66de000640 1 -- 172.21.15.16:0/1888899733 learned_addr learned my addr 172.21.15.16:0/1888899733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.426+0000 7f66d7400640 1 -- 172.21.15.16:0/1687216145 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f66c8010460 con 0x7f66e01036d0 2024-09-21T03:57:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.426+0000 7f66de000640 1 -- 172.21.15.16:0/1888899733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777451b0 msgr2=0x555e77aec0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.426+0000 7f66de000640 1 --2- 172.21.15.16:0/1888899733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777451b0 0x555e77aec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.426+0000 7f66dea00640 1 --2- 172.21.15.16:0/1888899733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e7771c2f0 0x555e77aec740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.427+0000 7f66de000640 1 -- 172.21.15.16:0/1888899733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e7771c2f0 msgr2=0x555e77aec740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.427+0000 7f66de000640 1 --2- 172.21.15.16:0/1888899733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e7771c2f0 0x555e77aec740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.427+0000 7f66de000640 1 -- 172.21.15.16:0/1888899733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e77a1b780 con 0x555e777244d0 2024-09-21T03:57:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.428+0000 7f66de000640 1 --2- 172.21.15.16:0/1888899733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e777244d0 0x555e77aed290 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f66d00115f0 tx=0x7f66d0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619561d2a63eaab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.428+0000 7f66cfe00640 1 -- 172.21.15.16:0/1888899733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d001e070 con 0x555e777244d0 2024-09-21T03:57:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.429+0000 7f66cfe00640 1 -- 172.21.15.16:0/1888899733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d000faf0 con 0x555e777244d0 2024-09-21T03:57:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.429+0000 7f66cfe00640 1 -- 172.21.15.16:0/1888899733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d001e070 con 0x555e777244d0 2024-09-21T03:57:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.430+0000 7f66ecc55740 1 -- 172.21.15.16:0/1888899733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e777244d0 msgr2=0x555e77aed290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.430+0000 7f66ecc55740 1 --2- 172.21.15.16:0/1888899733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e777244d0 0x555e77aed290 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f66d00115f0 tx=0x7f66d0011ab0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.430+0000 7f66d7400640 1 --2- 172.21.15.16:0/1687216145 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f66c40b66e0 0x7f66c40b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.431+0000 7f66df400640 1 --2- 172.21.15.16:0/1687216145 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f66c40b66e0 0x7f66c40b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.432+0000 7f66d7400640 1 -- 172.21.15.16:0/1687216145 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f66c80e71a0 con 0x7f66e01036d0 2024-09-21T03:57:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.435+0000 7f66ecc55740 1 -- 172.21.15.16:0/1888899733 shutdown_connections 2024-09-21T03:57:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.435+0000 7f66ecc55740 1 --2- 172.21.15.16:0/1888899733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e777244d0 0x555e77aed290 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.435+0000 7f66ecc55740 1 --2- 172.21.15.16:0/1888899733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e7771c2f0 0x555e77aec740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.435+0000 7f66ecc55740 1 --2- 172.21.15.16:0/1888899733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777451b0 0x555e77aec0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.435+0000 7f66ecc55740 1 -- 172.21.15.16:0/1888899733 >> 172.21.15.16:0/1888899733 conn(0x555e77739300 msgr2=0x555e77ae7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.435+0000 7f66ecc55740 1 -- 172.21.15.16:0/1888899733 shutdown_connections 2024-09-21T03:57:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.436+0000 7f66ecc55740 1 -- 172.21.15.16:0/1888899733 wait complete. 2024-09-21T03:57:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.437+0000 7f66ecc55740 1 -- start start 2024-09-21T03:57:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.437+0000 7f66ecc55740 1 Processor -- start 2024-09-21T03:57:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.438+0000 7f66df400640 1 --2- 172.21.15.16:0/1687216145 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f66c40b66e0 0x7f66c40b8bf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f66d8006010 tx=0x7f66d8009360 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.438+0000 7f66ecc55740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e7771c2f0 0x555e77af5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.438+0000 7f66ecc55740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777244d0 0x555e77af5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.438+0000 7f66ecc55740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e777451b0 0x555e77af6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.438+0000 7f66ecc55740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555e77986ab0 con 0x555e777451b0 2024-09-21T03:57:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.438+0000 7f66ecc55740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555e779f5570 con 0x555e7771c2f0 2024-09-21T03:57:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.438+0000 7f66ecc55740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555e77aeddb0 con 0x555e777244d0 2024-09-21T03:57:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.438+0000 7f66ecc55740 10 client.0 _ll_register_callbacks cb 0x555e77a04100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:57:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.439+0000 7f66df400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e777451b0 0x555e77af6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.439+0000 7f66df400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e777451b0 0x555e77af6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36012/0 (socket says 172.21.15.16:36012) 2024-09-21T03:57:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.439+0000 7f66df400640 1 -- 172.21.15.16:0/3943567242 learned_addr learned my addr 172.21.15.16:0/3943567242 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.439+0000 7f66de000640 1 --2- 172.21.15.16:0/3943567242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777244d0 0x555e77af5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.440+0000 7f66dea00640 1 --2- 172.21.15.16:0/3943567242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e7771c2f0 0x555e77af5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.440+0000 7f66df400640 1 -- 172.21.15.16:0/3943567242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777244d0 msgr2=0x555e77af5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.440+0000 7f66df400640 1 --2- 172.21.15.16:0/3943567242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777244d0 0x555e77af5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.440+0000 7f66df400640 1 -- 172.21.15.16:0/3943567242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e7771c2f0 msgr2=0x555e77af5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.440+0000 7f66df400640 1 --2- 172.21.15.16:0/3943567242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e7771c2f0 0x555e77af5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.440+0000 7f66df400640 1 -- 172.21.15.16:0/3943567242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e777b0790 con 0x555e777451b0 2024-09-21T03:57:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.440+0000 7f66dea00640 1 --2- 172.21.15.16:0/3943567242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e7771c2f0 0x555e77af5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:57:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.442+0000 7f66df400640 1 --2- 172.21.15.16:0/3943567242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e777451b0 0x555e77af6a90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f66d8035000 tx=0x7f66d802a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ab3adf109fc645 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.442+0000 7f66ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:57:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.444+0000 7f66ccc00640 1 -- 172.21.15.16:0/3943567242 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8030db0 con 0x555e777451b0 2024-09-21T03:57:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.444+0000 7f66ccc00640 1 -- 172.21.15.16:0/3943567242 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f66d8003970 con 0x555e777451b0 2024-09-21T03:57:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.445+0000 7f66ecc55740 1 -- 172.21.15.16:0/3943567242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66d000faf0 con 0x555e777451b0 2024-09-21T03:57:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.446+0000 7f66ccc00640 1 -- 172.21.15.16:0/3943567242 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f66d8003b70 con 0x555e777451b0 2024-09-21T03:57:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.446+0000 7f66ecc55740 10 client.16404 fetch_fsmap learned FSMap version 30 2024-09-21T03:57:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.446+0000 7f66ecc55740 1 -- 172.21.15.16:0/3943567242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555e77829620 con 0x555e777451b0 2024-09-21T03:57:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.447+0000 7f66ccc00640 1 -- 172.21.15.16:0/3943567242 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d000faf0 con 0x555e777451b0 2024-09-21T03:57:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.447+0000 7f66ccc00640 1 -- 172.21.15.16:0/3943567242 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f66d8051470 con 0x555e777451b0 2024-09-21T03:57:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.449+0000 7f66ecc55740 10 client.16404 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:57:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.449+0000 7f66ecc55740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:57:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.449+0000 7f66ecc55740 20 client.16404 populate_metadata read hostname 'smithi016' 2024-09-21T03:57:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.449+0000 7f66ecc55740 1 --2- 172.21.15.16:0/3943567242 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x555e77703d80 0x555e77b13ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.449+0000 7f66ecc55740 4 client.16404 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:57:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.449+0000 7f66ecc55740 1 -- 172.21.15.16:0/3943567242 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e77a61600 con 0x555e77703d80 2024-09-21T03:57:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.450+0000 7f66dea00640 1 --2- 172.21.15.16:0/3943567242 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x555e77703d80 0x555e77b13ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.452+0000 7f66dea00640 1 --2- 172.21.15.16:0/3943567242 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x555e77703d80 0x555e77b13ef0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50733a39381b5898 server_cookie=6dbbeee2e392ef6c in_seq=0 out_seq=0 2024-09-21T03:57:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.452+0000 7f66ccc00640 10 client.16404 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.454+0000 7f66ccc00640 1 -- 172.21.15.16:0/3943567242 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555e77a61600 con 0x555e77703d80 2024-09-21T03:57:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.455+0000 7f66ccc00640 10 client.16404 handle_command_reply: tid=0 2024-09-21T03:57:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.455+0000 7f66ecc55740 1 client.16404 shutdown 2024-09-21T03:57:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.457+0000 7f66ecc55740 1 -- 172.21.15.16:0/3943567242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e777451b0 msgr2=0x555e77af6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.457+0000 7f66ecc55740 1 --2- 172.21.15.16:0/3943567242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e777451b0 0x555e77af6a90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f66d8035000 tx=0x7f66d802a8f0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.459+0000 7f66ecc55740 1 -- 172.21.15.16:0/3943567242 shutdown_connections 2024-09-21T03:57:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.459+0000 7f66ecc55740 1 --2- 172.21.15.16:0/3943567242 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x555e77703d80 0x555e77b13ef0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.459+0000 7f66ecc55740 1 --2- 172.21.15.16:0/3943567242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e777451b0 0x555e77af6a90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.459+0000 7f66ecc55740 1 --2- 172.21.15.16:0/3943567242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777244d0 0x555e77af5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.459+0000 7f66ecc55740 1 --2- 172.21.15.16:0/3943567242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e7771c2f0 0x555e77af5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.459+0000 7f66ccc00640 0 client.16404 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.459+0000 7f66ecc55740 1 -- 172.21.15.16:0/3943567242 >> 172.21.15.16:0/3943567242 conn(0x555e77739300 msgr2=0x555e77ae60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.460+0000 7f66ecc55740 1 -- 172.21.15.16:0/3943567242 shutdown_connections 2024-09-21T03:57:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.460+0000 7f66ecc55740 1 -- 172.21.15.16:0/3943567242 wait complete. 2024-09-21T03:57:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.460+0000 7f66ecc55740 20 client.16404 trim_cache size 0 max 16384 2024-09-21T03:57:12.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.504+0000 7f66ecc55740 1 Processor -- start 2024-09-21T03:57:12.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.504+0000 7f66ecc55740 1 -- start start 2024-09-21T03:57:12.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.504+0000 7f66ecc55740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e7771c2f0 0x555e77af3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.504+0000 7f66ecc55740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e777244d0 0x555e77af42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.505+0000 7f66ecc55740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777451b0 0x555e77aebcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.505+0000 7f66ecc55740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66e01aa650 con 0x555e7771c2f0 2024-09-21T03:57:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.505+0000 7f66ecc55740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66e01ac190 con 0x555e777244d0 2024-09-21T03:57:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.505+0000 7f66ecc55740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66e01aad20 con 0x555e777451b0 2024-09-21T03:57:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.506+0000 7f66df400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777451b0 0x555e77aebcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.506+0000 7f66df400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777451b0 0x555e77aebcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:36266/0 (socket says 172.21.15.16:36266) 2024-09-21T03:57:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.506+0000 7f66df400640 1 -- 172.21.15.16:0/2480577133 learned_addr learned my addr 172.21.15.16:0/2480577133 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.507+0000 7f66df400640 1 -- 172.21.15.16:0/2480577133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e777244d0 msgr2=0x555e77af42d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:57:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.507+0000 7f66df400640 1 --2- 172.21.15.16:0/2480577133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e777244d0 0x555e77af42d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.507+0000 7f66df400640 1 -- 172.21.15.16:0/2480577133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e7771c2f0 msgr2=0x555e77af3d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:57:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.507+0000 7f66df400640 1 --2- 172.21.15.16:0/2480577133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e7771c2f0 0x555e77af3d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.507+0000 7f66df400640 1 -- 172.21.15.16:0/2480577133 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e77af55f0 con 0x555e777451b0 2024-09-21T03:57:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.507+0000 7f66df400640 1 --2- 172.21.15.16:0/2480577133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777451b0 0x555e77aebcc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f66d802aee0 tx=0x7f66d8031500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e4ff3baecd6eedd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.507+0000 7f66cf400640 1 -- 172.21.15.16:0/2480577133 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8051470 con 0x555e777451b0 2024-09-21T03:57:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.507+0000 7f66cf400640 1 -- 172.21.15.16:0/2480577133 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d8036040 con 0x555e777451b0 2024-09-21T03:57:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.508+0000 7f66cf400640 1 -- 172.21.15.16:0/2480577133 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d805b680 con 0x555e777451b0 2024-09-21T03:57:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.509+0000 7f66ecc55740 1 -- 172.21.15.16:0/2480577133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777451b0 msgr2=0x555e77aebcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:12.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.509+0000 7f66ecc55740 1 --2- 172.21.15.16:0/2480577133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777451b0 0x555e77aebcc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f66d802aee0 tx=0x7f66d8031500 comp rx=0 tx=0).stop 2024-09-21T03:57:12.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.509+0000 7f66ecc55740 1 -- 172.21.15.16:0/2480577133 shutdown_connections 2024-09-21T03:57:12.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.509+0000 7f66ecc55740 1 --2- 172.21.15.16:0/2480577133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777451b0 0x555e77aebcc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.509+0000 7f66ecc55740 1 --2- 172.21.15.16:0/2480577133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e777244d0 0x555e77af42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.509+0000 7f66ecc55740 1 --2- 172.21.15.16:0/2480577133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e7771c2f0 0x555e77af3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.509+0000 7f66ecc55740 1 -- 172.21.15.16:0/2480577133 >> 172.21.15.16:0/2480577133 conn(0x555e77739300 msgr2=0x555e77af4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:12.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.510+0000 7f66ecc55740 1 -- 172.21.15.16:0/2480577133 shutdown_connections 2024-09-21T03:57:12.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.510+0000 7f66ecc55740 1 -- 172.21.15.16:0/2480577133 wait complete. 2024-09-21T03:57:12.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.510+0000 7f66ecc55740 1 -- start start 2024-09-21T03:57:12.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.511+0000 7f66ecc55740 1 Processor -- start 2024-09-21T03:57:12.516 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.511+0000 7f66ecc55740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e7771c2f0 0x555e77afc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.516 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.511+0000 7f66ecc55740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777244d0 0x555e77afcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.511+0000 7f66ecc55740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e777451b0 0x555e77afb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.511+0000 7f66ecc55740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66e01abac0 con 0x555e7771c2f0 2024-09-21T03:57:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.511+0000 7f66ecc55740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66e01ab3f0 con 0x555e777451b0 2024-09-21T03:57:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.511+0000 7f66ecc55740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66e0075100 con 0x555e777244d0 2024-09-21T03:57:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.511+0000 7f66ecc55740 10 client.0 _ll_register_callbacks cb 0x7f66d8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:57:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.512+0000 7f66dea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e7771c2f0 0x555e77afc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.512+0000 7f66dea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e7771c2f0 0x555e77afc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36016/0 (socket says 172.21.15.16:36016) 2024-09-21T03:57:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.512+0000 7f66dea00640 1 -- 172.21.15.16:0/2693038061 learned_addr learned my addr 172.21.15.16:0/2693038061 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.512+0000 7f66df400640 1 --2- 172.21.15.16:0/2693038061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e777451b0 0x555e77afb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.512+0000 7f66de000640 1 --2- 172.21.15.16:0/2693038061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777244d0 0x555e77afcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.512+0000 7f66df400640 1 -- 172.21.15.16:0/2693038061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777244d0 msgr2=0x555e77afcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.512+0000 7f66df400640 1 --2- 172.21.15.16:0/2693038061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777244d0 0x555e77afcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.513+0000 7f66df400640 1 -- 172.21.15.16:0/2693038061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e7771c2f0 msgr2=0x555e77afc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.513+0000 7f66df400640 1 --2- 172.21.15.16:0/2693038061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e7771c2f0 0x555e77afc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.513+0000 7f66df400640 1 -- 172.21.15.16:0/2693038061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e77afb740 con 0x555e777451b0 2024-09-21T03:57:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.513+0000 7f66dea00640 1 --2- 172.21.15.16:0/2693038061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e7771c2f0 0x555e77afc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:57:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.513+0000 7f66de000640 1 --2- 172.21.15.16:0/2693038061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777244d0 0x555e77afcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:57:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.513+0000 7f66df400640 1 --2- 172.21.15.16:0/2693038061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e777451b0 0x555e77afb200 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f66d802ae10 tx=0x7f66d8009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=101f0c020c1ed0c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.513+0000 7f66ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:57:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.513+0000 7f66ccc00640 1 -- 172.21.15.16:0/2693038061 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8064420 con 0x555e777451b0 2024-09-21T03:57:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.513+0000 7f66ecc55740 1 -- 172.21.15.16:0/2693038061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66d8036040 con 0x555e777451b0 2024-09-21T03:57:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.513+0000 7f66ccc00640 1 -- 172.21.15.16:0/2693038061 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f66d80645c0 con 0x555e777451b0 2024-09-21T03:57:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.514+0000 7f66ccc00640 1 -- 172.21.15.16:0/2693038061 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f66d80647c0 con 0x555e777451b0 2024-09-21T03:57:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.514+0000 7f66ecc55740 10 client.26669 fetch_fsmap learned FSMap version 30 2024-09-21T03:57:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.514+0000 7f66ecc55740 1 -- 172.21.15.16:0/2693038061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555e77b041a0 con 0x555e777451b0 2024-09-21T03:57:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.514+0000 7f66ccc00640 1 -- 172.21.15.16:0/2693038061 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d8036040 con 0x555e777451b0 2024-09-21T03:57:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.514+0000 7f66ccc00640 1 -- 172.21.15.16:0/2693038061 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f66d806c030 con 0x555e777451b0 2024-09-21T03:57:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.514+0000 7f66ecc55740 10 client.26669 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:57:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.514+0000 7f66ecc55740 10 client.26669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:57:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.514+0000 7f66ecc55740 20 client.26669 populate_metadata read hostname 'smithi016' 2024-09-21T03:57:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.514+0000 7f66ecc55740 1 --2- 172.21.15.16:0/2693038061 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x555e776d91c0 0x555e77b06bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.514+0000 7f66ecc55740 4 client.26669 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:57:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.514+0000 7f66ecc55740 1 -- 172.21.15.16:0/2693038061 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e77a5d0b0 con 0x555e776d91c0 2024-09-21T03:57:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.515+0000 7f66dea00640 1 --2- 172.21.15.16:0/2693038061 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x555e776d91c0 0x555e77b06bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.515+0000 7f66dea00640 1 --2- 172.21.15.16:0/2693038061 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x555e776d91c0 0x555e77b06bc0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ec439140defc7b4 server_cookie=fb99553130eec906 in_seq=0 out_seq=0 2024-09-21T03:57:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.515+0000 7f66ccc00640 10 client.26669 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.534+0000 7f66ccc00640 1 -- 172.21.15.16:0/2693038061 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e77a5d0b0 con 0x555e776d91c0 2024-09-21T03:57:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.534+0000 7f66ccc00640 10 client.26669 handle_command_reply: tid=0 2024-09-21T03:57:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.537+0000 7f66ecc55740 1 client.26669 shutdown 2024-09-21T03:57:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.538+0000 7f66ecc55740 1 -- 172.21.15.16:0/2693038061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e777451b0 msgr2=0x555e77afb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.538+0000 7f66ecc55740 1 --2- 172.21.15.16:0/2693038061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e777451b0 0x555e77afb200 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f66d802ae10 tx=0x7f66d8009bd0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.540+0000 7f66ecc55740 1 -- 172.21.15.16:0/2693038061 shutdown_connections 2024-09-21T03:57:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.541+0000 7f66ecc55740 1 --2- 172.21.15.16:0/2693038061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555e777451b0 0x555e77afb200 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.541+0000 7f66ecc55740 1 --2- 172.21.15.16:0/2693038061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e777244d0 0x555e77afcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.541+0000 7f66ecc55740 1 --2- 172.21.15.16:0/2693038061 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x555e776d91c0 0x555e77b06bc0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.541+0000 7f66ecc55740 1 --2- 172.21.15.16:0/2693038061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e7771c2f0 0x555e77afc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.541+0000 7f66ecc55740 1 -- 172.21.15.16:0/2693038061 >> 172.21.15.16:0/2693038061 conn(0x555e77739300 msgr2=0x555e77af3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.541+0000 7f66ccc00640 0 client.26669 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.543+0000 7f66ecc55740 1 -- 172.21.15.16:0/2693038061 shutdown_connections 2024-09-21T03:57:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.543+0000 7f66ecc55740 1 -- 172.21.15.16:0/2693038061 wait complete. 2024-09-21T03:57:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.543+0000 7f66ecc55740 20 client.26669 trim_cache size 0 max 16384 2024-09-21T03:57:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:57:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:57:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:57:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:57:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.545+0000 7f66e5c00640 1 -- 172.21.15.16:0/1687216145 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f66c40b66e0 msgr2=0x7f66c40b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.545+0000 7f66e5c00640 1 --2- 172.21.15.16:0/1687216145 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f66c40b66e0 0x7f66c40b8bf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f66d8006010 tx=0x7f66d8009360 comp rx=0 tx=0).stop 2024-09-21T03:57:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.545+0000 7f66e5c00640 1 -- 172.21.15.16:0/1687216145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e01036d0 msgr2=0x7f66e01a0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.545+0000 7f66e5c00640 1 --2- 172.21.15.16:0/1687216145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e01036d0 0x7f66e01a0520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f66c800d950 tx=0x7f66c800de10 comp rx=0 tx=0).stop 2024-09-21T03:57:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.546+0000 7f66e5c00640 1 -- 172.21.15.16:0/1687216145 shutdown_connections 2024-09-21T03:57:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.546+0000 7f66e5c00640 1 --2- 172.21.15.16:0/1687216145 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f66c40b66e0 0x7f66c40b8bf0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.546+0000 7f66e5c00640 1 --2- 172.21.15.16:0/1687216145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e01050f0 0x7f66e01a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.546+0000 7f66e5c00640 1 --2- 172.21.15.16:0/1687216145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66e0104750 0x7f66e01a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.546+0000 7f66e5c00640 1 --2- 172.21.15.16:0/1687216145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e01036d0 0x7f66e01a0520 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.546+0000 7f66e5c00640 1 -- 172.21.15.16:0/1687216145 >> 172.21.15.16:0/1687216145 conn(0x7f66e00ff030 msgr2=0x7f66e0105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:12.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.550+0000 7f66e5c00640 1 -- 172.21.15.16:0/1687216145 shutdown_connections 2024-09-21T03:57:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:12.551+0000 7f66e5c00640 1 -- 172.21.15.16:0/1687216145 wait complete. 2024-09-21T03:57:13.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:12 smithi134 ceph-mon[35031]: pgmap v745: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 228 MiB/s wr, 1.53k op/s 2024-09-21T03:57:13.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:12 smithi123 ceph-mon[34673]: pgmap v745: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 228 MiB/s wr, 1.53k op/s 2024-09-21T03:57:13.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:12 smithi016 ceph-mon[30375]: pgmap v745: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 228 MiB/s wr, 1.53k op/s 2024-09-21T03:57:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.050+0000 7fa306600640 1 -- 172.21.15.16:0/2538402135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa30010dc20 msgr2=0x7fa300110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.050+0000 7fa306600640 1 --2- 172.21.15.16:0/2538402135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa30010dc20 0x7fa300110110 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009900 tx=0x7fa2ec02a410 comp rx=0 tx=0).stop 2024-09-21T03:57:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.053+0000 7fa306600640 1 -- 172.21.15.16:0/2538402135 shutdown_connections 2024-09-21T03:57:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.053+0000 7fa306600640 1 --2- 172.21.15.16:0/2538402135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa30010dc20 0x7fa300110110 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:15.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.053+0000 7fa306600640 1 --2- 172.21.15.16:0/2538402135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa300105110 0x7fa30010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.055+0000 7fa306600640 1 --2- 172.21.15.16:0/2538402135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3001047c0 0x7fa300104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.055+0000 7fa306600640 1 -- 172.21.15.16:0/2538402135 >> 172.21.15.16:0/2538402135 conn(0x7fa3000fe2c0 msgr2=0x7fa300100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.056+0000 7fa306600640 1 -- 172.21.15.16:0/2538402135 shutdown_connections 2024-09-21T03:57:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.057+0000 7fa306600640 1 -- 172.21.15.16:0/2538402135 wait complete. 2024-09-21T03:57:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.058+0000 7fa306600640 1 Processor -- start 2024-09-21T03:57:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.062+0000 7fa306600640 1 -- start start 2024-09-21T03:57:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.064+0000 7fa306600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3001047c0 0x7fa300109440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.064+0000 7fa306600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa300105110 0x7fa300105720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.064+0000 7fa306600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa30010dc20 0x7fa300105e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.064+0000 7fa306600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa300113900 con 0x7fa3001047c0 2024-09-21T03:57:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.064+0000 7fa306600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa300113780 con 0x7fa300105110 2024-09-21T03:57:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.065+0000 7fa305c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa30010dc20 0x7fa300105e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.065+0000 7fa305c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa30010dc20 0x7fa300105e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:36292/0 (socket says 172.21.15.16:36292) 2024-09-21T03:57:15.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.065+0000 7fa305c00640 1 -- 172.21.15.16:0/1595914065 learned_addr learned my addr 172.21.15.16:0/1595914065 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:15.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.065+0000 7fa306600640 1 -- 172.21.15.16:0/1595914065 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa300113a80 con 0x7fa30010dc20 2024-09-21T03:57:15.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.065+0000 7fa305200640 1 --2- 172.21.15.16:0/1595914065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3001047c0 0x7fa300109440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:15.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.066+0000 7fa2ffe00640 1 --2- 172.21.15.16:0/1595914065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa300105110 0x7fa300105720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:15.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.068+0000 7fa2ffe00640 1 -- 172.21.15.16:0/1595914065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa30010dc20 msgr2=0x7fa300105e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:15.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.068+0000 7fa2ffe00640 1 --2- 172.21.15.16:0/1595914065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa30010dc20 0x7fa300105e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:15.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.068+0000 7fa2ffe00640 1 -- 172.21.15.16:0/1595914065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3001047c0 msgr2=0x7fa300109440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:15.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.068+0000 7fa2ffe00640 1 --2- 172.21.15.16:0/1595914065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3001047c0 0x7fa300109440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:15.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.068+0000 7fa2ffe00640 1 -- 172.21.15.16:0/1595914065 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3001aca00 con 0x7fa300105110 2024-09-21T03:57:15.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.068+0000 7fa305c00640 1 --2- 172.21.15.16:0/1595914065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa30010dc20 0x7fa300105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:57:15.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.068+0000 7fa2ffe00640 1 --2- 172.21.15.16:0/1595914065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa300105110 0x7fa300105720 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa30006aa90 tx=0x7fa2f000cdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:15.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.068+0000 7fa305200640 1 --2- 172.21.15.16:0/1595914065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3001047c0 0x7fa300109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:57:15.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.069+0000 7fa2fd600640 1 -- 172.21.15.16:0/1595914065 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0007bb0 con 0x7fa300105110 2024-09-21T03:57:15.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.069+0000 7fa306600640 1 -- 172.21.15.16:0/1595914065 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3001acbd0 con 0x7fa300105110 2024-09-21T03:57:15.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.069+0000 7fa306600640 1 -- 172.21.15.16:0/1595914065 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3001ad130 con 0x7fa300105110 2024-09-21T03:57:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.069+0000 7fa2fd600640 1 -- 172.21.15.16:0/1595914065 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f0007d50 con 0x7fa300105110 2024-09-21T03:57:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.069+0000 7fa2fd600640 1 -- 172.21.15.16:0/1595914065 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f000e6e0 con 0x7fa300105110 2024-09-21T03:57:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.071+0000 7fa306600640 1 -- 172.21.15.16:0/1595914065 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e00052c0 con 0x7fa300105110 2024-09-21T03:57:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.074+0000 7fa2fd600640 1 -- 172.21.15.16:0/1595914065 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa2f0005030 con 0x7fa300105110 2024-09-21T03:57:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.077+0000 7fa2fd600640 1 --2- 172.21.15.16:0/1595914065 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa2e40b6610 0x7fa2e40b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.077+0000 7fa2fd600640 1 -- 172.21.15.16:0/1595914065 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fa2f00e7580 con 0x7fa300105110 2024-09-21T03:57:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.078+0000 7fa305200640 1 --2- 172.21.15.16:0/1595914065 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa2e40b6610 0x7fa2e40b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.080+0000 7fa2fd600640 1 -- 172.21.15.16:0/1595914065 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa2f001d060 con 0x7fa300105110 2024-09-21T03:57:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.081+0000 7fa305200640 1 --2- 172.21.15.16:0/1595914065 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa2e40b6610 0x7fa2e40b8b20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa300109190 tx=0x7fa2f8009570 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:15.321 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:14 smithi016 ceph-mon[30375]: pgmap v746: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 219 MiB/s wr, 1.60k op/s 2024-09-21T03:57:15.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:14 smithi134 ceph-mon[35031]: pgmap v746: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 219 MiB/s wr, 1.60k op/s 2024-09-21T03:57:15.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:14 smithi123 ceph-mon[34673]: pgmap v746: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 219 MiB/s wr, 1.60k op/s 2024-09-21T03:57:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.427+0000 7fa306600640 1 -- 172.21.15.16:0/1595914065 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2e00055b0 con 0x7fa300105110 2024-09-21T03:57:15.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.428+0000 7fa2fd600640 1 -- 172.21.15.16:0/1595914065 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7fa2f0014030 con 0x7fa300105110 2024-09-21T03:57:15.431 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:57:15.431 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:57:15.431 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:57:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.430+0000 7fa306600640 1 -- 172.21.15.16:0/1595914065 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa2e40b6610 msgr2=0x7fa2e40b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.430+0000 7fa306600640 1 --2- 172.21.15.16:0/1595914065 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa2e40b6610 0x7fa2e40b8b20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa300109190 tx=0x7fa2f8009570 comp rx=0 tx=0).stop 2024-09-21T03:57:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.430+0000 7fa306600640 1 -- 172.21.15.16:0/1595914065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa300105110 msgr2=0x7fa300105720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.430+0000 7fa306600640 1 --2- 172.21.15.16:0/1595914065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa300105110 0x7fa300105720 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa30006aa90 tx=0x7fa2f000cdd0 comp rx=0 tx=0).stop 2024-09-21T03:57:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.431+0000 7fa306600640 1 -- 172.21.15.16:0/1595914065 shutdown_connections 2024-09-21T03:57:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.431+0000 7fa306600640 1 --2- 172.21.15.16:0/1595914065 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa2e40b6610 0x7fa2e40b8b20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.431+0000 7fa306600640 1 --2- 172.21.15.16:0/1595914065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa30010dc20 0x7fa300105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.431+0000 7fa306600640 1 --2- 172.21.15.16:0/1595914065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa300105110 0x7fa300105720 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.431+0000 7fa306600640 1 --2- 172.21.15.16:0/1595914065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3001047c0 0x7fa300109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.431+0000 7fa306600640 1 -- 172.21.15.16:0/1595914065 >> 172.21.15.16:0/1595914065 conn(0x7fa3000fe2c0 msgr2=0x7fa3001023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.431+0000 7fa306600640 1 -- 172.21.15.16:0/1595914065 shutdown_connections 2024-09-21T03:57:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:15.431+0000 7fa306600640 1 -- 172.21.15.16:0/1595914065 wait complete. 2024-09-21T03:57:16.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:15 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1595914065' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:57:16.347 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db --set-id fc03fa01 --release --await' 2024-09-21T03:57:16.348 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db --set-id fc03fa01 --release --await 2024-09-21T03:57:16.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:15 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1595914065' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:57:16.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:15 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1595914065' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:57:16.769 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:57:17.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:16 smithi016 ceph-mon[30375]: pgmap v747: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 248 MiB/s wr, 2.00k op/s 2024-09-21T03:57:17.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:16 smithi134 ceph-mon[35031]: pgmap v747: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 248 MiB/s wr, 2.00k op/s 2024-09-21T03:57:17.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:16 smithi123 ceph-mon[34673]: pgmap v747: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 248 MiB/s wr, 2.00k op/s 2024-09-21T03:57:17.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.532+0000 7f6239800640 1 -- 172.21.15.16:0/3993168728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6234077380 msgr2=0x7f62340757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:17.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.532+0000 7f6239800640 1 --2- 172.21.15.16:0/3993168728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6234077380 0x7f62340757b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6224009900 tx=0x7f622402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.533+0000 7f6239800640 1 -- 172.21.15.16:0/3993168728 shutdown_connections 2024-09-21T03:57:17.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.533+0000 7f6239800640 1 --2- 172.21.15.16:0/3993168728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6234114430 0x7f6234116900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.533+0000 7f6239800640 1 --2- 172.21.15.16:0/3993168728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6234075dd0 0x7f62340761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.533+0000 7f6239800640 1 --2- 172.21.15.16:0/3993168728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6234077380 0x7f62340757b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.533+0000 7f6239800640 1 -- 172.21.15.16:0/3993168728 >> 172.21.15.16:0/3993168728 conn(0x7f623407b850 msgr2=0x7f623407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:17.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.533+0000 7f6239800640 1 -- 172.21.15.16:0/3993168728 shutdown_connections 2024-09-21T03:57:17.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.533+0000 7f6239800640 1 -- 172.21.15.16:0/3993168728 wait complete. 2024-09-21T03:57:17.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.534+0000 7f6239800640 1 Processor -- start 2024-09-21T03:57:17.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.534+0000 7f6239800640 1 -- start start 2024-09-21T03:57:17.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.534+0000 7f6239800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6234075dd0 0x7f62341a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.535+0000 7f6239800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6234077380 0x7f62341a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.535+0000 7f6239800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6234114430 0x7f62341a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.535+0000 7f6239800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f623411a0b0 con 0x7f6234077380 2024-09-21T03:57:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.535+0000 7f6239800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6234119f30 con 0x7f6234075dd0 2024-09-21T03:57:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.535+0000 7f6239800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f623411a230 con 0x7f6234114430 2024-09-21T03:57:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.535+0000 7f6232a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6234075dd0 0x7f62341a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.535+0000 7f6232000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6234077380 0x7f62341a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.535+0000 7f6232a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6234075dd0 0x7f62341a0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:52164/0 (socket says 172.21.15.16:52164) 2024-09-21T03:57:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.535+0000 7f6232a00640 1 -- 172.21.15.16:0/2121525849 learned_addr learned my addr 172.21.15.16:0/2121525849 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.535+0000 7f6233400640 1 --2- 172.21.15.16:0/2121525849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6234114430 0x7f62341a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.535+0000 7f6232a00640 1 -- 172.21.15.16:0/2121525849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6234114430 msgr2=0x7f62341a62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.535+0000 7f6232a00640 1 --2- 172.21.15.16:0/2121525849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6234114430 0x7f62341a62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.535+0000 7f6232a00640 1 -- 172.21.15.16:0/2121525849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6234077380 msgr2=0x7f62341a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.535+0000 7f6232a00640 1 --2- 172.21.15.16:0/2121525849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6234077380 0x7f62341a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.535+0000 7f6232a00640 1 -- 172.21.15.16:0/2121525849 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62341a69f0 con 0x7f6234075dd0 2024-09-21T03:57:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.535+0000 7f6232000640 1 --2- 172.21.15.16:0/2121525849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6234077380 0x7f62341a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:57:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.536+0000 7f6232a00640 1 --2- 172.21.15.16:0/2121525849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6234075dd0 0x7f62341a0a80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6224005ab0 tx=0x7f6224003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.536+0000 7f622b400640 1 -- 172.21.15.16:0/2121525849 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622403f070 con 0x7f6234075dd0 2024-09-21T03:57:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.536+0000 7f6239800640 1 -- 172.21.15.16:0/2121525849 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62341a6c80 con 0x7f6234075dd0 2024-09-21T03:57:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.536+0000 7f622b400640 1 -- 172.21.15.16:0/2121525849 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6224031d60 con 0x7f6234075dd0 2024-09-21T03:57:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.536+0000 7f622b400640 1 -- 172.21.15.16:0/2121525849 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224035030 con 0x7f6234075dd0 2024-09-21T03:57:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.536+0000 7f6239800640 1 -- 172.21.15.16:0/2121525849 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62341ad440 con 0x7f6234075dd0 2024-09-21T03:57:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.540+0000 7f622b400640 1 -- 172.21.15.16:0/2121525849 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f6224042070 con 0x7f6234075dd0 2024-09-21T03:57:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.541+0000 7f622b400640 1 --2- 172.21.15.16:0/2121525849 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f62180b6590 0x7f62180b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.542+0000 7f622b400640 1 -- 172.21.15.16:0/2121525849 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f6224107e90 con 0x7f6234075dd0 2024-09-21T03:57:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.542+0000 7f6232000640 1 --2- 172.21.15.16:0/2121525849 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f62180b6590 0x7f62180b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.542+0000 7f6232000640 1 --2- 172.21.15.16:0/2121525849 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f62180b6590 0x7f62180b8aa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f62341157f0 tx=0x7f622000b390 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:17.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.547+0000 7f62405cd740 1 Processor -- start 2024-09-21T03:57:17.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.547+0000 7f62405cd740 1 -- start start 2024-09-21T03:57:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.547+0000 7f62405cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f4610520 0x5601f49b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.547+0000 7f62405cd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f45e7660 0x5601f49b7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.547+0000 7f62405cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45ef840 0x5601f49b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.547+0000 7f62405cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5601f492e290 con 0x5601f4610520 2024-09-21T03:57:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.547+0000 7f62405cd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5601f491d6c0 con 0x5601f45e7660 2024-09-21T03:57:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.547+0000 7f62405cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5601f49214d0 con 0x5601f45ef840 2024-09-21T03:57:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.548+0000 7f6232a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f4610520 0x5601f49b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.548+0000 7f6232a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f4610520 0x5601f49b71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36076/0 (socket says 172.21.15.16:36076) 2024-09-21T03:57:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.548+0000 7f6232a00640 1 -- 172.21.15.16:0/3497544248 learned_addr learned my addr 172.21.15.16:0/3497544248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.548+0000 7f6232000640 1 --2- 172.21.15.16:0/3497544248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f45e7660 0x5601f49b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.548+0000 7f6233400640 1 --2- 172.21.15.16:0/3497544248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45ef840 0x5601f49b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.548+0000 7f6232a00640 1 -- 172.21.15.16:0/3497544248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45ef840 msgr2=0x5601f49b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.548+0000 7f6232a00640 1 --2- 172.21.15.16:0/3497544248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45ef840 0x5601f49b7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.548+0000 7f6232a00640 1 -- 172.21.15.16:0/3497544248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f45e7660 msgr2=0x5601f49b7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.548+0000 7f6232a00640 1 --2- 172.21.15.16:0/3497544248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f45e7660 0x5601f49b7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.548+0000 7f6232a00640 1 -- 172.21.15.16:0/3497544248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601f48aa460 con 0x5601f4610520 2024-09-21T03:57:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.548+0000 7f6232000640 1 --2- 172.21.15.16:0/3497544248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f45e7660 0x5601f49b7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:57:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.549+0000 7f6232a00640 1 --2- 172.21.15.16:0/3497544248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f4610520 0x5601f49b71c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f62241054f0 tx=0x7f6224031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa57a50b36cde78 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.549+0000 7f621fe00640 1 -- 172.21.15.16:0/3497544248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224049070 con 0x5601f4610520 2024-09-21T03:57:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.549+0000 7f621fe00640 1 -- 172.21.15.16:0/3497544248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62240b50b0 con 0x5601f4610520 2024-09-21T03:57:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.549+0000 7f621fe00640 1 -- 172.21.15.16:0/3497544248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62240d71a0 con 0x5601f4610520 2024-09-21T03:57:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.550+0000 7f62405cd740 1 -- 172.21.15.16:0/3497544248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f4610520 msgr2=0x5601f49b71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.550+0000 7f62405cd740 1 --2- 172.21.15.16:0/3497544248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f4610520 0x5601f49b71c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f62241054f0 tx=0x7f6224031400 comp rx=0 tx=0).stop 2024-09-21T03:57:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.550+0000 7f62405cd740 1 -- 172.21.15.16:0/3497544248 shutdown_connections 2024-09-21T03:57:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.550+0000 7f62405cd740 1 --2- 172.21.15.16:0/3497544248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45ef840 0x5601f49b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.550+0000 7f62405cd740 1 --2- 172.21.15.16:0/3497544248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f45e7660 0x5601f49b7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.550+0000 7f62405cd740 1 --2- 172.21.15.16:0/3497544248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f4610520 0x5601f49b71c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.550+0000 7f62405cd740 1 -- 172.21.15.16:0/3497544248 >> 172.21.15.16:0/3497544248 conn(0x5601f4604910 msgr2=0x5601f49b50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.550+0000 7f62405cd740 1 -- 172.21.15.16:0/3497544248 shutdown_connections 2024-09-21T03:57:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.550+0000 7f62405cd740 1 -- 172.21.15.16:0/3497544248 wait complete. 2024-09-21T03:57:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.550+0000 7f62405cd740 1 -- start start 2024-09-21T03:57:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.551+0000 7f62405cd740 1 Processor -- start 2024-09-21T03:57:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.551+0000 7f62405cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 0x5601f49c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.551+0000 7f62405cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 0x5601f49c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.551+0000 7f62405cd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f4610520 0x5601f49cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.551+0000 7f62405cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5601f477ebb0 con 0x5601f45ef840 2024-09-21T03:57:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.551+0000 7f62405cd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5601f48b86c0 con 0x5601f4610520 2024-09-21T03:57:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.551+0000 7f62405cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5601f49c5760 con 0x5601f45e7660 2024-09-21T03:57:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.551+0000 7f62405cd740 10 client.0 _ll_register_callbacks cb 0x5601f48d3490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:57:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.552+0000 7f6232000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 0x5601f49c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.552+0000 7f6232a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 0x5601f49c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.552+0000 7f6232a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 0x5601f49c2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:36332/0 (socket says 172.21.15.16:36332) 2024-09-21T03:57:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.552+0000 7f6232a00640 1 -- 172.21.15.16:0/1012849853 learned_addr learned my addr 172.21.15.16:0/1012849853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.552+0000 7f6233400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f4610520 0x5601f49cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.552+0000 7f6232a00640 1 -- 172.21.15.16:0/1012849853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f4610520 msgr2=0x5601f49cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.552+0000 7f6232a00640 1 --2- 172.21.15.16:0/1012849853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f4610520 0x5601f49cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.552+0000 7f6232a00640 1 -- 172.21.15.16:0/1012849853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 msgr2=0x5601f49c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.552+0000 7f6232a00640 1 --2- 172.21.15.16:0/1012849853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 0x5601f49c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.552+0000 7f6232a00640 1 -- 172.21.15.16:0/1012849853 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601f467bb00 con 0x5601f45e7660 2024-09-21T03:57:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.552+0000 7f6232a00640 1 --2- 172.21.15.16:0/1012849853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 0x5601f49c2d50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f62240cdba0 tx=0x7f62240ca110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0c392828c84d9e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.552+0000 7f621cc00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:57:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.553+0000 7f621cc00640 1 -- 172.21.15.16:0/1012849853 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62240f5020 con 0x5601f45e7660 2024-09-21T03:57:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.553+0000 7f621cc00640 1 -- 172.21.15.16:0/1012849853 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f62240d73a0 con 0x5601f45e7660 2024-09-21T03:57:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.553+0000 7f62405cd740 1 -- 172.21.15.16:0/1012849853 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62240b50b0 con 0x5601f45e7660 2024-09-21T03:57:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.553+0000 7f621cc00640 1 -- 172.21.15.16:0/1012849853 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6224049070 con 0x5601f45e7660 2024-09-21T03:57:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.553+0000 7f62405cd740 10 client.26641 fetch_fsmap learned FSMap version 30 2024-09-21T03:57:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.553+0000 7f62405cd740 1 -- 172.21.15.16:0/1012849853 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5601f46f2960 con 0x5601f45e7660 2024-09-21T03:57:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.553+0000 7f621cc00640 1 -- 172.21.15.16:0/1012849853 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62240b50b0 con 0x5601f45e7660 2024-09-21T03:57:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.553+0000 7f621cc00640 1 -- 172.21.15.16:0/1012849853 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f62240d21d0 con 0x5601f45e7660 2024-09-21T03:57:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.554+0000 7f62405cd740 10 client.26641 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:57:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.554+0000 7f62405cd740 10 client.26641 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:57:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.554+0000 7f62405cd740 20 client.26641 populate_metadata read hostname 'smithi016' 2024-09-21T03:57:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.554+0000 7f62405cd740 1 --2- 172.21.15.16:0/1012849853 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5601f45ceea0 0x5601f49d4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.554+0000 7f62405cd740 4 client.26641 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:57:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.554+0000 7f62405cd740 1 -- 172.21.15.16:0/1012849853 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601f49301e0 con 0x5601f45ceea0 2024-09-21T03:57:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.554+0000 7f6232000640 1 --2- 172.21.15.16:0/1012849853 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5601f45ceea0 0x5601f49d4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.555+0000 7f6232000640 1 --2- 172.21.15.16:0/1012849853 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5601f45ceea0 0x5601f49d4840 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e533eb0bd28580 server_cookie=92a7387d7640eec0 in_seq=0 out_seq=0 2024-09-21T03:57:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.555+0000 7f621cc00640 10 client.26641 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.555+0000 7f621cc00640 1 -- 172.21.15.16:0/1012849853 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5601f49301e0 con 0x5601f45ceea0 2024-09-21T03:57:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.555+0000 7f621cc00640 10 client.26641 handle_command_reply: tid=0 2024-09-21T03:57:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.556+0000 7f62405cd740 1 client.26641 shutdown 2024-09-21T03:57:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.556+0000 7f62405cd740 1 -- 172.21.15.16:0/1012849853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 msgr2=0x5601f49c2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.556+0000 7f62405cd740 1 --2- 172.21.15.16:0/1012849853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 0x5601f49c2d50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f62240cdba0 tx=0x7f62240ca110 comp rx=0 tx=0).stop 2024-09-21T03:57:17.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.556+0000 7f62405cd740 1 -- 172.21.15.16:0/1012849853 shutdown_connections 2024-09-21T03:57:17.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.556+0000 7f62405cd740 1 --2- 172.21.15.16:0/1012849853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f4610520 0x5601f49cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.556+0000 7f62405cd740 1 --2- 172.21.15.16:0/1012849853 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5601f45ceea0 0x5601f49d4840 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.556+0000 7f62405cd740 1 --2- 172.21.15.16:0/1012849853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 0x5601f49c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.556+0000 7f62405cd740 1 --2- 172.21.15.16:0/1012849853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 0x5601f49c2d50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.556+0000 7f62405cd740 1 -- 172.21.15.16:0/1012849853 >> 172.21.15.16:0/1012849853 conn(0x5601f4604910 msgr2=0x5601f49ba3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:17.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.556+0000 7f621cc00640 0 client.26641 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:17.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.556+0000 7f62405cd740 1 -- 172.21.15.16:0/1012849853 shutdown_connections 2024-09-21T03:57:17.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.556+0000 7f62405cd740 1 -- 172.21.15.16:0/1012849853 wait complete. 2024-09-21T03:57:17.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.556+0000 7f62405cd740 20 client.26641 trim_cache size 0 max 16384 2024-09-21T03:57:17.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.574+0000 7f62405cd740 1 Processor -- start 2024-09-21T03:57:17.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.574+0000 7f62405cd740 1 -- start start 2024-09-21T03:57:17.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.574+0000 7f62405cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 0x5601f49bac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.575+0000 7f62405cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 0x5601f49c4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.575+0000 7f62405cd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f4610520 0x5601f49bfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.575+0000 7f62405cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62341b7160 con 0x5601f45ef840 2024-09-21T03:57:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.575+0000 7f62405cd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62341b8ca0 con 0x5601f4610520 2024-09-21T03:57:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.575+0000 7f62405cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62341b7830 con 0x5601f45e7660 2024-09-21T03:57:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.575+0000 7f6232000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 0x5601f49c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.575+0000 7f6232000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 0x5601f49c4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36108/0 (socket says 172.21.15.16:36108) 2024-09-21T03:57:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.575+0000 7f6232000640 1 -- 172.21.15.16:0/476783862 learned_addr learned my addr 172.21.15.16:0/476783862 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.575+0000 7f6232a00640 1 --2- 172.21.15.16:0/476783862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 0x5601f49bac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.575+0000 7f6233400640 1 --2- 172.21.15.16:0/476783862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f4610520 0x5601f49bfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.575+0000 7f6232000640 1 -- 172.21.15.16:0/476783862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 msgr2=0x5601f49bac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.575+0000 7f6232000640 1 --2- 172.21.15.16:0/476783862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 0x5601f49bac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.575+0000 7f6232000640 1 -- 172.21.15.16:0/476783862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f4610520 msgr2=0x5601f49bfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.575+0000 7f6232000640 1 --2- 172.21.15.16:0/476783862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f4610520 0x5601f49bfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.575+0000 7f6232000640 1 -- 172.21.15.16:0/476783862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601f49b5c70 con 0x5601f45ef840 2024-09-21T03:57:17.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.575+0000 7f6232a00640 1 --2- 172.21.15.16:0/476783862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 0x5601f49bac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:57:17.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.576+0000 7f6232000640 1 --2- 172.21.15.16:0/476783862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 0x5601f49c4780 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6220011810 tx=0x7f6220013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afe4a9cedb045b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.576+0000 7f621f400640 1 -- 172.21.15.16:0/476783862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6220021190 con 0x5601f45ef840 2024-09-21T03:57:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.576+0000 7f621f400640 1 -- 172.21.15.16:0/476783862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6220021e90 con 0x5601f45ef840 2024-09-21T03:57:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.576+0000 7f621f400640 1 -- 172.21.15.16:0/476783862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6220004a00 con 0x5601f45ef840 2024-09-21T03:57:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.577+0000 7f62405cd740 1 -- 172.21.15.16:0/476783862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 msgr2=0x5601f49c4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.577+0000 7f62405cd740 1 --2- 172.21.15.16:0/476783862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 0x5601f49c4780 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6220011810 tx=0x7f6220013000 comp rx=0 tx=0).stop 2024-09-21T03:57:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.577+0000 7f62405cd740 1 -- 172.21.15.16:0/476783862 shutdown_connections 2024-09-21T03:57:17.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.577+0000 7f62405cd740 1 --2- 172.21.15.16:0/476783862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f4610520 0x5601f49bfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.577+0000 7f62405cd740 1 --2- 172.21.15.16:0/476783862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 0x5601f49c4780 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.577+0000 7f62405cd740 1 --2- 172.21.15.16:0/476783862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 0x5601f49bac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.577+0000 7f62405cd740 1 -- 172.21.15.16:0/476783862 >> 172.21.15.16:0/476783862 conn(0x5601f4604910 msgr2=0x5601f49c4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.577+0000 7f62405cd740 1 -- 172.21.15.16:0/476783862 shutdown_connections 2024-09-21T03:57:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.577+0000 7f62405cd740 1 -- 172.21.15.16:0/476783862 wait complete. 2024-09-21T03:57:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.577+0000 7f62405cd740 1 -- start start 2024-09-21T03:57:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.578+0000 7f62405cd740 1 Processor -- start 2024-09-21T03:57:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.578+0000 7f62405cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 0x5601f49cb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.578+0000 7f62405cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 0x5601f49c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.578+0000 7f62405cd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f4610520 0x5601f49c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.578+0000 7f62405cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62341b85d0 con 0x5601f45ef840 2024-09-21T03:57:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.578+0000 7f62405cd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62341b7f00 con 0x5601f4610520 2024-09-21T03:57:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.578+0000 7f62405cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62341b5140 con 0x5601f45e7660 2024-09-21T03:57:17.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.578+0000 7f62405cd740 10 client.0 _ll_register_callbacks cb 0x7f62240cd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:57:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.578+0000 7f6232000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 0x5601f49c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.578+0000 7f6232a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 0x5601f49cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.578+0000 7f6232a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 0x5601f49cb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:36350/0 (socket says 172.21.15.16:36350) 2024-09-21T03:57:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.578+0000 7f6232a00640 1 -- 172.21.15.16:0/2475386408 learned_addr learned my addr 172.21.15.16:0/2475386408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.578+0000 7f6233400640 1 --2- 172.21.15.16:0/2475386408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f4610520 0x5601f49c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.579+0000 7f6232000640 1 -- 172.21.15.16:0/2475386408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 msgr2=0x5601f49cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.579+0000 7f6232000640 1 --2- 172.21.15.16:0/2475386408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 0x5601f49cb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.579+0000 7f6232000640 1 -- 172.21.15.16:0/2475386408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f4610520 msgr2=0x5601f49c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.579+0000 7f6232000640 1 --2- 172.21.15.16:0/2475386408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f4610520 0x5601f49c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.579+0000 7f6232000640 1 -- 172.21.15.16:0/2475386408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601f49c8070 con 0x5601f45ef840 2024-09-21T03:57:17.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.579+0000 7f6233400640 1 --2- 172.21.15.16:0/2475386408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f4610520 0x5601f49c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:57:17.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.579+0000 7f6232000640 1 --2- 172.21.15.16:0/2475386408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 0x5601f49c74f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x5601f49b07d0 tx=0x7f62200058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ce7c666f5be13d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:17.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.579+0000 7f621cc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:57:17.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.580+0000 7f621cc00640 1 -- 172.21.15.16:0/2475386408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62200219e0 con 0x5601f45ef840 2024-09-21T03:57:17.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.580+0000 7f621cc00640 1 -- 172.21.15.16:0/2475386408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f6220021b80 con 0x5601f45ef840 2024-09-21T03:57:17.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.580+0000 7f62405cd740 1 -- 172.21.15.16:0/2475386408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6220021e90 con 0x5601f45ef840 2024-09-21T03:57:17.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.580+0000 7f621cc00640 1 -- 172.21.15.16:0/2475386408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6220031440 con 0x5601f45ef840 2024-09-21T03:57:17.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.580+0000 7f62405cd740 10 client.16428 fetch_fsmap learned FSMap version 30 2024-09-21T03:57:17.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.580+0000 7f62405cd740 1 -- 172.21.15.16:0/2475386408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5601f4a3ff80 con 0x5601f45ef840 2024-09-21T03:57:17.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.580+0000 7f621cc00640 1 -- 172.21.15.16:0/2475386408 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6220021e90 con 0x5601f45ef840 2024-09-21T03:57:17.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.580+0000 7f621cc00640 1 -- 172.21.15.16:0/2475386408 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6220042030 con 0x5601f45ef840 2024-09-21T03:57:17.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.581+0000 7f62405cd740 10 client.16428 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:57:17.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.581+0000 7f62405cd740 10 client.16428 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:57:17.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.581+0000 7f62405cd740 20 client.16428 populate_metadata read hostname 'smithi016' 2024-09-21T03:57:17.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.581+0000 7f62405cd740 1 --2- 172.21.15.16:0/2475386408 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5601f45a4250 0x5601f4a426d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:17.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.581+0000 7f62405cd740 4 client.16428 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fc03fa01", "release": true, "await": true}] 2024-09-21T03:57:17.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.581+0000 7f62405cd740 1 -- 172.21.15.16:0/2475386408 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fc03fa01", "release": true, "await": true}) -- 0x5601f492bc90 con 0x5601f45a4250 2024-09-21T03:57:17.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.581+0000 7f6232a00640 1 --2- 172.21.15.16:0/2475386408 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5601f45a4250 0x5601f4a426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:17.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.581+0000 7f6232a00640 1 --2- 172.21.15.16:0/2475386408 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5601f45a4250 0x5601f4a426d0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49024c66c7c3da9d server_cookie=5cbc6cd3ba295701 in_seq=0 out_seq=0 2024-09-21T03:57:17.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.581+0000 7f621cc00640 10 client.16428 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:17.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.585+0000 7f621cc00640 1 -- 172.21.15.16:0/2475386408 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5601f492bc90 con 0x5601f45a4250 2024-09-21T03:57:17.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.585+0000 7f621cc00640 10 client.16428 handle_command_reply: tid=0 2024-09-21T03:57:17.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.585+0000 7f62405cd740 1 client.16428 shutdown 2024-09-21T03:57:17.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.586+0000 7f62405cd740 1 -- 172.21.15.16:0/2475386408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 msgr2=0x5601f49c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:17.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.586+0000 7f62405cd740 1 --2- 172.21.15.16:0/2475386408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 0x5601f49c74f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x5601f49b07d0 tx=0x7f62200058d0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.586+0000 7f62405cd740 1 -- 172.21.15.16:0/2475386408 shutdown_connections 2024-09-21T03:57:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.586+0000 7f62405cd740 1 --2- 172.21.15.16:0/2475386408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5601f4610520 0x5601f49c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.586+0000 7f62405cd740 1 --2- 172.21.15.16:0/2475386408 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5601f45a4250 0x5601f4a426d0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.586+0000 7f62405cd740 1 --2- 172.21.15.16:0/2475386408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601f45ef840 0x5601f49c74f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.586+0000 7f62405cd740 1 --2- 172.21.15.16:0/2475386408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601f45e7660 0x5601f49cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.586+0000 7f62405cd740 1 -- 172.21.15.16:0/2475386408 >> 172.21.15.16:0/2475386408 conn(0x5601f4604910 msgr2=0x5601f49b8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.587+0000 7f621cc00640 0 client.16428 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.587+0000 7f62405cd740 1 -- 172.21.15.16:0/2475386408 shutdown_connections 2024-09-21T03:57:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.587+0000 7f62405cd740 1 -- 172.21.15.16:0/2475386408 wait complete. 2024-09-21T03:57:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.587+0000 7f62405cd740 20 client.16428 trim_cache size 0 max 16384 2024-09-21T03:57:17.591 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:57:17.591 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:57:17.592 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:57:17.592 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 40, 2024-09-21T03:57:17.592 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:57:17.592 INFO:teuthology.orchestra.run.smithi016.stdout: "fc03fa01": { 2024-09-21T03:57:17.592 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 40, 2024-09-21T03:57:17.592 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:57:17.592 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:57:17.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:57:17.592 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:57:17.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:57:17.592 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:57:17.592 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 144.5, 2024-09-21T03:57:17.592 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:57:17.593 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:57:17.593 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:57:17.593 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:57:17.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:57:17.593 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:57:17.593 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:57:17.593 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:57:17.593 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:57:17.593 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:57:17.593 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:57:17.593 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:57:17.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.589+0000 7f6239800640 1 -- 172.21.15.16:0/2121525849 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f62180b6590 msgr2=0x7f62180b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:17.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.589+0000 7f6239800640 1 --2- 172.21.15.16:0/2121525849 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f62180b6590 0x7f62180b8aa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f62341157f0 tx=0x7f622000b390 comp rx=0 tx=0).stop 2024-09-21T03:57:17.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.589+0000 7f6239800640 1 -- 172.21.15.16:0/2121525849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6234075dd0 msgr2=0x7f62341a0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:17.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.589+0000 7f6239800640 1 --2- 172.21.15.16:0/2121525849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6234075dd0 0x7f62341a0a80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6224005ab0 tx=0x7f6224003c80 comp rx=0 tx=0).stop 2024-09-21T03:57:17.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.589+0000 7f6239800640 1 -- 172.21.15.16:0/2121525849 shutdown_connections 2024-09-21T03:57:17.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.589+0000 7f6239800640 1 --2- 172.21.15.16:0/2121525849 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f62180b6590 0x7f62180b8aa0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.589+0000 7f6239800640 1 --2- 172.21.15.16:0/2121525849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6234114430 0x7f62341a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.589+0000 7f6239800640 1 --2- 172.21.15.16:0/2121525849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6234077380 0x7f62341a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.589+0000 7f6239800640 1 --2- 172.21.15.16:0/2121525849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6234075dd0 0x7f62341a0a80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:17.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.589+0000 7f6239800640 1 -- 172.21.15.16:0/2121525849 >> 172.21.15.16:0/2121525849 conn(0x7f623407b850 msgr2=0x7f6234113050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:17.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.590+0000 7f6239800640 1 -- 172.21.15.16:0/2121525849 shutdown_connections 2024-09-21T03:57:17.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:17.590+0000 7f6239800640 1 -- 172.21.15.16:0/2121525849 wait complete. 2024-09-21T03:57:18.596 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fc03fa01', total seconds elapsed: 31.7 2024-09-21T03:57:18.597 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-21T03:57:18.597 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:57:19.175 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:19 smithi016 ceph-mon[30375]: pgmap v748: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 206 MiB/s wr, 1.80k op/s 2024-09-21T03:57:19.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:19 smithi134 ceph-mon[35031]: pgmap v748: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 206 MiB/s wr, 1.80k op/s 2024-09-21T03:57:19.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:19 smithi123 ceph-mon[34673]: pgmap v748: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 206 MiB/s wr, 1.80k op/s 2024-09-21T03:57:19.730 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:57:21.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:21 smithi134 ceph-mon[35031]: pgmap v749: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 192 MiB/s wr, 1.79k op/s 2024-09-21T03:57:21.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:21 smithi123 ceph-mon[34673]: pgmap v749: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 192 MiB/s wr, 1.79k op/s 2024-09-21T03:57:21.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:21 smithi016 ceph-mon[30375]: pgmap v749: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 192 MiB/s wr, 1.79k op/s 2024-09-21T03:57:23.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:23 smithi134 ceph-mon[35031]: pgmap v750: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 153 MiB/s wr, 1.51k op/s 2024-09-21T03:57:23.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:23 smithi123 ceph-mon[34673]: pgmap v750: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 153 MiB/s wr, 1.51k op/s 2024-09-21T03:57:23.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:23 smithi016 ceph-mon[30375]: pgmap v750: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 153 MiB/s wr, 1.51k op/s 2024-09-21T03:57:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.301+0000 7fc1cf600640 1 -- 172.21.15.16:0/735318757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c8108d10 msgr2=0x7fc1c810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.301+0000 7fc1cf600640 1 --2- 172.21.15.16:0/735318757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c8108d10 0x7fc1c810f540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009900 tx=0x7fc1b402a410 comp rx=0 tx=0).stop 2024-09-21T03:57:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.302+0000 7fc1cf600640 1 -- 172.21.15.16:0/735318757 shutdown_connections 2024-09-21T03:57:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.302+0000 7fc1cf600640 1 --2- 172.21.15.16:0/735318757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c8108d10 0x7fc1c810f540 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.302+0000 7fc1cf600640 1 --2- 172.21.15.16:0/735318757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1c81083c0 0x7fc1c81087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.302+0000 7fc1cf600640 1 --2- 172.21.15.16:0/735318757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c8102a80 0x7fc1c8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.303+0000 7fc1cf600640 1 -- 172.21.15.16:0/735318757 >> 172.21.15.16:0/735318757 conn(0x7fc1c8079740 msgr2=0x7fc1c8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:25.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.303+0000 7fc1cf600640 1 -- 172.21.15.16:0/735318757 shutdown_connections 2024-09-21T03:57:25.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.303+0000 7fc1cf600640 1 -- 172.21.15.16:0/735318757 wait complete. 2024-09-21T03:57:25.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:25 smithi016 ceph-mon[30375]: pgmap v751: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 135 MiB/s wr, 1.40k op/s 2024-09-21T03:57:25.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.308+0000 7fc1cf600640 1 Processor -- start 2024-09-21T03:57:25.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.308+0000 7fc1cf600640 1 -- start start 2024-09-21T03:57:25.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.309+0000 7fc1cf600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c8102a80 0x7fc1c8199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:25.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.309+0000 7fc1cf600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81083c0 0x7fc1c8199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:25.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.309+0000 7fc1cf600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1c8108d10 0x7fc1c819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:25.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.309+0000 7fc1cf600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c8112a80 con 0x7fc1c8108d10 2024-09-21T03:57:25.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.309+0000 7fc1cf600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c8112900 con 0x7fc1c81083c0 2024-09-21T03:57:25.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.309+0000 7fc1cf600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c8112c00 con 0x7fc1c8102a80 2024-09-21T03:57:25.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.310+0000 7fc1c7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81083c0 0x7fc1c8199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:25.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.311+0000 7fc1c7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81083c0 0x7fc1c8199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:59992/0 (socket says 172.21.15.16:59992) 2024-09-21T03:57:25.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.311+0000 7fc1c7e00640 1 -- 172.21.15.16:0/3743027763 learned_addr learned my addr 172.21.15.16:0/3743027763 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:25.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.310+0000 7fc1ccc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c8102a80 0x7fc1c8199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:25.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.311+0000 7fc1c7e00640 1 -- 172.21.15.16:0/3743027763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c8102a80 msgr2=0x7fc1c8199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:25.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.311+0000 7fc1c7e00640 1 --2- 172.21.15.16:0/3743027763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c8102a80 0x7fc1c8199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.311+0000 7fc1c7e00640 1 -- 172.21.15.16:0/3743027763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1c8108d10 msgr2=0x7fc1c819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:25.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.311+0000 7fc1c7e00640 1 --2- 172.21.15.16:0/3743027763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1c8108d10 0x7fc1c819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.311+0000 7fc1c7e00640 1 -- 172.21.15.16:0/3743027763 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c819f4a0 con 0x7fc1c81083c0 2024-09-21T03:57:25.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.313+0000 7fc1c7e00640 1 --2- 172.21.15.16:0/3743027763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81083c0 0x7fc1c8199ad0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800b450 tx=0x7fc1b800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:25.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.315+0000 7fc1c5600640 1 -- 172.21.15.16:0/3743027763 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8015020 con 0x7fc1c81083c0 2024-09-21T03:57:25.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.315+0000 7fc1c5600640 1 -- 172.21.15.16:0/3743027763 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1b8011d40 con 0x7fc1c81083c0 2024-09-21T03:57:25.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.315+0000 7fc1c5600640 1 -- 172.21.15.16:0/3743027763 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b800f340 con 0x7fc1c81083c0 2024-09-21T03:57:25.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.319+0000 7fc1cf600640 1 -- 172.21.15.16:0/3743027763 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c819f790 con 0x7fc1c81083c0 2024-09-21T03:57:25.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.320+0000 7fc1cf600640 1 -- 172.21.15.16:0/3743027763 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c806bb40 con 0x7fc1c81083c0 2024-09-21T03:57:25.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.325+0000 7fc1cf600640 1 -- 172.21.15.16:0/3743027763 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c8103d00 con 0x7fc1c81083c0 2024-09-21T03:57:25.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.328+0000 7fc1c5600640 1 -- 172.21.15.16:0/3743027763 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc1b8018020 con 0x7fc1c81083c0 2024-09-21T03:57:25.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.334+0000 7fc1c5600640 1 --2- 172.21.15.16:0/3743027763 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc1ac0bab30 0x7fc1ac0bd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:25.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.334+0000 7fc1c5600640 1 -- 172.21.15.16:0/3743027763 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fc1b80e6d30 con 0x7fc1c81083c0 2024-09-21T03:57:25.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.335+0000 7fc1c5600640 1 -- 172.21.15.16:0/3743027763 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc1b80ec020 con 0x7fc1c81083c0 2024-09-21T03:57:25.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.335+0000 7fc1ccc00640 1 --2- 172.21.15.16:0/3743027763 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc1ac0bab30 0x7fc1ac0bd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:25.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.337+0000 7fc1ccc00640 1 --2- 172.21.15.16:0/3743027763 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc1ac0bab30 0x7fc1ac0bd040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009fd0 tx=0x7fc1c000d040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:25.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:25 smithi134 ceph-mon[35031]: pgmap v751: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 135 MiB/s wr, 1.40k op/s 2024-09-21T03:57:25.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:25 smithi123 ceph-mon[34673]: pgmap v751: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 135 MiB/s wr, 1.40k op/s 2024-09-21T03:57:25.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.816+0000 7fc1cf600640 1 -- 172.21.15.16:0/3743027763 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1c806c560 con 0x7fc1c81083c0 2024-09-21T03:57:25.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.820+0000 7fc1c5600640 1 -- 172.21.15.16:0/3743027763 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7fc1b80a8070 con 0x7fc1c81083c0 2024-09-21T03:57:25.828 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:57:25.828 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:57:25.828 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:57:25.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.825+0000 7fc1cf600640 1 -- 172.21.15.16:0/3743027763 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc1ac0bab30 msgr2=0x7fc1ac0bd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:25.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.825+0000 7fc1cf600640 1 --2- 172.21.15.16:0/3743027763 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc1ac0bab30 0x7fc1ac0bd040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009fd0 tx=0x7fc1c000d040 comp rx=0 tx=0).stop 2024-09-21T03:57:25.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.825+0000 7fc1cf600640 1 -- 172.21.15.16:0/3743027763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81083c0 msgr2=0x7fc1c8199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:25.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.825+0000 7fc1cf600640 1 --2- 172.21.15.16:0/3743027763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81083c0 0x7fc1c8199ad0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800b450 tx=0x7fc1b800b910 comp rx=0 tx=0).stop 2024-09-21T03:57:25.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.826+0000 7fc1cf600640 1 -- 172.21.15.16:0/3743027763 shutdown_connections 2024-09-21T03:57:25.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.826+0000 7fc1cf600640 1 --2- 172.21.15.16:0/3743027763 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc1ac0bab30 0x7fc1ac0bd040 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.826+0000 7fc1cf600640 1 --2- 172.21.15.16:0/3743027763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1c8108d10 0x7fc1c819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.826+0000 7fc1cf600640 1 --2- 172.21.15.16:0/3743027763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81083c0 0x7fc1c8199ad0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.826+0000 7fc1cf600640 1 --2- 172.21.15.16:0/3743027763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c8102a80 0x7fc1c8199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.826+0000 7fc1cf600640 1 -- 172.21.15.16:0/3743027763 >> 172.21.15.16:0/3743027763 conn(0x7fc1c8079740 msgr2=0x7fc1c81044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:25.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.827+0000 7fc1cf600640 1 -- 172.21.15.16:0/3743027763 shutdown_connections 2024-09-21T03:57:25.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:25.827+0000 7fc1cf600640 1 -- 172.21.15.16:0/3743027763 wait complete. 2024-09-21T03:57:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.945+0000 7f298d600640 1 -- 172.21.15.16:0/3296833105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29881100a0 msgr2=0x7f29881125b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.945+0000 7f298d600640 1 --2- 172.21.15.16:0/3296833105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29881100a0 0x7f29881125b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2974009860 tx=0x7f297402a380 comp rx=0 tx=0).stop 2024-09-21T03:57:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.946+0000 7f298d600640 1 -- 172.21.15.16:0/3296833105 shutdown_connections 2024-09-21T03:57:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.946+0000 7f298d600640 1 --2- 172.21.15.16:0/3296833105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29881100a0 0x7f29881125b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.946+0000 7f298d600640 1 --2- 172.21.15.16:0/3296833105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298810d480 0x7f298810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.946+0000 7f298d600640 1 --2- 172.21.15.16:0/3296833105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f298806be20 0x7f298810ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.946+0000 7f298d600640 1 -- 172.21.15.16:0/3296833105 >> 172.21.15.16:0/3296833105 conn(0x7f2988070aa0 msgr2=0x7f2988072f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.946+0000 7f298d600640 1 -- 172.21.15.16:0/3296833105 shutdown_connections 2024-09-21T03:57:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.946+0000 7f298d600640 1 -- 172.21.15.16:0/3296833105 wait complete. 2024-09-21T03:57:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.949+0000 7f298d600640 1 Processor -- start 2024-09-21T03:57:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.950+0000 7f298d600640 1 -- start start 2024-09-21T03:57:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.951+0000 7f298d600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f298806be20 0x7f29881a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.951+0000 7f298d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f298810d480 0x7f29881a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.951+0000 7f298d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29881100a0 0x7f29881a9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.951+0000 7f298d600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2988117940 con 0x7f298810d480 2024-09-21T03:57:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.951+0000 7f298d600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29881177c0 con 0x7f29881100a0 2024-09-21T03:57:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.951+0000 7f298d600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2988117ac0 con 0x7f298806be20 2024-09-21T03:57:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.952+0000 7f2987e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f298806be20 0x7f29881a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.952+0000 7f2987e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f298806be20 0x7f29881a0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:42302/0 (socket says 172.21.15.16:42302) 2024-09-21T03:57:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.952+0000 7f2987e00640 1 -- 172.21.15.16:0/2561297269 learned_addr learned my addr 172.21.15.16:0/2561297269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.952+0000 7f298cc00640 1 --2- 172.21.15.16:0/2561297269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29881100a0 0x7f29881a9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.953+0000 7f2987400640 1 --2- 172.21.15.16:0/2561297269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f298810d480 0x7f29881a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.955+0000 7f298cc00640 1 -- 172.21.15.16:0/2561297269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f298806be20 msgr2=0x7f29881a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.955+0000 7f298cc00640 1 --2- 172.21.15.16:0/2561297269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f298806be20 0x7f29881a0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.955+0000 7f298cc00640 1 -- 172.21.15.16:0/2561297269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f298810d480 msgr2=0x7f29881a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.955+0000 7f298cc00640 1 --2- 172.21.15.16:0/2561297269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f298810d480 0x7f29881a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.955+0000 7f298cc00640 1 -- 172.21.15.16:0/2561297269 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29881aa5e0 con 0x7f29881100a0 2024-09-21T03:57:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.955+0000 7f298cc00640 1 --2- 172.21.15.16:0/2561297269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29881100a0 0x7f29881a9dc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2974009780 tx=0x7f2974030b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.958+0000 7f2984c00640 1 -- 172.21.15.16:0/2561297269 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974037050 con 0x7f29881100a0 2024-09-21T03:57:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.958+0000 7f2984c00640 1 -- 172.21.15.16:0/2561297269 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2974038040 con 0x7f29881100a0 2024-09-21T03:57:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.958+0000 7f2984c00640 1 -- 172.21.15.16:0/2561297269 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974005310 con 0x7f29881100a0 2024-09-21T03:57:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.960+0000 7f298d600640 1 -- 172.21.15.16:0/2561297269 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29881aa870 con 0x7f29881100a0 2024-09-21T03:57:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.960+0000 7f298d600640 1 -- 172.21.15.16:0/2561297269 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29881b0dc0 con 0x7f29881100a0 2024-09-21T03:57:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.967+0000 7f2995dce740 1 Processor -- start 2024-09-21T03:57:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.969+0000 7f2984c00640 1 -- 172.21.15.16:0/2561297269 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2974045030 con 0x7f29881100a0 2024-09-21T03:57:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.969+0000 7f2995dce740 1 -- start start 2024-09-21T03:57:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.970+0000 7f2995dce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b621b0 0x55e7a1f092b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.970+0000 7f2995dce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b392f0 0x55e7a1f09920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.970+0000 7f2995dce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b414d0 0x55e7a1f0a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.970+0000 7f2995dce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e7a1e7c6b0 con 0x55e7a1b414d0 2024-09-21T03:57:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.970+0000 7f2995dce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e7a1e6bae0 con 0x55e7a1b392f0 2024-09-21T03:57:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.970+0000 7f2995dce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e7a1e6f8f0 con 0x55e7a1b621b0 2024-09-21T03:57:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.971+0000 7f2987400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b621b0 0x55e7a1f092b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.971+0000 7f2987400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b621b0 0x55e7a1f092b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:42304/0 (socket says 172.21.15.16:42304) 2024-09-21T03:57:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.971+0000 7f2987400640 1 -- 172.21.15.16:0/2912831382 learned_addr learned my addr 172.21.15.16:0/2912831382 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.971+0000 7f2987e00640 1 --2- 172.21.15.16:0/2912831382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b392f0 0x55e7a1f09920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.971+0000 7f2987400640 1 --2- 172.21.15.16:0/2912831382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b414d0 0x55e7a1f0a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.972+0000 7f2987e00640 1 -- 172.21.15.16:0/2912831382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b621b0 msgr2=0x55e7a1f092b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.972+0000 7f2987e00640 1 --2- 172.21.15.16:0/2912831382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b621b0 0x55e7a1f092b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.972+0000 7f2987e00640 1 -- 172.21.15.16:0/2912831382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b414d0 msgr2=0x55e7a1f0a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.972+0000 7f2987e00640 1 --2- 172.21.15.16:0/2912831382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b414d0 0x55e7a1f0a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.972+0000 7f2987e00640 1 -- 172.21.15.16:0/2912831382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7a1e38780 con 0x55e7a1b392f0 2024-09-21T03:57:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.973+0000 7f2987400640 1 --2- 172.21.15.16:0/2912831382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b414d0 0x55e7a1f0a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:57:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.973+0000 7f2984c00640 1 --2- 172.21.15.16:0/2561297269 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f296c0b6720 0x7f296c0b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.973+0000 7f2984c00640 1 -- 172.21.15.16:0/2561297269 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f2974107030 con 0x7f29881100a0 2024-09-21T03:57:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.974+0000 7f2987e00640 1 --2- 172.21.15.16:0/2912831382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b392f0 0x55e7a1f09920 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f29800114f0 tx=0x7f2980009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc65bf0c86307425 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.974+0000 7f2987400640 1 --2- 172.21.15.16:0/2561297269 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f296c0b6720 0x7f296c0b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.974+0000 7f297d200640 1 -- 172.21.15.16:0/2912831382 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298001e070 con 0x55e7a1b392f0 2024-09-21T03:57:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.975+0000 7f297d200640 1 -- 172.21.15.16:0/2912831382 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29800110f0 con 0x55e7a1b392f0 2024-09-21T03:57:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.975+0000 7f297d200640 1 -- 172.21.15.16:0/2912831382 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298001e070 con 0x55e7a1b392f0 2024-09-21T03:57:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.976+0000 7f2995dce740 1 -- 172.21.15.16:0/2912831382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b392f0 msgr2=0x55e7a1f09920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.976+0000 7f2995dce740 1 --2- 172.21.15.16:0/2912831382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b392f0 0x55e7a1f09920 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f29800114f0 tx=0x7f2980009f90 comp rx=0 tx=0).stop 2024-09-21T03:57:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.976+0000 7f2987400640 1 --2- 172.21.15.16:0/2561297269 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f296c0b6720 0x7f296c0b8c30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2978009fd0 tx=0x7f2978009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.977+0000 7f2995dce740 1 -- 172.21.15.16:0/2912831382 shutdown_connections 2024-09-21T03:57:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.977+0000 7f2995dce740 1 --2- 172.21.15.16:0/2912831382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b414d0 0x55e7a1f0a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.977+0000 7f2995dce740 1 --2- 172.21.15.16:0/2912831382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b392f0 0x55e7a1f09920 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.977+0000 7f2995dce740 1 --2- 172.21.15.16:0/2912831382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b621b0 0x55e7a1f092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.977+0000 7f2995dce740 1 -- 172.21.15.16:0/2912831382 >> 172.21.15.16:0/2912831382 conn(0x55e7a1b56300 msgr2=0x55e7a1f04d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.977+0000 7f2995dce740 1 -- 172.21.15.16:0/2912831382 shutdown_connections 2024-09-21T03:57:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.977+0000 7f2995dce740 1 -- 172.21.15.16:0/2912831382 wait complete. 2024-09-21T03:57:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.978+0000 7f2995dce740 1 -- start start 2024-09-21T03:57:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.978+0000 7f2995dce740 1 Processor -- start 2024-09-21T03:57:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.979+0000 7f2995dce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b392f0 0x55e7a1f12870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.979+0000 7f2995dce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b414d0 0x55e7a1f12f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.979+0000 7f2995dce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b621b0 0x55e7a1f13aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.979+0000 7f2995dce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e7a1da3ab0 con 0x55e7a1b392f0 2024-09-21T03:57:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.979+0000 7f2995dce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e7a1e12570 con 0x55e7a1b621b0 2024-09-21T03:57:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.979+0000 7f2987400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b414d0 0x55e7a1f12f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.979+0000 7f2987e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b392f0 0x55e7a1f12870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.979+0000 7f2987400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b414d0 0x55e7a1f12f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:42316/0 (socket says 172.21.15.16:42316) 2024-09-21T03:57:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.979+0000 7f2987400640 1 -- 172.21.15.16:0/3354013148 learned_addr learned my addr 172.21.15.16:0/3354013148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.979+0000 7f2995dce740 1 -- 172.21.15.16:0/3354013148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e7a1f0afe0 con 0x55e7a1b414d0 2024-09-21T03:57:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.979+0000 7f2995dce740 10 client.0 _ll_register_callbacks cb 0x55e7a1e21100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:57:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.980+0000 7f298cc00640 1 --2- 172.21.15.16:0/3354013148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b621b0 0x55e7a1f13aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.980+0000 7f2987400640 1 -- 172.21.15.16:0/3354013148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b621b0 msgr2=0x55e7a1f13aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.980+0000 7f2987400640 1 --2- 172.21.15.16:0/3354013148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b621b0 0x55e7a1f13aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.980+0000 7f2987400640 1 -- 172.21.15.16:0/3354013148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b392f0 msgr2=0x55e7a1f12870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.980+0000 7f2987400640 1 --2- 172.21.15.16:0/3354013148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b392f0 0x55e7a1f12870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.980+0000 7f2987400640 1 -- 172.21.15.16:0/3354013148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7a1bcd790 con 0x55e7a1b414d0 2024-09-21T03:57:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.980+0000 7f2987e00640 1 --2- 172.21.15.16:0/3354013148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b392f0 0x55e7a1f12870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:57:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.981+0000 7f2987400640 1 --2- 172.21.15.16:0/3354013148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b414d0 0x55e7a1f12f30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2978003b10 tx=0x7f2978004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3df448c069d84bfe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.981+0000 7f2971600640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:57:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.981+0000 7f2971600640 1 -- 172.21.15.16:0/3354013148 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978041060 con 0x55e7a1b414d0 2024-09-21T03:57:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.981+0000 7f2995dce740 1 -- 172.21.15.16:0/3354013148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29800110f0 con 0x55e7a1b414d0 2024-09-21T03:57:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.981+0000 7f2971600640 1 -- 172.21.15.16:0/3354013148 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f29780338c0 con 0x55e7a1b414d0 2024-09-21T03:57:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.981+0000 7f2971600640 1 -- 172.21.15.16:0/3354013148 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f297802a040 con 0x55e7a1b414d0 2024-09-21T03:57:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.981+0000 7f2995dce740 10 client.26665 fetch_fsmap learned FSMap version 30 2024-09-21T03:57:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.981+0000 7f2995dce740 1 -- 172.21.15.16:0/3354013148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e7a1c46620 con 0x55e7a1b414d0 2024-09-21T03:57:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.982+0000 7f2971600640 1 -- 172.21.15.16:0/3354013148 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29800110f0 con 0x55e7a1b414d0 2024-09-21T03:57:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.982+0000 7f2971600640 1 -- 172.21.15.16:0/3354013148 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2978031830 con 0x55e7a1b414d0 2024-09-21T03:57:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.982+0000 7f2995dce740 10 client.26665 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:57:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.982+0000 7f2995dce740 10 client.26665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:57:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.982+0000 7f2995dce740 20 client.26665 populate_metadata read hostname 'smithi016' 2024-09-21T03:57:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.982+0000 7f2995dce740 1 --2- 172.21.15.16:0/3354013148 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e7a1b20d80 0x55e7a1f200b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:25.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.982+0000 7f2995dce740 4 client.26665 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:57:25.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.983+0000 7f2995dce740 1 -- 172.21.15.16:0/3354013148 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7a1e7e600 con 0x55e7a1b20d80 2024-09-21T03:57:25.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.983+0000 7f2987e00640 1 --2- 172.21.15.16:0/3354013148 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e7a1b20d80 0x55e7a1f200b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:25.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.984+0000 7f2987e00640 1 --2- 172.21.15.16:0/3354013148 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e7a1b20d80 0x55e7a1f200b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3de260073a945b72 server_cookie=87e46da0ac2328d4 in_seq=0 out_seq=0 2024-09-21T03:57:25.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.984+0000 7f2971600640 10 client.26665 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.985+0000 7f2971600640 1 -- 172.21.15.16:0/3354013148 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e7a1e7e600 con 0x55e7a1b20d80 2024-09-21T03:57:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.985+0000 7f2971600640 10 client.26665 handle_command_reply: tid=0 2024-09-21T03:57:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.985+0000 7f2995dce740 1 client.26665 shutdown 2024-09-21T03:57:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.986+0000 7f2995dce740 1 -- 172.21.15.16:0/3354013148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b414d0 msgr2=0x55e7a1f12f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.986+0000 7f2995dce740 1 --2- 172.21.15.16:0/3354013148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b414d0 0x55e7a1f12f30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2978003b10 tx=0x7f2978004730 comp rx=0 tx=0).stop 2024-09-21T03:57:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.987+0000 7f2995dce740 1 -- 172.21.15.16:0/3354013148 shutdown_connections 2024-09-21T03:57:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.987+0000 7f2995dce740 1 --2- 172.21.15.16:0/3354013148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b621b0 0x55e7a1f13aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.987+0000 7f2995dce740 1 --2- 172.21.15.16:0/3354013148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b414d0 0x55e7a1f12f30 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.987+0000 7f2995dce740 1 --2- 172.21.15.16:0/3354013148 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e7a1b20d80 0x55e7a1f200b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.987+0000 7f2995dce740 1 --2- 172.21.15.16:0/3354013148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b392f0 0x55e7a1f12870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.987+0000 7f2995dce740 1 -- 172.21.15.16:0/3354013148 >> 172.21.15.16:0/3354013148 conn(0x55e7a1b56300 msgr2=0x55e7a1f03360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.987+0000 7f2971600640 0 client.26665 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.987+0000 7f2995dce740 1 -- 172.21.15.16:0/3354013148 shutdown_connections 2024-09-21T03:57:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.987+0000 7f2995dce740 1 -- 172.21.15.16:0/3354013148 wait complete. 2024-09-21T03:57:25.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:25.990+0000 7f2995dce740 20 client.26665 trim_cache size 0 max 16384 2024-09-21T03:57:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.030+0000 7f2995dce740 1 Processor -- start 2024-09-21T03:57:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.030+0000 7f2995dce740 1 -- start start 2024-09-21T03:57:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.031+0000 7f2995dce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b392f0 0x55e7a1f1fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.031+0000 7f2995dce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b414d0 0x55e7a1effa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.031+0000 7f2995dce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b621b0 0x55e7a1f11a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.031+0000 7f2995dce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29881b2a50 con 0x55e7a1b621b0 2024-09-21T03:57:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.031+0000 7f2995dce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29881b4590 con 0x55e7a1b414d0 2024-09-21T03:57:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.031+0000 7f2995dce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29881b3120 con 0x55e7a1b392f0 2024-09-21T03:57:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.032+0000 7f298cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b621b0 0x55e7a1f11a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.032+0000 7f2987400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b414d0 0x55e7a1effa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.033+0000 7f2987400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b414d0 0x55e7a1effa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:60026/0 (socket says 172.21.15.16:60026) 2024-09-21T03:57:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.033+0000 7f2987400640 1 -- 172.21.15.16:0/62833491 learned_addr learned my addr 172.21.15.16:0/62833491 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.032+0000 7f298cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b621b0 0x55e7a1f11a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51740/0 (socket says 172.21.15.16:51740) 2024-09-21T03:57:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.033+0000 7f2987e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b392f0 0x55e7a1f1fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.033+0000 7f298cc00640 1 -- 172.21.15.16:0/62833491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b392f0 msgr2=0x55e7a1f1fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.033+0000 7f298cc00640 1 --2- 172.21.15.16:0/62833491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b392f0 0x55e7a1f1fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.034+0000 7f298cc00640 1 -- 172.21.15.16:0/62833491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b414d0 msgr2=0x55e7a1effa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.034+0000 7f298cc00640 1 --2- 172.21.15.16:0/62833491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b414d0 0x55e7a1effa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.034+0000 7f298cc00640 1 -- 172.21.15.16:0/62833491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7a1b20fb0 con 0x55e7a1b621b0 2024-09-21T03:57:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.034+0000 7f298cc00640 1 --2- 172.21.15.16:0/62833491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b621b0 0x55e7a1f11a90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2974104720 tx=0x7f297402a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96c6d4a3547d47d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.034+0000 7f2973e00640 1 -- 172.21.15.16:0/62833491 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29740f4020 con 0x55e7a1b621b0 2024-09-21T03:57:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.034+0000 7f2973e00640 1 -- 172.21.15.16:0/62833491 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29740cd5c0 con 0x55e7a1b621b0 2024-09-21T03:57:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.034+0000 7f2973e00640 1 -- 172.21.15.16:0/62833491 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29740040d0 con 0x55e7a1b621b0 2024-09-21T03:57:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.035+0000 7f2995dce740 1 -- 172.21.15.16:0/62833491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b621b0 msgr2=0x55e7a1f11a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.035+0000 7f2995dce740 1 --2- 172.21.15.16:0/62833491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b621b0 0x55e7a1f11a90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2974104720 tx=0x7f297402a380 comp rx=0 tx=0).stop 2024-09-21T03:57:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.036+0000 7f2995dce740 1 -- 172.21.15.16:0/62833491 shutdown_connections 2024-09-21T03:57:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.036+0000 7f2995dce740 1 --2- 172.21.15.16:0/62833491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b621b0 0x55e7a1f11a90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.036+0000 7f2995dce740 1 --2- 172.21.15.16:0/62833491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b414d0 0x55e7a1effa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.036+0000 7f2995dce740 1 --2- 172.21.15.16:0/62833491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b392f0 0x55e7a1f1fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.036+0000 7f2995dce740 1 -- 172.21.15.16:0/62833491 >> 172.21.15.16:0/62833491 conn(0x55e7a1b56300 msgr2=0x55e7a1f09060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.036+0000 7f2995dce740 1 -- 172.21.15.16:0/62833491 shutdown_connections 2024-09-21T03:57:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.036+0000 7f2995dce740 1 -- 172.21.15.16:0/62833491 wait complete. 2024-09-21T03:57:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.037+0000 7f2995dce740 1 -- start start 2024-09-21T03:57:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.037+0000 7f2995dce740 1 Processor -- start 2024-09-21T03:57:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.038+0000 7f2995dce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b392f0 0x55e7a1f175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.038+0000 7f2995dce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b414d0 0x55e7a1f17c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.038+0000 7f2995dce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b621b0 0x55e7a1f18800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.038+0000 7f2995dce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29881b3ec0 con 0x55e7a1b414d0 2024-09-21T03:57:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.038+0000 7f2995dce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29881b37f0 con 0x55e7a1b621b0 2024-09-21T03:57:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.038+0000 7f2995dce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29881a9690 con 0x55e7a1b392f0 2024-09-21T03:57:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.038+0000 7f2995dce740 10 client.0 _ll_register_callbacks cb 0x7f29780039d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:57:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.039+0000 7f2987e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b392f0 0x55e7a1f175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.039+0000 7f2987e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b392f0 0x55e7a1f175d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:42332/0 (socket says 172.21.15.16:42332) 2024-09-21T03:57:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.039+0000 7f2987e00640 1 -- 172.21.15.16:0/3030627325 learned_addr learned my addr 172.21.15.16:0/3030627325 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.039+0000 7f298cc00640 1 --2- 172.21.15.16:0/3030627325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b621b0 0x55e7a1f18800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.039+0000 7f2987400640 1 --2- 172.21.15.16:0/3030627325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b414d0 0x55e7a1f17c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.040+0000 7f298cc00640 1 -- 172.21.15.16:0/3030627325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b392f0 msgr2=0x55e7a1f175d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.040+0000 7f298cc00640 1 --2- 172.21.15.16:0/3030627325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b392f0 0x55e7a1f175d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.040+0000 7f298cc00640 1 -- 172.21.15.16:0/3030627325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b414d0 msgr2=0x55e7a1f17c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.040+0000 7f298cc00640 1 --2- 172.21.15.16:0/3030627325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b414d0 0x55e7a1f17c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.040+0000 7f298cc00640 1 -- 172.21.15.16:0/3030627325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7a1f18f60 con 0x55e7a1b621b0 2024-09-21T03:57:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.041+0000 7f298cc00640 1 --2- 172.21.15.16:0/3030627325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b621b0 0x55e7a1f18800 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55e7a1f0a210 tx=0x7f2974005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91bf2cf975490daa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.041+0000 7f2971600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:57:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.041+0000 7f2971600640 1 -- 172.21.15.16:0/3030627325 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29740f4070 con 0x55e7a1b621b0 2024-09-21T03:57:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.041+0000 7f2971600640 1 -- 172.21.15.16:0/3030627325 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f29740d65c0 con 0x55e7a1b621b0 2024-09-21T03:57:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.042+0000 7f2995dce740 1 -- 172.21.15.16:0/3030627325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29740cd5c0 con 0x55e7a1b621b0 2024-09-21T03:57:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.042+0000 7f2971600640 1 -- 172.21.15.16:0/3030627325 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f29740d6850 con 0x55e7a1b621b0 2024-09-21T03:57:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.042+0000 7f2995dce740 10 client.26714 fetch_fsmap learned FSMap version 30 2024-09-21T03:57:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.043+0000 7f2995dce740 1 -- 172.21.15.16:0/3030627325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e7a1f1a250 con 0x55e7a1b621b0 2024-09-21T03:57:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.043+0000 7f2971600640 1 -- 172.21.15.16:0/3030627325 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29740cd5c0 con 0x55e7a1b621b0 2024-09-21T03:57:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.045+0000 7f2971600640 1 -- 172.21.15.16:0/3030627325 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f29740c8810 con 0x55e7a1b621b0 2024-09-21T03:57:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.045+0000 7f2995dce740 10 client.26714 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:57:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.045+0000 7f2995dce740 10 client.26714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:57:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.045+0000 7f2995dce740 20 client.26714 populate_metadata read hostname 'smithi016' 2024-09-21T03:57:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.045+0000 7f2995dce740 1 --2- 172.21.15.16:0/3030627325 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e7a1af61c0 0x55e7a1f8ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.045+0000 7f2995dce740 4 client.26714 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:57:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.045+0000 7f2995dce740 1 -- 172.21.15.16:0/3030627325 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e7a1e7a0b0 con 0x55e7a1af61c0 2024-09-21T03:57:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.045+0000 7f2987e00640 1 --2- 172.21.15.16:0/3030627325 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e7a1af61c0 0x55e7a1f8ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.046+0000 7f2971600640 10 client.26714 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.046+0000 7f2987e00640 1 --2- 172.21.15.16:0/3030627325 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e7a1af61c0 0x55e7a1f8ca90 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2efbdcd9f6d21882 server_cookie=f135724e92f4ffbf in_seq=0 out_seq=0 2024-09-21T03:57:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.059+0000 7f2971600640 1 -- 172.21.15.16:0/3030627325 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e7a1e7a0b0 con 0x55e7a1af61c0 2024-09-21T03:57:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.060+0000 7f2971600640 10 client.26714 handle_command_reply: tid=0 2024-09-21T03:57:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.060+0000 7f2995dce740 1 client.26714 shutdown 2024-09-21T03:57:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.060+0000 7f2995dce740 1 -- 172.21.15.16:0/3030627325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b621b0 msgr2=0x55e7a1f18800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.060+0000 7f2995dce740 1 --2- 172.21.15.16:0/3030627325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b621b0 0x55e7a1f18800 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55e7a1f0a210 tx=0x7f2974005f90 comp rx=0 tx=0).stop 2024-09-21T03:57:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.060+0000 7f2995dce740 1 -- 172.21.15.16:0/3030627325 shutdown_connections 2024-09-21T03:57:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.060+0000 7f2995dce740 1 --2- 172.21.15.16:0/3030627325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7a1b621b0 0x55e7a1f18800 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.060+0000 7f2995dce740 1 --2- 172.21.15.16:0/3030627325 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55e7a1af61c0 0x55e7a1f8ca90 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.060+0000 7f2995dce740 1 --2- 172.21.15.16:0/3030627325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7a1b414d0 0x55e7a1f17c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.060+0000 7f2995dce740 1 --2- 172.21.15.16:0/3030627325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7a1b392f0 0x55e7a1f175d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.060+0000 7f2995dce740 1 -- 172.21.15.16:0/3030627325 >> 172.21.15.16:0/3030627325 conn(0x55e7a1b56300 msgr2=0x55e7a1f1fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.060+0000 7f2971600640 0 client.26714 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.060+0000 7f2995dce740 1 -- 172.21.15.16:0/3030627325 shutdown_connections 2024-09-21T03:57:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.061+0000 7f2995dce740 1 -- 172.21.15.16:0/3030627325 wait complete. 2024-09-21T03:57:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.061+0000 7f2995dce740 20 client.26714 trim_cache size 0 max 16384 2024-09-21T03:57:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:57:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:57:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:57:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:57:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.062+0000 7f298d600640 1 -- 172.21.15.16:0/2561297269 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f296c0b6720 msgr2=0x7f296c0b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.062+0000 7f298d600640 1 --2- 172.21.15.16:0/2561297269 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f296c0b6720 0x7f296c0b8c30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2978009fd0 tx=0x7f2978009260 comp rx=0 tx=0).stop 2024-09-21T03:57:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.062+0000 7f298d600640 1 -- 172.21.15.16:0/2561297269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29881100a0 msgr2=0x7f29881a9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.062+0000 7f298d600640 1 --2- 172.21.15.16:0/2561297269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29881100a0 0x7f29881a9dc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2974009780 tx=0x7f2974030b40 comp rx=0 tx=0).stop 2024-09-21T03:57:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.062+0000 7f298d600640 1 -- 172.21.15.16:0/2561297269 shutdown_connections 2024-09-21T03:57:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.062+0000 7f298d600640 1 --2- 172.21.15.16:0/2561297269 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f296c0b6720 0x7f296c0b8c30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.062+0000 7f298d600640 1 --2- 172.21.15.16:0/2561297269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29881100a0 0x7f29881a9dc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.062+0000 7f298d600640 1 --2- 172.21.15.16:0/2561297269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f298810d480 0x7f29881a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.062+0000 7f298d600640 1 --2- 172.21.15.16:0/2561297269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f298806be20 0x7f29881a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.062+0000 7f298d600640 1 -- 172.21.15.16:0/2561297269 >> 172.21.15.16:0/2561297269 conn(0x7f2988070aa0 msgr2=0x7f2988114920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.062+0000 7f298d600640 1 -- 172.21.15.16:0/2561297269 shutdown_connections 2024-09-21T03:57:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:26.062+0000 7f298d600640 1 -- 172.21.15.16:0/2561297269 wait complete. 2024-09-21T03:57:26.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:26 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3743027763' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:57:26.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:26 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3743027763' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:57:26.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:26 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3743027763' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:57:27.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:27 smithi016 ceph-mon[30375]: pgmap v752: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 150 MiB/s wr, 1.53k op/s 2024-09-21T03:57:27.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:27 smithi134 ceph-mon[35031]: pgmap v752: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 150 MiB/s wr, 1.53k op/s 2024-09-21T03:57:27.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:27 smithi123 ceph-mon[34673]: pgmap v752: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 150 MiB/s wr, 1.53k op/s 2024-09-21T03:57:29.275 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-21T03:57:29.275 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-21T03:57:29.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:29 smithi134 ceph-mon[35031]: pgmap v753: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 125 MiB/s wr, 1.23k op/s 2024-09-21T03:57:29.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:29 smithi016 ceph-mon[30375]: pgmap v753: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 125 MiB/s wr, 1.23k op/s 2024-09-21T03:57:29.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:29 smithi123 ceph-mon[34673]: pgmap v753: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 125 MiB/s wr, 1.23k op/s 2024-09-21T03:57:30.238 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:57:31.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:31 smithi134 ceph-mon[35031]: pgmap v754: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 159 MiB/s wr, 1.43k op/s 2024-09-21T03:57:31.397 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:31 smithi016 ceph-mon[30375]: pgmap v754: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 159 MiB/s wr, 1.43k op/s 2024-09-21T03:57:31.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:31 smithi123 ceph-mon[34673]: pgmap v754: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 159 MiB/s wr, 1.43k op/s 2024-09-21T03:57:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.556+0000 7ff9aec00640 1 -- 172.21.15.16:0/391018256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9a8105140 msgr2=0x7ff9a810d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.556+0000 7ff9aec00640 1 --2- 172.21.15.16:0/391018256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9a8105140 0x7ff9a810d690 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff994009900 tx=0x7ff99402a420 comp rx=0 tx=0).stop 2024-09-21T03:57:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.557+0000 7ff9aec00640 1 -- 172.21.15.16:0/391018256 shutdown_connections 2024-09-21T03:57:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.557+0000 7ff9aec00640 1 --2- 172.21.15.16:0/391018256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a810dc50 0x7ff9a8110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.557+0000 7ff9aec00640 1 --2- 172.21.15.16:0/391018256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9a8105140 0x7ff9a810d690 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.557+0000 7ff9aec00640 1 --2- 172.21.15.16:0/391018256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a81047f0 0x7ff9a8104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.557+0000 7ff9aec00640 1 -- 172.21.15.16:0/391018256 >> 172.21.15.16:0/391018256 conn(0x7ff9a80fe330 msgr2=0x7ff9a8100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:32.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.558+0000 7ff9aec00640 1 -- 172.21.15.16:0/391018256 shutdown_connections 2024-09-21T03:57:32.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.558+0000 7ff9aec00640 1 -- 172.21.15.16:0/391018256 wait complete. 2024-09-21T03:57:32.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.560+0000 7ff9aec00640 1 Processor -- start 2024-09-21T03:57:32.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.560+0000 7ff9aec00640 1 -- start start 2024-09-21T03:57:32.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.561+0000 7ff9aec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a81047f0 0x7ff9a8109470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.561+0000 7ff9aec00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a8105140 0x7ff9a8105750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.561+0000 7ff9aec00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9a810dc50 0x7ff9a8105e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.561+0000 7ff9aec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81139e0 con 0x7ff9a81047f0 2024-09-21T03:57:32.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.561+0000 7ff9aec00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a8113860 con 0x7ff9a810dc50 2024-09-21T03:57:32.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.561+0000 7ff9aec00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a8113b60 con 0x7ff9a8105140 2024-09-21T03:57:32.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.564+0000 7ff9ace00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a8105140 0x7ff9a8105750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:32.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.564+0000 7ff9ace00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a8105140 0x7ff9a8105750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:44790/0 (socket says 172.21.15.16:44790) 2024-09-21T03:57:32.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.565+0000 7ff9ace00640 1 -- 172.21.15.16:0/1568834459 learned_addr learned my addr 172.21.15.16:0/1568834459 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:32.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.566+0000 7ff9ae200640 1 --2- 172.21.15.16:0/1568834459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9a810dc50 0x7ff9a8105e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:32.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.566+0000 7ff9ae200640 1 -- 172.21.15.16:0/1568834459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a8105140 msgr2=0x7ff9a8105750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:32.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.567+0000 7ff9ae200640 1 --2- 172.21.15.16:0/1568834459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a8105140 0x7ff9a8105750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.567+0000 7ff9ae200640 1 -- 172.21.15.16:0/1568834459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a81047f0 msgr2=0x7ff9a8109470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:32.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.567+0000 7ff9ae200640 1 --2- 172.21.15.16:0/1568834459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a81047f0 0x7ff9a8109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.567+0000 7ff9ae200640 1 -- 172.21.15.16:0/1568834459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a819c380 con 0x7ff9a810dc50 2024-09-21T03:57:32.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.571+0000 7ff9ae200640 1 --2- 172.21.15.16:0/1568834459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9a810dc50 0x7ff9a8105e70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00b410 tx=0x7ff99c00b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:32.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.572+0000 7ff9a6000640 1 -- 172.21.15.16:0/1568834459 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c015020 con 0x7ff9a810dc50 2024-09-21T03:57:32.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.572+0000 7ff9a6000640 1 -- 172.21.15.16:0/1568834459 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff99c011ce0 con 0x7ff9a810dc50 2024-09-21T03:57:32.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.572+0000 7ff9a6000640 1 -- 172.21.15.16:0/1568834459 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c015020 con 0x7ff9a810dc50 2024-09-21T03:57:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.573+0000 7ff9aec00640 1 -- 172.21.15.16:0/1568834459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a819c670 con 0x7ff9a810dc50 2024-09-21T03:57:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.575+0000 7ff9aec00640 1 -- 172.21.15.16:0/1568834459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a819cb50 con 0x7ff9a810dc50 2024-09-21T03:57:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.580+0000 7ff9a6000640 1 -- 172.21.15.16:0/1568834459 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff99c018020 con 0x7ff9a810dc50 2024-09-21T03:57:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.583+0000 7ff9a6000640 1 --2- 172.21.15.16:0/1568834459 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff9900b67c0 0x7ff9900b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.584+0000 7ff9a6000640 1 -- 172.21.15.16:0/1568834459 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7ff99c0a7c00 con 0x7ff9a810dc50 2024-09-21T03:57:32.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.583+0000 7ff9ad800640 1 --2- 172.21.15.16:0/1568834459 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff9900b67c0 0x7ff9900b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:32.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.624+0000 7ff9ad800640 1 --2- 172.21.15.16:0/1568834459 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff9900b67c0 0x7ff9900b8cd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff9a81091c0 tx=0x7ff9a0009250 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:32.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.627+0000 7ff9b6ea5740 1 Processor -- start 2024-09-21T03:57:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.627+0000 7ff9b6ea5740 1 -- start start 2024-09-21T03:57:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.628+0000 7ff9b6ea5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910034840 0x5559103dc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.628+0000 7ff9b6ea5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55591000b980 0x5559103dca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.628+0000 7ff9b6ea5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555910013b60 0x5559103dcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.628+0000 7ff9b6ea5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555910351ca0 con 0x555910013b60 2024-09-21T03:57:32.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.628+0000 7ff9b6ea5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5559103410d0 con 0x555910034840 2024-09-21T03:57:32.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.628+0000 7ff9b6ea5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555910344ee0 con 0x55591000b980 2024-09-21T03:57:32.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.629+0000 7ff9ad800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910034840 0x5559103dc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.629+0000 7ff9ad800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910034840 0x5559103dc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:35620/0 (socket says 172.21.15.16:35620) 2024-09-21T03:57:32.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.632+0000 7ff9ad800640 1 -- 172.21.15.16:0/495226016 learned_addr learned my addr 172.21.15.16:0/495226016 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:32.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.630+0000 7ff9ae200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555910013b60 0x5559103dcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.633+0000 7ff9ace00640 1 --2- 172.21.15.16:0/495226016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55591000b980 0x5559103dca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.633+0000 7ff9ad800640 1 -- 172.21.15.16:0/495226016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55591000b980 msgr2=0x5559103dca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.633+0000 7ff9ad800640 1 --2- 172.21.15.16:0/495226016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55591000b980 0x5559103dca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.633+0000 7ff9ad800640 1 -- 172.21.15.16:0/495226016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555910013b60 msgr2=0x5559103dcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.633+0000 7ff9ad800640 1 --2- 172.21.15.16:0/495226016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555910013b60 0x5559103dcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.633+0000 7ff9ad800640 1 -- 172.21.15.16:0/495226016 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559102f3400 con 0x555910034840 2024-09-21T03:57:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.637+0000 7ff9ad800640 1 --2- 172.21.15.16:0/495226016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910034840 0x5559103dc510 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009710 tx=0x7ff9a000f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc488d114d2c2e01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.637+0000 7ff99aa00640 1 -- 172.21.15.16:0/495226016 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0019830 con 0x555910034840 2024-09-21T03:57:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.637+0000 7ff99aa00640 1 -- 172.21.15.16:0/495226016 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9a0021400 con 0x555910034840 2024-09-21T03:57:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.637+0000 7ff99aa00640 1 -- 172.21.15.16:0/495226016 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a00175f0 con 0x555910034840 2024-09-21T03:57:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.638+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/495226016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910034840 msgr2=0x5559103dc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.638+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/495226016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910034840 0x5559103dc510 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009710 tx=0x7ff9a000f000 comp rx=0 tx=0).stop 2024-09-21T03:57:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.641+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/495226016 shutdown_connections 2024-09-21T03:57:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.641+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/495226016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555910013b60 0x5559103dcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.641+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/495226016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55591000b980 0x5559103dca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.641+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/495226016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910034840 0x5559103dc510 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.641+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/495226016 >> 172.21.15.16:0/495226016 conn(0x555910028990 msgr2=0x5559103da400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.643+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/495226016 shutdown_connections 2024-09-21T03:57:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.643+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/495226016 wait complete. 2024-09-21T03:57:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.644+0000 7ff9b6ea5740 1 -- start start 2024-09-21T03:57:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.645+0000 7ff9b6ea5740 1 Processor -- start 2024-09-21T03:57:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.645+0000 7ff9b6ea5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55591000b980 0x5559103e7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.648+0000 7ff9b6ea5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910013b60 0x5559103e80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.648+0000 7ff9b6ea5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555910034840 0x555910415ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.648+0000 7ff9b6ea5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559101a27a0 con 0x55591000b980 2024-09-21T03:57:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.648+0000 7ff9b6ea5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5559102e6a10 con 0x555910013b60 2024-09-21T03:57:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.648+0000 7ff9b6ea5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5559103eacf0 con 0x555910034840 2024-09-21T03:57:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.648+0000 7ff9b6ea5740 10 client.0 _ll_register_callbacks cb 0x5559102f6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:57:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.648+0000 7ff9ad800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55591000b980 0x5559103e7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.648+0000 7ff9ace00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910013b60 0x5559103e80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.648+0000 7ff9ace00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910013b60 0x5559103e80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:35632/0 (socket says 172.21.15.16:35632) 2024-09-21T03:57:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.648+0000 7ff9ace00640 1 -- 172.21.15.16:0/4252475874 learned_addr learned my addr 172.21.15.16:0/4252475874 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.649+0000 7ff9ace00640 1 -- 172.21.15.16:0/4252475874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555910034840 msgr2=0x555910415ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:57:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.649+0000 7ff9ace00640 1 --2- 172.21.15.16:0/4252475874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555910034840 0x555910415ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.649+0000 7ff9ace00640 1 -- 172.21.15.16:0/4252475874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55591000b980 msgr2=0x5559103e7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.649+0000 7ff9ace00640 1 --2- 172.21.15.16:0/4252475874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55591000b980 0x5559103e7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.649+0000 7ff9ace00640 1 -- 172.21.15.16:0/4252475874 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559100aeb90 con 0x555910013b60 2024-09-21T03:57:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.650+0000 7ff9ad800640 1 --2- 172.21.15.16:0/4252475874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55591000b980 0x5559103e7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:57:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.650+0000 7ff9ace00640 1 --2- 172.21.15.16:0/4252475874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910013b60 0x5559103e80c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff994009850 tx=0x7ff994004490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc32edba7661f5a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.650+0000 7ff98f400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:57:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.650+0000 7ff98f400640 1 -- 172.21.15.16:0/4252475874 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994035070 con 0x555910013b60 2024-09-21T03:57:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.650+0000 7ff98f400640 1 -- 172.21.15.16:0/4252475874 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7ff994005220 con 0x555910013b60 2024-09-21T03:57:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.650+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/4252475874 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a0021400 con 0x555910013b60 2024-09-21T03:57:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.651+0000 7ff98f400640 1 -- 172.21.15.16:0/4252475874 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff994005420 con 0x555910013b60 2024-09-21T03:57:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.651+0000 7ff9b6ea5740 10 client.26726 fetch_fsmap learned FSMap version 30 2024-09-21T03:57:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.651+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/4252475874 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559103eafd0 con 0x555910013b60 2024-09-21T03:57:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.651+0000 7ff98f400640 1 -- 172.21.15.16:0/4252475874 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9a0021400 con 0x555910013b60 2024-09-21T03:57:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.652+0000 7ff98f400640 1 -- 172.21.15.16:0/4252475874 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff994003b80 con 0x555910013b60 2024-09-21T03:57:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.652+0000 7ff9b6ea5740 10 client.26726 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:57:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.652+0000 7ff9b6ea5740 10 client.26726 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:57:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.652+0000 7ff9b6ea5740 20 client.26726 populate_metadata read hostname 'smithi016' 2024-09-21T03:57:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.652+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/4252475874 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55590fff31c0 0x5559103ee530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.652+0000 7ff9b6ea5740 4 client.26726 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:57:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.652+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/4252475874 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555910353bf0 con 0x55590fff31c0 2024-09-21T03:57:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.652+0000 7ff9ad800640 1 --2- 172.21.15.16:0/4252475874 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55590fff31c0 0x5559103ee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.653+0000 7ff9ad800640 1 --2- 172.21.15.16:0/4252475874 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55590fff31c0 0x5559103ee530 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5960d550635c525 server_cookie=8f6c8f120299efdd in_seq=0 out_seq=0 2024-09-21T03:57:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.653+0000 7ff98f400640 10 client.26726 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.654+0000 7ff98f400640 1 -- 172.21.15.16:0/4252475874 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555910353bf0 con 0x55590fff31c0 2024-09-21T03:57:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.654+0000 7ff98f400640 10 client.26726 handle_command_reply: tid=0 2024-09-21T03:57:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.654+0000 7ff9b6ea5740 1 client.26726 shutdown 2024-09-21T03:57:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.655+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/4252475874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910013b60 msgr2=0x5559103e80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.655+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/4252475874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910013b60 0x5559103e80c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff994009850 tx=0x7ff994004490 comp rx=0 tx=0).stop 2024-09-21T03:57:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.655+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/4252475874 shutdown_connections 2024-09-21T03:57:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.655+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/4252475874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555910034840 0x555910415ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.655+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/4252475874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910013b60 0x5559103e80c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.655+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/4252475874 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55590fff31c0 0x5559103ee530 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.655+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/4252475874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55591000b980 0x5559103e7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.655+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/4252475874 >> 172.21.15.16:0/4252475874 conn(0x555910028990 msgr2=0x5559103df6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.655+0000 7ff98f400640 0 client.26726 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.655+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/4252475874 shutdown_connections 2024-09-21T03:57:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.655+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/4252475874 wait complete. 2024-09-21T03:57:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.655+0000 7ff9b6ea5740 20 client.26726 trim_cache size 0 max 16384 2024-09-21T03:57:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.707+0000 7ff9b6ea5740 1 Processor -- start 2024-09-21T03:57:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.707+0000 7ff9b6ea5740 1 -- start start 2024-09-21T03:57:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.707+0000 7ff9b6ea5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55591000b980 0x5559103dffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.707+0000 7ff9b6ea5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910013b60 0x5559103ddc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.707+0000 7ff9b6ea5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555910034840 0x555910415cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.707+0000 7ff9b6ea5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81aebf0 con 0x555910034840 2024-09-21T03:57:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.707+0000 7ff9b6ea5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81b0730 con 0x555910013b60 2024-09-21T03:57:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.707+0000 7ff9b6ea5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81af2c0 con 0x55591000b980 2024-09-21T03:57:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.708+0000 7ff9ad800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55591000b980 0x5559103dffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.708+0000 7ff9ad800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55591000b980 0x5559103dffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:44820/0 (socket says 172.21.15.16:44820) 2024-09-21T03:57:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.708+0000 7ff9ad800640 1 -- 172.21.15.16:0/3508042433 learned_addr learned my addr 172.21.15.16:0/3508042433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.708+0000 7ff9ace00640 1 --2- 172.21.15.16:0/3508042433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910013b60 0x5559103ddc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.708+0000 7ff9ad800640 1 -- 172.21.15.16:0/3508042433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910013b60 msgr2=0x5559103ddc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.708+0000 7ff9ad800640 1 --2- 172.21.15.16:0/3508042433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910013b60 0x5559103ddc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.709+0000 7ff9ad800640 1 -- 172.21.15.16:0/3508042433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555910034840 msgr2=0x555910415cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.709+0000 7ff9ad800640 1 --2- 172.21.15.16:0/3508042433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555910034840 0x555910415cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.709+0000 7ff9ad800640 1 -- 172.21.15.16:0/3508042433 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559103e8ad0 con 0x55591000b980 2024-09-21T03:57:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.709+0000 7ff9ad800640 1 --2- 172.21.15.16:0/3508042433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55591000b980 0x5559103dffd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0021b90 tx=0x7ff9a0021d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afda6960991198f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.709+0000 7ff99a000640 1 -- 172.21.15.16:0/3508042433 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a00093a0 con 0x55591000b980 2024-09-21T03:57:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.709+0000 7ff99a000640 1 -- 172.21.15.16:0/3508042433 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9a0021400 con 0x55591000b980 2024-09-21T03:57:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.709+0000 7ff99a000640 1 -- 172.21.15.16:0/3508042433 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0004020 con 0x55591000b980 2024-09-21T03:57:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.711+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/3508042433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55591000b980 msgr2=0x5559103dffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.711+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/3508042433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55591000b980 0x5559103dffd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0021b90 tx=0x7ff9a0021d60 comp rx=0 tx=0).stop 2024-09-21T03:57:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.712+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/3508042433 shutdown_connections 2024-09-21T03:57:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.712+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/3508042433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555910034840 0x555910415cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.712+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/3508042433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555910013b60 0x5559103ddc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.712+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/3508042433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55591000b980 0x5559103dffd0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.712+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/3508042433 >> 172.21.15.16:0/3508042433 conn(0x555910028990 msgr2=0x5559103dd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.713+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/3508042433 shutdown_connections 2024-09-21T03:57:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.715+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/3508042433 wait complete. 2024-09-21T03:57:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.716+0000 7ff9b6ea5740 1 -- start start 2024-09-21T03:57:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.717+0000 7ff9b6ea5740 1 Processor -- start 2024-09-21T03:57:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.718+0000 7ff9b6ea5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55591000b980 0x555910400370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.719+0000 7ff9b6ea5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555910013b60 0x5559103fb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.719+0000 7ff9b6ea5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555910034840 0x5559103fbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.719+0000 7ff9b6ea5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81b0060 con 0x555910013b60 2024-09-21T03:57:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.719+0000 7ff9b6ea5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81af990 con 0x55591000b980 2024-09-21T03:57:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.719+0000 7ff9b6ea5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a8106a50 con 0x555910034840 2024-09-21T03:57:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.719+0000 7ff9b6ea5740 10 client.0 _ll_register_callbacks cb 0x7ff994005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:57:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.719+0000 7ff9ad800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55591000b980 0x555910400370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:32.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.720+0000 7ff9ad800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55591000b980 0x555910400370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:35644/0 (socket says 172.21.15.16:35644) 2024-09-21T03:57:32.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.720+0000 7ff9ad800640 1 -- 172.21.15.16:0/1275599493 learned_addr learned my addr 172.21.15.16:0/1275599493 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:32.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.720+0000 7ff9ace00640 1 --2- 172.21.15.16:0/1275599493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555910013b60 0x5559103fb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:32.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.720+0000 7ff9ad800640 1 -- 172.21.15.16:0/1275599493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555910034840 msgr2=0x5559103fbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:32.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.720+0000 7ff9ad800640 1 --2- 172.21.15.16:0/1275599493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555910034840 0x5559103fbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.720+0000 7ff9ad800640 1 -- 172.21.15.16:0/1275599493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555910013b60 msgr2=0x5559103fb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:32.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.721+0000 7ff9ad800640 1 --2- 172.21.15.16:0/1275599493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555910013b60 0x5559103fb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.721+0000 7ff9ad800640 1 -- 172.21.15.16:0/1275599493 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559103fc400 con 0x55591000b980 2024-09-21T03:57:32.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.721+0000 7ff9ace00640 1 --2- 172.21.15.16:0/1275599493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555910013b60 0x5559103fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:57:32.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.722+0000 7ff9ad800640 1 --2- 172.21.15.16:0/1275599493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55591000b980 0x555910400370 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0019bb0 tx=0x7ff9a0009430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd3e1d5a7cfda18f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.722+0000 7ff98f400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:57:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.724+0000 7ff98f400640 1 -- 172.21.15.16:0/1275599493 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0044020 con 0x55591000b980 2024-09-21T03:57:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.724+0000 7ff98f400640 1 -- 172.21.15.16:0/1275599493 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7ff9a0019630 con 0x55591000b980 2024-09-21T03:57:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.726+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/1275599493 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a0021400 con 0x55591000b980 2024-09-21T03:57:32.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.727+0000 7ff98f400640 1 -- 172.21.15.16:0/1275599493 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff9a0019e50 con 0x55591000b980 2024-09-21T03:57:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.732+0000 7ff9b6ea5740 10 client.26732 fetch_fsmap learned FSMap version 30 2024-09-21T03:57:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.732+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/1275599493 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555910464970 con 0x55591000b980 2024-09-21T03:57:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.733+0000 7ff98f400640 1 -- 172.21.15.16:0/1275599493 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9a0021400 con 0x55591000b980 2024-09-21T03:57:32.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.733+0000 7ff98f400640 1 -- 172.21.15.16:0/1275599493 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff9a0051020 con 0x55591000b980 2024-09-21T03:57:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.734+0000 7ff9b6ea5740 10 client.26732 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:57:32.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.734+0000 7ff9b6ea5740 10 client.26732 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:57:32.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.734+0000 7ff9b6ea5740 20 client.26732 populate_metadata read hostname 'smithi016' 2024-09-21T03:57:32.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.734+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/1275599493 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55590ffc8570 0x555910467060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:32.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.734+0000 7ff9b6ea5740 4 client.26732 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-21T03:57:32.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.734+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/1275599493 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x55591034f6a0 con 0x55590ffc8570 2024-09-21T03:57:32.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.735+0000 7ff9ace00640 1 --2- 172.21.15.16:0/1275599493 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55590ffc8570 0x555910467060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:32.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.736+0000 7ff9ace00640 1 --2- 172.21.15.16:0/1275599493 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55590ffc8570 0x555910467060 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b98a8fd489aa1c6f server_cookie=87e57567b36a3d5b in_seq=0 out_seq=0 2024-09-21T03:57:32.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.738+0000 7ff98f400640 10 client.26732 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.957+0000 7ff98f400640 1 -- 172.21.15.16:0/1275599493 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55591034f6a0 con 0x55590ffc8570 2024-09-21T03:57:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.957+0000 7ff98f400640 10 client.26732 handle_command_reply: tid=0 2024-09-21T03:57:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.957+0000 7ff9b6ea5740 1 client.26732 shutdown 2024-09-21T03:57:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.958+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/1275599493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55591000b980 msgr2=0x555910400370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.958+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/1275599493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55591000b980 0x555910400370 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0019bb0 tx=0x7ff9a0009430 comp rx=0 tx=0).stop 2024-09-21T03:57:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.958+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/1275599493 shutdown_connections 2024-09-21T03:57:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.958+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/1275599493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555910034840 0x5559103fbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.958+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/1275599493 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55590ffc8570 0x555910467060 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.959+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/1275599493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555910013b60 0x5559103fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.959+0000 7ff98f400640 0 client.26732 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:32.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.959+0000 7ff9b6ea5740 1 --2- 172.21.15.16:0/1275599493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55591000b980 0x555910400370 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.959+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/1275599493 >> 172.21.15.16:0/1275599493 conn(0x555910028990 msgr2=0x5559103e6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:32.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.959+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/1275599493 shutdown_connections 2024-09-21T03:57:32.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.960+0000 7ff9b6ea5740 1 -- 172.21.15.16:0/1275599493 wait complete. 2024-09-21T03:57:32.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.960+0000 7ff9b6ea5740 20 client.26732 trim_cache size 0 max 16384 2024-09-21T03:57:32.965 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:57:32.965 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:57:32.965 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:57:32.965 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 42, 2024-09-21T03:57:32.965 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:57:32.965 INFO:teuthology.orchestra.run.smithi016.stdout: "ce95ca56": { 2024-09-21T03:57:32.965 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 42, 2024-09-21T03:57:32.965 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:57:32.965 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:57:32.965 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:57:32.965 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:57:32.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:57:32.965 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:57:32.965 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 144.5, 2024-09-21T03:57:32.966 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:57:32.966 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:57:32.966 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:57:32.966 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:57:32.966 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-21T03:57:32.966 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:57:32.966 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:57:32.966 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:57:32.966 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:57:32.966 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:57:32.966 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:57:32.966 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:57:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.962+0000 7ff9aec00640 1 -- 172.21.15.16:0/1568834459 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff9900b67c0 msgr2=0x7ff9900b8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.962+0000 7ff9aec00640 1 --2- 172.21.15.16:0/1568834459 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff9900b67c0 0x7ff9900b8cd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff9a81091c0 tx=0x7ff9a0009250 comp rx=0 tx=0).stop 2024-09-21T03:57:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.962+0000 7ff9aec00640 1 -- 172.21.15.16:0/1568834459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9a810dc50 msgr2=0x7ff9a8105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.962+0000 7ff9aec00640 1 --2- 172.21.15.16:0/1568834459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9a810dc50 0x7ff9a8105e70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00b410 tx=0x7ff99c00b8d0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.963+0000 7ff9aec00640 1 -- 172.21.15.16:0/1568834459 shutdown_connections 2024-09-21T03:57:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.963+0000 7ff9aec00640 1 --2- 172.21.15.16:0/1568834459 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7ff9900b67c0 0x7ff9900b8cd0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.963+0000 7ff9aec00640 1 --2- 172.21.15.16:0/1568834459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9a810dc50 0x7ff9a8105e70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.963+0000 7ff9aec00640 1 --2- 172.21.15.16:0/1568834459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a8105140 0x7ff9a8105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.963+0000 7ff9aec00640 1 --2- 172.21.15.16:0/1568834459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a81047f0 0x7ff9a8109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.963+0000 7ff9aec00640 1 -- 172.21.15.16:0/1568834459 >> 172.21.15.16:0/1568834459 conn(0x7ff9a80fe330 msgr2=0x7ff9a8102410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.965+0000 7ff9aec00640 1 -- 172.21.15.16:0/1568834459 shutdown_connections 2024-09-21T03:57:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:32.966+0000 7ff9aec00640 1 -- 172.21.15.16:0/1568834459 wait complete. 2024-09-21T03:57:33.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:33 smithi123 ceph-mon[34673]: pgmap v755: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 179 MiB/s wr, 1.50k op/s 2024-09-21T03:57:33.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:33 smithi016 ceph-mon[30375]: pgmap v755: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 179 MiB/s wr, 1.50k op/s 2024-09-21T03:57:33.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:33 smithi134 ceph-mon[35031]: pgmap v755: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 179 MiB/s wr, 1.50k op/s 2024-09-21T03:57:35.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:35 smithi123 ceph-mon[34673]: pgmap v756: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 184 MiB/s wr, 1.49k op/s 2024-09-21T03:57:35.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:35 smithi016 ceph-mon[30375]: pgmap v756: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 184 MiB/s wr, 1.49k op/s 2024-09-21T03:57:35.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:35 smithi134 ceph-mon[35031]: pgmap v756: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 184 MiB/s wr, 1.49k op/s 2024-09-21T03:57:37.360 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ce95ca56', quiesce took 18.8 seconds. Will release after: 5.699999999999999 2024-09-21T03:57:37.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:37 smithi123 ceph-mon[34673]: pgmap v757: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 214 MiB/s wr, 1.62k op/s 2024-09-21T03:57:37.417 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:37 smithi016 ceph-mon[30375]: pgmap v757: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 214 MiB/s wr, 1.62k op/s 2024-09-21T03:57:37.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:37 smithi134 ceph-mon[35031]: pgmap v757: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 214 MiB/s wr, 1.62k op/s 2024-09-21T03:57:39.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:39 smithi123 ceph-mon[34673]: pgmap v758: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 210 MiB/s wr, 1.47k op/s 2024-09-21T03:57:39.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:39 smithi016 ceph-mon[30375]: pgmap v758: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 210 MiB/s wr, 1.47k op/s 2024-09-21T03:57:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.423+0000 7fb06da00640 1 -- 172.21.15.16:0/2922303262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb068111f90 msgr2=0x7fb0681143b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.423+0000 7fb06da00640 1 --2- 172.21.15.16:0/2922303262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb068111f90 0x7fb0681143b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb060009be0 tx=0x7fb06002a6c0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.426+0000 7fb06da00640 1 -- 172.21.15.16:0/2922303262 shutdown_connections 2024-09-21T03:57:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.426+0000 7fb06da00640 1 --2- 172.21.15.16:0/2922303262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb068111f90 0x7fb0681143b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.426+0000 7fb06da00640 1 --2- 172.21.15.16:0/2922303262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb06810ada0 0x7fb06810b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.426+0000 7fb06da00640 1 --2- 172.21.15.16:0/2922303262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb06810a3c0 0x7fb06810a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.426+0000 7fb06da00640 1 -- 172.21.15.16:0/2922303262 >> 172.21.15.16:0/2922303262 conn(0x7fb068070810 msgr2=0x7fb068072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.426+0000 7fb06da00640 1 -- 172.21.15.16:0/2922303262 shutdown_connections 2024-09-21T03:57:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.427+0000 7fb06da00640 1 -- 172.21.15.16:0/2922303262 wait complete. 2024-09-21T03:57:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.428+0000 7fb06da00640 1 Processor -- start 2024-09-21T03:57:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.428+0000 7fb06da00640 1 -- start start 2024-09-21T03:57:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.429+0000 7fb06da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb06810a3c0 0x7fb068110ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.429+0000 7fb06da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb06810ada0 0x7fb068111540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.429+0000 7fb06da00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb068111f90 0x7fb0681a9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.429+0000 7fb06da00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb068117b10 con 0x7fb06810a3c0 2024-09-21T03:57:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.429+0000 7fb06da00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb068117990 con 0x7fb06810ada0 2024-09-21T03:57:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.429+0000 7fb06da00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb068117c90 con 0x7fb068111f90 2024-09-21T03:57:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.430+0000 7fb05fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb06810ada0 0x7fb068111540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.430+0000 7fb067400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb068111f90 0x7fb0681a9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.430+0000 7fb05fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb06810ada0 0x7fb068111540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:35654/0 (socket says 172.21.15.16:35654) 2024-09-21T03:57:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.430+0000 7fb067400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb068111f90 0x7fb0681a9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:44856/0 (socket says 172.21.15.16:44856) 2024-09-21T03:57:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.430+0000 7fb05fe00640 1 -- 172.21.15.16:0/3319729144 learned_addr learned my addr 172.21.15.16:0/3319729144 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.430+0000 7fb05fe00640 1 -- 172.21.15.16:0/3319729144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb068111f90 msgr2=0x7fb0681a9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.430+0000 7fb05fe00640 1 --2- 172.21.15.16:0/3319729144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb068111f90 0x7fb0681a9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.430+0000 7fb05fe00640 1 -- 172.21.15.16:0/3319729144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb06810a3c0 msgr2=0x7fb068110ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.430+0000 7fb05fe00640 1 --2- 172.21.15.16:0/3319729144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb06810a3c0 0x7fb068110ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.430+0000 7fb05fe00640 1 -- 172.21.15.16:0/3319729144 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0681aa750 con 0x7fb06810ada0 2024-09-21T03:57:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.430+0000 7fb067400640 1 --2- 172.21.15.16:0/3319729144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb068111f90 0x7fb0681a9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T03:57:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.431+0000 7fb05fe00640 1 --2- 172.21.15.16:0/3319729144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb06810ada0 0x7fb068111540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb05000e9f0 tx=0x7fb05000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.433+0000 7fb05f400640 1 -- 172.21.15.16:0/3319729144 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05000cd80 con 0x7fb06810ada0 2024-09-21T03:57:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.433+0000 7fb05f400640 1 -- 172.21.15.16:0/3319729144 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0500108e0 con 0x7fb06810ada0 2024-09-21T03:57:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.434+0000 7fb05f400640 1 -- 172.21.15.16:0/3319729144 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050018880 con 0x7fb06810ada0 2024-09-21T03:57:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.435+0000 7fb06da00640 1 -- 172.21.15.16:0/3319729144 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0681aaa40 con 0x7fb06810ada0 2024-09-21T03:57:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.435+0000 7fb06da00640 1 -- 172.21.15.16:0/3319729144 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0681b1130 con 0x7fb06810ada0 2024-09-21T03:57:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.441+0000 7fb074ac8740 1 Processor -- start 2024-09-21T03:57:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.443+0000 7fb074ac8740 1 -- start start 2024-09-21T03:57:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.444+0000 7fb074ac8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 0x5565db8a90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.445+0000 7fb074ac8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8a9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.445+0000 7fb074ac8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8aa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.445+0000 7fb074ac8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5565db81c6b0 con 0x5565db5021b0 2024-09-21T03:57:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.445+0000 7fb074ac8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5565db80bae0 con 0x5565db4d92f0 2024-09-21T03:57:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.445+0000 7fb074ac8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5565db80f8f0 con 0x5565db4e14d0 2024-09-21T03:57:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.445+0000 7fb05f400640 1 -- 172.21.15.16:0/3319729144 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb050010430 con 0x7fb06810ada0 2024-09-21T03:57:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.445+0000 7fb066a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.445+0000 7fb066a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8a9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:35658/0 (socket says 172.21.15.16:35658) 2024-09-21T03:57:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.445+0000 7fb066a00640 1 -- 172.21.15.16:0/1832949388 learned_addr learned my addr 172.21.15.16:0/1832949388 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.445+0000 7fb067400640 1 --2- 172.21.15.16:0/1832949388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 0x5565db8a90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.445+0000 7fb066a00640 1 -- 172.21.15.16:0/1832949388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 msgr2=0x5565db8aa290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:57:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.445+0000 7fb066a00640 1 --2- 172.21.15.16:0/1832949388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8aa290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.445+0000 7fb066a00640 1 -- 172.21.15.16:0/1832949388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 msgr2=0x5565db8a90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.445+0000 7fb066a00640 1 --2- 172.21.15.16:0/1832949388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 0x5565db8a90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.445+0000 7fb066a00640 1 -- 172.21.15.16:0/1832949388 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565db7d8780 con 0x5565db4d92f0 2024-09-21T03:57:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.445+0000 7fb067400640 1 --2- 172.21.15.16:0/1832949388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 0x5565db8a90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:57:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.446+0000 7fb066a00640 1 --2- 172.21.15.16:0/1832949388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8a9740 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb058009b30 tx=0x7fb05800e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9839d7f81a3ebd60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.447+0000 7fb057e00640 1 -- 172.21.15.16:0/1832949388 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05800ed80 con 0x5565db4d92f0 2024-09-21T03:57:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.447+0000 7fb057e00640 1 -- 172.21.15.16:0/1832949388 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0580054d0 con 0x5565db4d92f0 2024-09-21T03:57:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.447+0000 7fb057e00640 1 -- 172.21.15.16:0/1832949388 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05800ed80 con 0x5565db4d92f0 2024-09-21T03:57:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.452+0000 7fb05f400640 1 --2- 172.21.15.16:0/3319729144 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb04c0b65b0 0x7fb04c0b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.452+0000 7fb05f400640 1 -- 172.21.15.16:0/3319729144 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb0500e79e0 con 0x7fb06810ada0 2024-09-21T03:57:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.453+0000 7fb074ac8740 1 -- 172.21.15.16:0/1832949388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 msgr2=0x5565db8a9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.453+0000 7fb074ac8740 1 --2- 172.21.15.16:0/1832949388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8a9740 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb058009b30 tx=0x7fb05800e350 comp rx=0 tx=0).stop 2024-09-21T03:57:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.454+0000 7fb074ac8740 1 -- 172.21.15.16:0/1832949388 shutdown_connections 2024-09-21T03:57:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.454+0000 7fb074ac8740 1 --2- 172.21.15.16:0/1832949388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8aa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.454+0000 7fb074ac8740 1 --2- 172.21.15.16:0/1832949388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8a9740 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.454+0000 7fb074ac8740 1 --2- 172.21.15.16:0/1832949388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 0x5565db8a90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.454+0000 7fb074ac8740 1 -- 172.21.15.16:0/1832949388 >> 172.21.15.16:0/1832949388 conn(0x5565db4f6300 msgr2=0x5565db8a4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.454+0000 7fb067400640 1 --2- 172.21.15.16:0/3319729144 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb04c0b65b0 0x7fb04c0b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.456+0000 7fb074ac8740 1 -- 172.21.15.16:0/1832949388 shutdown_connections 2024-09-21T03:57:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.457+0000 7fb074ac8740 1 -- 172.21.15.16:0/1832949388 wait complete. 2024-09-21T03:57:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.458+0000 7fb074ac8740 1 -- start start 2024-09-21T03:57:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.458+0000 7fb074ac8740 1 Processor -- start 2024-09-21T03:57:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.459+0000 7fb074ac8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.459+0000 7fb074ac8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.459+0000 7fb074ac8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 0x5565db8b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.460+0000 7fb066a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.460+0000 7fb05fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.460+0000 7fb05fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8b2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:44880/0 (socket says 172.21.15.16:44880) 2024-09-21T03:57:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.460+0000 7fb067400640 1 --2- 172.21.15.16:0/3319729144 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb04c0b65b0 0x7fb04c0b8ac0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb060034040 tx=0x7fb060005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.460+0000 7fb066a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:35662/0 (socket says 172.21.15.16:35662) 2024-09-21T03:57:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.461+0000 7fb05fe00640 1 -- 172.21.15.16:0/2285050984 learned_addr learned my addr 172.21.15.16:0/2285050984 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.461+0000 7fb074ac8740 1 -- 172.21.15.16:0/2285050984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5565db743ab0 con 0x5565db5021b0 2024-09-21T03:57:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.461+0000 7fb074ac8740 1 -- 172.21.15.16:0/2285050984 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5565db7b2570 con 0x5565db4d92f0 2024-09-21T03:57:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.461+0000 7fb074ac8740 1 -- 172.21.15.16:0/2285050984 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5565db8aadb0 con 0x5565db4e14d0 2024-09-21T03:57:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.461+0000 7fb074ac8740 10 client.0 _ll_register_callbacks cb 0x5565db7c1100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:57:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.461+0000 7fb067400640 1 --2- 172.21.15.16:0/2285050984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 0x5565db8b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.461+0000 7fb066a00640 1 -- 172.21.15.16:0/2285050984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 msgr2=0x5565db8b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.461+0000 7fb066a00640 1 --2- 172.21.15.16:0/2285050984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.461+0000 7fb066a00640 1 -- 172.21.15.16:0/2285050984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 msgr2=0x5565db8b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.461+0000 7fb066a00640 1 --2- 172.21.15.16:0/2285050984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 0x5565db8b39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.461+0000 7fb066a00640 1 -- 172.21.15.16:0/2285050984 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565db56d790 con 0x5565db4d92f0 2024-09-21T03:57:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.461+0000 7fb05fe00640 1 --2- 172.21.15.16:0/2285050984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:57:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.462+0000 7fb066a00640 1 --2- 172.21.15.16:0/2285050984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b27b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb0580092a0 tx=0x7fb0580098f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a256b59ccc024872 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.462+0000 7fb054c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:57:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.465+0000 7fb054c00640 1 -- 172.21.15.16:0/2285050984 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05801d040 con 0x5565db4d92f0 2024-09-21T03:57:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.465+0000 7fb054c00640 1 -- 172.21.15.16:0/2285050984 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb058004030 con 0x5565db4d92f0 2024-09-21T03:57:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.465+0000 7fb074ac8740 1 -- 172.21.15.16:0/2285050984 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0580054d0 con 0x5565db4d92f0 2024-09-21T03:57:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.466+0000 7fb054c00640 1 -- 172.21.15.16:0/2285050984 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb05800ee80 con 0x5565db4d92f0 2024-09-21T03:57:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.467+0000 7fb074ac8740 10 client.26744 fetch_fsmap learned FSMap version 30 2024-09-21T03:57:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.468+0000 7fb074ac8740 1 -- 172.21.15.16:0/2285050984 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565db5e6620 con 0x5565db4d92f0 2024-09-21T03:57:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.468+0000 7fb054c00640 1 -- 172.21.15.16:0/2285050984 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0580054d0 con 0x5565db4d92f0 2024-09-21T03:57:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.468+0000 7fb054c00640 1 -- 172.21.15.16:0/2285050984 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb058018430 con 0x5565db4d92f0 2024-09-21T03:57:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.469+0000 7fb074ac8740 10 client.26744 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:57:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.469+0000 7fb074ac8740 10 client.26744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:57:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.469+0000 7fb074ac8740 20 client.26744 populate_metadata read hostname 'smithi016' 2024-09-21T03:57:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.470+0000 7fb074ac8740 1 --2- 172.21.15.16:0/2285050984 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5565db4c0d80 0x5565db8bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.470+0000 7fb074ac8740 4 client.26744 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:57:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.470+0000 7fb074ac8740 1 -- 172.21.15.16:0/2285050984 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565db81e600 con 0x5565db4c0d80 2024-09-21T03:57:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.470+0000 7fb05fe00640 1 --2- 172.21.15.16:0/2285050984 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5565db4c0d80 0x5565db8bfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.471+0000 7fb05fe00640 1 --2- 172.21.15.16:0/2285050984 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5565db4c0d80 0x5565db8bfb70 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55d0f731bce76914 server_cookie=52908551d200dafe in_seq=0 out_seq=0 2024-09-21T03:57:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.472+0000 7fb054c00640 10 client.26744 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.472+0000 7fb054c00640 1 -- 172.21.15.16:0/2285050984 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5565db81e600 con 0x5565db4c0d80 2024-09-21T03:57:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.472+0000 7fb054c00640 10 client.26744 handle_command_reply: tid=0 2024-09-21T03:57:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.473+0000 7fb074ac8740 1 client.26744 shutdown 2024-09-21T03:57:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.475+0000 7fb074ac8740 1 -- 172.21.15.16:0/2285050984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 msgr2=0x5565db8b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.475+0000 7fb074ac8740 1 --2- 172.21.15.16:0/2285050984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b27b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb0580092a0 tx=0x7fb0580098f0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.476+0000 7fb074ac8740 1 -- 172.21.15.16:0/2285050984 shutdown_connections 2024-09-21T03:57:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.476+0000 7fb074ac8740 1 --2- 172.21.15.16:0/2285050984 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5565db4c0d80 0x5565db8bfb70 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.476+0000 7fb054c00640 0 client.26744 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.476+0000 7fb074ac8740 1 --2- 172.21.15.16:0/2285050984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 0x5565db8b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.476+0000 7fb074ac8740 1 --2- 172.21.15.16:0/2285050984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.476+0000 7fb074ac8740 1 --2- 172.21.15.16:0/2285050984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b27b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.476+0000 7fb074ac8740 1 -- 172.21.15.16:0/2285050984 >> 172.21.15.16:0/2285050984 conn(0x5565db4f6300 msgr2=0x5565db8a30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.476+0000 7fb074ac8740 1 -- 172.21.15.16:0/2285050984 shutdown_connections 2024-09-21T03:57:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.476+0000 7fb074ac8740 1 -- 172.21.15.16:0/2285050984 wait complete. 2024-09-21T03:57:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.477+0000 7fb074ac8740 20 client.26744 trim_cache size 0 max 16384 2024-09-21T03:57:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.522+0000 7fb074ac8740 1 Processor -- start 2024-09-21T03:57:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.522+0000 7fb074ac8740 1 -- start start 2024-09-21T03:57:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.523+0000 7fb074ac8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.523+0000 7fb074ac8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8a9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.523+0000 7fb074ac8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 0x5565db8a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.523+0000 7fb074ac8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0681bb390 con 0x5565db5021b0 2024-09-21T03:57:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.523+0000 7fb074ac8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0681bced0 con 0x5565db4d92f0 2024-09-21T03:57:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.523+0000 7fb074ac8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0681bba60 con 0x5565db4e14d0 2024-09-21T03:57:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.523+0000 7fb066a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.524+0000 7fb066a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:35668/0 (socket says 172.21.15.16:35668) 2024-09-21T03:57:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.524+0000 7fb066a00640 1 -- 172.21.15.16:0/3206917173 learned_addr learned my addr 172.21.15.16:0/3206917173 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.523+0000 7fb05fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8a9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.524+0000 7fb067400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 0x5565db8a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.525+0000 7fb066a00640 1 -- 172.21.15.16:0/3206917173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 msgr2=0x5565db8a9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.525+0000 7fb066a00640 1 --2- 172.21.15.16:0/3206917173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8a9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.525+0000 7fb066a00640 1 -- 172.21.15.16:0/3206917173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 msgr2=0x5565db8a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.525+0000 7fb066a00640 1 --2- 172.21.15.16:0/3206917173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 0x5565db8a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.525+0000 7fb066a00640 1 -- 172.21.15.16:0/3206917173 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565db8aa450 con 0x5565db4d92f0 2024-09-21T03:57:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.525+0000 7fb066a00640 1 --2- 172.21.15.16:0/3206917173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b19b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb058018950 tx=0x7fb058018a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddb73ba25a7ed863 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.527+0000 7fb057400640 1 -- 172.21.15.16:0/3206917173 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05803a070 con 0x5565db4d92f0 2024-09-21T03:57:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.528+0000 7fb057400640 1 -- 172.21.15.16:0/3206917173 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb058027070 con 0x5565db4d92f0 2024-09-21T03:57:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.528+0000 7fb057400640 1 -- 172.21.15.16:0/3206917173 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058005470 con 0x5565db4d92f0 2024-09-21T03:57:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.529+0000 7fb074ac8740 1 -- 172.21.15.16:0/3206917173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 msgr2=0x5565db8b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.529+0000 7fb074ac8740 1 --2- 172.21.15.16:0/3206917173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b19b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb058018950 tx=0x7fb058018a90 comp rx=0 tx=0).stop 2024-09-21T03:57:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.530+0000 7fb074ac8740 1 -- 172.21.15.16:0/3206917173 shutdown_connections 2024-09-21T03:57:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.530+0000 7fb074ac8740 1 --2- 172.21.15.16:0/3206917173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 0x5565db8a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.530+0000 7fb074ac8740 1 --2- 172.21.15.16:0/3206917173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8a9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.530+0000 7fb074ac8740 1 --2- 172.21.15.16:0/3206917173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b19b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.530+0000 7fb074ac8740 1 -- 172.21.15.16:0/3206917173 >> 172.21.15.16:0/3206917173 conn(0x5565db4f6300 msgr2=0x5565db8a44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.530+0000 7fb074ac8740 1 -- 172.21.15.16:0/3206917173 shutdown_connections 2024-09-21T03:57:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.530+0000 7fb074ac8740 1 -- 172.21.15.16:0/3206917173 wait complete. 2024-09-21T03:57:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.532+0000 7fb074ac8740 1 -- start start 2024-09-21T03:57:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.532+0000 7fb074ac8740 1 Processor -- start 2024-09-21T03:57:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.533+0000 7fb074ac8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.533+0000 7fb074ac8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.533+0000 7fb074ac8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 0x5565db8b8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.533+0000 7fb05fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.533+0000 7fb066a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.533+0000 7fb074ac8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0681bc800 con 0x5565db5021b0 2024-09-21T03:57:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.533+0000 7fb074ac8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0681bc130 con 0x5565db4d92f0 2024-09-21T03:57:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.533+0000 7fb074ac8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0681b9370 con 0x5565db4e14d0 2024-09-21T03:57:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.534+0000 7fb066a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:35682/0 (socket says 172.21.15.16:35682) 2024-09-21T03:57:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.534+0000 7fb066a00640 1 -- 172.21.15.16:0/3872598246 learned_addr learned my addr 172.21.15.16:0/3872598246 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.534+0000 7fb074ac8740 10 client.0 _ll_register_callbacks cb 0x7fb058009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:57:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.534+0000 7fb05fe00640 1 -- 172.21.15.16:0/3872598246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 msgr2=0x5565db8b71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.534+0000 7fb05fe00640 1 --2- 172.21.15.16:0/3872598246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.534+0000 7fb05fe00640 1 -- 172.21.15.16:0/3872598246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 msgr2=0x5565db8b8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.534+0000 7fb05fe00640 1 --2- 172.21.15.16:0/3872598246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 0x5565db8b8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.534+0000 7fb05fe00640 1 -- 172.21.15.16:0/3872598246 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565db8b8a90 con 0x5565db4e14d0 2024-09-21T03:57:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.535+0000 7fb066a00640 1 --2- 172.21.15.16:0/3872598246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:57:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.535+0000 7fb05fe00640 1 --2- 172.21.15.16:0/3872598246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8b78a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb05000c6a0 tx=0x7fb0500107f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc5b72c93e30617 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.535+0000 7fb054c00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:57:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.536+0000 7fb054c00640 1 -- 172.21.15.16:0/3872598246 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0500a7e60 con 0x5565db4e14d0 2024-09-21T03:57:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.536+0000 7fb054c00640 1 -- 172.21.15.16:0/3872598246 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb0500a90b0 con 0x5565db4e14d0 2024-09-21T03:57:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.536+0000 7fb074ac8740 1 -- 172.21.15.16:0/3872598246 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb058027070 con 0x5565db4e14d0 2024-09-21T03:57:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.538+0000 7fb054c00640 1 -- 172.21.15.16:0/3872598246 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb0500b2240 con 0x5565db4e14d0 2024-09-21T03:57:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.538+0000 7fb074ac8740 10 client.26689 fetch_fsmap learned FSMap version 30 2024-09-21T03:57:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.538+0000 7fb074ac8740 1 -- 172.21.15.16:0/3872598246 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565db8b9e50 con 0x5565db4e14d0 2024-09-21T03:57:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.538+0000 7fb054c00640 1 -- 172.21.15.16:0/3872598246 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb058027070 con 0x5565db4e14d0 2024-09-21T03:57:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.539+0000 7fb054c00640 1 -- 172.21.15.16:0/3872598246 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb0500aea90 con 0x5565db4e14d0 2024-09-21T03:57:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.541+0000 7fb074ac8740 10 client.26689 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:57:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.541+0000 7fb074ac8740 10 client.26689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:57:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.541+0000 7fb074ac8740 20 client.26689 populate_metadata read hostname 'smithi016' 2024-09-21T03:57:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.541+0000 7fb074ac8740 1 --2- 172.21.15.16:0/3872598246 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5565db4961c0 0x5565db92c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.541+0000 7fb074ac8740 4 client.26689 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:57:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.542+0000 7fb066a00640 1 --2- 172.21.15.16:0/3872598246 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5565db4961c0 0x5565db92c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.543+0000 7fb074ac8740 1 -- 172.21.15.16:0/3872598246 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565db81a0b0 con 0x5565db4961c0 2024-09-21T03:57:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.544+0000 7fb054c00640 10 client.26689 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.544+0000 7fb066a00640 1 --2- 172.21.15.16:0/3872598246 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5565db4961c0 0x5565db92c480 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8aa2f10d74acd0f server_cookie=3dc3992cf1ed10a9 in_seq=0 out_seq=0 2024-09-21T03:57:39.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:39 smithi134 ceph-mon[35031]: pgmap v758: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 210 MiB/s wr, 1.47k op/s 2024-09-21T03:57:39.634 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.629+0000 7fb054c00640 1 -- 172.21.15.16:0/3872598246 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565db81a0b0 con 0x5565db4961c0 2024-09-21T03:57:39.634 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.632+0000 7fb054c00640 10 client.26689 handle_command_reply: tid=0 2024-09-21T03:57:39.634 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.632+0000 7fb074ac8740 1 client.26689 shutdown 2024-09-21T03:57:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.634+0000 7fb074ac8740 1 -- 172.21.15.16:0/3872598246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 msgr2=0x5565db8b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.634+0000 7fb074ac8740 1 --2- 172.21.15.16:0/3872598246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8b78a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb05000c6a0 tx=0x7fb0500107f0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.634+0000 7fb074ac8740 1 -- 172.21.15.16:0/3872598246 shutdown_connections 2024-09-21T03:57:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.634+0000 7fb074ac8740 1 --2- 172.21.15.16:0/3872598246 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5565db4961c0 0x5565db92c480 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.634+0000 7fb074ac8740 1 --2- 172.21.15.16:0/3872598246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565db5021b0 0x5565db8b8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.634+0000 7fb054c00640 0 client.26689 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.635+0000 7fb074ac8740 1 --2- 172.21.15.16:0/3872598246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565db4e14d0 0x5565db8b78a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.635+0000 7fb074ac8740 1 --2- 172.21.15.16:0/3872598246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565db4d92f0 0x5565db8b71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.635+0000 7fb074ac8740 1 -- 172.21.15.16:0/3872598246 >> 172.21.15.16:0/3872598246 conn(0x5565db4f6300 msgr2=0x5565db8b19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.636+0000 7fb074ac8740 1 -- 172.21.15.16:0/3872598246 shutdown_connections 2024-09-21T03:57:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.636+0000 7fb074ac8740 1 -- 172.21.15.16:0/3872598246 wait complete. 2024-09-21T03:57:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.636+0000 7fb074ac8740 20 client.26689 trim_cache size 0 max 16384 2024-09-21T03:57:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:57:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:57:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:57:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:57:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.637+0000 7fb06da00640 1 -- 172.21.15.16:0/3319729144 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb04c0b65b0 msgr2=0x7fb04c0b8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.637+0000 7fb06da00640 1 --2- 172.21.15.16:0/3319729144 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb04c0b65b0 0x7fb04c0b8ac0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb060034040 tx=0x7fb060005fd0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.637+0000 7fb06da00640 1 -- 172.21.15.16:0/3319729144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb06810ada0 msgr2=0x7fb068111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.637+0000 7fb06da00640 1 --2- 172.21.15.16:0/3319729144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb06810ada0 0x7fb068111540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb05000e9f0 tx=0x7fb05000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.637+0000 7fb06da00640 1 -- 172.21.15.16:0/3319729144 shutdown_connections 2024-09-21T03:57:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.637+0000 7fb06da00640 1 --2- 172.21.15.16:0/3319729144 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb04c0b65b0 0x7fb04c0b8ac0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.637+0000 7fb06da00640 1 --2- 172.21.15.16:0/3319729144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb068111f90 0x7fb0681a9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.637+0000 7fb06da00640 1 --2- 172.21.15.16:0/3319729144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb06810ada0 0x7fb068111540 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.637+0000 7fb06da00640 1 --2- 172.21.15.16:0/3319729144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb06810a3c0 0x7fb068110ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.637+0000 7fb06da00640 1 -- 172.21.15.16:0/3319729144 >> 172.21.15.16:0/3319729144 conn(0x7fb068070810 msgr2=0x7fb068070c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.638+0000 7fb06da00640 1 -- 172.21.15.16:0/3319729144 shutdown_connections 2024-09-21T03:57:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:39.639+0000 7fb06da00640 1 -- 172.21.15.16:0/3319729144 wait complete. 2024-09-21T03:57:41.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:41 smithi123 ceph-mon[34673]: pgmap v759: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 234 MiB/s wr, 1.57k op/s 2024-09-21T03:57:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:41 smithi016 ceph-mon[30375]: pgmap v759: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 234 MiB/s wr, 1.57k op/s 2024-09-21T03:57:41.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:41 smithi134 ceph-mon[35031]: pgmap v759: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 234 MiB/s wr, 1.57k op/s 2024-09-21T03:57:42.424 INFO:tasks.workunit.client.0.smithi016.stdout:stopping iogen 2024-09-21T03:57:42.425 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'stopping iogen' 2024-09-21T03:57:42.425 INFO:tasks.workunit.client.0.smithi016.stderr:+ ./iogen -k 2024-09-21T03:57:42.432 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo OK 2024-09-21T03:57:42.433 INFO:tasks.workunit.client.0.smithi016.stdout:OK 2024-09-21T03:57:43.061 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:57:43.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:43 smithi016 ceph-mon[30375]: pgmap v760: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 217 MiB/s wr, 1.41k op/s 2024-09-21T03:57:43.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:43 smithi134 ceph-mon[35031]: pgmap v760: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 217 MiB/s wr, 1.41k op/s 2024-09-21T03:57:43.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:43 smithi123 ceph-mon[34673]: pgmap v760: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 217 MiB/s wr, 1.41k op/s 2024-09-21T03:57:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:57:45.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:45 smithi016 ceph-mon[30375]: pgmap v761: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 209 MiB/s wr, 1.36k op/s 2024-09-21T03:57:45.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:45 smithi134 ceph-mon[35031]: pgmap v761: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 209 MiB/s wr, 1.36k op/s 2024-09-21T03:57:45.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:45 smithi123 ceph-mon[34673]: pgmap v761: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 209 MiB/s wr, 1.36k op/s 2024-09-21T03:57:47.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.041+0000 7fb6cfe00640 1 -- 172.21.15.16:0/608857967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d01025b0 msgr2=0x7fb6d01029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:47.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.041+0000 7fb6cfe00640 1 --2- 172.21.15.16:0/608857967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d01025b0 0x7fb6d01029c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009890 tx=0x7fb6c002a390 comp rx=0 tx=0).stop 2024-09-21T03:57:47.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.044+0000 7fb6cfe00640 1 -- 172.21.15.16:0/608857967 shutdown_connections 2024-09-21T03:57:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.045+0000 7fb6cfe00640 1 --2- 172.21.15.16:0/608857967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6d01079f0 0x7fb6d0107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.045+0000 7fb6cfe00640 1 --2- 172.21.15.16:0/608857967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6d0102f00 0x7fb6d0107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.045+0000 7fb6cfe00640 1 --2- 172.21.15.16:0/608857967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d01025b0 0x7fb6d01029c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.045+0000 7fb6cfe00640 1 -- 172.21.15.16:0/608857967 >> 172.21.15.16:0/608857967 conn(0x7fb6d00fe2c0 msgr2=0x7fb6d0100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.048+0000 7fb6cfe00640 1 -- 172.21.15.16:0/608857967 shutdown_connections 2024-09-21T03:57:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.048+0000 7fb6cfe00640 1 -- 172.21.15.16:0/608857967 wait complete. 2024-09-21T03:57:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.050+0000 7fb6cfe00640 1 Processor -- start 2024-09-21T03:57:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.051+0000 7fb6cfe00640 1 -- start start 2024-09-21T03:57:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.051+0000 7fb6cfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d01025b0 0x7fb6d0116b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.051+0000 7fb6cfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6d0102f00 0x7fb6d010fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.051+0000 7fb6cfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6d01079f0 0x7fb6d01103e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.051+0000 7fb6cfe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d0119e90 con 0x7fb6d01025b0 2024-09-21T03:57:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.051+0000 7fb6cfe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d0119d10 con 0x7fb6d01079f0 2024-09-21T03:57:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.051+0000 7fb6cfe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d011a010 con 0x7fb6d0102f00 2024-09-21T03:57:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.052+0000 7fb6ce000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6d0102f00 0x7fb6d010fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.052+0000 7fb6cea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d01025b0 0x7fb6d0116b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.052+0000 7fb6cea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d01025b0 0x7fb6d0116b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46718/0 (socket says 172.21.15.16:46718) 2024-09-21T03:57:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.052+0000 7fb6cea00640 1 -- 172.21.15.16:0/1701700363 learned_addr learned my addr 172.21.15.16:0/1701700363 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.052+0000 7fb6cf400640 1 --2- 172.21.15.16:0/1701700363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6d01079f0 0x7fb6d01103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.053+0000 7fb6cea00640 1 -- 172.21.15.16:0/1701700363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6d0102f00 msgr2=0x7fb6d010fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.053+0000 7fb6cea00640 1 --2- 172.21.15.16:0/1701700363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6d0102f00 0x7fb6d010fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.053+0000 7fb6cea00640 1 -- 172.21.15.16:0/1701700363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6d01079f0 msgr2=0x7fb6d01103e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.053+0000 7fb6cea00640 1 --2- 172.21.15.16:0/1701700363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6d01079f0 0x7fb6d01103e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.053+0000 7fb6cea00640 1 -- 172.21.15.16:0/1701700363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d0110c00 con 0x7fb6d01025b0 2024-09-21T03:57:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.053+0000 7fb6cf400640 1 --2- 172.21.15.16:0/1701700363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6d01079f0 0x7fb6d01103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:57:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.053+0000 7fb6cea00640 1 --2- 172.21.15.16:0/1701700363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d01025b0 0x7fb6d0116b20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0035040 tx=0x7fb6c0033ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:47.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.057+0000 7fb6c7400640 1 -- 172.21.15.16:0/1701700363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c003f020 con 0x7fb6d01025b0 2024-09-21T03:57:47.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.057+0000 7fb6c7400640 1 -- 172.21.15.16:0/1701700363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6c0005040 con 0x7fb6d01025b0 2024-09-21T03:57:47.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.058+0000 7fb6c7400640 1 -- 172.21.15.16:0/1701700363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0037040 con 0x7fb6d01025b0 2024-09-21T03:57:47.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.061+0000 7fb6cfe00640 1 -- 172.21.15.16:0/1701700363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d0115850 con 0x7fb6d01025b0 2024-09-21T03:57:47.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.061+0000 7fb6cfe00640 1 -- 172.21.15.16:0/1701700363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d0115cb0 con 0x7fb6d01025b0 2024-09-21T03:57:47.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.066+0000 7fb6cfe00640 1 -- 172.21.15.16:0/1701700363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d0107430 con 0x7fb6d01025b0 2024-09-21T03:57:47.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.068+0000 7fb6c7400640 1 -- 172.21.15.16:0/1701700363 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb6c0042020 con 0x7fb6d01025b0 2024-09-21T03:57:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.071+0000 7fb6c7400640 1 --2- 172.21.15.16:0/1701700363 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6b40b6660 0x7fb6b40b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.071+0000 7fb6c7400640 1 -- 172.21.15.16:0/1701700363 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb6c0106900 con 0x7fb6d01025b0 2024-09-21T03:57:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.072+0000 7fb6ce000640 1 --2- 172.21.15.16:0/1701700363 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6b40b6660 0x7fb6b40b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:47.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.074+0000 7fb6ce000640 1 --2- 172.21.15.16:0/1701700363 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6b40b6660 0x7fb6b40b8b70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0108310 tx=0x7fb6bc005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.081+0000 7fb6c7400640 1 -- 172.21.15.16:0/1701700363 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb6c00cce90 con 0x7fb6d01025b0 2024-09-21T03:57:47.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:47 smithi016 ceph-mon[30375]: pgmap v762: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 241 MiB/s wr, 1.61k op/s 2024-09-21T03:57:47.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:47 smithi134 ceph-mon[35031]: pgmap v762: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 241 MiB/s wr, 1.61k op/s 2024-09-21T03:57:47.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:47 smithi123 ceph-mon[34673]: pgmap v762: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 241 MiB/s wr, 1.61k op/s 2024-09-21T03:57:47.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.726+0000 7fb6cfe00640 1 -- 172.21.15.16:0/1701700363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6d0115fd0 con 0x7fb6d01025b0 2024-09-21T03:57:47.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.727+0000 7fb6c7400640 1 -- 172.21.15.16:0/1701700363 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7fb6c00d1270 con 0x7fb6d01025b0 2024-09-21T03:57:47.729 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:57:47.729 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:57:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:57:47.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.729+0000 7fb6cfe00640 1 -- 172.21.15.16:0/1701700363 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6b40b6660 msgr2=0x7fb6b40b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.729+0000 7fb6cfe00640 1 --2- 172.21.15.16:0/1701700363 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6b40b6660 0x7fb6b40b8b70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0108310 tx=0x7fb6bc005f90 comp rx=0 tx=0).stop 2024-09-21T03:57:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.729+0000 7fb6cfe00640 1 -- 172.21.15.16:0/1701700363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d01025b0 msgr2=0x7fb6d0116b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.729+0000 7fb6cfe00640 1 --2- 172.21.15.16:0/1701700363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d01025b0 0x7fb6d0116b20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0035040 tx=0x7fb6c0033ce0 comp rx=0 tx=0).stop 2024-09-21T03:57:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.729+0000 7fb6cfe00640 1 -- 172.21.15.16:0/1701700363 shutdown_connections 2024-09-21T03:57:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.729+0000 7fb6cfe00640 1 --2- 172.21.15.16:0/1701700363 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6b40b6660 0x7fb6b40b8b70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.729+0000 7fb6cfe00640 1 --2- 172.21.15.16:0/1701700363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6d01079f0 0x7fb6d01103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.729+0000 7fb6cfe00640 1 --2- 172.21.15.16:0/1701700363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6d0102f00 0x7fb6d010fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.729+0000 7fb6cfe00640 1 --2- 172.21.15.16:0/1701700363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d01025b0 0x7fb6d0116b20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.730+0000 7fb6cfe00640 1 -- 172.21.15.16:0/1701700363 >> 172.21.15.16:0/1701700363 conn(0x7fb6d00fe2c0 msgr2=0x7fb6d0105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.730+0000 7fb6cfe00640 1 -- 172.21.15.16:0/1701700363 shutdown_connections 2024-09-21T03:57:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:47.730+0000 7fb6cfe00640 1 -- 172.21.15.16:0/1701700363 wait complete. 2024-09-21T03:57:48.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:48 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1701700363' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:57:48.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:48 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1701700363' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:57:48.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:48 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1701700363' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:57:49.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:49 smithi134 ceph-mon[35031]: pgmap v763: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 228 MiB/s wr, 1.57k op/s 2024-09-21T03:57:49.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:49 smithi123 ceph-mon[34673]: pgmap v763: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 228 MiB/s wr, 1.57k op/s 2024-09-21T03:57:49.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:49 smithi016 ceph-mon[30375]: pgmap v763: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 228 MiB/s wr, 1.57k op/s 2024-09-21T03:57:49.903 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db --set-id ce95ca56 --release --await' 2024-09-21T03:57:49.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db --set-id ce95ca56 --release --await 2024-09-21T03:57:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:57:51.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:51 smithi016 ceph-mon[30375]: pgmap v764: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 236 MiB/s wr, 1.70k op/s 2024-09-21T03:57:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.544+0000 7fc9c0e00640 1 -- 172.21.15.16:0/1156773813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9bc105d80 msgr2=0x7fc9bc106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.544+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/1156773813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9bc105d80 0x7fc9bc106190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009900 tx=0x7fc9ac02a410 comp rx=0 tx=0).stop 2024-09-21T03:57:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.544+0000 7fc9c0e00640 1 -- 172.21.15.16:0/1156773813 shutdown_connections 2024-09-21T03:57:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.544+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/1156773813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9bc10c010 0x7fc9bc112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.544+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/1156773813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9bc10b6c0 0x7fc9bc10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.544+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/1156773813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9bc105d80 0x7fc9bc106190 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.544+0000 7fc9c0e00640 1 -- 172.21.15.16:0/1156773813 >> 172.21.15.16:0/1156773813 conn(0x7fc9bc101810 msgr2=0x7fc9bc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.544+0000 7fc9c0e00640 1 -- 172.21.15.16:0/1156773813 shutdown_connections 2024-09-21T03:57:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.544+0000 7fc9c0e00640 1 -- 172.21.15.16:0/1156773813 wait complete. 2024-09-21T03:57:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.545+0000 7fc9c0e00640 1 Processor -- start 2024-09-21T03:57:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.545+0000 7fc9c0e00640 1 -- start start 2024-09-21T03:57:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.545+0000 7fc9c0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9bc105d80 0x7fc9bc19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.545+0000 7fc9c0e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9bc10b6c0 0x7fc9bc19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.545+0000 7fc9c0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9bc10c010 0x7fc9bc1a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.545+0000 7fc9c0e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc115cd0 con 0x7fc9bc10c010 2024-09-21T03:57:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.545+0000 7fc9c0e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc115b50 con 0x7fc9bc105d80 2024-09-21T03:57:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.545+0000 7fc9c0e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc115e50 con 0x7fc9bc10b6c0 2024-09-21T03:57:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.546+0000 7fc9b9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9bc105d80 0x7fc9bc19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.546+0000 7fc9b9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9bc105d80 0x7fc9bc19c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:51002/0 (socket says 172.21.15.16:51002) 2024-09-21T03:57:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.546+0000 7fc9b9e00640 1 -- 172.21.15.16:0/2458890650 learned_addr learned my addr 172.21.15.16:0/2458890650 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.546+0000 7fc9ba800640 1 --2- 172.21.15.16:0/2458890650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9bc10c010 0x7fc9bc1a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.546+0000 7fc9b9400640 1 --2- 172.21.15.16:0/2458890650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9bc10b6c0 0x7fc9bc19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.546+0000 7fc9b9e00640 1 -- 172.21.15.16:0/2458890650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9bc10b6c0 msgr2=0x7fc9bc19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.546+0000 7fc9b9e00640 1 --2- 172.21.15.16:0/2458890650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9bc10b6c0 0x7fc9bc19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.546+0000 7fc9b9e00640 1 -- 172.21.15.16:0/2458890650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9bc10c010 msgr2=0x7fc9bc1a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.546+0000 7fc9b9e00640 1 --2- 172.21.15.16:0/2458890650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9bc10c010 0x7fc9bc1a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.546+0000 7fc9b9e00640 1 -- 172.21.15.16:0/2458890650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9bc1a2650 con 0x7fc9bc105d80 2024-09-21T03:57:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.547+0000 7fc9b9e00640 1 --2- 172.21.15.16:0/2458890650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9bc105d80 0x7fc9bc19c6e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac005940 tx=0x7fc9ac0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.547+0000 7fc9b2a00640 1 -- 172.21.15.16:0/2458890650 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac03f030 con 0x7fc9bc105d80 2024-09-21T03:57:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.547+0000 7fc9c0e00640 1 -- 172.21.15.16:0/2458890650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9bc1a28e0 con 0x7fc9bc105d80 2024-09-21T03:57:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.547+0000 7fc9b2a00640 1 -- 172.21.15.16:0/2458890650 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9ac0044c0 con 0x7fc9bc105d80 2024-09-21T03:57:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.547+0000 7fc9b2a00640 1 -- 172.21.15.16:0/2458890650 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac035070 con 0x7fc9bc105d80 2024-09-21T03:57:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.547+0000 7fc9c0e00640 1 -- 172.21.15.16:0/2458890650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9bc1a90a0 con 0x7fc9bc105d80 2024-09-21T03:57:51.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.551+0000 7fc9b2a00640 1 -- 172.21.15.16:0/2458890650 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fc9ac042030 con 0x7fc9bc105d80 2024-09-21T03:57:51.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.552+0000 7fc9b2a00640 1 --2- 172.21.15.16:0/2458890650 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc9a00b6590 0x7fc9a00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:51.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.552+0000 7fc9b2a00640 1 -- 172.21.15.16:0/2458890650 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fc9ac1079d0 con 0x7fc9bc105d80 2024-09-21T03:57:51.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.552+0000 7fc9b9400640 1 --2- 172.21.15.16:0/2458890650 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc9a00b6590 0x7fc9a00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:51.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:51.553+0000 7fc9b9400640 1 --2- 172.21.15.16:0/2458890650 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc9a00b6590 0x7fc9a00b8aa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80095f0 tx=0x7fc9a8005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:51.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:51 smithi134 ceph-mon[35031]: pgmap v764: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 236 MiB/s wr, 1.70k op/s 2024-09-21T03:57:51.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:51 smithi123 ceph-mon[34673]: pgmap v764: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 236 MiB/s wr, 1.70k op/s 2024-09-21T03:57:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.328+0000 7fc9c7bea740 1 Processor -- start 2024-09-21T03:57:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.328+0000 7fc9c7bea740 1 -- start start 2024-09-21T03:57:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.328+0000 7fc9c7bea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56139680d520 0x561396bb41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.328+0000 7fc9c7bea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5613967e4660 0x561396bb4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.328+0000 7fc9c7bea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613967ec840 0x561396bb4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.328+0000 7fc9c7bea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561396b2b290 con 0x56139680d520 2024-09-21T03:57:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.328+0000 7fc9c7bea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561396b1a6c0 con 0x5613967ec840 2024-09-21T03:57:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.328+0000 7fc9c7bea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561396b1e4d0 con 0x5613967e4660 2024-09-21T03:57:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.329+0000 7fc9b9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56139680d520 0x561396bb41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.329+0000 7fc9b9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56139680d520 0x561396bb41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34980/0 (socket says 172.21.15.16:34980) 2024-09-21T03:57:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.329+0000 7fc9b9e00640 1 -- 172.21.15.16:0/2612806288 learned_addr learned my addr 172.21.15.16:0/2612806288 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.329+0000 7fc9b9400640 1 --2- 172.21.15.16:0/2612806288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5613967e4660 0x561396bb4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.329+0000 7fc9ba800640 1 --2- 172.21.15.16:0/2612806288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613967ec840 0x561396bb4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.330+0000 7fc9b9e00640 1 -- 172.21.15.16:0/2612806288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5613967e4660 msgr2=0x561396bb4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.330+0000 7fc9b9e00640 1 --2- 172.21.15.16:0/2612806288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5613967e4660 0x561396bb4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.330+0000 7fc9b9e00640 1 -- 172.21.15.16:0/2612806288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613967ec840 msgr2=0x561396bb4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.330+0000 7fc9b9e00640 1 --2- 172.21.15.16:0/2612806288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613967ec840 0x561396bb4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.330+0000 7fc9b9e00640 1 -- 172.21.15.16:0/2612806288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561396aa7460 con 0x56139680d520 2024-09-21T03:57:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.330+0000 7fc9ba800640 1 --2- 172.21.15.16:0/2612806288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613967ec840 0x561396bb4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:57:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.330+0000 7fc9b9e00640 1 --2- 172.21.15.16:0/2612806288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56139680d520 0x561396bb41c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0caa70 tx=0x7fc9ac02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c4b31746849b94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.330+0000 7fc9a7400640 1 -- 172.21.15.16:0/2612806288 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac049070 con 0x56139680d520 2024-09-21T03:57:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.330+0000 7fc9a7400640 1 -- 172.21.15.16:0/2612806288 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9ac0d7170 con 0x56139680d520 2024-09-21T03:57:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.331+0000 7fc9a7400640 1 -- 172.21.15.16:0/2612806288 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac0c9150 con 0x56139680d520 2024-09-21T03:57:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.332+0000 7fc9c7bea740 1 -- 172.21.15.16:0/2612806288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56139680d520 msgr2=0x561396bb41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.332+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/2612806288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56139680d520 0x561396bb41c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0caa70 tx=0x7fc9ac02a410 comp rx=0 tx=0).stop 2024-09-21T03:57:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.332+0000 7fc9c7bea740 1 -- 172.21.15.16:0/2612806288 shutdown_connections 2024-09-21T03:57:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.332+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/2612806288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613967ec840 0x561396bb4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.332+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/2612806288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5613967e4660 0x561396bb4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.332+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/2612806288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56139680d520 0x561396bb41c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.332+0000 7fc9c7bea740 1 -- 172.21.15.16:0/2612806288 >> 172.21.15.16:0/2612806288 conn(0x561396801910 msgr2=0x561396bb20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.332+0000 7fc9c7bea740 1 -- 172.21.15.16:0/2612806288 shutdown_connections 2024-09-21T03:57:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.332+0000 7fc9c7bea740 1 -- 172.21.15.16:0/2612806288 wait complete. 2024-09-21T03:57:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.333+0000 7fc9c7bea740 1 -- start start 2024-09-21T03:57:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.333+0000 7fc9c7bea740 1 Processor -- start 2024-09-21T03:57:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.334+0000 7fc9c7bea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5613967e4660 0x561396bbfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.334+0000 7fc9c7bea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613967ec840 0x561396bc0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.334+0000 7fc9c7bea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139680d520 0x561396bcb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.334+0000 7fc9c7bea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56139697bbb0 con 0x5613967ec840 2024-09-21T03:57:52.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.334+0000 7fc9c7bea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561396ab56c0 con 0x56139680d520 2024-09-21T03:57:52.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.334+0000 7fc9c7bea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561396bc2760 con 0x5613967e4660 2024-09-21T03:57:52.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.334+0000 7fc9c7bea740 10 client.0 _ll_register_callbacks cb 0x561396ad0490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:57:52.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.334+0000 7fc9b9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613967ec840 0x561396bc0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.334+0000 7fc9b9e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5613967e4660 0x561396bbfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.334+0000 7fc9b9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613967ec840 0x561396bc0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34988/0 (socket says 172.21.15.16:34988) 2024-09-21T03:57:52.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.334+0000 7fc9b9400640 1 -- 172.21.15.16:0/1692671362 learned_addr learned my addr 172.21.15.16:0/1692671362 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:52.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.334+0000 7fc9ba800640 1 --2- 172.21.15.16:0/1692671362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139680d520 0x561396bcb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.335+0000 7fc9b9e00640 1 -- 172.21.15.16:0/1692671362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139680d520 msgr2=0x561396bcb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:52.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.335+0000 7fc9b9e00640 1 --2- 172.21.15.16:0/1692671362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139680d520 0x561396bcb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.335+0000 7fc9b9e00640 1 -- 172.21.15.16:0/1692671362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613967ec840 msgr2=0x561396bc0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:52.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.335+0000 7fc9b9e00640 1 --2- 172.21.15.16:0/1692671362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613967ec840 0x561396bc0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.335+0000 7fc9b9e00640 1 -- 172.21.15.16:0/1692671362 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561396878b00 con 0x5613967e4660 2024-09-21T03:57:52.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.335+0000 7fc9b9400640 1 --2- 172.21.15.16:0/1692671362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613967ec840 0x561396bc0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:57:52.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.335+0000 7fc9b9e00640 1 --2- 172.21.15.16:0/1692671362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5613967e4660 0x561396bbfde0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac006010 tx=0x7fc9ac0d2360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e9a3d59c5fd383f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:52.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.335+0000 7fc99be00640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:57:52.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.335+0000 7fc99be00640 1 -- 172.21.15.16:0/1692671362 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac0f3020 con 0x5613967e4660 2024-09-21T03:57:52.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.336+0000 7fc99be00640 1 -- 172.21.15.16:0/1692671362 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fc9ac0ccc60 con 0x5613967e4660 2024-09-21T03:57:52.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.336+0000 7fc9c7bea740 1 -- 172.21.15.16:0/1692671362 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9ac0d7170 con 0x5613967e4660 2024-09-21T03:57:52.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.336+0000 7fc99be00640 1 -- 172.21.15.16:0/1692671362 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc9ac049070 con 0x5613967e4660 2024-09-21T03:57:52.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.336+0000 7fc9c7bea740 10 client.26707 fetch_fsmap learned FSMap version 30 2024-09-21T03:57:52.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.336+0000 7fc9c7bea740 1 -- 172.21.15.16:0/1692671362 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5613968ef960 con 0x5613967e4660 2024-09-21T03:57:52.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.336+0000 7fc99be00640 1 -- 172.21.15.16:0/1692671362 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9ac0d7170 con 0x5613967e4660 2024-09-21T03:57:52.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.336+0000 7fc99be00640 1 -- 172.21.15.16:0/1692671362 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc9ac0cc4d0 con 0x5613967e4660 2024-09-21T03:57:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.336+0000 7fc9c7bea740 10 client.26707 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:57:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.336+0000 7fc9c7bea740 10 client.26707 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:57:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.336+0000 7fc9c7bea740 20 client.26707 populate_metadata read hostname 'smithi016' 2024-09-21T03:57:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.337+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/1692671362 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5613967cbea0 0x561396bd18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.337+0000 7fc9c7bea740 4 client.26707 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:57:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.337+0000 7fc9c7bea740 1 -- 172.21.15.16:0/1692671362 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561396b2d1e0 con 0x5613967cbea0 2024-09-21T03:57:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.337+0000 7fc9b9400640 1 --2- 172.21.15.16:0/1692671362 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5613967cbea0 0x561396bd18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.338+0000 7fc9b9400640 1 --2- 172.21.15.16:0/1692671362 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5613967cbea0 0x561396bd18c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b182d62dc36e3692 server_cookie=f44513e5f52a3f2e in_seq=0 out_seq=0 2024-09-21T03:57:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.338+0000 7fc99be00640 10 client.26707 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.339+0000 7fc99be00640 1 -- 172.21.15.16:0/1692671362 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561396b2d1e0 con 0x5613967cbea0 2024-09-21T03:57:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.339+0000 7fc99be00640 10 client.26707 handle_command_reply: tid=0 2024-09-21T03:57:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.339+0000 7fc9c7bea740 1 client.26707 shutdown 2024-09-21T03:57:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.339+0000 7fc9c7bea740 1 -- 172.21.15.16:0/1692671362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5613967e4660 msgr2=0x561396bbfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.339+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/1692671362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5613967e4660 0x561396bbfde0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac006010 tx=0x7fc9ac0d2360 comp rx=0 tx=0).stop 2024-09-21T03:57:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.339+0000 7fc9c7bea740 1 -- 172.21.15.16:0/1692671362 shutdown_connections 2024-09-21T03:57:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.339+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/1692671362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139680d520 0x561396bcb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.339+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/1692671362 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5613967cbea0 0x561396bd18c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.339+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/1692671362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613967ec840 0x561396bc0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.339+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/1692671362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5613967e4660 0x561396bbfde0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.339+0000 7fc99be00640 0 client.26707 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.339+0000 7fc9c7bea740 1 -- 172.21.15.16:0/1692671362 >> 172.21.15.16:0/1692671362 conn(0x561396801910 msgr2=0x561396bb73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.339+0000 7fc9c7bea740 1 -- 172.21.15.16:0/1692671362 shutdown_connections 2024-09-21T03:57:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.339+0000 7fc9c7bea740 1 -- 172.21.15.16:0/1692671362 wait complete. 2024-09-21T03:57:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.340+0000 7fc9c7bea740 20 client.26707 trim_cache size 0 max 16384 2024-09-21T03:57:52.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.359+0000 7fc9c7bea740 1 Processor -- start 2024-09-21T03:57:52.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.359+0000 7fc9c7bea740 1 -- start start 2024-09-21T03:57:52.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.359+0000 7fc9c7bea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5613967e4660 0x561396bb99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.359+0000 7fc9c7bea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613967ec840 0x561396bb19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.360+0000 7fc9c7bea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56139680d520 0x561396bb1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.360+0000 7fc9c7bea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc1b2da0 con 0x56139680d520 2024-09-21T03:57:52.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.360+0000 7fc9c7bea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc1b48e0 con 0x5613967ec840 2024-09-21T03:57:52.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.360+0000 7fc9c7bea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc1b3470 con 0x5613967e4660 2024-09-21T03:57:52.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.360+0000 7fc9ba800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56139680d520 0x561396bb1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.360+0000 7fc9ba800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56139680d520 0x561396bb1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35000/0 (socket says 172.21.15.16:35000) 2024-09-21T03:57:52.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.360+0000 7fc9ba800640 1 -- 172.21.15.16:0/688663195 learned_addr learned my addr 172.21.15.16:0/688663195 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:52.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.360+0000 7fc9b9e00640 1 --2- 172.21.15.16:0/688663195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5613967e4660 0x561396bb99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.360+0000 7fc9b9400640 1 --2- 172.21.15.16:0/688663195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613967ec840 0x561396bb19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.360+0000 7fc9b9e00640 1 -- 172.21.15.16:0/688663195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613967ec840 msgr2=0x561396bb19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:52.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.361+0000 7fc9b9e00640 1 --2- 172.21.15.16:0/688663195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613967ec840 0x561396bb19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.361+0000 7fc9b9e00640 1 -- 172.21.15.16:0/688663195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56139680d520 msgr2=0x561396bb1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:52.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.361+0000 7fc9b9e00640 1 --2- 172.21.15.16:0/688663195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56139680d520 0x561396bb1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.361+0000 7fc9b9e00640 1 -- 172.21.15.16:0/688663195 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561396bbb470 con 0x5613967e4660 2024-09-21T03:57:52.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.361+0000 7fc9ba800640 1 --2- 172.21.15.16:0/688663195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56139680d520 0x561396bb1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:57:52.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.361+0000 7fc9b9e00640 1 --2- 172.21.15.16:0/688663195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5613967e4660 0x561396bb99e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0cd500 tx=0x7fc9ac0d22f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28019d3c63ef5da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:52.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.361+0000 7fc9a6a00640 1 -- 172.21.15.16:0/688663195 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac0f3070 con 0x5613967e4660 2024-09-21T03:57:52.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.361+0000 7fc9a6a00640 1 -- 172.21.15.16:0/688663195 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9ac0d6ce0 con 0x5613967e4660 2024-09-21T03:57:52.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.361+0000 7fc9a6a00640 1 -- 172.21.15.16:0/688663195 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac0035f0 con 0x5613967e4660 2024-09-21T03:57:52.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.362+0000 7fc9c7bea740 1 -- 172.21.15.16:0/688663195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5613967e4660 msgr2=0x561396bb99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:52.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.362+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/688663195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5613967e4660 0x561396bb99e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0cd500 tx=0x7fc9ac0d22f0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.362+0000 7fc9c7bea740 1 -- 172.21.15.16:0/688663195 shutdown_connections 2024-09-21T03:57:52.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.362+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/688663195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56139680d520 0x561396bb1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.362+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/688663195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613967ec840 0x561396bb19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.362+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/688663195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5613967e4660 0x561396bb99e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.362+0000 7fc9c7bea740 1 -- 172.21.15.16:0/688663195 >> 172.21.15.16:0/688663195 conn(0x561396801910 msgr2=0x561396bb92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:52.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.363+0000 7fc9c7bea740 1 -- 172.21.15.16:0/688663195 shutdown_connections 2024-09-21T03:57:52.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.363+0000 7fc9c7bea740 1 -- 172.21.15.16:0/688663195 wait complete. 2024-09-21T03:57:52.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.363+0000 7fc9c7bea740 1 -- start start 2024-09-21T03:57:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.363+0000 7fc9c7bea740 1 Processor -- start 2024-09-21T03:57:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.364+0000 7fc9c7bea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613967e4660 0x561396bc7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.364+0000 7fc9c7bea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613967ec840 0x561396bc7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.364+0000 7fc9c7bea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56139680d520 0x561396bc2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.364+0000 7fc9c7bea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc1b4210 con 0x5613967ec840 2024-09-21T03:57:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.364+0000 7fc9c7bea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc1b3b40 con 0x5613967e4660 2024-09-21T03:57:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.364+0000 7fc9c7bea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc1b0d80 con 0x56139680d520 2024-09-21T03:57:52.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.364+0000 7fc9c7bea740 10 client.0 _ll_register_callbacks cb 0x7fc9b4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:57:52.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.364+0000 7fc9b9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613967ec840 0x561396bc7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.364+0000 7fc9b9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613967e4660 0x561396bc7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.364+0000 7fc9b9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613967ec840 0x561396bc7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35004/0 (socket says 172.21.15.16:35004) 2024-09-21T03:57:52.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.364+0000 7fc9b9400640 1 -- 172.21.15.16:0/3931238020 learned_addr learned my addr 172.21.15.16:0/3931238020 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:52.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.364+0000 7fc9ba800640 1 --2- 172.21.15.16:0/3931238020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56139680d520 0x561396bc2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.365+0000 7fc9b9e00640 1 -- 172.21.15.16:0/3931238020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56139680d520 msgr2=0x561396bc2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:52.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.365+0000 7fc9b9e00640 1 --2- 172.21.15.16:0/3931238020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56139680d520 0x561396bc2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.365+0000 7fc9b9e00640 1 -- 172.21.15.16:0/3931238020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613967ec840 msgr2=0x561396bc7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:52.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.365+0000 7fc9b9e00640 1 --2- 172.21.15.16:0/3931238020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613967ec840 0x561396bc7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.365+0000 7fc9b9e00640 1 -- 172.21.15.16:0/3931238020 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561396bc2d70 con 0x5613967e4660 2024-09-21T03:57:52.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.365+0000 7fc9b9e00640 1 --2- 172.21.15.16:0/3931238020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613967e4660 0x561396bc7880 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac005a90 tx=0x7fc9ac03b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db1560f6c4cfdd9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:52.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.365+0000 7fc99be00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:57:52.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.366+0000 7fc99be00640 1 -- 172.21.15.16:0/3931238020 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac0e7070 con 0x5613967e4660 2024-09-21T03:57:52.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.366+0000 7fc99be00640 1 -- 172.21.15.16:0/3931238020 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fc9ac0cca60 con 0x5613967e4660 2024-09-21T03:57:52.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.366+0000 7fc9c7bea740 1 -- 172.21.15.16:0/3931238020 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9ac0d6ce0 con 0x5613967e4660 2024-09-21T03:57:52.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.366+0000 7fc99be00640 1 -- 172.21.15.16:0/3931238020 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc9ac0f3070 con 0x5613967e4660 2024-09-21T03:57:52.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.366+0000 7fc9c7bea740 10 client.26771 fetch_fsmap learned FSMap version 30 2024-09-21T03:57:52.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.366+0000 7fc9c7bea740 1 -- 172.21.15.16:0/3931238020 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561396bc2d70 con 0x5613967e4660 2024-09-21T03:57:52.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.366+0000 7fc99be00640 1 -- 172.21.15.16:0/3931238020 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9ac0d6ce0 con 0x5613967e4660 2024-09-21T03:57:52.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.366+0000 7fc99be00640 1 -- 172.21.15.16:0/3931238020 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc9ac031ac0 con 0x5613967e4660 2024-09-21T03:57:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.367+0000 7fc9c7bea740 10 client.26771 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:57:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.367+0000 7fc9c7bea740 10 client.26771 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:57:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.367+0000 7fc9c7bea740 20 client.26771 populate_metadata read hostname 'smithi016' 2024-09-21T03:57:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.367+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/3931238020 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5613967a1250 0x561396c3ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.367+0000 7fc9c7bea740 4 client.26771 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ce95ca56", "release": true, "await": true}] 2024-09-21T03:57:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.367+0000 7fc9c7bea740 1 -- 172.21.15.16:0/3931238020 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ce95ca56", "release": true, "await": true}) -- 0x561396b28c90 con 0x5613967a1250 2024-09-21T03:57:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.367+0000 7fc9b9400640 1 --2- 172.21.15.16:0/3931238020 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5613967a1250 0x561396c3ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.368+0000 7fc9b9400640 1 --2- 172.21.15.16:0/3931238020 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5613967a1250 0x561396c3ebf0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=509c64eb54db07dc server_cookie=bddaf0296a31786e in_seq=0 out_seq=0 2024-09-21T03:57:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.368+0000 7fc99be00640 10 client.26771 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.371+0000 7fc99be00640 1 -- 172.21.15.16:0/3931238020 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561396b28c90 con 0x5613967a1250 2024-09-21T03:57:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.371+0000 7fc99be00640 10 client.26771 handle_command_reply: tid=0 2024-09-21T03:57:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.371+0000 7fc9c7bea740 1 client.26771 shutdown 2024-09-21T03:57:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.372+0000 7fc9c7bea740 1 -- 172.21.15.16:0/3931238020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613967e4660 msgr2=0x561396bc7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.372+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/3931238020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613967e4660 0x561396bc7880 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac005a90 tx=0x7fc9ac03b460 comp rx=0 tx=0).stop 2024-09-21T03:57:52.375 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T03:57:52.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T03:57:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.372+0000 7fc9c7bea740 1 -- 172.21.15.16:0/3931238020 shutdown_connections 2024-09-21T03:57:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.372+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/3931238020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56139680d520 0x561396bc2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.372+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/3931238020 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x5613967a1250 0x561396c3ebf0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.372+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/3931238020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613967ec840 0x561396bc7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.372+0000 7fc9c7bea740 1 --2- 172.21.15.16:0/3931238020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5613967e4660 0x561396bc7880 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.372+0000 7fc99be00640 0 client.26771 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.372+0000 7fc9c7bea740 1 -- 172.21.15.16:0/3931238020 >> 172.21.15.16:0/3931238020 conn(0x561396801910 msgr2=0x561396bb18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.384+0000 7fc9c7bea740 1 -- 172.21.15.16:0/3931238020 shutdown_connections 2024-09-21T03:57:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.384+0000 7fc9c7bea740 1 -- 172.21.15.16:0/3931238020 wait complete. 2024-09-21T03:57:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.384+0000 7fc9c7bea740 20 client.26771 trim_cache size 0 max 16384 2024-09-21T03:57:52.387 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:57:52.387 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:57:52.387 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:57:52.387 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 44, 2024-09-21T03:57:52.387 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-21T03:57:52.387 INFO:teuthology.orchestra.run.smithi016.stdout: "ce95ca56": { 2024-09-21T03:57:52.387 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 44, 2024-09-21T03:57:52.387 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-21T03:57:52.387 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:57:52.388 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:57:52.388 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:57:52.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:57:52.388 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-21T03:57:52.388 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 144.5, 2024-09-21T03:57:52.388 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-21T03:57:52.388 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-21T03:57:52.388 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-21T03:57:52.388 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-21T03:57:52.388 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-21T03:57:52.388 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-21T03:57:52.389 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:57:52.389 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:57:52.389 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:57:52.389 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:57:52.389 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:57:52.389 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:57:52.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.385+0000 7fc9c0e00640 1 -- 172.21.15.16:0/2458890650 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc9a00b6590 msgr2=0x7fc9a00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:52.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.385+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/2458890650 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc9a00b6590 0x7fc9a00b8aa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80095f0 tx=0x7fc9a8005f90 comp rx=0 tx=0).stop 2024-09-21T03:57:52.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.385+0000 7fc9c0e00640 1 -- 172.21.15.16:0/2458890650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9bc105d80 msgr2=0x7fc9bc19c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:52.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.385+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/2458890650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9bc105d80 0x7fc9bc19c6e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac005940 tx=0x7fc9ac0040c0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.385+0000 7fc9c0e00640 1 -- 172.21.15.16:0/2458890650 shutdown_connections 2024-09-21T03:57:52.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.385+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/2458890650 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fc9a00b6590 0x7fc9a00b8aa0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.385+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/2458890650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9bc10c010 0x7fc9bc1a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.385+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/2458890650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9bc10b6c0 0x7fc9bc19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.385+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/2458890650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9bc105d80 0x7fc9bc19c6e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.385+0000 7fc9c0e00640 1 -- 172.21.15.16:0/2458890650 >> 172.21.15.16:0/2458890650 conn(0x7fc9bc101810 msgr2=0x7fc9bc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:52.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.386+0000 7fc9c0e00640 1 -- 172.21.15.16:0/2458890650 shutdown_connections 2024-09-21T03:57:52.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:52.386+0000 7fc9c0e00640 1 -- 172.21.15.16:0/2458890650 wait complete. 2024-09-21T03:57:52.727 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-09-21T03:57:52.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T03:57:52.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.870+0000 7f0497e00640 1 -- 172.21.15.16:0/3852438199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f049010fe50 msgr2=0x7f0490112270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:52.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.870+0000 7f0497e00640 1 --2- 172.21.15.16:0/3852438199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f049010fe50 0x7f0490112270 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0488009be0 tx=0x7f048802a6c0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.870+0000 7f0497e00640 1 -- 172.21.15.16:0/3852438199 shutdown_connections 2024-09-21T03:57:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.870+0000 7f0497e00640 1 --2- 172.21.15.16:0/3852438199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f049010fe50 0x7f0490112270 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.870+0000 7f0497e00640 1 --2- 172.21.15.16:0/3852438199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0490075cf0 0x7f0490076100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.870+0000 7f0497e00640 1 --2- 172.21.15.16:0/3852438199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04900770f0 0x7f04900757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.870+0000 7f0497e00640 1 -- 172.21.15.16:0/3852438199 >> 172.21.15.16:0/3852438199 conn(0x7f04900ff030 msgr2=0x7f04901014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.870+0000 7f0497e00640 1 -- 172.21.15.16:0/3852438199 shutdown_connections 2024-09-21T03:57:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.870+0000 7f0497e00640 1 -- 172.21.15.16:0/3852438199 wait complete. 2024-09-21T03:57:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.871+0000 7f0497e00640 1 Processor -- start 2024-09-21T03:57:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.871+0000 7f0497e00640 1 -- start start 2024-09-21T03:57:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.872+0000 7f0497e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0490075cf0 0x7f04901a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.872+0000 7f0497e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04900770f0 0x7f04901a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.872+0000 7f0497e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049010fe50 0x7f04901a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.872+0000 7f0497e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04901158f0 con 0x7f049010fe50 2024-09-21T03:57:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.872+0000 7f0497e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0490115770 con 0x7f0490075cf0 2024-09-21T03:57:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.872+0000 7f0497e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0490115a70 con 0x7f04900770f0 2024-09-21T03:57:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.872+0000 7f0495400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0490075cf0 0x7f04901a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.872+0000 7f0495e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049010fe50 0x7f04901a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.872+0000 7f0495400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0490075cf0 0x7f04901a04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:36202/0 (socket says 172.21.15.16:36202) 2024-09-21T03:57:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.872+0000 7f0495400640 1 -- 172.21.15.16:0/698002130 learned_addr learned my addr 172.21.15.16:0/698002130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.872+0000 7f0494a00640 1 --2- 172.21.15.16:0/698002130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04900770f0 0x7f04901a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.873+0000 7f0495400640 1 -- 172.21.15.16:0/698002130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04900770f0 msgr2=0x7f04901a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.873+0000 7f0495400640 1 --2- 172.21.15.16:0/698002130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04900770f0 0x7f04901a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.873+0000 7f0495400640 1 -- 172.21.15.16:0/698002130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049010fe50 msgr2=0x7f04901a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.873+0000 7f0495400640 1 --2- 172.21.15.16:0/698002130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049010fe50 0x7f04901a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.873+0000 7f0495400640 1 -- 172.21.15.16:0/698002130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04901a1f10 con 0x7f0490075cf0 2024-09-21T03:57:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.873+0000 7f0495400640 1 --2- 172.21.15.16:0/698002130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0490075cf0 0x7f04901a04d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f048400e9f0 tx=0x7f048400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.873+0000 7f048e000640 1 -- 172.21.15.16:0/698002130 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048400ccc0 con 0x7f0490075cf0 2024-09-21T03:57:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.873+0000 7f0497e00640 1 -- 172.21.15.16:0/698002130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04901a2200 con 0x7f0490075cf0 2024-09-21T03:57:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.873+0000 7f048e000640 1 -- 172.21.15.16:0/698002130 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04840108e0 con 0x7f0490075cf0 2024-09-21T03:57:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.873+0000 7f048e000640 1 -- 172.21.15.16:0/698002130 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484018880 con 0x7f0490075cf0 2024-09-21T03:57:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.873+0000 7f0497e00640 1 -- 172.21.15.16:0/698002130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04901a88f0 con 0x7f0490075cf0 2024-09-21T03:57:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.876+0000 7f049f015740 1 Processor -- start 2024-09-21T03:57:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.876+0000 7f049f015740 1 -- start start 2024-09-21T03:57:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.877+0000 7f049f015740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7e51b0 0x564d0fb8c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.877+0000 7f049f015740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7bc2f0 0x564d0fb8c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.877+0000 7f049f015740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7c44d0 0x564d0fb8d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.877+0000 7f049f015740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564d0faff6b0 con 0x564d0f7e51b0 2024-09-21T03:57:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.877+0000 7f049f015740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564d0faeeae0 con 0x564d0f7c44d0 2024-09-21T03:57:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.877+0000 7f049f015740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564d0faf28f0 con 0x564d0f7bc2f0 2024-09-21T03:57:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.877+0000 7f0495e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7e51b0 0x564d0fb8c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.877+0000 7f0495e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7e51b0 0x564d0fb8c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35036/0 (socket says 172.21.15.16:35036) 2024-09-21T03:57:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.877+0000 7f0495e00640 1 -- 172.21.15.16:0/2287276447 learned_addr learned my addr 172.21.15.16:0/2287276447 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.877+0000 7f0494a00640 1 --2- 172.21.15.16:0/2287276447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7c44d0 0x564d0fb8d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.877+0000 7f0495e00640 1 -- 172.21.15.16:0/2287276447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7bc2f0 msgr2=0x564d0fb8c740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:57:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.877+0000 7f0495e00640 1 --2- 172.21.15.16:0/2287276447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7bc2f0 0x564d0fb8c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.877+0000 7f0495e00640 1 -- 172.21.15.16:0/2287276447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7c44d0 msgr2=0x564d0fb8d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.877+0000 7f0495e00640 1 --2- 172.21.15.16:0/2287276447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7c44d0 0x564d0fb8d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.877+0000 7f0495e00640 1 -- 172.21.15.16:0/2287276447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0fabb780 con 0x564d0f7e51b0 2024-09-21T03:57:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.878+0000 7f0495e00640 1 --2- 172.21.15.16:0/2287276447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7e51b0 0x564d0fb8c0d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0488009be0 tx=0x7f04880093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70475fccfadad1e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.878+0000 7f0482600640 1 -- 172.21.15.16:0/2287276447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488041030 con 0x564d0f7e51b0 2024-09-21T03:57:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.878+0000 7f0482600640 1 -- 172.21.15.16:0/2287276447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04880305e0 con 0x564d0f7e51b0 2024-09-21T03:57:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.878+0000 7f0482600640 1 -- 172.21.15.16:0/2287276447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488036040 con 0x564d0f7e51b0 2024-09-21T03:57:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.878+0000 7f048e000640 1 -- 172.21.15.16:0/698002130 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0484010430 con 0x7f0490075cf0 2024-09-21T03:57:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.879+0000 7f049f015740 1 -- 172.21.15.16:0/2287276447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7e51b0 msgr2=0x564d0fb8c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.879+0000 7f049f015740 1 --2- 172.21.15.16:0/2287276447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7e51b0 0x564d0fb8c0d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0488009be0 tx=0x7f04880093a0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.879+0000 7f049f015740 1 -- 172.21.15.16:0/2287276447 shutdown_connections 2024-09-21T03:57:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.879+0000 7f049f015740 1 --2- 172.21.15.16:0/2287276447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7c44d0 0x564d0fb8d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.879+0000 7f049f015740 1 --2- 172.21.15.16:0/2287276447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7bc2f0 0x564d0fb8c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.879+0000 7f049f015740 1 --2- 172.21.15.16:0/2287276447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7e51b0 0x564d0fb8c0d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.879+0000 7f049f015740 1 -- 172.21.15.16:0/2287276447 >> 172.21.15.16:0/2287276447 conn(0x564d0f7d9300 msgr2=0x564d0fb87b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.879+0000 7f049f015740 1 -- 172.21.15.16:0/2287276447 shutdown_connections 2024-09-21T03:57:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.879+0000 7f049f015740 1 -- 172.21.15.16:0/2287276447 wait complete. 2024-09-21T03:57:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.880+0000 7f049f015740 1 -- start start 2024-09-21T03:57:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.880+0000 7f049f015740 1 Processor -- start 2024-09-21T03:57:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.880+0000 7f048e000640 1 --2- 172.21.15.16:0/698002130 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04780b6420 0x7f04780b8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.880+0000 7f048e000640 1 -- 172.21.15.16:0/698002130 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f04840e7b10 con 0x7f0490075cf0 2024-09-21T03:57:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.880+0000 7f0495e00640 1 --2- 172.21.15.16:0/698002130 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04780b6420 0x7f04780b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.880+0000 7f049f015740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7bc2f0 0x564d0fb95700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.880+0000 7f049f015740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7c44d0 0x564d0fb95dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.880+0000 7f049f015740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7e51b0 0x564d0fb96930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.880+0000 7f049f015740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564d0fa26ab0 con 0x564d0f7e51b0 2024-09-21T03:57:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.880+0000 7f049f015740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564d0fa95570 con 0x564d0f7c44d0 2024-09-21T03:57:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.880+0000 7f049f015740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564d0fb8dd50 con 0x564d0f7bc2f0 2024-09-21T03:57:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.880+0000 7f049f015740 10 client.0 _ll_register_callbacks cb 0x564d0faa4100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:57:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.881+0000 7f0495e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7e51b0 0x564d0fb96930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.881+0000 7f0494a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7c44d0 0x564d0fb95dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.881+0000 7f0494a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7c44d0 0x564d0fb95dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:36216/0 (socket says 172.21.15.16:36216) 2024-09-21T03:57:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.881+0000 7f0494a00640 1 -- 172.21.15.16:0/1878196355 learned_addr learned my addr 172.21.15.16:0/1878196355 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.881+0000 7f0495400640 1 --2- 172.21.15.16:0/1878196355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7bc2f0 0x564d0fb95700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.881+0000 7f0494a00640 1 -- 172.21.15.16:0/1878196355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7bc2f0 msgr2=0x564d0fb95700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.881+0000 7f0494a00640 1 --2- 172.21.15.16:0/1878196355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7bc2f0 0x564d0fb95700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.881+0000 7f0494a00640 1 -- 172.21.15.16:0/1878196355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7e51b0 msgr2=0x564d0fb96930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.881+0000 7f0494a00640 1 --2- 172.21.15.16:0/1878196355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7e51b0 0x564d0fb96930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.881+0000 7f0494a00640 1 -- 172.21.15.16:0/1878196355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0f850790 con 0x564d0f7c44d0 2024-09-21T03:57:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.881+0000 7f0495400640 1 --2- 172.21.15.16:0/1878196355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7bc2f0 0x564d0fb95700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:57:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.881+0000 7f0495e00640 1 --2- 172.21.15.16:0/698002130 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04780b6420 0x7f04780b8930 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0488009b30 tx=0x7f048802a6c0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.881+0000 7f0494a00640 1 --2- 172.21.15.16:0/1878196355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7c44d0 0x564d0fb95dc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f047c011780 tx=0x7f047c011c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d603c5d74b57e2cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.881+0000 7f0476a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:57:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.882+0000 7f0476a00640 1 -- 172.21.15.16:0/1878196355 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c00fb30 con 0x564d0f7c44d0 2024-09-21T03:57:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.882+0000 7f049f015740 1 -- 172.21.15.16:0/1878196355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04880305e0 con 0x564d0f7c44d0 2024-09-21T03:57:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.882+0000 7f0476a00640 1 -- 172.21.15.16:0/1878196355 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f047c00fcd0 con 0x564d0f7c44d0 2024-09-21T03:57:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.882+0000 7f0476a00640 1 -- 172.21.15.16:0/1878196355 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f047c027050 con 0x564d0f7c44d0 2024-09-21T03:57:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.882+0000 7f049f015740 10 client.26783 fetch_fsmap learned FSMap version 30 2024-09-21T03:57:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.882+0000 7f049f015740 1 -- 172.21.15.16:0/1878196355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564d0f8c9620 con 0x564d0f7c44d0 2024-09-21T03:57:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.882+0000 7f0476a00640 1 -- 172.21.15.16:0/1878196355 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04880305e0 con 0x564d0f7c44d0 2024-09-21T03:57:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.883+0000 7f0476a00640 1 -- 172.21.15.16:0/1878196355 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f047c022030 con 0x564d0f7c44d0 2024-09-21T03:57:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.883+0000 7f049f015740 10 client.26783 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:57:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.883+0000 7f049f015740 10 client.26783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:57:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.883+0000 7f049f015740 20 client.26783 populate_metadata read hostname 'smithi016' 2024-09-21T03:57:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.883+0000 7f049f015740 1 --2- 172.21.15.16:0/1878196355 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x564d0f7a3d80 0x564d0fba2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.883+0000 7f049f015740 4 client.26783 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:57:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.883+0000 7f049f015740 1 -- 172.21.15.16:0/1878196355 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d0fb01600 con 0x564d0f7a3d80 2024-09-21T03:57:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.883+0000 7f0495400640 1 --2- 172.21.15.16:0/1878196355 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x564d0f7a3d80 0x564d0fba2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.884+0000 7f0495400640 1 --2- 172.21.15.16:0/1878196355 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x564d0f7a3d80 0x564d0fba2eb0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a7e95580f03e253 server_cookie=17ef4c175f1c5f7f in_seq=0 out_seq=0 2024-09-21T03:57:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.884+0000 7f0476a00640 10 client.26783 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.884+0000 7f0476a00640 1 -- 172.21.15.16:0/1878196355 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564d0fb01600 con 0x564d0f7a3d80 2024-09-21T03:57:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.884+0000 7f0476a00640 10 client.26783 handle_command_reply: tid=0 2024-09-21T03:57:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.884+0000 7f049f015740 1 client.26783 shutdown 2024-09-21T03:57:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.885+0000 7f049f015740 1 -- 172.21.15.16:0/1878196355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7c44d0 msgr2=0x564d0fb95dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.885+0000 7f049f015740 1 --2- 172.21.15.16:0/1878196355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7c44d0 0x564d0fb95dc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f047c011780 tx=0x7f047c011c40 comp rx=0 tx=0).stop 2024-09-21T03:57:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.885+0000 7f049f015740 1 -- 172.21.15.16:0/1878196355 shutdown_connections 2024-09-21T03:57:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.885+0000 7f049f015740 1 --2- 172.21.15.16:0/1878196355 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x564d0f7a3d80 0x564d0fba2eb0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.885+0000 7f049f015740 1 --2- 172.21.15.16:0/1878196355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7e51b0 0x564d0fb96930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.885+0000 7f0476a00640 0 client.26783 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.885+0000 7f049f015740 1 --2- 172.21.15.16:0/1878196355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7c44d0 0x564d0fb95dc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.885+0000 7f049f015740 1 --2- 172.21.15.16:0/1878196355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7bc2f0 0x564d0fb95700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.885+0000 7f049f015740 1 -- 172.21.15.16:0/1878196355 >> 172.21.15.16:0/1878196355 conn(0x564d0f7d9300 msgr2=0x564d0fb860e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.885+0000 7f049f015740 1 -- 172.21.15.16:0/1878196355 shutdown_connections 2024-09-21T03:57:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.885+0000 7f049f015740 1 -- 172.21.15.16:0/1878196355 wait complete. 2024-09-21T03:57:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.885+0000 7f049f015740 20 client.26783 trim_cache size 0 max 16384 2024-09-21T03:57:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.900+0000 7f049f015740 1 Processor -- start 2024-09-21T03:57:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.900+0000 7f049f015740 1 -- start start 2024-09-21T03:57:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.900+0000 7f049f015740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7bc2f0 0x564d0fb95e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.901+0000 7f049f015740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7c44d0 0x564d0fb94900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.901+0000 7f049f015740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7e51b0 0x564d0fb94e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.901+0000 7f049f015740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04901b2cc0 con 0x564d0f7c44d0 2024-09-21T03:57:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.901+0000 7f049f015740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04901b4800 con 0x564d0f7bc2f0 2024-09-21T03:57:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.901+0000 7f049f015740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04901b3390 con 0x564d0f7e51b0 2024-09-21T03:57:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.901+0000 7f0494a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7c44d0 0x564d0fb94900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.901+0000 7f0494a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7c44d0 0x564d0fb94900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35068/0 (socket says 172.21.15.16:35068) 2024-09-21T03:57:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.901+0000 7f0494a00640 1 -- 172.21.15.16:0/3026360196 learned_addr learned my addr 172.21.15.16:0/3026360196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.901+0000 7f0495400640 1 --2- 172.21.15.16:0/3026360196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7bc2f0 0x564d0fb95e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.901+0000 7f0495e00640 1 --2- 172.21.15.16:0/3026360196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7e51b0 0x564d0fb94e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.901+0000 7f0494a00640 1 -- 172.21.15.16:0/3026360196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7e51b0 msgr2=0x564d0fb94e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.901+0000 7f0494a00640 1 --2- 172.21.15.16:0/3026360196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7e51b0 0x564d0fb94e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.901+0000 7f0494a00640 1 -- 172.21.15.16:0/3026360196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7bc2f0 msgr2=0x564d0fb95e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.901+0000 7f0494a00640 1 --2- 172.21.15.16:0/3026360196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7bc2f0 0x564d0fb95e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.901+0000 7f0494a00640 1 -- 172.21.15.16:0/3026360196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0fb94650 con 0x564d0f7c44d0 2024-09-21T03:57:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.901+0000 7f0495e00640 1 --2- 172.21.15.16:0/3026360196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7e51b0 0x564d0fb94e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:57:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.901+0000 7f0495400640 1 --2- 172.21.15.16:0/3026360196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7bc2f0 0x564d0fb95e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:57:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.902+0000 7f0494a00640 1 --2- 172.21.15.16:0/3026360196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7c44d0 0x564d0fb94900 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f047c011750 tx=0x7f047c00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1977a28614bfe882 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.902+0000 7f0481c00640 1 -- 172.21.15.16:0/3026360196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c00fb30 con 0x564d0f7c44d0 2024-09-21T03:57:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.902+0000 7f0481c00640 1 -- 172.21.15.16:0/3026360196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047c019d90 con 0x564d0f7c44d0 2024-09-21T03:57:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.902+0000 7f0481c00640 1 -- 172.21.15.16:0/3026360196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c0035f0 con 0x564d0f7c44d0 2024-09-21T03:57:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.903+0000 7f049f015740 1 -- 172.21.15.16:0/3026360196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7c44d0 msgr2=0x564d0fb94900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.903+0000 7f049f015740 1 --2- 172.21.15.16:0/3026360196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7c44d0 0x564d0fb94900 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f047c011750 tx=0x7f047c00f810 comp rx=0 tx=0).stop 2024-09-21T03:57:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.903+0000 7f049f015740 1 -- 172.21.15.16:0/3026360196 shutdown_connections 2024-09-21T03:57:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.903+0000 7f049f015740 1 --2- 172.21.15.16:0/3026360196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7e51b0 0x564d0fb94e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.903+0000 7f049f015740 1 --2- 172.21.15.16:0/3026360196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7c44d0 0x564d0fb94900 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.903+0000 7f049f015740 1 --2- 172.21.15.16:0/3026360196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7bc2f0 0x564d0fb95e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.903+0000 7f049f015740 1 -- 172.21.15.16:0/3026360196 >> 172.21.15.16:0/3026360196 conn(0x564d0f7d9300 msgr2=0x564d0fb95700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.903+0000 7f049f015740 1 -- 172.21.15.16:0/3026360196 shutdown_connections 2024-09-21T03:57:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.903+0000 7f049f015740 1 -- 172.21.15.16:0/3026360196 wait complete. 2024-09-21T03:57:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.903+0000 7f049f015740 1 -- start start 2024-09-21T03:57:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.903+0000 7f049f015740 1 Processor -- start 2024-09-21T03:57:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f049f015740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7bc2f0 0x564d0fb9a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f049f015740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7c44d0 0x564d0fb9ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f049f015740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7e51b0 0x564d0fb9b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f049f015740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04901b4130 con 0x564d0f7c44d0 2024-09-21T03:57:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f049f015740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04901b3a60 con 0x564d0f7e51b0 2024-09-21T03:57:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f049f015740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04901b0ca0 con 0x564d0f7bc2f0 2024-09-21T03:57:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f049f015740 10 client.0 _ll_register_callbacks cb 0x7f047c009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:57:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f0494a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7c44d0 0x564d0fb9ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f0494a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7c44d0 0x564d0fb9ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35084/0 (socket says 172.21.15.16:35084) 2024-09-21T03:57:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f0494a00640 1 -- 172.21.15.16:0/999853925 learned_addr learned my addr 172.21.15.16:0/999853925 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f0495e00640 1 --2- 172.21.15.16:0/999853925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7e51b0 0x564d0fb9b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f0495400640 1 --2- 172.21.15.16:0/999853925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7bc2f0 0x564d0fb9a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f0494a00640 1 -- 172.21.15.16:0/999853925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7bc2f0 msgr2=0x564d0fb9a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f0494a00640 1 --2- 172.21.15.16:0/999853925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7bc2f0 0x564d0fb9a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f0494a00640 1 -- 172.21.15.16:0/999853925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7e51b0 msgr2=0x564d0fb9b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f0494a00640 1 --2- 172.21.15.16:0/999853925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7e51b0 0x564d0fb9b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f0494a00640 1 -- 172.21.15.16:0/999853925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0fb9be50 con 0x564d0f7c44d0 2024-09-21T03:57:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f0495e00640 1 --2- 172.21.15.16:0/999853925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7e51b0 0x564d0fb9b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:57:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.904+0000 7f0495400640 1 --2- 172.21.15.16:0/999853925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7bc2f0 0x564d0fb9a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:57:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.905+0000 7f0494a00640 1 --2- 172.21.15.16:0/999853925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7c44d0 0x564d0fb9ab80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f047c0114a0 tx=0x7f047c005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50980ac21975bb58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.905+0000 7f0476a00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:57:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.905+0000 7f0476a00640 1 -- 172.21.15.16:0/999853925 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c011240 con 0x564d0f7c44d0 2024-09-21T03:57:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.905+0000 7f0476a00640 1 -- 172.21.15.16:0/999853925 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f047c017040 con 0x564d0f7c44d0 2024-09-21T03:57:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.905+0000 7f049f015740 1 -- 172.21.15.16:0/999853925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f047c019d90 con 0x564d0f7c44d0 2024-09-21T03:57:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.905+0000 7f0476a00640 1 -- 172.21.15.16:0/999853925 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f047c0198e0 con 0x564d0f7c44d0 2024-09-21T03:57:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.905+0000 7f049f015740 10 client.16500 fetch_fsmap learned FSMap version 30 2024-09-21T03:57:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.905+0000 7f049f015740 1 -- 172.21.15.16:0/999853925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564d0fb9d140 con 0x564d0f7c44d0 2024-09-21T03:57:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.906+0000 7f0476a00640 1 -- 172.21.15.16:0/999853925 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047c019d90 con 0x564d0f7c44d0 2024-09-21T03:57:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.906+0000 7f0476a00640 1 -- 172.21.15.16:0/999853925 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f047c022040 con 0x564d0f7c44d0 2024-09-21T03:57:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.906+0000 7f049f015740 10 client.16500 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:57:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.906+0000 7f049f015740 10 client.16500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:57:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.906+0000 7f049f015740 20 client.16500 populate_metadata read hostname 'smithi016' 2024-09-21T03:57:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.906+0000 7f049f015740 1 --2- 172.21.15.16:0/999853925 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x564d0f7791c0 0x564d0fc0fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.906+0000 7f049f015740 4 client.16500 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:57:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.906+0000 7f049f015740 1 -- 172.21.15.16:0/999853925 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d0fafd0b0 con 0x564d0f7791c0 2024-09-21T03:57:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.906+0000 7f0495400640 1 --2- 172.21.15.16:0/999853925 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x564d0f7791c0 0x564d0fc0fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.907+0000 7f0495400640 1 --2- 172.21.15.16:0/999853925 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x564d0f7791c0 0x564d0fc0fbe0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=945f4ecb28e10b6a server_cookie=883571d6b8b602c3 in_seq=0 out_seq=0 2024-09-21T03:57:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.907+0000 7f0476a00640 10 client.16500 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.934+0000 7f0476a00640 1 -- 172.21.15.16:0/999853925 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d0fafd0b0 con 0x564d0f7791c0 2024-09-21T03:57:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.934+0000 7f0476a00640 10 client.16500 handle_command_reply: tid=0 2024-09-21T03:57:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.934+0000 7f049f015740 1 client.16500 shutdown 2024-09-21T03:57:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.934+0000 7f049f015740 1 -- 172.21.15.16:0/999853925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7c44d0 msgr2=0x564d0fb9ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.934+0000 7f049f015740 1 --2- 172.21.15.16:0/999853925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7c44d0 0x564d0fb9ab80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f047c0114a0 tx=0x7f047c005300 comp rx=0 tx=0).stop 2024-09-21T03:57:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.934+0000 7f049f015740 1 -- 172.21.15.16:0/999853925 shutdown_connections 2024-09-21T03:57:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.934+0000 7f049f015740 1 --2- 172.21.15.16:0/999853925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d0f7e51b0 0x564d0fb9b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.934+0000 7f049f015740 1 --2- 172.21.15.16:0/999853925 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x564d0f7791c0 0x564d0fc0fbe0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.934+0000 7f049f015740 1 --2- 172.21.15.16:0/999853925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d0f7c44d0 0x564d0fb9ab80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.934+0000 7f049f015740 1 --2- 172.21.15.16:0/999853925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0f7bc2f0 0x564d0fb9a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.934+0000 7f0476a00640 0 client.16500 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.934+0000 7f049f015740 1 -- 172.21.15.16:0/999853925 >> 172.21.15.16:0/999853925 conn(0x564d0f7d9300 msgr2=0x564d0fb94900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.935+0000 7f049f015740 1 -- 172.21.15.16:0/999853925 shutdown_connections 2024-09-21T03:57:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.935+0000 7f049f015740 1 -- 172.21.15.16:0/999853925 wait complete. 2024-09-21T03:57:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.935+0000 7f049f015740 20 client.16500 trim_cache size 0 max 16384 2024-09-21T03:57:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-21T03:57:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-21T03:57:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-21T03:57:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-21T03:57:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.935+0000 7f0497e00640 1 -- 172.21.15.16:0/698002130 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04780b6420 msgr2=0x7f04780b8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.935+0000 7f0497e00640 1 --2- 172.21.15.16:0/698002130 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04780b6420 0x7f04780b8930 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0488009b30 tx=0x7f048802a6c0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.935+0000 7f0497e00640 1 -- 172.21.15.16:0/698002130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0490075cf0 msgr2=0x7f04901a04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.935+0000 7f0497e00640 1 --2- 172.21.15.16:0/698002130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0490075cf0 0x7f04901a04d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f048400e9f0 tx=0x7f048400edf0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.935+0000 7f0497e00640 1 -- 172.21.15.16:0/698002130 shutdown_connections 2024-09-21T03:57:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.935+0000 7f0497e00640 1 --2- 172.21.15.16:0/698002130 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f04780b6420 0x7f04780b8930 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.935+0000 7f0497e00640 1 --2- 172.21.15.16:0/698002130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049010fe50 0x7f04901a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.935+0000 7f0497e00640 1 --2- 172.21.15.16:0/698002130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04900770f0 0x7f04901a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.935+0000 7f0497e00640 1 --2- 172.21.15.16:0/698002130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0490075cf0 0x7f04901a04d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.936+0000 7f0497e00640 1 -- 172.21.15.16:0/698002130 >> 172.21.15.16:0/698002130 conn(0x7f04900ff030 msgr2=0x7f04900ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.936+0000 7f0497e00640 1 -- 172.21.15.16:0/698002130 shutdown_connections 2024-09-21T03:57:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-21T03:57:52.936+0000 7f0497e00640 1 -- 172.21.15.16:0/698002130 wait complete. 2024-09-21T03:57:53.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:53 smithi134 ceph-mon[35031]: pgmap v765: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 222 MiB/s wr, 1.70k op/s 2024-09-21T03:57:53.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:53 smithi123 ceph-mon[34673]: pgmap v765: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 222 MiB/s wr, 1.70k op/s 2024-09-21T03:57:53.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:53 smithi016 ceph-mon[30375]: pgmap v765: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 222 MiB/s wr, 1.70k op/s 2024-09-21T03:57:53.752 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ce95ca56', total seconds elapsed: 35.2 2024-09-21T03:57:54.159 DEBUG:teuthology.parallel:result is None 2024-09-21T03:57:54.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T03:57:54.222 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T03:57:54.223 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-21T03:57:54.237 INFO:tasks.quiescer:joining Quiescers 2024-09-21T03:57:54.237 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-21T03:57:54.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:57:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:57:55.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.255+0000 7f7dc0c00640 1 -- 172.21.15.16:0/1509363983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc109c40 msgr2=0x7f7dbc110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:55.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.255+0000 7f7dc0c00640 1 --2- 172.21.15.16:0/1509363983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc109c40 0x7f7dbc110470 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009ca0 tx=0x7f7db402a8b0 comp rx=0 tx=0).stop 2024-09-21T03:57:55.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.255+0000 7f7dc0c00640 1 -- 172.21.15.16:0/1509363983 shutdown_connections 2024-09-21T03:57:55.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.255+0000 7f7dc0c00640 1 --2- 172.21.15.16:0/1509363983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc109c40 0x7f7dbc110470 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:55.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.255+0000 7f7dc0c00640 1 --2- 172.21.15.16:0/1509363983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dbc1092f0 0x7f7dbc109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:55.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.255+0000 7f7dc0c00640 1 --2- 172.21.15.16:0/1509363983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dbc1039b0 0x7f7dbc103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:55.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.255+0000 7f7dc0c00640 1 -- 172.21.15.16:0/1509363983 >> 172.21.15.16:0/1509363983 conn(0x7f7dbc0ff440 msgr2=0x7f7dbc101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:55.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.256+0000 7f7dc0c00640 1 -- 172.21.15.16:0/1509363983 shutdown_connections 2024-09-21T03:57:55.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.256+0000 7f7dc0c00640 1 -- 172.21.15.16:0/1509363983 wait complete. 2024-09-21T03:57:55.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.257+0000 7f7dc0c00640 1 Processor -- start 2024-09-21T03:57:55.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.257+0000 7f7dc0c00640 1 -- start start 2024-09-21T03:57:55.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.257+0000 7f7dc0c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dbc1039b0 0x7f7dbc19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:55.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.257+0000 7f7dc0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc1092f0 0x7f7dbc19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:55.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.257+0000 7f7dc0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dbc109c40 0x7f7dbc1a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:55.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.257+0000 7f7dc0c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc113900 con 0x7f7dbc1092f0 2024-09-21T03:57:55.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.257+0000 7f7dc0c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc113780 con 0x7f7dbc109c40 2024-09-21T03:57:55.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.257+0000 7f7dc0c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc113a80 con 0x7f7dbc1039b0 2024-09-21T03:57:55.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.258+0000 7f7db3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc1092f0 0x7f7dbc19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:55.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.258+0000 7f7dbb400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dbc1039b0 0x7f7dbc19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:55.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.258+0000 7f7db3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc1092f0 0x7f7dbc19ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35088/0 (socket says 172.21.15.16:35088) 2024-09-21T03:57:55.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.258+0000 7f7dbb400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dbc1039b0 0x7f7dbc19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:60488/0 (socket says 172.21.15.16:60488) 2024-09-21T03:57:55.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.258+0000 7f7db3e00640 1 -- 172.21.15.16:0/1994343013 learned_addr learned my addr 172.21.15.16:0/1994343013 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:55.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.258+0000 7f7dbbe00640 1 --2- 172.21.15.16:0/1994343013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dbc109c40 0x7f7dbc1a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:55.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.258+0000 7f7dbb400640 1 -- 172.21.15.16:0/1994343013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dbc109c40 msgr2=0x7f7dbc1a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:55.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.258+0000 7f7dbb400640 1 --2- 172.21.15.16:0/1994343013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dbc109c40 0x7f7dbc1a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:55.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.258+0000 7f7dbb400640 1 -- 172.21.15.16:0/1994343013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc1092f0 msgr2=0x7f7dbc19ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.258+0000 7f7dbb400640 1 --2- 172.21.15.16:0/1994343013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc1092f0 0x7f7dbc19ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.258+0000 7f7dbb400640 1 -- 172.21.15.16:0/1994343013 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc1a2460 con 0x7f7dbc1039b0 2024-09-21T03:57:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.258+0000 7f7dbbe00640 1 --2- 172.21.15.16:0/1994343013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dbc109c40 0x7f7dbc1a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:57:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.259+0000 7f7dbb400640 1 --2- 172.21.15.16:0/1994343013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dbc1039b0 0x7f7dbc19c4f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00d6d0 tx=0x7f7dac00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.259+0000 7f7db8c00640 1 -- 172.21.15.16:0/1994343013 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac017070 con 0x7f7dbc1039b0 2024-09-21T03:57:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.259+0000 7f7dc0c00640 1 -- 172.21.15.16:0/1994343013 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dbc1a2750 con 0x7f7dbc1039b0 2024-09-21T03:57:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.259+0000 7f7db8c00640 1 -- 172.21.15.16:0/1994343013 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dac009820 con 0x7f7dbc1039b0 2024-09-21T03:57:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.259+0000 7f7db8c00640 1 -- 172.21.15.16:0/1994343013 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac004c40 con 0x7f7dbc1039b0 2024-09-21T03:57:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.259+0000 7f7dc0c00640 1 -- 172.21.15.16:0/1994343013 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dbc1a8eb0 con 0x7f7dbc1039b0 2024-09-21T03:57:55.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.260+0000 7f7dc0c00640 1 -- 172.21.15.16:0/1994343013 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dbc103dc0 con 0x7f7dbc1039b0 2024-09-21T03:57:55.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.262+0000 7f7db8c00640 1 -- 172.21.15.16:0/1994343013 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f7dac005220 con 0x7f7dbc1039b0 2024-09-21T03:57:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.264+0000 7f7db8c00640 1 --2- 172.21.15.16:0/1994343013 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7da00b6510 0x7f7da00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.264+0000 7f7db8c00640 1 -- 172.21.15.16:0/1994343013 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f7dac0e7190 con 0x7f7dbc1039b0 2024-09-21T03:57:55.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.264+0000 7f7db3e00640 1 --2- 172.21.15.16:0/1994343013 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7da00b6510 0x7f7da00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:55.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.265+0000 7f7db3e00640 1 --2- 172.21.15.16:0/1994343013 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7da00b6510 0x7f7da00b8a20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc1059f0 tx=0x7f7da4005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:55.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.268+0000 7f7db8c00640 1 -- 172.21.15.16:0/1994343013 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7dac0b19f0 con 0x7f7dbc1039b0 2024-09-21T03:57:55.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:55 smithi016 ceph-mon[30375]: pgmap v766: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 201 MiB/s wr, 1.60k op/s 2024-09-21T03:57:55.459 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:55 smithi123 ceph-mon[34673]: pgmap v766: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 201 MiB/s wr, 1.60k op/s 2024-09-21T03:57:55.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.520+0000 7f7dc0c00640 1 -- 172.21.15.16:0/1994343013 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dbc1a9270 con 0x7f7dbc1039b0 2024-09-21T03:57:55.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.521+0000 7f7db8c00640 1 -- 172.21.15.16:0/1994343013 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f7dac0b6930 con 0x7f7dbc1039b0 2024-09-21T03:57:55.524 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:57:55.524 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:57:55.524 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:57:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.524+0000 7f7dc0c00640 1 -- 172.21.15.16:0/1994343013 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7da00b6510 msgr2=0x7f7da00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.524+0000 7f7dc0c00640 1 --2- 172.21.15.16:0/1994343013 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7da00b6510 0x7f7da00b8a20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc1059f0 tx=0x7f7da4005f90 comp rx=0 tx=0).stop 2024-09-21T03:57:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.524+0000 7f7dc0c00640 1 -- 172.21.15.16:0/1994343013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dbc1039b0 msgr2=0x7f7dbc19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.524+0000 7f7dc0c00640 1 --2- 172.21.15.16:0/1994343013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dbc1039b0 0x7f7dbc19c4f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00d6d0 tx=0x7f7dac00db90 comp rx=0 tx=0).stop 2024-09-21T03:57:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.524+0000 7f7dc0c00640 1 -- 172.21.15.16:0/1994343013 shutdown_connections 2024-09-21T03:57:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.524+0000 7f7dc0c00640 1 --2- 172.21.15.16:0/1994343013 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7da00b6510 0x7f7da00b8a20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.524+0000 7f7dc0c00640 1 --2- 172.21.15.16:0/1994343013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dbc109c40 0x7f7dbc1a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.524+0000 7f7dc0c00640 1 --2- 172.21.15.16:0/1994343013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dbc1092f0 0x7f7dbc19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.524+0000 7f7dc0c00640 1 --2- 172.21.15.16:0/1994343013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dbc1039b0 0x7f7dbc19c4f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.524+0000 7f7dc0c00640 1 -- 172.21.15.16:0/1994343013 >> 172.21.15.16:0/1994343013 conn(0x7f7dbc0ff440 msgr2=0x7f7dbc108980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.524+0000 7f7dc0c00640 1 -- 172.21.15.16:0/1994343013 shutdown_connections 2024-09-21T03:57:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:55.524+0000 7f7dc0c00640 1 -- 172.21.15.16:0/1994343013 wait complete. 2024-09-21T03:57:55.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:55 smithi134 ceph-mon[35031]: pgmap v766: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 201 MiB/s wr, 1.60k op/s 2024-09-21T03:57:56.330 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24731 quiesce db --cancel --all' 2024-09-21T03:57:56.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24731 quiesce db --cancel --all 2024-09-21T03:57:56.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:56 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1994343013' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:57:56.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:56 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1994343013' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:57:56.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:56 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1994343013' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:57:56.708 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:57:57.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.377+0000 7f5a18a00640 1 -- 172.21.15.16:0/2401517642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a14109e10 msgr2=0x7f5a14110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.377+0000 7f5a18a00640 1 --2- 172.21.15.16:0/2401517642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a14109e10 0x7f5a14110640 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009ca0 tx=0x7f5a0c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.377+0000 7f5a18a00640 1 -- 172.21.15.16:0/2401517642 shutdown_connections 2024-09-21T03:57:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.377+0000 7f5a18a00640 1 --2- 172.21.15.16:0/2401517642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a14109e10 0x7f5a14110640 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.377+0000 7f5a18a00640 1 --2- 172.21.15.16:0/2401517642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a141094c0 0x7f5a141098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.377+0000 7f5a18a00640 1 --2- 172.21.15.16:0/2401517642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a14103b80 0x7f5a14103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.377+0000 7f5a18a00640 1 -- 172.21.15.16:0/2401517642 >> 172.21.15.16:0/2401517642 conn(0x7f5a140ff610 msgr2=0x7f5a14101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.377+0000 7f5a18a00640 1 -- 172.21.15.16:0/2401517642 shutdown_connections 2024-09-21T03:57:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.377+0000 7f5a18a00640 1 -- 172.21.15.16:0/2401517642 wait complete. 2024-09-21T03:57:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.378+0000 7f5a18a00640 1 Processor -- start 2024-09-21T03:57:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.379+0000 7f5a18a00640 1 -- start start 2024-09-21T03:57:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.379+0000 7f5a18a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a14103b80 0x7f5a1419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.379+0000 7f5a18a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a141094c0 0x7f5a1419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.379+0000 7f5a18a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a14109e10 0x7f5a141a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.379+0000 7f5a18a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a14113b20 con 0x7f5a14109e10 2024-09-21T03:57:57.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.379+0000 7f5a18a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141139a0 con 0x7f5a141094c0 2024-09-21T03:57:57.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.379+0000 7f5a18a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a14113ca0 con 0x7f5a14103b80 2024-09-21T03:57:57.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.379+0000 7f5a11e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a14103b80 0x7f5a1419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.379+0000 7f5a11e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a14103b80 0x7f5a1419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:60504/0 (socket says 172.21.15.16:60504) 2024-09-21T03:57:57.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.379+0000 7f5a11e00640 1 -- 172.21.15.16:0/229425412 learned_addr learned my addr 172.21.15.16:0/229425412 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:57.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.379+0000 7f5a11400640 1 --2- 172.21.15.16:0/229425412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a141094c0 0x7f5a1419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.380+0000 7f5a12800640 1 --2- 172.21.15.16:0/229425412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a14109e10 0x7f5a141a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.380+0000 7f5a11e00640 1 -- 172.21.15.16:0/229425412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a141094c0 msgr2=0x7f5a1419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:57.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.380+0000 7f5a11e00640 1 --2- 172.21.15.16:0/229425412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a141094c0 0x7f5a1419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.380+0000 7f5a11e00640 1 -- 172.21.15.16:0/229425412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a14109e10 msgr2=0x7f5a141a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:57.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.380+0000 7f5a11e00640 1 --2- 172.21.15.16:0/229425412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a14109e10 0x7f5a141a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.380+0000 7f5a11e00640 1 -- 172.21.15.16:0/229425412 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a141a25e0 con 0x7f5a14103b80 2024-09-21T03:57:57.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.380+0000 7f5a11400640 1 --2- 172.21.15.16:0/229425412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a141094c0 0x7f5a1419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:57:57.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.380+0000 7f5a11e00640 1 --2- 172.21.15.16:0/229425412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a14103b80 0x7f5a1419c670 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400e9f0 tx=0x7f5a0400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:57.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.381+0000 7f5a0aa00640 1 -- 172.21.15.16:0/229425412 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0400cd80 con 0x7f5a14103b80 2024-09-21T03:57:57.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.381+0000 7f5a0aa00640 1 -- 172.21.15.16:0/229425412 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a040108e0 con 0x7f5a14103b80 2024-09-21T03:57:57.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.381+0000 7f5a18a00640 1 -- 172.21.15.16:0/229425412 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a141a28d0 con 0x7f5a14103b80 2024-09-21T03:57:57.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.381+0000 7f5a0aa00640 1 -- 172.21.15.16:0/229425412 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04018980 con 0x7f5a14103b80 2024-09-21T03:57:57.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.381+0000 7f5a18a00640 1 -- 172.21.15.16:0/229425412 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a141a9030 con 0x7f5a14103b80 2024-09-21T03:57:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.384+0000 7f5a1f65a740 1 Processor -- start 2024-09-21T03:57:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.384+0000 7f5a1f65a740 1 -- start start 2024-09-21T03:57:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.385+0000 7f5a1f65a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 0x55a8d86c9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.385+0000 7f5a1f65a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d82fa410 0x55a8d86c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.385+0000 7f5a1f65a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d83025f0 0x55a8d86c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.385+0000 7f5a1f65a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a8d8640ff0 con 0x55a8d83232d0 2024-09-21T03:57:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.385+0000 7f5a1f65a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a8d8630420 con 0x55a8d83025f0 2024-09-21T03:57:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.385+0000 7f5a1f65a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a8d8634230 con 0x55a8d82fa410 2024-09-21T03:57:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.385+0000 7f5a0aa00640 1 -- 172.21.15.16:0/229425412 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f5a04010430 con 0x7f5a14103b80 2024-09-21T03:57:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.385+0000 7f5a12800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 0x55a8d86c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.385+0000 7f5a11400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d83025f0 0x55a8d86c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.385+0000 7f5a12800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 0x55a8d86c9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35108/0 (socket says 172.21.15.16:35108) 2024-09-21T03:57:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.385+0000 7f5a12800640 1 -- 172.21.15.16:0/1298383770 learned_addr learned my addr 172.21.15.16:0/1298383770 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.385+0000 7f5a11e00640 1 --2- 172.21.15.16:0/1298383770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d82fa410 0x55a8d86c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.385+0000 7f5a12800640 1 -- 172.21.15.16:0/1298383770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d82fa410 msgr2=0x55a8d86c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.385+0000 7f5a12800640 1 --2- 172.21.15.16:0/1298383770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d82fa410 0x55a8d86c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.385+0000 7f5a12800640 1 -- 172.21.15.16:0/1298383770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d83025f0 msgr2=0x55a8d86c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:57.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.386+0000 7f5a12800640 1 --2- 172.21.15.16:0/1298383770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d83025f0 0x55a8d86c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.386+0000 7f5a12800640 1 -- 172.21.15.16:0/1298383770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8d838e8b0 con 0x55a8d83232d0 2024-09-21T03:57:57.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.386+0000 7f5a11e00640 1 --2- 172.21.15.16:0/1298383770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d82fa410 0x55a8d86c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:57:57.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.386+0000 7f5a12800640 1 --2- 172.21.15.16:0/1298383770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 0x55a8d86c9290 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009ca0 tx=0x7f5a0c009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78189a154e2369b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:57.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.386+0000 7f59ff000640 1 -- 172.21.15.16:0/1298383770 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c031cf0 con 0x55a8d83232d0 2024-09-21T03:57:57.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.386+0000 7f59ff000640 1 -- 172.21.15.16:0/1298383770 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a0c036030 con 0x55a8d83232d0 2024-09-21T03:57:57.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.386+0000 7f59ff000640 1 -- 172.21.15.16:0/1298383770 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c004260 con 0x55a8d83232d0 2024-09-21T03:57:57.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.387+0000 7f5a0aa00640 1 --2- 172.21.15.16:0/229425412 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f59f80b6510 0x7f59f80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.387+0000 7f5a0aa00640 1 -- 172.21.15.16:0/229425412 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f5a040e7a00 con 0x7f5a14103b80 2024-09-21T03:57:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.387+0000 7f5a11e00640 1 --2- 172.21.15.16:0/229425412 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f59f80b6510 0x7f59f80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.387+0000 7f5a1f65a740 1 -- 172.21.15.16:0/1298383770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 msgr2=0x55a8d86c9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.387+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/1298383770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 0x55a8d86c9290 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009ca0 tx=0x7f5a0c009430 comp rx=0 tx=0).stop 2024-09-21T03:57:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.387+0000 7f5a1f65a740 1 -- 172.21.15.16:0/1298383770 shutdown_connections 2024-09-21T03:57:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.387+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/1298383770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d83025f0 0x55a8d86c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.387+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/1298383770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d82fa410 0x55a8d86c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.387+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/1298383770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 0x55a8d86c9290 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.387+0000 7f5a1f65a740 1 -- 172.21.15.16:0/1298383770 >> 172.21.15.16:0/1298383770 conn(0x55a8d83176c0 msgr2=0x55a8d86c7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.388+0000 7f5a1f65a740 1 -- 172.21.15.16:0/1298383770 shutdown_connections 2024-09-21T03:57:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.388+0000 7f5a1f65a740 1 -- 172.21.15.16:0/1298383770 wait complete. 2024-09-21T03:57:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.388+0000 7f5a11e00640 1 --2- 172.21.15.16:0/229425412 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f59f80b6510 0x7f59f80b8a20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5a040aaaa0 tx=0x7f5a04000bc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.388+0000 7f5a1f65a740 1 -- start start 2024-09-21T03:57:57.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.388+0000 7f5a1f65a740 1 Processor -- start 2024-09-21T03:57:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.389+0000 7f5a1f65a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d82fa410 0x55a8d86d4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.389+0000 7f5a1f65a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d83025f0 0x55a8d86d4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.389+0000 7f5a1f65a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 0x55a8d86e0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.389+0000 7f5a1f65a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a8d8491810 con 0x55a8d83232d0 2024-09-21T03:57:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.389+0000 7f5a1f65a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a8d85cb550 con 0x55a8d82fa410 2024-09-21T03:57:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.389+0000 7f5a1f65a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a8d86d7890 con 0x55a8d83025f0 2024-09-21T03:57:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.389+0000 7f5a1f65a740 10 client.0 _ll_register_callbacks cb 0x55a8d85db040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:57:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.389+0000 7f5a11400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d83025f0 0x55a8d86d4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.389+0000 7f5a12800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d82fa410 0x55a8d86d4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.389+0000 7f5a11400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d83025f0 0x55a8d86d4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:60532/0 (socket says 172.21.15.16:60532) 2024-09-21T03:57:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.389+0000 7f5a11400640 1 -- 172.21.15.16:0/2248617021 learned_addr learned my addr 172.21.15.16:0/2248617021 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.389+0000 7f5a12800640 1 --2- 172.21.15.16:0/2248617021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 0x55a8d86e0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.390+0000 7f5a11400640 1 -- 172.21.15.16:0/2248617021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d82fa410 msgr2=0x55a8d86d4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.390+0000 7f5a11400640 1 --2- 172.21.15.16:0/2248617021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d82fa410 0x55a8d86d4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.390+0000 7f5a11400640 1 -- 172.21.15.16:0/2248617021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 msgr2=0x55a8d86e0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.390+0000 7f5a11400640 1 --2- 172.21.15.16:0/2248617021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 0x55a8d86e0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.390+0000 7f5a11400640 1 -- 172.21.15.16:0/2248617021 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8d85bd200 con 0x55a8d83025f0 2024-09-21T03:57:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.390+0000 7f5a11400640 1 --2- 172.21.15.16:0/2248617021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d83025f0 0x55a8d86d4e50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5a00012d20 tx=0x7f5a0000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e8a1ee521bdd6d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.390+0000 7f59f3400640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T03:57:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.390+0000 7f59f3400640 1 -- 172.21.15.16:0/2248617021 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a00012110 con 0x55a8d83025f0 2024-09-21T03:57:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.390+0000 7f5a1f65a740 1 -- 172.21.15.16:0/2248617021 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a0c036030 con 0x55a8d83025f0 2024-09-21T03:57:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.390+0000 7f59f3400640 1 -- 172.21.15.16:0/2248617021 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f5a00004050 con 0x55a8d83025f0 2024-09-21T03:57:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.391+0000 7f59f3400640 1 -- 172.21.15.16:0/2248617021 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5a00025050 con 0x55a8d83025f0 2024-09-21T03:57:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.391+0000 7f5a1f65a740 10 client.26746 fetch_fsmap learned FSMap version 30 2024-09-21T03:57:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.391+0000 7f5a1f65a740 1 -- 172.21.15.16:0/2248617021 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a8d86e3610 con 0x55a8d83025f0 2024-09-21T03:57:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.391+0000 7f59f3400640 1 -- 172.21.15.16:0/2248617021 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a0c036030 con 0x55a8d83025f0 2024-09-21T03:57:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.391+0000 7f59f3400640 1 -- 172.21.15.16:0/2248617021 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5a00010430 con 0x55a8d83025f0 2024-09-21T03:57:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.391+0000 7f5a1f65a740 10 client.26746 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:57:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.391+0000 7f5a1f65a740 10 client.26746 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:57:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.391+0000 7f5a1f65a740 20 client.26746 populate_metadata read hostname 'smithi016' 2024-09-21T03:57:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.391+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/2248617021 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a8d82e1c50 0x55a8d86e6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.391+0000 7f5a1f65a740 4 client.26746 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:57:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.391+0000 7f5a1f65a740 1 -- 172.21.15.16:0/2248617021 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8d8642f40 con 0x55a8d82e1c50 2024-09-21T03:57:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.392+0000 7f5a12800640 1 --2- 172.21.15.16:0/2248617021 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a8d82e1c50 0x55a8d86e6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.392+0000 7f5a12800640 1 --2- 172.21.15.16:0/2248617021 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a8d82e1c50 0x55a8d86e6b60 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26bcf9fbe68ce1e1 server_cookie=6de9675ab55ce8a5 in_seq=0 out_seq=0 2024-09-21T03:57:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.392+0000 7f59f3400640 10 client.26746 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.393+0000 7f59f3400640 1 -- 172.21.15.16:0/2248617021 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a8d8642f40 con 0x55a8d82e1c50 2024-09-21T03:57:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.393+0000 7f59f3400640 10 client.26746 handle_command_reply: tid=0 2024-09-21T03:57:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.393+0000 7f5a1f65a740 1 client.26746 shutdown 2024-09-21T03:57:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.394+0000 7f5a1f65a740 1 -- 172.21.15.16:0/2248617021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d83025f0 msgr2=0x55a8d86d4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.394+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/2248617021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d83025f0 0x55a8d86d4e50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5a00012d20 tx=0x7f5a0000f790 comp rx=0 tx=0).stop 2024-09-21T03:57:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.394+0000 7f5a1f65a740 1 -- 172.21.15.16:0/2248617021 shutdown_connections 2024-09-21T03:57:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.394+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/2248617021 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a8d82e1c50 0x55a8d86e6b60 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.394+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/2248617021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 0x55a8d86e0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.394+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/2248617021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d83025f0 0x55a8d86d4e50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.394+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/2248617021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d82fa410 0x55a8d86d4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.394+0000 7f59f3400640 0 client.26746 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.394+0000 7f5a1f65a740 1 -- 172.21.15.16:0/2248617021 >> 172.21.15.16:0/2248617021 conn(0x55a8d83176c0 msgr2=0x55a8d86cc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.394+0000 7f5a1f65a740 1 -- 172.21.15.16:0/2248617021 shutdown_connections 2024-09-21T03:57:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.394+0000 7f5a1f65a740 1 -- 172.21.15.16:0/2248617021 wait complete. 2024-09-21T03:57:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.394+0000 7f5a1f65a740 20 client.26746 trim_cache size 0 max 16384 2024-09-21T03:57:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.412+0000 7f5a1f65a740 1 Processor -- start 2024-09-21T03:57:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.412+0000 7f5a1f65a740 1 -- start start 2024-09-21T03:57:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.412+0000 7f5a1f65a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d82fa410 0x55a8d86d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.412+0000 7f5a1f65a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d83025f0 0x55a8d86d2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.412+0000 7f5a1f65a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 0x55a8d86ec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.412+0000 7f5a1f65a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141b2d20 con 0x55a8d83232d0 2024-09-21T03:57:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.412+0000 7f5a1f65a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141b4860 con 0x55a8d83025f0 2024-09-21T03:57:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.412+0000 7f5a1f65a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141b33f0 con 0x55a8d82fa410 2024-09-21T03:57:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.413+0000 7f5a12800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 0x55a8d86ec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.413+0000 7f5a12800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 0x55a8d86ec340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35122/0 (socket says 172.21.15.16:35122) 2024-09-21T03:57:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.413+0000 7f5a12800640 1 -- 172.21.15.16:0/2865349329 learned_addr learned my addr 172.21.15.16:0/2865349329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.413+0000 7f5a11400640 1 --2- 172.21.15.16:0/2865349329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d83025f0 0x55a8d86d2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.413+0000 7f5a12800640 1 --2- 172.21.15.16:0/2865349329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d82fa410 0x55a8d86d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.413+0000 7f5a12800640 1 -- 172.21.15.16:0/2865349329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d82fa410 msgr2=0x55a8d86d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.413+0000 7f5a12800640 1 --2- 172.21.15.16:0/2865349329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d82fa410 0x55a8d86d1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.413+0000 7f5a12800640 1 -- 172.21.15.16:0/2865349329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d83025f0 msgr2=0x55a8d86d2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.413+0000 7f5a12800640 1 --2- 172.21.15.16:0/2865349329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d83025f0 0x55a8d86d2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.413+0000 7f5a12800640 1 -- 172.21.15.16:0/2865349329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8d86d51a0 con 0x55a8d83232d0 2024-09-21T03:57:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.414+0000 7f5a12800640 1 --2- 172.21.15.16:0/2865349329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 0x55a8d86ec340 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c004af0 tx=0x7f5a0c005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d2713b9cc72cee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.414+0000 7f59fe600640 1 -- 172.21.15.16:0/2865349329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c031af0 con 0x55a8d83232d0 2024-09-21T03:57:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.414+0000 7f59fe600640 1 -- 172.21.15.16:0/2865349329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a0c036070 con 0x55a8d83232d0 2024-09-21T03:57:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.414+0000 7f59fe600640 1 -- 172.21.15.16:0/2865349329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c004200 con 0x55a8d83232d0 2024-09-21T03:57:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.415+0000 7f5a1f65a740 1 -- 172.21.15.16:0/2865349329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 msgr2=0x55a8d86ec340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.415+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/2865349329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 0x55a8d86ec340 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c004af0 tx=0x7f5a0c005d80 comp rx=0 tx=0).stop 2024-09-21T03:57:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.415+0000 7f5a1f65a740 1 -- 172.21.15.16:0/2865349329 shutdown_connections 2024-09-21T03:57:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.415+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/2865349329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83232d0 0x55a8d86ec340 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.415+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/2865349329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d83025f0 0x55a8d86d2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.415+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/2865349329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d82fa410 0x55a8d86d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.415+0000 7f5a1f65a740 1 -- 172.21.15.16:0/2865349329 >> 172.21.15.16:0/2865349329 conn(0x55a8d83176c0 msgr2=0x55a8d86d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.415+0000 7f5a1f65a740 1 -- 172.21.15.16:0/2865349329 shutdown_connections 2024-09-21T03:57:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.415+0000 7f5a1f65a740 1 -- 172.21.15.16:0/2865349329 wait complete. 2024-09-21T03:57:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.415+0000 7f5a1f65a740 1 -- start start 2024-09-21T03:57:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.415+0000 7f5a1f65a740 1 Processor -- start 2024-09-21T03:57:57.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.416+0000 7f5a1f65a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d82fa410 0x55a8d86dccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.416+0000 7f5a1f65a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83025f0 0x55a8d86dd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.416+0000 7f5a1f65a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d83232d0 0x55a8d86d7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.416+0000 7f5a1f65a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141b4190 con 0x55a8d83025f0 2024-09-21T03:57:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.416+0000 7f5a1f65a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141b3ac0 con 0x55a8d82fa410 2024-09-21T03:57:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.416+0000 7f5a1f65a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141b0d00 con 0x55a8d83232d0 2024-09-21T03:57:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.416+0000 7f5a1f65a740 10 client.0 _ll_register_callbacks cb 0x7f5a00012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:57:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.416+0000 7f5a11400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83025f0 0x55a8d86dd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.416+0000 7f5a12800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d82fa410 0x55a8d86dccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.416+0000 7f5a11400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83025f0 0x55a8d86dd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35128/0 (socket says 172.21.15.16:35128) 2024-09-21T03:57:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.416+0000 7f5a11400640 1 -- 172.21.15.16:0/4022458924 learned_addr learned my addr 172.21.15.16:0/4022458924 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.416+0000 7f5a12800640 1 --2- 172.21.15.16:0/4022458924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d83232d0 0x55a8d86d7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.417+0000 7f5a11400640 1 -- 172.21.15.16:0/4022458924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d83232d0 msgr2=0x55a8d86d7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.417+0000 7f5a11400640 1 --2- 172.21.15.16:0/4022458924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d83232d0 0x55a8d86d7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.417+0000 7f5a11400640 1 -- 172.21.15.16:0/4022458924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d82fa410 msgr2=0x55a8d86dccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:57:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.417+0000 7f5a11400640 1 --2- 172.21.15.16:0/4022458924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d82fa410 0x55a8d86dccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.417+0000 7f5a11400640 1 -- 172.21.15.16:0/4022458924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8d86d8190 con 0x55a8d83025f0 2024-09-21T03:57:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.417+0000 7f5a11400640 1 --2- 172.21.15.16:0/4022458924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83025f0 0x55a8d86dd270 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5a000109f0 tx=0x7f5a000123c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d380aef9fcbfebb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.417+0000 7f59f3400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:57:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.417+0000 7f59f3400640 1 -- 172.21.15.16:0/4022458924 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a00022040 con 0x55a8d83025f0 2024-09-21T03:57:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.417+0000 7f59f3400640 1 -- 172.21.15.16:0/4022458924 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f5a000050e0 con 0x55a8d83025f0 2024-09-21T03:57:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.417+0000 7f5a1f65a740 1 -- 172.21.15.16:0/4022458924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a0c036070 con 0x55a8d83025f0 2024-09-21T03:57:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.418+0000 7f59f3400640 1 -- 172.21.15.16:0/4022458924 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5a00038050 con 0x55a8d83025f0 2024-09-21T03:57:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.418+0000 7f5a1f65a740 10 client.16524 fetch_fsmap learned FSMap version 30 2024-09-21T03:57:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.418+0000 7f5a1f65a740 1 -- 172.21.15.16:0/4022458924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a8d8751eb0 con 0x55a8d83025f0 2024-09-21T03:57:57.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.418+0000 7f59f3400640 1 -- 172.21.15.16:0/4022458924 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a0c036070 con 0x55a8d83025f0 2024-09-21T03:57:57.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.418+0000 7f59f3400640 1 -- 172.21.15.16:0/4022458924 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5a000038f0 con 0x55a8d83025f0 2024-09-21T03:57:57.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.418+0000 7f5a1f65a740 10 client.16524 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:57:57.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.418+0000 7f5a1f65a740 10 client.16524 resolve_mds: validated gid 24731 aka daemon mds.b 2024-09-21T03:57:57.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.418+0000 7f5a1f65a740 20 client.16524 populate_metadata read hostname 'smithi016' 2024-09-21T03:57:57.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.418+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/4022458924 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a8d82b7000 0x55a8d8754720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:57.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.418+0000 7f5a1f65a740 4 client.16524 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-21T03:57:57.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.418+0000 7f5a1f65a740 1 -- 172.21.15.16:0/4022458924 --> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55a8d863e9f0 con 0x55a8d82b7000 2024-09-21T03:57:57.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.419+0000 7f5a12800640 1 --2- 172.21.15.16:0/4022458924 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a8d82b7000 0x55a8d8754720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:57.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.419+0000 7f5a12800640 1 --2- 172.21.15.16:0/4022458924 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a8d82b7000 0x55a8d8754720 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38b57d6843a3772d server_cookie=92d2df74594f9753 in_seq=0 out_seq=0 2024-09-21T03:57:57.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.419+0000 7f59f3400640 10 client.16524 ms_handle_connect on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:57.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.420+0000 7f59f3400640 1 -- 172.21.15.16:0/4022458924 <== mds.0 v2:172.21.15.123:6838/459522816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55a8d863e9f0 con 0x55a8d82b7000 2024-09-21T03:57:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.420+0000 7f59f3400640 10 client.16524 handle_command_reply: tid=0 2024-09-21T03:57:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.420+0000 7f5a1f65a740 1 client.16524 shutdown 2024-09-21T03:57:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.420+0000 7f5a1f65a740 1 -- 172.21.15.16:0/4022458924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83025f0 msgr2=0x55a8d86dd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.420+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/4022458924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83025f0 0x55a8d86dd270 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5a000109f0 tx=0x7f5a000123c0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.420+0000 7f5a1f65a740 1 -- 172.21.15.16:0/4022458924 shutdown_connections 2024-09-21T03:57:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.420+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/4022458924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8d83232d0 0x55a8d86d7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.420+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/4022458924 >> [v2:172.21.15.123:6838/459522816,v1:172.21.15.123:6839/459522816] conn(0x55a8d82b7000 0x55a8d8754720 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.420+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/4022458924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8d83025f0 0x55a8d86dd270 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.420+0000 7f5a1f65a740 1 --2- 172.21.15.16:0/4022458924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8d82fa410 0x55a8d86dccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.420+0000 7f5a1f65a740 1 -- 172.21.15.16:0/4022458924 >> 172.21.15.16:0/4022458924 conn(0x55a8d83176c0 msgr2=0x55a8d86d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.420+0000 7f59f3400640 0 client.16524 ms_handle_reset on v2:172.21.15.123:6838/459522816 2024-09-21T03:57:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.420+0000 7f5a1f65a740 1 -- 172.21.15.16:0/4022458924 shutdown_connections 2024-09-21T03:57:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.420+0000 7f5a1f65a740 1 -- 172.21.15.16:0/4022458924 wait complete. 2024-09-21T03:57:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.420+0000 7f5a1f65a740 20 client.16524 trim_cache size 0 max 16384 2024-09-21T03:57:57.426 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:57:57.426 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-09-21T03:57:57.426 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24731, 2024-09-21T03:57:57.426 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 44, 2024-09-21T03:57:57.426 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": {} 2024-09-21T03:57:57.426 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:57:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.421+0000 7f5a18a00640 1 -- 172.21.15.16:0/229425412 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f59f80b6510 msgr2=0x7f59f80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.421+0000 7f5a18a00640 1 --2- 172.21.15.16:0/229425412 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f59f80b6510 0x7f59f80b8a20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5a040aaaa0 tx=0x7f5a04000bc0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.421+0000 7f5a18a00640 1 -- 172.21.15.16:0/229425412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a14103b80 msgr2=0x7f5a1419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.421+0000 7f5a18a00640 1 --2- 172.21.15.16:0/229425412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a14103b80 0x7f5a1419c670 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400e9f0 tx=0x7f5a0400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.421+0000 7f5a18a00640 1 -- 172.21.15.16:0/229425412 shutdown_connections 2024-09-21T03:57:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.421+0000 7f5a18a00640 1 --2- 172.21.15.16:0/229425412 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f59f80b6510 0x7f59f80b8a20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.421+0000 7f5a18a00640 1 --2- 172.21.15.16:0/229425412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a14109e10 0x7f5a141a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.421+0000 7f5a18a00640 1 --2- 172.21.15.16:0/229425412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a141094c0 0x7f5a1419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.421+0000 7f5a18a00640 1 --2- 172.21.15.16:0/229425412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a14103b80 0x7f5a1419c670 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.421+0000 7f5a18a00640 1 -- 172.21.15.16:0/229425412 >> 172.21.15.16:0/229425412 conn(0x7f5a140ff610 msgr2=0x7f5a141069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.421+0000 7f5a18a00640 1 -- 172.21.15.16:0/229425412 shutdown_connections 2024-09-21T03:57:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:57.421+0000 7f5a18a00640 1 -- 172.21.15.16:0/229425412 wait complete. 2024-09-21T03:57:57.445 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:57 smithi016 ceph-mon[30375]: pgmap v767: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 194 MiB/s wr, 1.63k op/s 2024-09-21T03:57:57.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:57 smithi134 ceph-mon[35031]: pgmap v767: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 194 MiB/s wr, 1.63k op/s 2024-09-21T03:57:57.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:57 smithi123 ceph-mon[34673]: pgmap v767: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 194 MiB/s wr, 1.63k op/s 2024-09-21T03:57:58.529 INFO:tasks.quiescer:done joining Quiescers 2024-09-21T03:57:58.529 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-21T03:57:58.538 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi016.front.sepia.ceph.com 2024-09-21T03:57:58.550 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T03:57:58.559 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:57:58.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.820+0000 7fef62800640 1 -- 172.21.15.16:0/3236296753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef5c078ca0 msgr2=0x7fef5c0790b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:58.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.820+0000 7fef62800640 1 --2- 172.21.15.16:0/3236296753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef5c078ca0 0x7fef5c0790b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fef48009900 tx=0x7fef4802a410 comp rx=0 tx=0).stop 2024-09-21T03:57:58.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.821+0000 7fef62800640 1 -- 172.21.15.16:0/3236296753 shutdown_connections 2024-09-21T03:57:58.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.821+0000 7fef62800640 1 --2- 172.21.15.16:0/3236296753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef5c079640 0x7fef5c079fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:58.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.821+0000 7fef62800640 1 --2- 172.21.15.16:0/3236296753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef5c078ca0 0x7fef5c0790b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:58.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.821+0000 7fef62800640 1 --2- 172.21.15.16:0/3236296753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef5c077c20 0x7fef5c078090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:58.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.821+0000 7fef62800640 1 -- 172.21.15.16:0/3236296753 >> 172.21.15.16:0/3236296753 conn(0x7fef5c101050 msgr2=0x7fef5c1034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:58.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.821+0000 7fef62800640 1 -- 172.21.15.16:0/3236296753 shutdown_connections 2024-09-21T03:57:58.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.821+0000 7fef62800640 1 -- 172.21.15.16:0/3236296753 wait complete. 2024-09-21T03:57:58.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.822+0000 7fef62800640 1 Processor -- start 2024-09-21T03:57:58.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.822+0000 7fef62800640 1 -- start start 2024-09-21T03:57:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.822+0000 7fef62800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef5c077c20 0x7fef5c1a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.822+0000 7fef62800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef5c078ca0 0x7fef5c1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.822+0000 7fef62800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef5c079640 0x7fef5c1a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.822+0000 7fef62800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c115790 con 0x7fef5c079640 2024-09-21T03:57:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.822+0000 7fef62800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c115610 con 0x7fef5c077c20 2024-09-21T03:57:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.822+0000 7fef62800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c115910 con 0x7fef5c078ca0 2024-09-21T03:57:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.822+0000 7fef61400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef5c077c20 0x7fef5c1a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.822+0000 7fef61400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef5c077c20 0x7fef5c1a0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:36328/0 (socket says 172.21.15.16:36328) 2024-09-21T03:57:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.822+0000 7fef61400640 1 -- 172.21.15.16:0/409805080 learned_addr learned my addr 172.21.15.16:0/409805080 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.822+0000 7fef61e00640 1 --2- 172.21.15.16:0/409805080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef5c079640 0x7fef5c1a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.822+0000 7fef60a00640 1 --2- 172.21.15.16:0/409805080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef5c078ca0 0x7fef5c1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.823+0000 7fef60a00640 1 -- 172.21.15.16:0/409805080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef5c077c20 msgr2=0x7fef5c1a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.823+0000 7fef60a00640 1 --2- 172.21.15.16:0/409805080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef5c077c20 0x7fef5c1a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.823+0000 7fef60a00640 1 -- 172.21.15.16:0/409805080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef5c079640 msgr2=0x7fef5c1a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.823+0000 7fef60a00640 1 --2- 172.21.15.16:0/409805080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef5c079640 0x7fef5c1a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.823+0000 7fef60a00640 1 -- 172.21.15.16:0/409805080 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5c1a1d70 con 0x7fef5c078ca0 2024-09-21T03:57:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.823+0000 7fef61400640 1 --2- 172.21.15.16:0/409805080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef5c077c20 0x7fef5c1a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:57:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.823+0000 7fef60a00640 1 --2- 172.21.15.16:0/409805080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef5c078ca0 0x7fef5c1a09a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fef4802aa00 tx=0x7fef48003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.824+0000 7fef5a000640 1 -- 172.21.15.16:0/409805080 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4803f070 con 0x7fef5c078ca0 2024-09-21T03:57:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.824+0000 7fef62800640 1 -- 172.21.15.16:0/409805080 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5c1a2000 con 0x7fef5c078ca0 2024-09-21T03:57:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.824+0000 7fef5a000640 1 -- 172.21.15.16:0/409805080 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef48031c50 con 0x7fef5c078ca0 2024-09-21T03:57:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.824+0000 7fef5a000640 1 -- 172.21.15.16:0/409805080 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48035030 con 0x7fef5c078ca0 2024-09-21T03:57:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.824+0000 7fef62800640 1 -- 172.21.15.16:0/409805080 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef5c1a8570 con 0x7fef5c078ca0 2024-09-21T03:57:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.825+0000 7fef62800640 1 -- 172.21.15.16:0/409805080 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef5c1121f0 con 0x7fef5c078ca0 2024-09-21T03:57:58.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.828+0000 7fef5a000640 1 -- 172.21.15.16:0/409805080 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fef48042070 con 0x7fef5c078ca0 2024-09-21T03:57:58.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.830+0000 7fef5a000640 1 --2- 172.21.15.16:0/409805080 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fef440b6510 0x7fef440b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.830+0000 7fef5a000640 1 -- 172.21.15.16:0/409805080 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fef481087b0 con 0x7fef5c078ca0 2024-09-21T03:57:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.830+0000 7fef61400640 1 --2- 172.21.15.16:0/409805080 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fef440b6510 0x7fef440b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.831+0000 7fef61400640 1 --2- 172.21.15.16:0/409805080 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fef440b6510 0x7fef440b8a20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fef50011550 tx=0x7fef50005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:58.831+0000 7fef5a000640 1 -- 172.21.15.16:0/409805080 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fef480d3010 con 0x7fef5c078ca0 2024-09-21T03:57:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.033+0000 7fef62800640 1 -- 172.21.15.16:0/409805080 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef5c063fc0 con 0x7fef5c078ca0 2024-09-21T03:57:59.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.035+0000 7fef5a000640 1 -- 172.21.15.16:0/409805080 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24902 (secure 0 0 0) 0x7fef480d7f50 con 0x7fef5c078ca0 2024-09-21T03:57:59.037 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:57:59.037 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:33:53.220358+0000","modified":"2024-09-21T03:44:54.658912+0000","last_up_change":"2024-09-21T03:42:06.614247+0000","last_in_change":"2024-09-21T03:41:43.379404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:37:55.708915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:43:03.567522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:43:06.048083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:43:16.061356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6a7e4b9e-078f-4ae9-ab9e-37be995e4deb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6803","nonce":311974551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6805","nonce":311974551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6809","nonce":311974551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6807","nonce":311974551}]},"public_addr":"172.21.15.16:6803/311974551","cluster_addr":"172.21.15.16:6805/311974551","heartbeat_back_addr":"172.21.15.16:6809/311974551","heartbeat_front_addr":"172.21.15.16:6807/311974551","state":["exists","up"]},{"osd":1,"uuid":"3a497a64-e1a8-43ef-8509-0a79515da0aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6801","nonce":2152486571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6803","nonce":2152486571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6807","nonce":2152486571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6805","nonce":2152486571}]},"public_addr":"172.21.15.123:6801/2152486571","cluster_addr":"172.21.15.123:6803/2152486571","heartbeat_back_addr":"172.21.15.123:6807/2152486571","heartbeat_front_addr":"172.21.15.123:6805/2152486571","state":["exists","up"]},{"osd":2,"uuid":"1c39518e-6cce-4a0f-9ee8-55c60ad325f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6801","nonce":2812351279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6803","nonce":2812351279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6807","nonce":2812351279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6805","nonce":2812351279}]},"public_addr":"172.21.15.134:6801/2812351279","cluster_addr":"172.21.15.134:6803/2812351279","heartbeat_back_addr":"172.21.15.134:6807/2812351279","heartbeat_front_addr":"172.21.15.134:6805/2812351279","state":["exists","up"]},{"osd":3,"uuid":"0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6811","nonce":2366216026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6813","nonce":2366216026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6817","nonce":2366216026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6815","nonce":2366216026}]},"public_addr":"172.21.15.16:6811/2366216026","cluster_addr":"172.21.15.16:6813/2366216026","heartbeat_back_addr":"172.21.15.16:6817/2366216026","heartbeat_front_addr":"172.21.15.16:6815/2366216026","state":["exists","up"]},{"osd":4,"uuid":"ad5e6d0e-31e8-4779-9e74-f215032777ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6809","nonce":2354279116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6811","nonce":2354279116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6815","nonce":2354279116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6813","nonce":2354279116}]},"public_addr":"172.21.15.123:6809/2354279116","cluster_addr":"172.21.15.123:6811/2354279116","heartbeat_back_addr":"172.21.15.123:6815/2354279116","heartbeat_front_addr":"172.21.15.123:6813/2354279116","state":["exists","up"]},{"osd":5,"uuid":"ca09bf88-1772-4dc0-8711-cbf9a5b31977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6809","nonce":2678416182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6811","nonce":2678416182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6815","nonce":2678416182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6813","nonce":2678416182}]},"public_addr":"172.21.15.134:6809/2678416182","cluster_addr":"172.21.15.134:6811/2678416182","heartbeat_back_addr":"172.21.15.134:6815/2678416182","heartbeat_front_addr":"172.21.15.134:6813/2678416182","state":["exists","up"]},{"osd":6,"uuid":"0d41ed4e-8699-4d88-9254-12d3b84941e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6819","nonce":3377807744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6821","nonce":3377807744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6825","nonce":3377807744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6823","nonce":3377807744}]},"public_addr":"172.21.15.16:6819/3377807744","cluster_addr":"172.21.15.16:6821/3377807744","heartbeat_back_addr":"172.21.15.16:6825/3377807744","heartbeat_front_addr":"172.21.15.16:6823/3377807744","state":["exists","up"]},{"osd":7,"uuid":"1541a272-c524-4f12-9b94-763f69bb1c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6817","nonce":1463412739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6819","nonce":1463412739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6823","nonce":1463412739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6821","nonce":1463412739}]},"public_addr":"172.21.15.123:6817/1463412739","cluster_addr":"172.21.15.123:6819/1463412739","heartbeat_back_addr":"172.21.15.123:6823/1463412739","heartbeat_front_addr":"172.21.15.123:6821/1463412739","state":["exists","up"]},{"osd":8,"uuid":"cc00014c-eb99-49ab-984c-0bc26ef85130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6817","nonce":2644021450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6819","nonce":2644021450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6823","nonce":2644021450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6821","nonce":2644021450}]},"public_addr":"172.21.15.134:6817/2644021450","cluster_addr":"172.21.15.134:6819/2644021450","heartbeat_back_addr":"172.21.15.134:6823/2644021450","heartbeat_front_addr":"172.21.15.134:6821/2644021450","state":["exists","up"]},{"osd":9,"uuid":"3cbfff74-31ff-4ffd-844b-54ec8563ba32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6827","nonce":1653164371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6829","nonce":1653164371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6833","nonce":1653164371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6831","nonce":1653164371}]},"public_addr":"172.21.15.16:6827/1653164371","cluster_addr":"172.21.15.16:6829/1653164371","heartbeat_back_addr":"172.21.15.16:6833/1653164371","heartbeat_front_addr":"172.21.15.16:6831/1653164371","state":["exists","up"]},{"osd":10,"uuid":"0a1caaad-70cd-4c2c-88d3-e9a5cc885655","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6825","nonce":3682234496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6827","nonce":3682234496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6831","nonce":3682234496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6829","nonce":3682234496}]},"public_addr":"172.21.15.123:6825/3682234496","cluster_addr":"172.21.15.123:6827/3682234496","heartbeat_back_addr":"172.21.15.123:6831/3682234496","heartbeat_front_addr":"172.21.15.123:6829/3682234496","state":["exists","up"]},{"osd":11,"uuid":"3aff3258-5a38-41ff-aac3-4a082feffdae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6825","nonce":584758685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6827","nonce":584758685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6831","nonce":584758685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6829","nonce":584758685}]},"public_addr":"172.21.15.134:6825/584758685","cluster_addr":"172.21.15.134:6827/584758685","heartbeat_back_addr":"172.21.15.134:6831/584758685","heartbeat_front_addr":"172.21.15.134:6829/584758685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:02.177731+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:28.847562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:54.272384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:24.338596+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:52.430371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:17.832737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:50.397890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:16.842245+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:42.102933+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:13.074732+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:40.057905+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:04.898190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/557742783":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/3238676548":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6801/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/2357336233":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6801/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6800/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/3806844703":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/2351581621":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6800/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/355731724":"2024-09-22T03:34:22.549156+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:57:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.037+0000 7fef62800640 1 -- 172.21.15.16:0/409805080 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fef440b6510 msgr2=0x7fef440b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.037+0000 7fef62800640 1 --2- 172.21.15.16:0/409805080 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fef440b6510 0x7fef440b8a20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fef50011550 tx=0x7fef50005f90 comp rx=0 tx=0).stop 2024-09-21T03:57:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.037+0000 7fef62800640 1 -- 172.21.15.16:0/409805080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef5c078ca0 msgr2=0x7fef5c1a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.037+0000 7fef62800640 1 --2- 172.21.15.16:0/409805080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef5c078ca0 0x7fef5c1a09a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fef4802aa00 tx=0x7fef48003c10 comp rx=0 tx=0).stop 2024-09-21T03:57:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.037+0000 7fef62800640 1 -- 172.21.15.16:0/409805080 shutdown_connections 2024-09-21T03:57:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.037+0000 7fef62800640 1 --2- 172.21.15.16:0/409805080 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fef440b6510 0x7fef440b8a20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.037+0000 7fef62800640 1 --2- 172.21.15.16:0/409805080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef5c079640 0x7fef5c1a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.037+0000 7fef62800640 1 --2- 172.21.15.16:0/409805080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef5c078ca0 0x7fef5c1a09a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.037+0000 7fef62800640 1 --2- 172.21.15.16:0/409805080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef5c077c20 0x7fef5c1a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.037+0000 7fef62800640 1 -- 172.21.15.16:0/409805080 >> 172.21.15.16:0/409805080 conn(0x7fef5c101050 msgr2=0x7fef5c103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.037+0000 7fef62800640 1 -- 172.21.15.16:0/409805080 shutdown_connections 2024-09-21T03:57:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.038+0000 7fef62800640 1 -- 172.21.15.16:0/409805080 wait complete. 2024-09-21T03:57:59.067 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:37:55.708915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T03:43:03.567522+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T03:43:06.048083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-21T03:43:16.061356+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-21T03:57:59.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:57:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.308+0000 7f9c77200640 1 -- 172.21.15.16:0/1336963269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c78106600 msgr2=0x7f9c78108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.308+0000 7f9c77200640 1 --2- 172.21.15.16:0/1336963269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c78106600 0x7f9c78108a80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009900 tx=0x7f9c6002a410 comp rx=0 tx=0).stop 2024-09-21T03:57:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.308+0000 7f9c77200640 1 -- 172.21.15.16:0/1336963269 shutdown_connections 2024-09-21T03:57:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.308+0000 7f9c77200640 1 --2- 172.21.15.16:0/1336963269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c781091c0 0x7f9c7810b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.308+0000 7f9c77200640 1 --2- 172.21.15.16:0/1336963269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c78106600 0x7f9c78108a80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.308+0000 7f9c77200640 1 --2- 172.21.15.16:0/1336963269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c7806c1f0 0x7f9c781060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.308+0000 7f9c77200640 1 -- 172.21.15.16:0/1336963269 >> 172.21.15.16:0/1336963269 conn(0x7f9c780fdcc0 msgr2=0x7f9c78100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.308+0000 7f9c77200640 1 -- 172.21.15.16:0/1336963269 shutdown_connections 2024-09-21T03:57:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.308+0000 7f9c77200640 1 -- 172.21.15.16:0/1336963269 wait complete. 2024-09-21T03:57:59.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.309+0000 7f9c77200640 1 Processor -- start 2024-09-21T03:57:59.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.310+0000 7f9c77200640 1 -- start start 2024-09-21T03:57:59.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.310+0000 7f9c77200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c7806c1f0 0x7f9c7819bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:59.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.310+0000 7f9c77200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c78106600 0x7f9c7819c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:59.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.310+0000 7f9c77200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c781091c0 0x7f9c7819d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.310+0000 7f9c77200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7810f0e0 con 0x7f9c7806c1f0 2024-09-21T03:57:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.310+0000 7f9c77200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7810ef60 con 0x7f9c78106600 2024-09-21T03:57:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.310+0000 7f9c77200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7810f260 con 0x7f9c781091c0 2024-09-21T03:57:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.310+0000 7f9c75e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c7806c1f0 0x7f9c7819bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.310+0000 7f9c75e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c7806c1f0 0x7f9c7819bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35180/0 (socket says 172.21.15.16:35180) 2024-09-21T03:57:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.310+0000 7f9c75e00640 1 -- 172.21.15.16:0/2751223749 learned_addr learned my addr 172.21.15.16:0/2751223749 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.310+0000 7f9c6d400640 1 --2- 172.21.15.16:0/2751223749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c78106600 0x7f9c7819c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.311+0000 7f9c76800640 1 --2- 172.21.15.16:0/2751223749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c781091c0 0x7f9c7819d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.311+0000 7f9c75e00640 1 -- 172.21.15.16:0/2751223749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c781091c0 msgr2=0x7f9c7819d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.311+0000 7f9c75e00640 1 --2- 172.21.15.16:0/2751223749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c781091c0 0x7f9c7819d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.311+0000 7f9c75e00640 1 -- 172.21.15.16:0/2751223749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c78106600 msgr2=0x7f9c7819c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.311+0000 7f9c75e00640 1 --2- 172.21.15.16:0/2751223749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c78106600 0x7f9c7819c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.311+0000 7f9c75e00640 1 -- 172.21.15.16:0/2751223749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7819d9a0 con 0x7f9c7806c1f0 2024-09-21T03:57:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.311+0000 7f9c76800640 1 --2- 172.21.15.16:0/2751223749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c781091c0 0x7f9c7819d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:57:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.311+0000 7f9c75e00640 1 --2- 172.21.15.16:0/2751223749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c7806c1f0 0x7f9c7819bf60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800e9f0 tx=0x7f9c6800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.311+0000 7f9c6f400640 1 -- 172.21.15.16:0/2751223749 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6800cd80 con 0x7f9c7806c1f0 2024-09-21T03:57:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.311+0000 7f9c6f400640 1 -- 172.21.15.16:0/2751223749 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c680054d0 con 0x7f9c7806c1f0 2024-09-21T03:57:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.311+0000 7f9c77200640 1 -- 172.21.15.16:0/2751223749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7819dc90 con 0x7f9c7806c1f0 2024-09-21T03:57:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.311+0000 7f9c6f400640 1 -- 172.21.15.16:0/2751223749 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68010690 con 0x7f9c7806c1f0 2024-09-21T03:57:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.312+0000 7f9c77200640 1 -- 172.21.15.16:0/2751223749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c781a4380 con 0x7f9c7806c1f0 2024-09-21T03:57:59.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.313+0000 7f9c77200640 1 -- 172.21.15.16:0/2751223749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c78106a70 con 0x7f9c7806c1f0 2024-09-21T03:57:59.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.315+0000 7f9c6f400640 1 -- 172.21.15.16:0/2751223749 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f9c68004e30 con 0x7f9c7806c1f0 2024-09-21T03:57:59.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.317+0000 7f9c6f400640 1 --2- 172.21.15.16:0/2751223749 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9c5c0b6510 0x7f9c5c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:59.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.317+0000 7f9c6f400640 1 -- 172.21.15.16:0/2751223749 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f9c680e7db0 con 0x7f9c7806c1f0 2024-09-21T03:57:59.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.317+0000 7f9c6d400640 1 --2- 172.21.15.16:0/2751223749 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9c5c0b6510 0x7f9c5c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:59.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.318+0000 7f9c6d400640 1 --2- 172.21.15.16:0/2751223749 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9c5c0b6510 0x7f9c5c0b8a20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009230 tx=0x7f9c60031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:59.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.322+0000 7f9c6f400640 1 -- 172.21.15.16:0/2751223749 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9c68056710 con 0x7f9c7806c1f0 2024-09-21T03:57:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:59 smithi016 ceph-mon[30375]: pgmap v768: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 145 MiB/s wr, 1.26k op/s 2024-09-21T03:57:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:57:59 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/409805080' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:57:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.519+0000 7f9c77200640 1 -- 172.21.15.16:0/2751223749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9c781a4550 con 0x7f9c7806c1f0 2024-09-21T03:57:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.531+0000 7f9c6f400640 1 -- 172.21.15.16:0/2751223749 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7f9c680ae230 con 0x7f9c7806c1f0 2024-09-21T03:57:59.533 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-21T03:57:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.533+0000 7f9c77200640 1 -- 172.21.15.16:0/2751223749 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9c5c0b6510 msgr2=0x7f9c5c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.533+0000 7f9c77200640 1 --2- 172.21.15.16:0/2751223749 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9c5c0b6510 0x7f9c5c0b8a20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009230 tx=0x7f9c60031400 comp rx=0 tx=0).stop 2024-09-21T03:57:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.533+0000 7f9c77200640 1 -- 172.21.15.16:0/2751223749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c7806c1f0 msgr2=0x7f9c7819bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.533+0000 7f9c77200640 1 --2- 172.21.15.16:0/2751223749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c7806c1f0 0x7f9c7819bf60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800e9f0 tx=0x7f9c6800edf0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.533+0000 7f9c77200640 1 -- 172.21.15.16:0/2751223749 shutdown_connections 2024-09-21T03:57:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.533+0000 7f9c77200640 1 --2- 172.21.15.16:0/2751223749 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f9c5c0b6510 0x7f9c5c0b8a20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.533+0000 7f9c77200640 1 --2- 172.21.15.16:0/2751223749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c781091c0 0x7f9c7819d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.533+0000 7f9c77200640 1 --2- 172.21.15.16:0/2751223749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c78106600 0x7f9c7819c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.534+0000 7f9c77200640 1 --2- 172.21.15.16:0/2751223749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c7806c1f0 0x7f9c7819bf60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.534+0000 7f9c77200640 1 -- 172.21.15.16:0/2751223749 >> 172.21.15.16:0/2751223749 conn(0x7f9c780fdcc0 msgr2=0x7f9c781000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.534+0000 7f9c77200640 1 -- 172.21.15.16:0/2751223749 shutdown_connections 2024-09-21T03:57:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.534+0000 7f9c77200640 1 -- 172.21.15.16:0/2751223749 wait complete. 2024-09-21T03:57:59.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T03:57:59.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:59 smithi134 ceph-mon[35031]: pgmap v768: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 145 MiB/s wr, 1.26k op/s 2024-09-21T03:57:59.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:57:59 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/409805080' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:57:59.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:59 smithi123 ceph-mon[34673]: pgmap v768: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 145 MiB/s wr, 1.26k op/s 2024-09-21T03:57:59.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:57:59 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/409805080' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:57:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.788+0000 7fbdfaa00640 1 -- 172.21.15.16:0/1535892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf41093e0 msgr2=0x7fbdf410b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.788+0000 7fbdfaa00640 1 --2- 172.21.15.16:0/1535892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf41093e0 0x7fbdf410b8f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbdec009ca0 tx=0x7fbdec02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.788+0000 7fbdfaa00640 1 -- 172.21.15.16:0/1535892 shutdown_connections 2024-09-21T03:57:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.788+0000 7fbdfaa00640 1 --2- 172.21.15.16:0/1535892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf41093e0 0x7fbdf410b8f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.788+0000 7fbdfaa00640 1 --2- 172.21.15.16:0/1535892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdf4106880 0x7fbdf4108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.788+0000 7fbdfaa00640 1 --2- 172.21.15.16:0/1535892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdf406c370 0x7fbdf4106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.788+0000 7fbdfaa00640 1 -- 172.21.15.16:0/1535892 >> 172.21.15.16:0/1535892 conn(0x7fbdf40fdee0 msgr2=0x7fbdf4100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.788+0000 7fbdfaa00640 1 -- 172.21.15.16:0/1535892 shutdown_connections 2024-09-21T03:57:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.788+0000 7fbdfaa00640 1 -- 172.21.15.16:0/1535892 wait complete. 2024-09-21T03:57:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.789+0000 7fbdfaa00640 1 Processor -- start 2024-09-21T03:57:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.789+0000 7fbdfaa00640 1 -- start start 2024-09-21T03:57:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.790+0000 7fbdfaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf406c370 0x7fbdf419c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.790+0000 7fbdfaa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdf4106880 0x7fbdf419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.790+0000 7fbdfaa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdf41093e0 0x7fbdf419d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.790+0000 7fbdfaa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf410f370 con 0x7fbdf406c370 2024-09-21T03:57:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.790+0000 7fbdfaa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf410f1f0 con 0x7fbdf41093e0 2024-09-21T03:57:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.790+0000 7fbdfaa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf410f4f0 con 0x7fbdf4106880 2024-09-21T03:57:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.790+0000 7fbdf3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf406c370 0x7fbdf419c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.790+0000 7fbdf3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf406c370 0x7fbdf419c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35198/0 (socket says 172.21.15.16:35198) 2024-09-21T03:57:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.790+0000 7fbdf3e00640 1 -- 172.21.15.16:0/1365552298 learned_addr learned my addr 172.21.15.16:0/1365552298 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:57:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.790+0000 7fbdebe00640 1 --2- 172.21.15.16:0/1365552298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdf4106880 0x7fbdf419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.790+0000 7fbdf8a00640 1 --2- 172.21.15.16:0/1365552298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdf41093e0 0x7fbdf419d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.790+0000 7fbdf3e00640 1 -- 172.21.15.16:0/1365552298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdf4106880 msgr2=0x7fbdf419c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.790+0000 7fbdf3e00640 1 --2- 172.21.15.16:0/1365552298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdf4106880 0x7fbdf419c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.790+0000 7fbdf3e00640 1 -- 172.21.15.16:0/1365552298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdf41093e0 msgr2=0x7fbdf419d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.790+0000 7fbdf3e00640 1 --2- 172.21.15.16:0/1365552298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdf41093e0 0x7fbdf419d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.790+0000 7fbdf3e00640 1 -- 172.21.15.16:0/1365552298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf419dbe0 con 0x7fbdf406c370 2024-09-21T03:57:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.791+0000 7fbdf3e00640 1 --2- 172.21.15.16:0/1365552298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf406c370 0x7fbdf419c1a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbde400e9f0 tx=0x7fbde400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.791+0000 7fbdf1600640 1 -- 172.21.15.16:0/1365552298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde400cd80 con 0x7fbdf406c370 2024-09-21T03:57:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.791+0000 7fbdfaa00640 1 -- 172.21.15.16:0/1365552298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf419ded0 con 0x7fbdf406c370 2024-09-21T03:57:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.791+0000 7fbdf1600640 1 -- 172.21.15.16:0/1365552298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbde40054d0 con 0x7fbdf406c370 2024-09-21T03:57:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.791+0000 7fbdf1600640 1 -- 172.21.15.16:0/1365552298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4018630 con 0x7fbdf406c370 2024-09-21T03:57:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.791+0000 7fbdfaa00640 1 -- 172.21.15.16:0/1365552298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf41a45c0 con 0x7fbdf406c370 2024-09-21T03:57:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.792+0000 7fbdfaa00640 1 -- 172.21.15.16:0/1365552298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf4106c90 con 0x7fbdf406c370 2024-09-21T03:57:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.795+0000 7fbdf1600640 1 -- 172.21.15.16:0/1365552298 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fbde4004e70 con 0x7fbdf406c370 2024-09-21T03:57:59.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.796+0000 7fbdf1600640 1 --2- 172.21.15.16:0/1365552298 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbdd80b6510 0x7fbdd80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:59.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.796+0000 7fbdf1600640 1 -- 172.21.15.16:0/1365552298 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fbde40e82c0 con 0x7fbdf406c370 2024-09-21T03:57:59.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.797+0000 7fbdebe00640 1 --2- 172.21.15.16:0/1365552298 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbdd80b6510 0x7fbdd80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:59.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.797+0000 7fbdebe00640 1 --2- 172.21.15.16:0/1365552298 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbdd80b6510 0x7fbdd80b8a20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0095f0 tx=0x7fbddc005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:59.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.798+0000 7fbdf1600640 1 -- 172.21.15.16:0/1365552298 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbde40ae740 con 0x7fbdf406c370 2024-09-21T03:58:00.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.999+0000 7fbdfaa00640 1 -- 172.21.15.16:0/1365552298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbdf41a4790 con 0x7fbdf406c370 2024-09-21T03:58:00.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:57:59.999+0000 7fbdf1600640 1 -- 172.21.15.16:0/1365552298 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) ==== 104+0+11 (secure 0 0 0) 0x7fbde40b2b20 con 0x7fbdf406c370 2024-09-21T03:58:00.001 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-21T03:58:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.002+0000 7fbdfaa00640 1 -- 172.21.15.16:0/1365552298 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbdd80b6510 msgr2=0x7fbdd80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.002+0000 7fbdfaa00640 1 --2- 172.21.15.16:0/1365552298 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbdd80b6510 0x7fbdd80b8a20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0095f0 tx=0x7fbddc005f90 comp rx=0 tx=0).stop 2024-09-21T03:58:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.002+0000 7fbdfaa00640 1 -- 172.21.15.16:0/1365552298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf406c370 msgr2=0x7fbdf419c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.002+0000 7fbdfaa00640 1 --2- 172.21.15.16:0/1365552298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf406c370 0x7fbdf419c1a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbde400e9f0 tx=0x7fbde400edf0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.002+0000 7fbdfaa00640 1 -- 172.21.15.16:0/1365552298 shutdown_connections 2024-09-21T03:58:00.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.002+0000 7fbdfaa00640 1 --2- 172.21.15.16:0/1365552298 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fbdd80b6510 0x7fbdd80b8a20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.002+0000 7fbdfaa00640 1 --2- 172.21.15.16:0/1365552298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdf41093e0 0x7fbdf419d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.002+0000 7fbdfaa00640 1 --2- 172.21.15.16:0/1365552298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdf4106880 0x7fbdf419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.002+0000 7fbdfaa00640 1 --2- 172.21.15.16:0/1365552298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdf406c370 0x7fbdf419c1a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.002+0000 7fbdfaa00640 1 -- 172.21.15.16:0/1365552298 >> 172.21.15.16:0/1365552298 conn(0x7fbdf40fdee0 msgr2=0x7fbdf4100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:00.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.002+0000 7fbdfaa00640 1 -- 172.21.15.16:0/1365552298 shutdown_connections 2024-09-21T03:58:00.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.002+0000 7fbdfaa00640 1 -- 172.21.15.16:0/1365552298 wait complete. 2024-09-21T03:58:00.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T03:58:00.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.261+0000 7f2504800640 1 -- 172.21.15.16:0/2416304642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc104700 msgr2=0x7f24fc104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:00.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.261+0000 7f2504800640 1 --2- 172.21.15.16:0/2416304642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc104700 0x7f24fc104b10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009900 tx=0x7f24ec02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.261+0000 7f2504800640 1 -- 172.21.15.16:0/2416304642 shutdown_connections 2024-09-21T03:58:00.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.261+0000 7f2504800640 1 --2- 172.21.15.16:0/2416304642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24fc1050a0 0x7f24fc10ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.261+0000 7f2504800640 1 --2- 172.21.15.16:0/2416304642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc104700 0x7f24fc104b10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.261+0000 7f2504800640 1 --2- 172.21.15.16:0/2416304642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24fc103680 0x7f24fc103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.261+0000 7f2504800640 1 -- 172.21.15.16:0/2416304642 >> 172.21.15.16:0/2416304642 conn(0x7f24fc0fefe0 msgr2=0x7f24fc101470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:00.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.261+0000 7f2504800640 1 -- 172.21.15.16:0/2416304642 shutdown_connections 2024-09-21T03:58:00.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.261+0000 7f2504800640 1 -- 172.21.15.16:0/2416304642 wait complete. 2024-09-21T03:58:00.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.263+0000 7f2504800640 1 Processor -- start 2024-09-21T03:58:00.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.263+0000 7f2504800640 1 -- start start 2024-09-21T03:58:00.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.263+0000 7f2504800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24fc103680 0x7f24fc076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:00.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.263+0000 7f2504800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc104700 0x7f24fc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:00.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.264+0000 7f2504800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24fc1050a0 0x7f24fc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:00.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.264+0000 7f2504800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc10f2b0 con 0x7f24fc104700 2024-09-21T03:58:00.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.264+0000 7f2504800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc10f130 con 0x7f24fc103680 2024-09-21T03:58:00.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.264+0000 7f2504800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc10f430 con 0x7f24fc1050a0 2024-09-21T03:58:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.264+0000 7f2501400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc104700 0x7f24fc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.264+0000 7f2501e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24fc103680 0x7f24fc076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.264+0000 7f2501400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc104700 0x7f24fc075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35210/0 (socket says 172.21.15.16:35210) 2024-09-21T03:58:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.264+0000 7f2501400640 1 -- 172.21.15.16:0/441704293 learned_addr learned my addr 172.21.15.16:0/441704293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:58:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.264+0000 7f2502800640 1 --2- 172.21.15.16:0/441704293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24fc1050a0 0x7f24fc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.264+0000 7f2501400640 1 -- 172.21.15.16:0/441704293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24fc1050a0 msgr2=0x7f24fc075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.264+0000 7f2501400640 1 --2- 172.21.15.16:0/441704293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24fc1050a0 0x7f24fc075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.264+0000 7f2501400640 1 -- 172.21.15.16:0/441704293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24fc103680 msgr2=0x7f24fc076e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.264+0000 7f2501400640 1 --2- 172.21.15.16:0/441704293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24fc103680 0x7f24fc076e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.264+0000 7f2501400640 1 -- 172.21.15.16:0/441704293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24fc076380 con 0x7f24fc104700 2024-09-21T03:58:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.265+0000 7f2501400640 1 --2- 172.21.15.16:0/441704293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc104700 0x7f24fc075710 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f24ec006010 tx=0x7f24ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.265+0000 7f24f6a00640 1 -- 172.21.15.16:0/441704293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec03f030 con 0x7f24fc104700 2024-09-21T03:58:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.265+0000 7f24f6a00640 1 -- 172.21.15.16:0/441704293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24ec032e50 con 0x7f24fc104700 2024-09-21T03:58:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.265+0000 7f2504800640 1 -- 172.21.15.16:0/441704293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24fc1a3f60 con 0x7f24fc104700 2024-09-21T03:58:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.265+0000 7f24f6a00640 1 -- 172.21.15.16:0/441704293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec035070 con 0x7f24fc104700 2024-09-21T03:58:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.265+0000 7f2504800640 1 -- 172.21.15.16:0/441704293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24fc1a4310 con 0x7f24fc104700 2024-09-21T03:58:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.267+0000 7f2504800640 1 -- 172.21.15.16:0/441704293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24fc10a850 con 0x7f24fc104700 2024-09-21T03:58:00.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.269+0000 7f24f6a00640 1 -- 172.21.15.16:0/441704293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f24ec042030 con 0x7f24fc104700 2024-09-21T03:58:00.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.271+0000 7f24f6a00640 1 --2- 172.21.15.16:0/441704293 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24e40b6590 0x7f24e40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:00.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.272+0000 7f24f6a00640 1 -- 172.21.15.16:0/441704293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f24ec107bd0 con 0x7f24fc104700 2024-09-21T03:58:00.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.272+0000 7f2501e00640 1 --2- 172.21.15.16:0/441704293 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24e40b6590 0x7f24e40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:00.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.272+0000 7f2501e00640 1 --2- 172.21.15.16:0/441704293 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24e40b6590 0x7f24e40b8aa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f24f00095f0 tx=0x7f24f0005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:00.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.276+0000 7f24f6a00640 1 -- 172.21.15.16:0/441704293 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f24ec0d2430 con 0x7f24fc104700 2024-09-21T03:58:00.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.480+0000 7f2504800640 1 -- 172.21.15.16:0/441704293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f24fc063fc0 con 0x7f24fc104700 2024-09-21T03:58:00.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.481+0000 7f24f6a00640 1 -- 172.21.15.16:0/441704293 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) ==== 100+0+11 (secure 0 0 0) 0x7f24ec0d7370 con 0x7f24fc104700 2024-09-21T03:58:00.482 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-21T03:58:00.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.483+0000 7f2504800640 1 -- 172.21.15.16:0/441704293 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24e40b6590 msgr2=0x7f24e40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:00.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.483+0000 7f2504800640 1 --2- 172.21.15.16:0/441704293 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24e40b6590 0x7f24e40b8aa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f24f00095f0 tx=0x7f24f0005f90 comp rx=0 tx=0).stop 2024-09-21T03:58:00.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.483+0000 7f2504800640 1 -- 172.21.15.16:0/441704293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc104700 msgr2=0x7f24fc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:00.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.483+0000 7f2504800640 1 --2- 172.21.15.16:0/441704293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc104700 0x7f24fc075710 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f24ec006010 tx=0x7f24ec0042f0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.483+0000 7f2504800640 1 -- 172.21.15.16:0/441704293 shutdown_connections 2024-09-21T03:58:00.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.483+0000 7f2504800640 1 --2- 172.21.15.16:0/441704293 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f24e40b6590 0x7f24e40b8aa0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.483+0000 7f2504800640 1 --2- 172.21.15.16:0/441704293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24fc1050a0 0x7f24fc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.483+0000 7f2504800640 1 --2- 172.21.15.16:0/441704293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc104700 0x7f24fc075710 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.483+0000 7f2504800640 1 --2- 172.21.15.16:0/441704293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24fc103680 0x7f24fc076e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.483+0000 7f2504800640 1 -- 172.21.15.16:0/441704293 >> 172.21.15.16:0/441704293 conn(0x7f24fc0fefe0 msgr2=0x7f24fc107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:00.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.483+0000 7f2504800640 1 -- 172.21.15.16:0/441704293 shutdown_connections 2024-09-21T03:58:00.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.483+0000 7f2504800640 1 -- 172.21.15.16:0/441704293 wait complete. 2024-09-21T03:58:00.507 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:00 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2751223749' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:58:00.507 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:00 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1365552298' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:58:00.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-21T03:58:00.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:00 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2751223749' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:58:00.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:00 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1365552298' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:58:00.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:00 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2751223749' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:58:00.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:00 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1365552298' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:58:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.738+0000 7f0e75e00640 1 -- 172.21.15.16:0/321126384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e70104540 msgr2=0x7f0e70104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.738+0000 7f0e75e00640 1 --2- 172.21.15.16:0/321126384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e70104540 0x7f0e70104950 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009900 tx=0x7f0e5c02a410 comp rx=0 tx=0).stop 2024-09-21T03:58:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.739+0000 7f0e75e00640 1 -- 172.21.15.16:0/321126384 shutdown_connections 2024-09-21T03:58:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.739+0000 7f0e75e00640 1 --2- 172.21.15.16:0/321126384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e70104ee0 0x7f0e7010b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.739+0000 7f0e75e00640 1 --2- 172.21.15.16:0/321126384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e70104540 0x7f0e70104950 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.739+0000 7f0e75e00640 1 --2- 172.21.15.16:0/321126384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e701034c0 0x7f0e70103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.739+0000 7f0e75e00640 1 -- 172.21.15.16:0/321126384 >> 172.21.15.16:0/321126384 conn(0x7f0e700fee00 msgr2=0x7f0e701012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.739+0000 7f0e75e00640 1 -- 172.21.15.16:0/321126384 shutdown_connections 2024-09-21T03:58:00.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.739+0000 7f0e75e00640 1 -- 172.21.15.16:0/321126384 wait complete. 2024-09-21T03:58:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.740+0000 7f0e75e00640 1 Processor -- start 2024-09-21T03:58:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.740+0000 7f0e75e00640 1 -- start start 2024-09-21T03:58:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.740+0000 7f0e75e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e701034c0 0x7f0e70105bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.740+0000 7f0e75e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e70104540 0x7f0e70106260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.740+0000 7f0e75e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e70104ee0 0x7f0e70106e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.740+0000 7f0e75e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7010f160 con 0x7f0e70104ee0 2024-09-21T03:58:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.740+0000 7f0e75e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7010efe0 con 0x7f0e70104540 2024-09-21T03:58:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.740+0000 7f0e75e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7010f2e0 con 0x7f0e701034c0 2024-09-21T03:58:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.740+0000 7f0e75400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e70104ee0 0x7f0e70106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.740+0000 7f0e75400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e70104ee0 0x7f0e70106e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35228/0 (socket says 172.21.15.16:35228) 2024-09-21T03:58:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.740+0000 7f0e75400640 1 -- 172.21.15.16:0/626190644 learned_addr learned my addr 172.21.15.16:0/626190644 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:58:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.740+0000 7f0e6fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e70104540 0x7f0e70106260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.740+0000 7f0e74a00640 1 --2- 172.21.15.16:0/626190644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e701034c0 0x7f0e70105bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.741+0000 7f0e75400640 1 -- 172.21.15.16:0/626190644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e701034c0 msgr2=0x7f0e70105bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.741+0000 7f0e75400640 1 --2- 172.21.15.16:0/626190644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e701034c0 0x7f0e70105bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.741+0000 7f0e75400640 1 -- 172.21.15.16:0/626190644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e70104540 msgr2=0x7f0e70106260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.741+0000 7f0e75400640 1 --2- 172.21.15.16:0/626190644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e70104540 0x7f0e70106260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.741+0000 7f0e75400640 1 -- 172.21.15.16:0/626190644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e70107630 con 0x7f0e70104ee0 2024-09-21T03:58:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.741+0000 7f0e75400640 1 --2- 172.21.15.16:0/626190644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e70104ee0 0x7f0e70106e10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800eb10 tx=0x7f0e6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:00.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.742+0000 7f0e6d600640 1 -- 172.21.15.16:0/626190644 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68004b20 con 0x7f0e70104ee0 2024-09-21T03:58:00.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.742+0000 7f0e6d600640 1 -- 172.21.15.16:0/626190644 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e68004cc0 con 0x7f0e70104ee0 2024-09-21T03:58:00.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.742+0000 7f0e75e00640 1 -- 172.21.15.16:0/626190644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e70107920 con 0x7f0e70104ee0 2024-09-21T03:58:00.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.742+0000 7f0e6d600640 1 -- 172.21.15.16:0/626190644 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68010670 con 0x7f0e70104ee0 2024-09-21T03:58:00.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.742+0000 7f0e75e00640 1 -- 172.21.15.16:0/626190644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e70107fe0 con 0x7f0e70104ee0 2024-09-21T03:58:00.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.743+0000 7f0e75e00640 1 -- 172.21.15.16:0/626190644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e70103930 con 0x7f0e70104ee0 2024-09-21T03:58:00.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.747+0000 7f0e6d600640 1 -- 172.21.15.16:0/626190644 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0e68005000 con 0x7f0e70104ee0 2024-09-21T03:58:00.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.749+0000 7f0e6d600640 1 --2- 172.21.15.16:0/626190644 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0e580b6510 0x7f0e580b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:00.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.749+0000 7f0e6d600640 1 -- 172.21.15.16:0/626190644 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f0e680e7e50 con 0x7f0e70104ee0 2024-09-21T03:58:00.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.749+0000 7f0e74a00640 1 --2- 172.21.15.16:0/626190644 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0e580b6510 0x7f0e580b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:00.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.749+0000 7f0e6d600640 1 -- 172.21.15.16:0/626190644 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0e680ae2d0 con 0x7f0e70104ee0 2024-09-21T03:58:00.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.750+0000 7f0e74a00640 1 --2- 172.21.15.16:0/626190644 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0e580b6510 0x7f0e580b8a20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009bd0 tx=0x7f0e64005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.945+0000 7f0e75e00640 1 -- 172.21.15.16:0/626190644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0e70108430 con 0x7f0e70104ee0 2024-09-21T03:58:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.946+0000 7f0e6d600640 1 -- 172.21.15.16:0/626190644 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) ==== 103+0+11 (secure 0 0 0) 0x7f0e680b26b0 con 0x7f0e70104ee0 2024-09-21T03:58:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-21T03:58:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.949+0000 7f0e75e00640 1 -- 172.21.15.16:0/626190644 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0e580b6510 msgr2=0x7f0e580b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.949+0000 7f0e75e00640 1 --2- 172.21.15.16:0/626190644 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0e580b6510 0x7f0e580b8a20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009bd0 tx=0x7f0e64005f90 comp rx=0 tx=0).stop 2024-09-21T03:58:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.949+0000 7f0e75e00640 1 -- 172.21.15.16:0/626190644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e70104ee0 msgr2=0x7f0e70106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.949+0000 7f0e75e00640 1 --2- 172.21.15.16:0/626190644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e70104ee0 0x7f0e70106e10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800eb10 tx=0x7f0e6800c2d0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.949+0000 7f0e75e00640 1 -- 172.21.15.16:0/626190644 shutdown_connections 2024-09-21T03:58:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.949+0000 7f0e75e00640 1 --2- 172.21.15.16:0/626190644 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0e580b6510 0x7f0e580b8a20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.949+0000 7f0e75e00640 1 --2- 172.21.15.16:0/626190644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e70104ee0 0x7f0e70106e10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.949+0000 7f0e75e00640 1 --2- 172.21.15.16:0/626190644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e70104540 0x7f0e70106260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.949+0000 7f0e75e00640 1 --2- 172.21.15.16:0/626190644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e701034c0 0x7f0e70105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.949+0000 7f0e75e00640 1 -- 172.21.15.16:0/626190644 >> 172.21.15.16:0/626190644 conn(0x7f0e700fee00 msgr2=0x7f0e700755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.949+0000 7f0e75e00640 1 -- 172.21.15.16:0/626190644 shutdown_connections 2024-09-21T03:58:00.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:00.949+0000 7f0e75e00640 1 -- 172.21.15.16:0/626190644 wait complete. 2024-09-21T03:58:00.974 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T03:58:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.200+0000 7f64c5200640 1 -- 172.21.15.16:0/774341681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c0106750 msgr2=0x7f64c0106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.200+0000 7f64c5200640 1 --2- 172.21.15.16:0/774341681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c0106750 0x7f64c0106b60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009900 tx=0x7f64ac02a410 comp rx=0 tx=0).stop 2024-09-21T03:58:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.200+0000 7f64c5200640 1 -- 172.21.15.16:0/774341681 shutdown_connections 2024-09-21T03:58:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.200+0000 7f64c5200640 1 --2- 172.21.15.16:0/774341681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64c01070f0 0x7f64c010da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.200+0000 7f64c5200640 1 --2- 172.21.15.16:0/774341681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c0106750 0x7f64c0106b60 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.200+0000 7f64c5200640 1 --2- 172.21.15.16:0/774341681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64c01056d0 0x7f64c0105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.200+0000 7f64c5200640 1 -- 172.21.15.16:0/774341681 >> 172.21.15.16:0/774341681 conn(0x7f64c0101050 msgr2=0x7f64c01034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.201+0000 7f64c5200640 1 -- 172.21.15.16:0/774341681 shutdown_connections 2024-09-21T03:58:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.201+0000 7f64c5200640 1 -- 172.21.15.16:0/774341681 wait complete. 2024-09-21T03:58:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.201+0000 7f64c5200640 1 Processor -- start 2024-09-21T03:58:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.202+0000 7f64c5200640 1 -- start start 2024-09-21T03:58:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.202+0000 7f64c5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64c01056d0 0x7f64c019bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.202+0000 7f64c5200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64c0106750 0x7f64c019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.202+0000 7f64c5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c01070f0 0x7f64c019d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.202+0000 7f64c5200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64c0111320 con 0x7f64c01070f0 2024-09-21T03:58:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.202+0000 7f64c5200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64c01111a0 con 0x7f64c01056d0 2024-09-21T03:58:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.202+0000 7f64c5200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64c01114a0 con 0x7f64c0106750 2024-09-21T03:58:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.202+0000 7f64bea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64c0106750 0x7f64c019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.202+0000 7f64bfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c01070f0 0x7f64c019d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.202+0000 7f64bf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64c01056d0 0x7f64c019bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.202+0000 7f64bea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64c0106750 0x7f64c019c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:60664/0 (socket says 172.21.15.16:60664) 2024-09-21T03:58:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.202+0000 7f64bea00640 1 -- 172.21.15.16:0/2153629282 learned_addr learned my addr 172.21.15.16:0/2153629282 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:58:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.203+0000 7f64bea00640 1 -- 172.21.15.16:0/2153629282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64c01056d0 msgr2=0x7f64c019bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.203+0000 7f64bea00640 1 --2- 172.21.15.16:0/2153629282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64c01056d0 0x7f64c019bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.203+0000 7f64bea00640 1 -- 172.21.15.16:0/2153629282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c01070f0 msgr2=0x7f64c019d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.203+0000 7f64bea00640 1 --2- 172.21.15.16:0/2153629282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c01070f0 0x7f64c019d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.203+0000 7f64bea00640 1 -- 172.21.15.16:0/2153629282 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c019d950 con 0x7f64c0106750 2024-09-21T03:58:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.203+0000 7f64bea00640 1 --2- 172.21.15.16:0/2153629282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64c0106750 0x7f64c019c580 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f64ac006010 tx=0x7f64ac0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.203+0000 7f64b7e00640 1 -- 172.21.15.16:0/2153629282 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac03f030 con 0x7f64c0106750 2024-09-21T03:58:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.203+0000 7f64c5200640 1 -- 172.21.15.16:0/2153629282 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c019dbe0 con 0x7f64c0106750 2024-09-21T03:58:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.203+0000 7f64b7e00640 1 -- 172.21.15.16:0/2153629282 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64ac032e50 con 0x7f64c0106750 2024-09-21T03:58:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.203+0000 7f64b7e00640 1 -- 172.21.15.16:0/2153629282 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac035070 con 0x7f64c0106750 2024-09-21T03:58:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.204+0000 7f64c5200640 1 -- 172.21.15.16:0/2153629282 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c01a4330 con 0x7f64c0106750 2024-09-21T03:58:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.204+0000 7f64c5200640 1 -- 172.21.15.16:0/2153629282 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c010c880 con 0x7f64c0106750 2024-09-21T03:58:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.207+0000 7f64b7e00640 1 -- 172.21.15.16:0/2153629282 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f64ac042030 con 0x7f64c0106750 2024-09-21T03:58:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.209+0000 7f64b7e00640 1 --2- 172.21.15.16:0/2153629282 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64a40b6510 0x7f64a40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.209+0000 7f64b7e00640 1 -- 172.21.15.16:0/2153629282 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f64ac1082c0 con 0x7f64c0106750 2024-09-21T03:58:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.209+0000 7f64bf400640 1 --2- 172.21.15.16:0/2153629282 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64a40b6510 0x7f64a40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.210+0000 7f64bf400640 1 --2- 172.21.15.16:0/2153629282 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64a40b6510 0x7f64a40b8a20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f64b00095f0 tx=0x7f64b0005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.211+0000 7f64b7e00640 1 -- 172.21.15.16:0/2153629282 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f64ac0d2b20 con 0x7f64c0106750 2024-09-21T03:58:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.467+0000 7f64c5200640 1 -- 172.21.15.16:0/2153629282 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f64c0063fc0 con 0x7f64c0106750 2024-09-21T03:58:01.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.477+0000 7f64b7e00640 1 -- 172.21.15.16:0/2153629282 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+833523 (secure 0 0 0) 0x7f64ac0d7a60 con 0x7f64c0106750 2024-09-21T03:58:01.492 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:58:01.492 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:58:01.492 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 18, 2024-09-21T03:58:01.493 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.493 INFO:teuthology.orchestra.run.smithi016.stdout: "active_gid": 14154, 2024-09-21T03:58:01.493 INFO:teuthology.orchestra.run.smithi016.stdout: "active_name": "x", 2024-09-21T03:58:01.493 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addrs": { 2024-09-21T03:58:01.493 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-21T03:58:01.493 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.493 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-21T03:58:01.493 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6800", 2024-09-21T03:58:01.493 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3097741949 2024-09-21T03:58:01.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.493 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.493 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-21T03:58:01.494 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6801", 2024-09-21T03:58:01.494 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3097741949 2024-09-21T03:58:01.494 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.494 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:58:01.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.494 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addr": "172.21.15.16:6801/3097741949", 2024-09-21T03:58:01.494 INFO:teuthology.orchestra.run.smithi016.stdout: "active_change": "2024-09-21T03:34:53.217379+0000", 2024-09-21T03:58:01.494 INFO:teuthology.orchestra.run.smithi016.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T03:58:01.494 INFO:teuthology.orchestra.run.smithi016.stdout: "available": true, 2024-09-21T03:58:01.494 INFO:teuthology.orchestra.run.smithi016.stdout: "standbys": [ 2024-09-21T03:58:01.495 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.495 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 24175, 2024-09-21T03:58:01.495 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "z", 2024-09-21T03:58:01.495 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540719139924082687, 2024-09-21T03:58:01.495 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-21T03:58:01.495 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.495 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-21T03:58:01.495 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.495 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.495 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.495 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:58:01.495 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:58:01.495 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.495 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.496 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:58:01.496 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.496 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.496 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.496 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:58:01.496 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.496 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.496 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.497 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.497 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.497 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.497 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.497 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.497 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.497 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.497 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.497 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.497 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.497 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.497 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.497 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.498 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.498 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.498 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.498 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.498 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.498 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.498 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.499 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.499 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.499 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.499 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.499 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.499 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.499 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.499 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.499 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.499 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.499 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.500 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.500 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.500 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.500 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.500 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.500 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.500 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.500 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.501 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.501 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.501 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.501 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.501 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.501 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.501 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.501 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.501 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.501 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.501 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.502 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.502 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.502 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.502 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.502 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.502 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.502 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.503 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.503 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.503 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.503 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.503 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.503 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-21T03:58:01.503 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-21T03:58:01.503 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.503 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.504 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.504 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.504 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.504 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.504 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.504 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:58:01.504 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.504 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.504 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.504 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-21T03:58:01.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-21T03:58:01.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.505 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-21T03:58:01.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-21T03:58:01.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.505 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-21T03:58:01.505 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-21T03:58:01.506 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.506 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.506 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.506 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.506 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-21T03:58:01.506 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.506 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.506 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.507 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-21T03:58:01.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-21T03:58:01.507 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.507 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.507 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.507 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.507 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.507 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.507 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.507 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-21T03:58:01.507 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.507 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.508 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.508 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-21T03:58:01.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-21T03:58:01.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.508 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.508 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.508 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-21T03:58:01.508 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.508 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.508 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.508 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-21T03:58:01.508 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.508 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.508 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.509 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-21T03:58:01.509 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-21T03:58:01.509 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.509 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.509 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.509 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.509 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.509 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:58:01.509 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.509 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-21T03:58:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-21T03:58:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:58:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-21T03:58:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-21T03:58:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-21T03:58:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-21T03:58:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-21T03:58:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-21T03:58:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:58:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-21T03:58:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-21T03:58:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-21T03:58:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:58:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:58:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-21T03:58:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-21T03:58:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:58:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-21T03:58:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-21T03:58:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:58:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:58:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-21T03:58:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:58:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:58:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-21T03:58:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:58:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-21T03:58:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:58:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-21T03:58:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-21T03:58:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:58:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:58:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:58:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-21T03:58:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-21T03:58:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-21T03:58:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-21T03:58:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-21T03:58:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:58:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:58:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-21T03:58:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-21T03:58:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.520 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-21T03:58:01.520 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.520 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.520 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.520 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:58:01.520 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:58:01.520 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.520 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.520 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-21T03:58:01.520 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-21T03:58:01.520 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:58:01.520 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.520 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.521 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.521 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-21T03:58:01.521 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-21T03:58:01.521 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.521 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:58:01.521 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:58:01.521 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.521 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.521 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.521 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.521 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.521 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.521 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.522 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.522 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.522 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.522 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.522 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.522 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.522 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.522 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.522 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.522 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.522 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.522 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.523 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.523 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.523 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.523 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.523 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.523 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.523 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.523 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.523 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.523 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.523 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.523 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.524 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.524 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.524 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.524 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.524 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.524 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.524 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.524 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.524 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.524 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.524 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.524 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.524 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.525 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.525 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.525 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.525 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.525 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.525 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.525 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.525 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.525 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.525 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.525 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.525 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.526 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.526 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.526 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.526 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.526 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.526 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.526 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.526 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.526 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.526 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-21T03:58:01.526 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-21T03:58:01.526 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:01.527 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.527 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.527 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.527 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.527 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.527 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.527 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:58:01.527 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.527 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.527 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.527 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-21T03:58:01.527 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-21T03:58:01.527 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.527 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.528 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.528 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-21T03:58:01.528 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.528 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.528 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.528 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-21T03:58:01.528 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-21T03:58:01.528 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-21T03:58:01.529 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-21T03:58:01.529 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-21T03:58:01.529 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.529 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-21T03:58:01.529 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.529 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.529 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.529 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-21T03:58:01.529 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-21T03:58:01.529 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.529 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.529 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.530 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.530 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.530 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.530 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.530 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:58:01.530 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.530 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.530 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.531 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:58:01.531 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:58:01.531 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.531 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.531 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.531 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:58:01.531 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.531 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.531 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.531 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:58:01.531 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.531 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.531 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.532 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.532 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.532 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.532 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.532 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.532 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.532 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.532 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.532 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.532 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.533 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.533 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.533 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.533 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-21T03:58:01.533 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-21T03:58:01.533 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.533 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.533 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.533 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:58:01.533 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:58:01.533 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.533 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.533 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:58:01.534 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:58:01.534 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.534 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.534 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-21T03:58:01.534 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:58:01.534 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:58:01.534 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.534 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.535 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:01.535 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.535 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.535 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.535 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:58:01.535 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.535 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.535 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.535 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.535 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.535 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.535 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-21T03:58:01.536 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.536 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.536 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.536 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-21T03:58:01.536 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-21T03:58:01.536 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:01.536 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.536 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.536 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-21T03:58:01.536 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.537 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.537 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.537 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:58:01.537 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.537 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.537 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.537 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-21T03:58:01.537 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-21T03:58:01.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.538 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.538 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.538 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-21T03:58:01.538 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.538 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.538 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.538 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:58:01.538 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.539 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.539 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.539 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-21T03:58:01.539 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-21T03:58:01.539 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.539 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.539 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.539 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-21T03:58:01.540 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.540 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.540 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.540 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:58:01.540 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.540 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.540 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.540 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-21T03:58:01.540 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-21T03:58:01.541 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.541 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.541 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.541 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.541 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.541 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.541 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:58:01.541 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:58:01.541 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.541 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.541 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-21T03:58:01.541 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-21T03:58:01.542 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.542 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.542 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.542 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:58:01.542 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.542 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.542 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.542 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:58:01.542 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.542 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.542 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.542 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:58:01.542 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:58:01.542 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:01.543 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.543 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.543 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-21T03:58:01.543 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.543 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.543 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.543 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:58:01.543 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.543 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.543 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.543 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-21T03:58:01.543 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-21T03:58:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-21T03:58:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-21T03:58:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-21T03:58:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:58:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.545 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.545 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.545 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-21T03:58:01.545 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-21T03:58:01.545 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.545 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.545 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.545 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.545 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.545 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.545 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.545 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:58:01.545 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.545 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.546 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.546 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.546 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-21T03:58:01.546 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-21T03:58:01.546 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.546 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.546 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.546 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.546 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.546 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.546 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.546 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:58:01.546 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.546 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.547 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.547 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.547 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-21T03:58:01.547 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-21T03:58:01.547 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.547 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.547 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.547 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.547 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.547 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.547 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.547 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:58:01.547 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.547 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.547 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.548 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.548 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-21T03:58:01.548 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-21T03:58:01.548 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.548 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.548 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.548 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:58:01.548 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.548 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.548 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.548 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:58:01.548 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.548 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.548 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.549 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-21T03:58:01.549 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-21T03:58:01.549 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.549 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.549 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.549 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:58:01.549 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.549 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.549 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.549 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:58:01.549 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.549 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.549 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.550 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-21T03:58:01.550 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:58:01.550 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.550 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.550 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.550 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:58:01.550 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.550 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.550 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.550 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-21T03:58:01.550 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.550 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.551 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.551 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-21T03:58:01.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-21T03:58:01.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.551 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.551 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.551 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:58:01.551 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.551 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.551 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:58:01.551 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.551 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.552 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.552 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-21T03:58:01.552 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-21T03:58:01.552 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.552 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.552 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.552 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:58:01.552 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.552 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.552 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.552 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-21T03:58:01.552 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.552 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-21T03:58:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:58:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:58:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:58:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-21T03:58:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:58:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:58:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:58:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-21T03:58:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:58:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:58:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-21T03:58:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-21T03:58:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-21T03:58:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:58:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-21T03:58:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-21T03:58:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-21T03:58:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:58:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-21T03:58:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.558 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.558 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.558 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nginx": { 2024-09-21T03:58:01.558 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nginx", 2024-09-21T03:58:01.558 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.558 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.558 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.558 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:58:01.558 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.558 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.558 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.558 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nginx container image", 2024-09-21T03:58:01.558 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.558 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-21T03:58:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-21T03:58:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:58:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:58:01.559 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.560 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.560 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.560 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-21T03:58:01.560 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-21T03:58:01.560 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.560 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.560 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.560 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:58:01.560 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.560 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.560 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.560 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-21T03:58:01.560 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.560 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.561 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.561 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:58:01.561 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:58:01.561 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.561 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.561 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.561 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:58:01.561 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.561 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.561 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.562 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:58:01.562 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.562 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.562 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-21T03:58:01.562 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-21T03:58:01.562 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.562 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.562 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.562 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:58:01.562 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.563 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.563 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:58:01.563 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.563 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.563 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.563 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.563 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-21T03:58:01.563 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-21T03:58:01.563 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.563 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.563 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.564 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:58:01.564 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.564 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.564 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.564 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-21T03:58:01.564 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.564 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.564 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.564 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-21T03:58:01.564 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-21T03:58:01.565 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.565 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.565 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.565 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:58:01.565 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.565 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.565 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.565 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:58:01.565 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.565 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.565 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.565 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba_metrics": { 2024-09-21T03:58:01.566 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:58:01.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.566 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.566 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.566 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:58:01.566 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.566 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.566 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.566 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:58:01.567 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.567 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.567 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.567 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.567 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-21T03:58:01.567 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:58:01.567 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.567 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.567 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.567 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:58:01.567 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.567 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.567 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.568 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:58:01.568 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.568 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.568 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.568 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.568 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-21T03:58:01.568 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-21T03:58:01.568 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.568 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.568 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.568 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.569 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.569 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.569 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:58:01.569 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.569 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.569 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.569 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.569 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-21T03:58:01.569 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:58:01.569 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.570 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.570 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.570 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:58:01.570 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.570 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.570 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.570 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:58:01.570 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.570 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.570 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.571 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:58:01.571 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:58:01.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.571 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.571 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.571 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-21T03:58:01.571 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.571 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.571 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.571 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:58:01.571 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.572 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.572 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.572 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.572 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-21T03:58:01.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-21T03:58:01.572 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.572 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.572 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.572 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-21T03:58:01.573 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.573 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.573 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.573 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:58:01.573 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.573 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.573 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.573 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.573 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-21T03:58:01.574 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-21T03:58:01.574 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.574 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.574 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.574 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-21T03:58:01.574 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.574 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.574 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.574 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:58:01.574 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.574 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.575 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.575 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.575 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-21T03:58:01.575 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-21T03:58:01.575 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.575 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.575 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.575 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.575 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.575 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.575 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.575 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:58:01.575 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.575 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.575 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.576 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.576 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-21T03:58:01.576 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-21T03:58:01.576 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.576 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.576 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.576 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:58:01.576 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.576 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.576 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.576 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:58:01.576 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.576 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.576 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.577 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.577 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-21T03:58:01.577 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:58:01.577 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.577 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.577 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.577 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:58:01.577 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.577 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.577 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:58:01.577 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.577 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.578 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.578 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.578 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-21T03:58:01.578 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-21T03:58:01.578 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.578 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.578 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.578 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:58:01.578 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.578 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.578 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.578 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:58:01.578 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.578 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.579 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.579 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.579 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-21T03:58:01.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-21T03:58:01.579 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.579 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.579 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.579 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.579 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.579 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.579 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:58:01.579 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.580 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.580 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.580 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-21T03:58:01.580 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-21T03:58:01.580 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.580 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.580 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.580 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.580 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.580 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.580 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.580 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:58:01.581 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.581 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.581 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.581 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.581 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.581 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.581 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.581 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.581 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.581 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.581 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.582 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.582 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.582 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.582 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.582 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.582 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.582 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.582 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.582 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.582 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.582 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.582 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-21T03:58:01.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-21T03:58:01.582 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.583 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.583 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.583 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.583 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.583 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.583 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:58:01.583 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.583 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.583 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.583 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.583 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.583 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.583 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.584 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.584 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.584 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.584 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.584 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:58:01.584 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.584 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.584 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.584 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.584 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.584 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.584 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.584 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.585 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.585 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.585 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.585 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.585 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.585 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.585 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.585 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.585 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.585 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.585 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.585 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.585 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.585 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.585 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.586 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.586 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.586 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.586 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.586 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.586 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.586 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.586 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.586 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.586 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.586 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.586 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.587 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.587 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.587 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:58:01.587 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:58:01.587 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.587 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.587 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.587 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.587 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:58:01.587 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.587 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.588 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.588 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.588 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:58:01.588 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:58:01.588 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.588 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.588 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.588 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-21T03:58:01.588 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.588 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.588 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.588 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:58:01.588 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.588 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.589 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.589 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-21T03:58:01.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-21T03:58:01.589 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.589 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.589 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.589 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:01.589 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.589 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.589 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.589 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:58:01.589 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.589 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.590 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.590 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-21T03:58:01.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-21T03:58:01.590 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.590 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.590 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.590 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:01.590 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.590 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.590 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.590 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:58:01.590 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.591 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.591 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.591 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.591 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-21T03:58:01.591 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-21T03:58:01.591 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.591 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.591 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.591 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.591 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.591 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.591 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.591 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-21T03:58:01.591 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.591 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.592 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.592 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-21T03:58:01.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-21T03:58:01.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.592 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-21T03:58:01.592 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.592 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.592 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.592 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-21T03:58:01.592 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-21T03:58:01.592 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.593 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:58:01.593 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.593 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.593 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.593 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-21T03:58:01.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-21T03:58:01.593 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.593 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.593 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.593 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-21T03:58:01.593 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.593 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.594 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:58:01.594 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.594 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.594 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.594 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-21T03:58:01.594 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:58:01.594 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.594 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.594 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.594 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:58:01.595 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.595 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.595 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.595 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:58:01.595 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.595 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.595 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.595 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-21T03:58:01.595 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-21T03:58:01.596 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.596 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.596 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.596 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.596 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.596 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.596 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:58:01.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.596 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.596 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.596 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-21T03:58:01.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-21T03:58:01.596 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.597 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.597 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.597 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.597 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.597 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.597 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.597 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:58:01.597 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.597 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.597 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-21T03:58:01.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-21T03:58:01.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.598 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.598 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.598 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.598 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.598 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.598 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.598 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:58:01.598 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.598 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.598 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.598 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-21T03:58:01.598 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-21T03:58:01.598 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.599 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.599 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.599 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.599 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.599 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.599 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.599 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:58:01.599 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.600 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.600 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.600 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-21T03:58:01.600 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:58:01.600 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.600 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.600 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.600 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.600 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.600 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.600 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:58:01.600 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.601 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.601 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.601 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-21T03:58:01.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-21T03:58:01.601 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.601 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.601 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.601 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-21T03:58:01.601 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.602 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.602 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:58:01.602 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.602 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.602 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.602 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.602 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.602 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.602 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.602 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.602 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.602 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.603 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.603 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.603 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.603 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.603 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.603 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.603 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-21T03:58:01.603 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-21T03:58:01.603 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.604 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.604 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.604 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.604 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.604 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.604 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.604 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:58:01.604 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.604 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.604 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.605 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:58:01.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:58:01.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.605 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.605 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-21T03:58:01.605 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.605 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.605 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:58:01.606 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.606 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.606 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_keepalive_interval": { 2024-09-21T03:58:01.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:58:01.606 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.606 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.606 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-21T03:58:01.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.607 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.607 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.607 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:58:01.607 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.607 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.607 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.607 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-21T03:58:01.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-21T03:58:01.608 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.608 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.608 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.608 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.608 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:58:01.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.609 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.609 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.609 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-21T03:58:01.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-21T03:58:01.609 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.609 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.609 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.609 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.609 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.610 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:58:01.610 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.610 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.610 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.610 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-21T03:58:01.610 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:58:01.610 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.611 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:58:01.611 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.611 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.611 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-21T03:58:01.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:58:01.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.611 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.611 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.611 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.611 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.611 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.611 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:58:01.612 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.612 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.612 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.612 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.612 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-21T03:58:01.612 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:58:01.612 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.612 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.612 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.612 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.612 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.612 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:58:01.613 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.613 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.613 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.613 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.613 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.613 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-21T03:58:01.613 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.613 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.613 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.613 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.613 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.614 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.614 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.614 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.614 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.614 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.614 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.614 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.614 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.614 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.614 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.614 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.614 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.614 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.614 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.616 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.616 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.616 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.616 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.616 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.616 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.616 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.616 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.616 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.616 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.618 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.618 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.618 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.618 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.618 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.618 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.618 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-21T03:58:01.618 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-21T03:58:01.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-21T03:58:01.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.619 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:58:01.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.619 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.619 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.619 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.620 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.620 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.620 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.620 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.620 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.620 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.620 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-21T03:58:01.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-21T03:58:01.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.621 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.621 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-21T03:58:01.621 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.621 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.621 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:58:01.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.621 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.621 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.622 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-21T03:58:01.622 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.622 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.622 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.622 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:58:01.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:58:01.622 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.622 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:01.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.622 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.623 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.623 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.623 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:58:01.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:58:01.623 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.623 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.623 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.623 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.623 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.623 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.623 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:58:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:58:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.625 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.625 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.625 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.625 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.625 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:58:01.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:58:01.625 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.625 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.625 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.626 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:58:01.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:58:01.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.626 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.626 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.626 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.627 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:58:01.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:58:01.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.627 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.628 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:58:01.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:58:01.628 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.628 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.628 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.628 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.628 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.628 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.629 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.629 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.629 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.629 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:58:01.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:58:01.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.629 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.630 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.630 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.630 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.630 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.630 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:58:01.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:58:01.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.630 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.631 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.631 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.631 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.631 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:58:01.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:58:01.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.632 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.632 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.632 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.632 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.632 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.632 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:58:01.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:58:01.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.633 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.634 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:58:01.634 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:58:01.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.635 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.635 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.635 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.635 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.635 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:58:01.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:58:01.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.635 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.636 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.636 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:58:01.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:58:01.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.637 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.637 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.638 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:58:01.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:58:01.638 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:58:01.638 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.638 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.638 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.639 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.639 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:58:01.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:58:01.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.639 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.640 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.640 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.640 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.640 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.640 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.640 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.640 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.641 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-21T03:58:01.641 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:58:01.641 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.641 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.641 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.641 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.641 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.641 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.641 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.642 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.642 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.642 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:58:01.642 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:58:01.642 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.642 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.642 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:58:01.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.642 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.642 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.642 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.643 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.643 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.643 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.643 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:58:01.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:58:01.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.643 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.643 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.643 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.643 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.643 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.643 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.644 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.644 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.644 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.644 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:58:01.644 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:58:01.644 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.644 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.644 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.644 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.644 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.644 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.644 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.644 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.645 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.645 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.645 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.645 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:58:01.645 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:58:01.645 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.645 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.645 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.645 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.645 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.645 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.645 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.646 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.646 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.646 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.646 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:58:01.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:58:01.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.646 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.646 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.647 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.647 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.647 INFO:teuthology.orchestra.run.smithi016.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:58:01.647 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:58:01.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.647 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.647 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "[]", 2024-09-21T03:58:01.647 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.647 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.647 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.647 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.648 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.648 INFO:teuthology.orchestra.run.smithi016.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:58:01.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:58:01.648 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.648 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.648 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.648 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "{}", 2024-09-21T03:58:01.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.649 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.649 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.649 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.649 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.649 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:58:01.649 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:58:01.649 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.649 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.649 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.649 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.649 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.650 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.650 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.650 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:58:01.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:58:01.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.651 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.651 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.651 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.651 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.651 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.651 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.651 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:58:01.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:58:01.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.651 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.651 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.651 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.651 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.652 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.652 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.652 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.652 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.652 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.652 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.652 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.652 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:58:01.652 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:58:01.652 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.652 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.652 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.652 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.652 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.653 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.653 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.653 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.653 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.653 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.653 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:58:01.653 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:58:01.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.653 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.653 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.654 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.654 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.654 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.654 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.654 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:58:01.654 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:58:01.654 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.654 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.654 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.654 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.655 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:58:01.655 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:58:01.655 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.655 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.655 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.655 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.655 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.655 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.656 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.656 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.656 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.656 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.656 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:58:01.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:58:01.656 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.656 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.656 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.656 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.656 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.656 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.656 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.657 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.657 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.657 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.657 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.657 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:58:01.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:58:01.657 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.657 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.657 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.657 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.657 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.657 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:58:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:58:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:58:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:58:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:58:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.660 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.660 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.660 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.660 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.660 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:58:01.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:58:01.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:01.661 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.661 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.661 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.661 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.661 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.661 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:58:01.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:58:01.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-21T03:58:01.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.662 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.662 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.662 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.663 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:58:01.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:58:01.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.663 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.663 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-21T03:58:01.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.663 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.663 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.664 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.664 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:58:01.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:58:01.664 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.664 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.664 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.664 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.664 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.664 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.664 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.664 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.665 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:58:01.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:58:01.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.665 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.665 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:58:01.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.666 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.666 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.666 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:58:01.666 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:58:01.666 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.666 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.666 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.666 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.666 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.667 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.667 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.667 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.667 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:58:01.667 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:58:01.667 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.667 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.667 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.667 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.667 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.667 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.667 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.667 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.668 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.668 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.668 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:58:01.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:58:01.668 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.668 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.668 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.668 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.668 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.669 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.669 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.669 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.669 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:58:01.669 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:58:01.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.670 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.670 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.670 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.670 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.670 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.670 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:58:01.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:58:01.671 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.671 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.671 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.671 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:01.671 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.671 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.671 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.671 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.671 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.672 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:58:01.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:58:01.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.672 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:58:01.672 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.672 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.672 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.672 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.672 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.673 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.673 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.673 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-21T03:58:01.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-21T03:58:01.673 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.673 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.673 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.674 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.674 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.674 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.674 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-21T03:58:01.674 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-21T03:58:01.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.674 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.675 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.675 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.675 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.675 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.675 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.675 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.676 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-21T03:58:01.676 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-21T03:58:01.676 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.676 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.676 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.676 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.676 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.676 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:58:01.676 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.677 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-21T03:58:01.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-21T03:58:01.677 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.677 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.677 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.677 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-21T03:58:01.677 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.677 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.677 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.678 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.678 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.678 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.678 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-21T03:58:01.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-21T03:58:01.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.678 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.678 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.678 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.678 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.678 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.678 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.679 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.679 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.679 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.679 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.679 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.679 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.679 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.679 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.679 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.679 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.679 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.680 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.680 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.680 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.680 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.680 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.680 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.680 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.680 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.680 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.680 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.680 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.680 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.681 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.681 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.681 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.681 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.681 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.681 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.681 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.681 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.681 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.681 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.682 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.682 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.682 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.682 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.682 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.682 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.682 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.682 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.682 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.682 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.682 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.683 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.683 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.683 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.683 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.683 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.683 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.683 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.683 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.683 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.684 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.684 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.684 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.684 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.684 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.684 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.684 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.684 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.684 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-21T03:58:01.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-21T03:58:01.684 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.684 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.684 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.685 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.685 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.685 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.685 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.685 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-21T03:58:01.685 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.685 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.685 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:58:01.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:58:01.685 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.685 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.686 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.686 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.686 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.686 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.686 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.686 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.686 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.686 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.686 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.686 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:58:01.686 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:58:01.686 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.686 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.687 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.687 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-21T03:58:01.687 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.687 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.687 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.687 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.687 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.687 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:58:01.687 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:58:01.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.687 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.687 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.688 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-21T03:58:01.688 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.688 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.688 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.688 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.688 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.688 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.688 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.688 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.688 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.688 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.688 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.688 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.689 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.689 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.689 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.689 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.689 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.690 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-21T03:58:01.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-21T03:58:01.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.690 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.690 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.690 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.690 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.690 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.690 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.690 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.690 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.690 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.690 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.691 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-21T03:58:01.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-21T03:58:01.691 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.691 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.691 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.691 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-21T03:58:01.691 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.691 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.691 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.691 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.691 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.691 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.692 INFO:teuthology.orchestra.run.smithi016.stdout: "sso_oauth2": { 2024-09-21T03:58:01.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sso_oauth2", 2024-09-21T03:58:01.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.692 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.692 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.692 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.692 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.692 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.692 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.692 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.692 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.692 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.692 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.693 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.693 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-21T03:58:01.693 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-21T03:58:01.693 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.693 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.693 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.693 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-21T03:58:01.693 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.693 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.693 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.693 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-21T03:58:01.693 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.694 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.694 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.694 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.694 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.694 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-21T03:58:01.694 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-21T03:58:01.694 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.694 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.694 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.694 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:58:01.694 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-21T03:58:01.694 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-21T03:58:01.694 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.695 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.695 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.695 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.695 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.695 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-21T03:58:01.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-21T03:58:01.695 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.695 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.695 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.695 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.695 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.695 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.695 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.695 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.696 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.696 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.696 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.696 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.696 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.696 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.696 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-21T03:58:01.696 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.696 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.696 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.696 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-21T03:58:01.696 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-21T03:58:01.697 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.697 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.697 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.697 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.697 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.697 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.697 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.697 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-21T03:58:01.697 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.697 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.697 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.697 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.697 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.698 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.698 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.698 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.698 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.698 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.698 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.698 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.698 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.699 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.699 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.699 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.699 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.699 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.699 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.699 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.699 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.699 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.699 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.699 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.699 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.699 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.699 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.700 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.700 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.700 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.700 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.700 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.700 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.700 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.700 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.700 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.700 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.701 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.701 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.701 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.701 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.701 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.701 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.701 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.701 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.701 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.701 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.701 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.702 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.702 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.702 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.702 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.702 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.702 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.702 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.702 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.703 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.703 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.703 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.703 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.703 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.703 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-21T03:58:01.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-21T03:58:01.703 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.703 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.703 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.704 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-21T03:58:01.704 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.704 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:58:01.704 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.704 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.704 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.704 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-21T03:58:01.704 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-21T03:58:01.704 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.704 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.705 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.705 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-21T03:58:01.705 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.705 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:58:01.705 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.705 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.705 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.705 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-21T03:58:01.705 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-21T03:58:01.706 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.706 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.706 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-21T03:58:01.706 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.706 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.706 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:58:01.706 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.707 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.707 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.707 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-21T03:58:01.707 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-21T03:58:01.707 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.707 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.707 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.707 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-21T03:58:01.708 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.708 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.708 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.708 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:58:01.708 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.708 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.708 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.708 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-21T03:58:01.708 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-21T03:58:01.709 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.709 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.709 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.709 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.709 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.709 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.709 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:58:01.710 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.710 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.710 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.710 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:58:01.710 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:58:01.710 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.710 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.710 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.711 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:58:01.711 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.711 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.711 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.711 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:58:01.711 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.711 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.711 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.711 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.711 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.711 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.712 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.712 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.712 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.712 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.712 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.712 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.712 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.712 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.712 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.712 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-21T03:58:01.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-21T03:58:01.712 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.713 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.713 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.713 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-21T03:58:01.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.713 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.713 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.713 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:58:01.713 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.713 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.713 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.713 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.713 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.713 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.714 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-21T03:58:01.714 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.714 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.714 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.714 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.714 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.714 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.714 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.714 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.714 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.714 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.714 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.714 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.714 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.715 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.715 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.715 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.715 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.715 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.715 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.715 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.715 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.715 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.715 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.715 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.715 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.715 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.715 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.716 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.716 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.716 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.716 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.716 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.716 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.716 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.716 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.716 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.716 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.716 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.716 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.716 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.716 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.717 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.717 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.717 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.717 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.717 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.717 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.717 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.717 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.717 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.717 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.717 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.717 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.717 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.717 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.718 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.718 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.718 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.718 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.718 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.718 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.718 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.718 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.718 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.718 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.718 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.719 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.719 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.719 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.719 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.719 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-21T03:58:01.719 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-21T03:58:01.719 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.719 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.719 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.719 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-21T03:58:01.719 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.719 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.719 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.719 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.720 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.720 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.720 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.720 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-21T03:58:01.720 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-21T03:58:01.720 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.720 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.720 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.720 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-21T03:58:01.720 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.720 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.721 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.721 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.721 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.721 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.721 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.721 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:58:01.721 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:58:01.721 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.721 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.721 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:58:01.721 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.722 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.722 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.722 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.722 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.722 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.722 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.722 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.722 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.722 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.722 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.722 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.723 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.723 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.723 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.723 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.723 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.723 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.723 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.723 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.723 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.723 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.723 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-21T03:58:01.724 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-21T03:58:01.724 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:58:01.724 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.724 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-21T03:58:01.724 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-21T03:58:01.724 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.724 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.724 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.724 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-21T03:58:01.724 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.724 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.724 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.724 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:58:01.725 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.725 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.725 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.725 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-21T03:58:01.725 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-21T03:58:01.725 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.725 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.725 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.725 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-21T03:58:01.725 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.725 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.725 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.725 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:58:01.725 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.726 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.726 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.726 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-21T03:58:01.726 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-21T03:58:01.726 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.726 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.726 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.726 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.726 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.726 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.726 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.726 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:58:01.726 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.727 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.727 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.727 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:58:01.727 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:58:01.727 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.727 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.727 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.727 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-21T03:58:01.727 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-21T03:58:01.727 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.727 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.727 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:58:01.727 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.728 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.728 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.728 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.728 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.728 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.728 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.728 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.728 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.728 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.728 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.728 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.728 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.728 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.729 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.729 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.729 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.729 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.729 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.729 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.729 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.729 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.729 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.729 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.729 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.730 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.730 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.730 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.730 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.730 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.730 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.730 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.730 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.730 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.730 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.730 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.730 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.731 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.731 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.731 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.731 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.731 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.731 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.731 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.731 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.731 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.731 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.732 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.732 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.732 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.732 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.732 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.732 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.732 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.732 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.732 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.732 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.732 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.732 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.733 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.733 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.733 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.733 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.733 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.733 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.733 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.733 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-21T03:58:01.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-21T03:58:01.733 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.733 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.734 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.734 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.734 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.734 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.734 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.734 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:58:01.734 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.734 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.734 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.734 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.734 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-21T03:58:01.735 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-21T03:58:01.735 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.735 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.735 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.735 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-21T03:58:01.735 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.735 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.735 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.735 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-21T03:58:01.735 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.735 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.735 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.736 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.736 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.736 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.736 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.736 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.736 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.736 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.736 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.736 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.736 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.737 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.737 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.737 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.737 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.737 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.737 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-21T03:58:01.737 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-21T03:58:01.737 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.737 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.737 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.737 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-21T03:58:01.737 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.738 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.738 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.738 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:58:01.738 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.738 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.738 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.738 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.738 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-21T03:58:01.738 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-21T03:58:01.738 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.738 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.738 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.739 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:58:01.739 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:58:01.739 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-21T03:58:01.739 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.739 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:58:01.739 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.739 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.739 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.739 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-21T03:58:01.739 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-21T03:58:01.739 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.739 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.739 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.739 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.740 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.740 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.740 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.740 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:58:01.740 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.740 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.740 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.740 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.740 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-21T03:58:01.740 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-21T03:58:01.740 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.740 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.740 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.741 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-21T03:58:01.741 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.741 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.741 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.741 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:58:01.741 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.741 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.741 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.741 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.741 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.741 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.742 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.742 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-21T03:58:01.742 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.742 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.742 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.742 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.742 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.742 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.742 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.742 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.743 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.743 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.743 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.743 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.743 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.743 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.743 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.743 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.743 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.743 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.743 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.744 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.744 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.744 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.744 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.744 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.744 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.744 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.744 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.744 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.745 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.745 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.745 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.745 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.745 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.745 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.745 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.745 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.745 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.745 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.746 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.746 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.746 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.746 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.746 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.746 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.746 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.746 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.746 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.746 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.746 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.746 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.747 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.747 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.747 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.747 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.747 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.747 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.747 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.747 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.747 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.747 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.747 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.747 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.747 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.747 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.747 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.748 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.748 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.748 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.748 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.748 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.748 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.748 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.748 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.748 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.748 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.748 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.748 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.748 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.749 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.749 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.749 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.749 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.749 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.749 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.749 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.749 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.749 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.749 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.749 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-21T03:58:01.749 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.749 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.749 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.750 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.750 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.750 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.750 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.750 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.750 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.750 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.750 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.750 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.750 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.750 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.750 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.750 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.750 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.751 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.751 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.751 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.751 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.751 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.751 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.751 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.751 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.751 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.751 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.751 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.751 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.752 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.752 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.752 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.752 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.752 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.752 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.752 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.752 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.752 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.752 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.752 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.752 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.753 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.753 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.753 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.753 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.753 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.753 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.753 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.753 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.753 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.753 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.753 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.753 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.753 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.753 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.753 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.754 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.754 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.754 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.754 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.754 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.754 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.754 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.754 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.754 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.754 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.754 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.754 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.754 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.754 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.755 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.755 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.755 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.755 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.755 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.755 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.755 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.755 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.755 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.755 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.755 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.755 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.755 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.755 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.756 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.756 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-21T03:58:01.756 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.756 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.756 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.756 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-21T03:58:01.756 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:58:01.756 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.756 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.756 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.756 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-21T03:58:01.756 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.756 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.756 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.756 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:58:01.757 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.757 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.757 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.757 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.757 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-21T03:58:01.757 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-21T03:58:01.757 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.757 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.757 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.757 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:01.757 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-21T03:58:01.757 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.757 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.757 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:58:01.757 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.758 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.758 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.758 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.758 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.758 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.758 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.758 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.758 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.758 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.758 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.758 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.758 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.758 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.759 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.759 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.759 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.759 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.759 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.759 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.759 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.759 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.759 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.759 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.759 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.759 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.759 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.759 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.759 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.760 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.760 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.760 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.760 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.760 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.760 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.760 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.760 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.760 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.760 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.760 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.760 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.760 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.760 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.760 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.761 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.761 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.761 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.761 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.761 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.761 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.761 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.761 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.761 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.761 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.761 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.761 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.761 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.761 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.761 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-21T03:58:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-21T03:58:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-21T03:58:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-21T03:58:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:58:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:58:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.765 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.765 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.765 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.765 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.765 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.765 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.765 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.765 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.765 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.765 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.765 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.765 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.765 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.765 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.766 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.766 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.766 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.766 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.766 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.766 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.766 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.766 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.766 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.766 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.766 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.766 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.767 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.767 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.767 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.767 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.767 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.767 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.767 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.767 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.767 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.767 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.767 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.767 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.767 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.768 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.768 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.768 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.768 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.768 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.768 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.768 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.768 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.768 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.768 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.768 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.768 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.768 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.769 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.769 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.769 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.769 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.769 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.769 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.769 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.770 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.770 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.770 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.770 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.770 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.770 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.770 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.770 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.770 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.770 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-21T03:58:01.770 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-21T03:58:01.770 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.771 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.771 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.771 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.771 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.771 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.771 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.771 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:58:01.771 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:58:01.771 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.771 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.772 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.772 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-21T03:58:01.772 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-21T03:58:01.772 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.772 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.772 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.772 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-21T03:58:01.772 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.772 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.772 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.772 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:58:01.772 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.772 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.773 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.773 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-21T03:58:01.773 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-21T03:58:01.773 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.773 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.773 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.773 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-21T03:58:01.773 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.773 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.773 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.774 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:58:01.774 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.774 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.774 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.774 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-21T03:58:01.774 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-21T03:58:01.774 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.774 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.774 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.774 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.774 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.775 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.775 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.775 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:58:01.775 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.775 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.775 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.775 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.775 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.775 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.776 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.776 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.776 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.776 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.776 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.776 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.776 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.776 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.776 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.776 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.776 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.776 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-21T03:58:01.777 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-21T03:58:01.777 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.777 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.777 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.777 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-21T03:58:01.777 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.777 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.777 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.777 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:58:01.777 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:58:01.777 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.778 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.778 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.778 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.778 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.778 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.778 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-21T03:58:01.778 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.778 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.778 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.778 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.778 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.779 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.779 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.779 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.779 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.779 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.779 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.779 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.779 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.779 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.779 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.780 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.780 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.780 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.780 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.780 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.780 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.780 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.780 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.780 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.780 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.780 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.780 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.780 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.781 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.781 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.781 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.781 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.781 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.781 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.781 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.781 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.781 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.781 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.781 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.781 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.781 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.781 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.782 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.782 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.782 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.782 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.782 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.782 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.782 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.782 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.782 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.782 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.782 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.782 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.782 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.782 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.783 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.783 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.783 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.783 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.783 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.783 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.783 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.783 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.783 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.783 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.783 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.783 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.783 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.783 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.784 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.784 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.784 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.784 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.784 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.784 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.784 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.784 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.784 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.784 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.784 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.784 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.784 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.784 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.785 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.785 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.785 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.785 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.785 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.785 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-21T03:58:01.785 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.785 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.785 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.785 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.785 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.785 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.785 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.785 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.786 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.786 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.786 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.786 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.786 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.786 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.786 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.786 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.786 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.786 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.786 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.786 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.786 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.786 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.786 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.787 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.787 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.787 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.787 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.787 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.787 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.787 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.787 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.787 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.787 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.787 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.787 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.787 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.787 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.789 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.789 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.789 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.789 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.789 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.789 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.789 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.789 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.789 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.789 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.789 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.789 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.789 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.789 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.789 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.789 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.789 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.790 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.790 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.790 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.790 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.790 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.790 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.790 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.790 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.790 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.790 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.790 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.790 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.790 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.790 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.790 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.791 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.791 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.791 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.791 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-21T03:58:01.791 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.791 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.791 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.791 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.791 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.791 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.791 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.791 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.791 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.791 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.791 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.791 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.795 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.795 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.795 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.795 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.795 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.795 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.795 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.795 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.795 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.795 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.795 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.795 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.795 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.796 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.796 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.796 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.796 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.796 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.796 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.796 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.796 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.796 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.796 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.796 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.796 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.796 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.796 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.796 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.797 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.797 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.797 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.797 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-21T03:58:01.797 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.797 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.797 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.797 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-21T03:58:01.797 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-21T03:58:01.797 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.797 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.797 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.797 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.797 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.797 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.798 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.798 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:58:01.798 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.798 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.798 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.798 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.798 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.798 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.798 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.798 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.798 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.798 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.798 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.798 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.799 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.799 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.799 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.799 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.799 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.799 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.799 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.799 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.799 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.799 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.799 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.799 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-21T03:58:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-21T03:58:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-21T03:58:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-21T03:58:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-21T03:58:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-21T03:58:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.806 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.806 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.806 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.806 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.806 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.806 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.806 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.806 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.806 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-21T03:58:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-21T03:58:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.819 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.819 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.819 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.819 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.819 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.819 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.819 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.819 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.819 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.819 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.819 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.819 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.819 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.820 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.820 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.820 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.820 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.820 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.820 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.820 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.820 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.820 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.820 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.820 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.820 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.820 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.821 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.821 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.821 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.821 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.821 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.821 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.821 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.821 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.821 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.821 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.821 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.821 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.821 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.821 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.821 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.821 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-21T03:58:01.822 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.822 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.822 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.822 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.822 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.822 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.822 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.822 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.822 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.822 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.822 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.822 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.822 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.822 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.822 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.823 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.823 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.823 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.823 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.823 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.823 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.823 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.823 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.823 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.823 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.823 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.823 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.824 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.824 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.824 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.824 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.824 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.824 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.824 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.824 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.824 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.824 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.824 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.824 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.825 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.825 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.825 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.825 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.825 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.825 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.825 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.825 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.825 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.825 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.825 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.825 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.825 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.825 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.826 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.826 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.826 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.826 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.826 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.826 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.826 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.826 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.826 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.826 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.826 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.826 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.827 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.827 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.827 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.827 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:58:01.827 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:58:01.827 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.827 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.827 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.827 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:58:01.827 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.827 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.827 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.827 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.828 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.828 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.828 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.828 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.828 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.828 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.828 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.828 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.828 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.828 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.828 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.828 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.828 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.829 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.829 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.829 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.829 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-21T03:58:01.829 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-21T03:58:01.829 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:01.829 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.829 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.829 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-21T03:58:01.829 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-21T03:58:01.829 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.829 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.829 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-21T03:58:01.830 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:58:01.830 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.830 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.830 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.830 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.830 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.830 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-21T03:58:01.830 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.830 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.830 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.830 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-21T03:58:01.830 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:58:01.830 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.831 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.831 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.831 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.831 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.831 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.831 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.831 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:58:01.831 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.831 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.831 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.831 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-21T03:58:01.831 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-21T03:58:01.831 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.831 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.832 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.832 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.832 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.832 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.832 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.832 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.832 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.832 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.832 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.832 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.832 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.832 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.832 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.833 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.833 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.833 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.833 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.833 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.833 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.833 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.833 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.833 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.833 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.833 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.833 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.833 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.833 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.834 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.834 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.834 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.834 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.834 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.834 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.834 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.834 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.834 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.834 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.834 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.834 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.835 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.835 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.835 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.835 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.835 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.835 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.835 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.835 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.835 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.836 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.836 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.836 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.836 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.836 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.836 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.836 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.836 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.836 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.836 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.836 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.836 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.836 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.836 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.837 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.837 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.837 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.837 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.837 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.837 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.837 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.837 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.837 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.837 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.838 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.838 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.838 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.838 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.838 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-21T03:58:01.838 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-21T03:58:01.838 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.838 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.838 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.838 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-21T03:58:01.839 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.839 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.839 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.839 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:58:01.839 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.839 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.839 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.839 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:58:01.839 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:58:01.839 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.840 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.840 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.840 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:58:01.840 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.840 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.840 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.840 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:58:01.840 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.840 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.840 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.840 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.840 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.841 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.841 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.841 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.841 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.841 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.841 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.841 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.841 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.841 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.841 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.841 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.842 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.842 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.842 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.842 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.842 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-21T03:58:01.842 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.842 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.842 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.842 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-21T03:58:01.842 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-21T03:58:01.842 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.842 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.843 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.843 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.843 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.843 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.843 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.843 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.843 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.843 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.843 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.843 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.843 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-21T03:58:01.844 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-21T03:58:01.844 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.844 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.844 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.844 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.844 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.844 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.844 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.844 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:58:01.844 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:58:01.845 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.845 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.845 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.845 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.845 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.845 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.845 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.846 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.846 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.846 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.846 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.846 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.846 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.846 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.846 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.846 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.846 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.847 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.847 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.847 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.847 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.847 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.847 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.847 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.847 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.847 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.847 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.848 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.848 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.848 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.848 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.848 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.848 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.848 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.848 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.848 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.848 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.848 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.848 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.848 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.848 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.849 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.849 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.849 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.849 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.849 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.849 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.849 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.849 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.849 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.849 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.849 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.849 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.849 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.849 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.850 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.850 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.850 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.850 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.850 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.850 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.850 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.850 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.850 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.850 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.850 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.850 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.850 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.851 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.851 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.851 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.851 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.851 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-21T03:58:01.851 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-21T03:58:01.851 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.851 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.851 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.851 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.851 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.851 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.851 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.851 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.851 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.852 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.852 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.852 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.852 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:58:01.852 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:58:01.852 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.852 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.852 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.852 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-21T03:58:01.852 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.852 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.852 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.852 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.852 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.852 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.853 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.853 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.853 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-21T03:58:01.853 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-21T03:58:01.853 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:01.853 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.853 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.853 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-21T03:58:01.853 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.853 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.853 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.853 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.853 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.853 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.853 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.853 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.854 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:58:01.854 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:58:01.854 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.854 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.854 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.854 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-21T03:58:01.854 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.854 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.854 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.854 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:58:01.854 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.854 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.854 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.854 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:58:01.855 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:58:01.855 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.855 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.855 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.855 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-21T03:58:01.855 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.855 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.855 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.855 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:58:01.855 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.855 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.855 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.855 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.855 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.856 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.856 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.856 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.856 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.856 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.856 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.856 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.856 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.856 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.856 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.856 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.856 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.856 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-21T03:58:01.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-21T03:58:01.857 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.857 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.857 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.857 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-21T03:58:01.857 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.857 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.857 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.857 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.857 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.857 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.857 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.857 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.857 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-21T03:58:01.857 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-21T03:58:01.858 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.858 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.858 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.858 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-21T03:58:01.858 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.858 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.858 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.858 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-21T03:58:01.858 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-21T03:58:01.858 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.858 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.858 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.858 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.858 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.859 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.859 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-21T03:58:01.859 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-21T03:58:01.859 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.859 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.859 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.859 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:58:01.859 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-21T03:58:01.859 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-21T03:58:01.859 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.859 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.859 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.859 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.860 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.860 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.860 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.860 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.860 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.860 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-21T03:58:01.860 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.860 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.860 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.860 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.860 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.860 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.860 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.860 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.860 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.861 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.861 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.861 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.861 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.861 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.861 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.861 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.861 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.861 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.861 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.861 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.861 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.861 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.863 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.863 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.863 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.863 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.863 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.863 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.863 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.863 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.863 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.863 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.863 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.863 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.863 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.863 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.864 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.864 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.864 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.864 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.864 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.864 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.864 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.864 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.864 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.864 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.864 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.864 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.864 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.865 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.865 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.865 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.865 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.865 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.865 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.865 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.865 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.865 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.865 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.865 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-21T03:58:01.865 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:58:01.865 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.865 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.866 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.866 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:01.866 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.866 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.866 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.866 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.866 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.866 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.866 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.866 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.866 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:58:01.866 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:58:01.866 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.867 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.867 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.867 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.867 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.867 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.867 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.867 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.867 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.867 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.867 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.868 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.868 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.868 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.868 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.868 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.868 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.868 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.868 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.868 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.868 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.868 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.868 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.869 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.869 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.869 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.869 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-21T03:58:01.869 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-21T03:58:01.869 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.869 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.869 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.869 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.869 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.869 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.869 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.870 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.870 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.870 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.870 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.870 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.870 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.870 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.870 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.870 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-21T03:58:01.870 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.870 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.871 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.871 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-21T03:58:01.871 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-21T03:58:01.871 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.871 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.871 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.871 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.871 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.871 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.871 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.871 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.871 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.872 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.872 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.872 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.872 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-21T03:58:01.872 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-21T03:58:01.872 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.872 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.872 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.872 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.873 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.873 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.873 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.873 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.873 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.873 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.873 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.873 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.873 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.873 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.873 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.874 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.874 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.874 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.874 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.874 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.874 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.874 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.874 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.874 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.874 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.875 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.875 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.875 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.875 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.875 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.875 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.875 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.875 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.875 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.876 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.876 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.876 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.876 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.876 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.876 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.876 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.876 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.876 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.876 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.876 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.877 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.877 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.877 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.877 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.877 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.877 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.877 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.877 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.877 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.878 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.878 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.878 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.878 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.878 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.878 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.878 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.878 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.878 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.878 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.879 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.879 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.879 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.879 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.879 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.879 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.879 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.879 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.879 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.879 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.879 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.879 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.880 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.880 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.880 INFO:teuthology.orchestra.run.smithi016.stdout: "max_requests": { 2024-09-21T03:58:01.880 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_requests", 2024-09-21T03:58:01.880 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.880 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.880 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.880 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:58:01.880 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.880 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.880 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.880 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:58:01.880 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.881 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.881 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.881 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:58:01.881 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:58:01.881 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.881 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.881 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.881 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.882 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.882 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.882 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.882 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.882 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.882 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.882 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.882 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.882 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:58:01.883 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:58:01.883 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.883 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.883 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.883 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.883 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.883 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.883 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.883 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.883 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.883 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.883 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.883 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.883 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.883 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.884 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.884 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.884 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.884 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.884 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.884 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.884 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.884 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.884 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.884 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.884 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.884 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.884 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.885 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-21T03:58:01.885 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.885 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.885 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.885 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.885 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.885 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.885 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.885 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.885 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.885 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.887 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.887 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.887 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.887 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.887 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.887 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.887 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.887 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.887 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.887 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.887 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.887 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.888 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.888 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.888 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.888 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.888 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.888 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.888 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.888 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.888 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.888 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.888 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.888 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.888 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.889 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.889 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.889 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.889 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.889 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.889 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.889 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.889 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.889 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.889 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.889 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.889 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.889 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.890 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.890 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.890 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.890 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.890 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.890 INFO:teuthology.orchestra.run.smithi016.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:58:01.890 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:58:01.890 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.890 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.890 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.890 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:58:01.890 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.890 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.891 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.891 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:58:01.891 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.891 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.891 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.891 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.891 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.891 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.891 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.891 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.891 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.891 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.891 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.892 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.892 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.892 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.892 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.892 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.892 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.892 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.892 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.892 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-21T03:58:01.892 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.892 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.892 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.892 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.892 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.893 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.893 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.893 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.893 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.893 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.893 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.893 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.893 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.893 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.893 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.893 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.893 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.893 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.893 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.894 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.894 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.894 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.894 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.894 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.894 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.894 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.894 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.894 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.894 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.894 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.894 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.894 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.895 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.895 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.895 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.895 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.895 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.895 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.895 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.895 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.895 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.895 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.895 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.895 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.896 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.896 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.896 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.896 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.896 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.896 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.896 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.896 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.896 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.896 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.896 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.896 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.896 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.896 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.896 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.897 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.897 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.897 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.897 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.897 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.897 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.897 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.897 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.897 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.897 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.897 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.897 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.898 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.898 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.898 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.898 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.898 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.898 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.898 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.898 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.898 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.898 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.898 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.898 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-21T03:58:01.899 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-21T03:58:01.899 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.899 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.899 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.899 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-21T03:58:01.899 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.899 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.899 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.899 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:58:01.899 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.899 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.899 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.899 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.899 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.900 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.900 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.900 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-21T03:58:01.900 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.900 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.900 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.900 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.900 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.900 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.900 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.900 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.900 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.900 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.900 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.901 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.901 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.901 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.901 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.901 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.901 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.901 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.901 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.901 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.901 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.901 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.901 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.901 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.902 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.902 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.902 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.902 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.902 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.902 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.902 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.902 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.902 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.902 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.903 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.903 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.903 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.903 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.903 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.903 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.903 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.903 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.903 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.903 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.903 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.903 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.903 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.904 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.904 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.904 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.904 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.904 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.904 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.904 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.904 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.904 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.904 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.905 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.905 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.905 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.905 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.905 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.905 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.905 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.905 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.905 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.905 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.905 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.906 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.906 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.906 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-21T03:58:01.906 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-21T03:58:01.906 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.906 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.906 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.906 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.906 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.906 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.906 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.906 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.907 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.907 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.907 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.907 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.907 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-21T03:58:01.907 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-21T03:58:01.907 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.907 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.907 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.907 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-21T03:58:01.908 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.908 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.908 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.908 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.908 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.908 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.908 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.908 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.908 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-21T03:58:01.908 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-21T03:58:01.908 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.908 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.909 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.909 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.909 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.909 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.909 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.909 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.909 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.909 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.909 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.909 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.909 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-21T03:58:01.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-21T03:58:01.909 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.910 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.910 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.910 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.910 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.910 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.910 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.910 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.910 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.910 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.910 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.911 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.911 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-21T03:58:01.911 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-21T03:58:01.911 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:01.911 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.911 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.911 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.911 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.911 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.911 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.912 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.912 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.912 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.912 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.912 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.912 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-21T03:58:01.912 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-21T03:58:01.912 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.912 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.912 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.913 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.913 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.913 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.913 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.913 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.913 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.913 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.913 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.913 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.913 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-21T03:58:01.914 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-21T03:58:01.914 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.914 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.914 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.914 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.914 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.914 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.914 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.914 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.915 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.915 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.915 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.915 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.915 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-21T03:58:01.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-21T03:58:01.915 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.915 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.915 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.915 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.916 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.916 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.916 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.916 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.916 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.916 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.916 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.916 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-21T03:58:01.916 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-21T03:58:01.916 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.916 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.916 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.916 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.916 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:58:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-21T03:58:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.917 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-21T03:58:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-21T03:58:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.918 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-21T03:58:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-21T03:58:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.919 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-21T03:58:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-21T03:58:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.920 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smb", 2024-09-21T03:58:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "internal_store_backend": { 2024-09-21T03:58:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "internal_store_backend", 2024-09-21T03:58:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:58:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.922 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.923 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.924 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.925 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.926 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.927 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "update_orchestration": { 2024-09-21T03:58:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "update_orchestration", 2024-09-21T03:58:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:58:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.929 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-21T03:58:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-21T03:58:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-21T03:58:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.930 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:58:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-21T03:58:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-21T03:58:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.931 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:58:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.932 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.933 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.934 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.934 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.934 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.934 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.934 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.934 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.934 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.934 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.934 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.934 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.934 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.934 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.934 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.935 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.935 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.935 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.935 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.935 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.935 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.935 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.935 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.935 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.936 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.937 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.937 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.937 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.937 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.937 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.937 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.937 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.937 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.937 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.937 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.937 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.938 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.938 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.938 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.938 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.938 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.938 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.938 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.938 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.938 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.938 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.938 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.938 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.938 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.939 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.939 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.939 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.939 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.939 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.939 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-21T03:58:01.939 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.939 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.939 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.939 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.939 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.939 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.939 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.940 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.940 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.940 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.940 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.940 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.940 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.940 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.940 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.940 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.940 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.940 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.940 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.941 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.941 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.941 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.941 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.941 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.941 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.941 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.941 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.941 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.941 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.942 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.942 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.942 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.942 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.942 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.942 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.942 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.942 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.942 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.942 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.942 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.942 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.942 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.942 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.943 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.943 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.943 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.943 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.943 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.943 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.943 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.943 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.943 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.943 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.944 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.944 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.944 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.944 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.944 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.944 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.944 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.944 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.944 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.944 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.944 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.945 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.945 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.945 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.945 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.945 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.945 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.945 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.945 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.945 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.945 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.946 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.946 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.946 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.946 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.946 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.946 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.946 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.946 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.946 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.947 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.947 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.947 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.947 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.947 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.947 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.947 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.947 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-21T03:58:01.947 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.947 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.947 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.947 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.947 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.947 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.948 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.948 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.948 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.948 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.948 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.948 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.948 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.948 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.948 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.948 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.948 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.948 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.948 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.948 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.948 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.948 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.949 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.949 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.949 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.949 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.949 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.949 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.949 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.949 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.949 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.949 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.949 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.949 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.949 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.949 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.949 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.950 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.950 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.950 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.950 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.950 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.950 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.950 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.950 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.950 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.950 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.950 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.950 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.950 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.951 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.951 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.951 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.951 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.951 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.951 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.951 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.951 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.951 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.951 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.951 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.952 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.952 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.952 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.952 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.952 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.952 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.952 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.952 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.952 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.952 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.952 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.952 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.952 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.952 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.952 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.953 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.953 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.953 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.953 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.953 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.953 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.953 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.953 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.953 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.953 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.953 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.953 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.953 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-21T03:58:01.953 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.953 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.953 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.954 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-21T03:58:01.954 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-21T03:58:01.954 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.954 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.954 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.954 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:58:01.954 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.954 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.954 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.954 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.954 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.954 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.954 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.954 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:58:01.955 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:58:01.955 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:01.955 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.955 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.955 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-21T03:58:01.955 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.955 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.955 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.955 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.955 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.955 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.955 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.955 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.955 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.955 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.956 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.956 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.956 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.956 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.956 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.956 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.956 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.956 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.956 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.956 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.956 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.956 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.956 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.956 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.956 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.957 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.957 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.957 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.957 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.957 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.957 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.957 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.957 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.957 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.957 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.957 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.958 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.958 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.958 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.958 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.958 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.958 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.958 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.958 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.958 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.958 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.958 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.958 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.958 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.959 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.959 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.959 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.959 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.959 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.959 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.959 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.959 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.959 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.959 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.959 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.959 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.960 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.960 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.960 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.960 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.960 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.960 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.960 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.960 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.960 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.960 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.960 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.960 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.961 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.961 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.961 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.961 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.961 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.961 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.961 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.961 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.961 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.961 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.961 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.961 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.962 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.962 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.962 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.962 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.962 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-21T03:58:01.962 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.962 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.962 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.962 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-21T03:58:01.962 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-21T03:58:01.962 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.962 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.963 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.963 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.963 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.963 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.963 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.963 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:58:01.963 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.963 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.963 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.963 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.963 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-21T03:58:01.963 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-21T03:58:01.963 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.964 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.964 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.964 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.964 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.964 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.964 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.964 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:58:01.964 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.964 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.964 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.964 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.964 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-21T03:58:01.964 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-21T03:58:01.965 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.965 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.965 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.965 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:01.965 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.965 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.965 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.965 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:58:01.965 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.965 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.965 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.965 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-21T03:58:01.965 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-21T03:58:01.966 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.966 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.966 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.966 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.966 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.966 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.966 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.966 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:58:01.966 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.966 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.966 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.966 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-21T03:58:01.967 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-21T03:58:01.967 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.967 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.967 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.967 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.967 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.967 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.967 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.967 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:58:01.967 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.967 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.967 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.967 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-21T03:58:01.968 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-21T03:58:01.968 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.968 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.968 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.968 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.968 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.968 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.968 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.968 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.968 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.968 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.968 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.968 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-21T03:58:01.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-21T03:58:01.969 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.969 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.969 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.969 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.969 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.969 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.969 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.969 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.969 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.969 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.969 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.969 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-21T03:58:01.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-21T03:58:01.970 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.970 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.970 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.970 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:58:01.970 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.970 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.970 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.970 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.970 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.970 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.970 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.970 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-21T03:58:01.970 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-21T03:58:01.970 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.971 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.971 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.971 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.971 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.971 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.971 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.971 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.971 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.971 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.971 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.971 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:58:01.971 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:58:01.971 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.972 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.972 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.972 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-21T03:58:01.972 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-21T03:58:01.972 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.972 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.972 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.972 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.972 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.972 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.973 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.973 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-21T03:58:01.973 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-21T03:58:01.973 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.973 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.973 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.973 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-21T03:58:01.973 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.973 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.973 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.973 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.973 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.973 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.973 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.974 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.974 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-21T03:58:01.974 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-21T03:58:01.974 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.974 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.974 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.974 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.974 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.974 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.974 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.974 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.974 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.974 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.974 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.975 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.975 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-21T03:58:01.975 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-21T03:58:01.975 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.975 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.975 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.975 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.975 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.975 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.975 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.975 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.975 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.975 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.975 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.976 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.976 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.976 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.976 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.976 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.976 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.976 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.976 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.976 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.976 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.976 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.976 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.976 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.976 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.977 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.977 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.977 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.977 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.977 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.977 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.977 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.977 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.977 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.977 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.977 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.977 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.977 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.977 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.977 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.978 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.978 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.978 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.978 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.978 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.978 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.978 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.978 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.978 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.978 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.978 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.978 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.978 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.980 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.980 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.980 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.980 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.980 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.980 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.980 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.980 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.980 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.980 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.980 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.980 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.980 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.981 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-21T03:58:01.981 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-21T03:58:01.981 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.981 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.981 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.981 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.981 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.981 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.981 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.981 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.981 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.981 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.981 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.982 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-21T03:58:01.982 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-21T03:58:01.982 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.982 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.982 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.982 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.982 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.982 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.982 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.982 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.982 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.982 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.983 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.983 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.983 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.983 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.983 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.983 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.983 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.983 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.983 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.983 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.983 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.983 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.983 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.984 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.984 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.984 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.984 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-21T03:58:01.984 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-21T03:58:01.984 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.984 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.984 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.984 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:58:01.984 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.984 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.984 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.984 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.984 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.984 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.984 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.985 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.985 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.985 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.985 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.985 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-21T03:58:01.985 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.985 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.985 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.985 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.985 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.985 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.985 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.985 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.985 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.985 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.985 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.986 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.986 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.986 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.986 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.986 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.986 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.986 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.986 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.986 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.986 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.986 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.986 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.986 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.986 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.986 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.987 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.987 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.987 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.987 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.987 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.987 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.987 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.987 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.987 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.987 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.987 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.987 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.987 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.987 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.987 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.988 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.988 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.988 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.988 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.988 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.988 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.988 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.988 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.988 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.988 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.988 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.988 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.988 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.988 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.988 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.989 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.989 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.989 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.989 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.989 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.989 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.989 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.989 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.989 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.989 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.989 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.989 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.989 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.989 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.990 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.990 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.990 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.990 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:01.990 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:01.990 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.990 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:01.990 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.990 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.990 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.990 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.990 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.990 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.990 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.991 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.991 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.991 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.991 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:01.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.991 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:01.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-21T03:58:01.991 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:01.991 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:01.991 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:01.991 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:01.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:01.991 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.991 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.991 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.992 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:01.992 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.992 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.992 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.992 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.992 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.992 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.992 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.992 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.992 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.992 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.992 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.992 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.992 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.992 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.993 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.993 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:01.993 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:01.993 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.993 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.993 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.993 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.993 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.993 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.993 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.993 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.993 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.994 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.994 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.994 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:01.994 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:01.994 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:01.994 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.994 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.994 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.996 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:01.996 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:01.996 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.996 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.996 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:01.996 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.996 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.996 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.996 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.996 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:01.996 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.996 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.996 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.997 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-21T03:58:01.997 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-21T03:58:01.997 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.997 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.997 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.997 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-21T03:58:01.997 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.997 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.997 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.997 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:58:01.997 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.997 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.997 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.997 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.997 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-21T03:58:01.998 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-21T03:58:01.998 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:01.998 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.998 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.998 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:01.998 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.998 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.998 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.998 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-21T03:58:01.998 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.998 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.998 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.998 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.998 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-21T03:58:01.998 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:58:01.999 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:01.999 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:01.999 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:01.999 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:01.999 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:01.999 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:01.999 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:01.999 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:58:01.999 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:01.999 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:01.999 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:01.999 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:01.999 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:58:01.999 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:58:02.000 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.000 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.000 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.000 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.000 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.000 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.000 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.000 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:58:02.000 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.000 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.000 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.000 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.000 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.001 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.001 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.001 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.001 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.001 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.001 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.001 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.001 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.001 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.001 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.001 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.001 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.001 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.001 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.002 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.002 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.002 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-21T03:58:02.002 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.002 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.002 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.002 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-21T03:58:02.002 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-21T03:58:02.002 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:58:02.002 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.002 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.002 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-21T03:58:02.002 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.002 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.002 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.003 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.003 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.003 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.003 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.003 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.003 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-21T03:58:02.003 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-21T03:58:02.003 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.003 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.003 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.003 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.003 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.004 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.004 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.004 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.004 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.004 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.004 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.004 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.004 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:58:02.004 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:58:02.004 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.004 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.004 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.004 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:58:02.005 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.005 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.005 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.005 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.005 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.005 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.005 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.005 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.005 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.005 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.005 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.005 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.005 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.005 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.005 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.006 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.006 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.006 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.006 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.006 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.006 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.006 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.006 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.006 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.006 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.006 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.006 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.006 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.006 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.007 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.007 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.007 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.007 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.007 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.007 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.007 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.007 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.007 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.007 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.007 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.007 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.007 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.007 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.008 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.008 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.008 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.008 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.008 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.008 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.008 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.008 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.008 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.008 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.008 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.008 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.008 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.009 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.009 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.009 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.009 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.009 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.009 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.009 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.009 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.009 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.009 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.009 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.009 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.009 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.010 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.010 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.010 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.010 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.010 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.010 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.010 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.010 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.010 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.010 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.010 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.010 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.010 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.011 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.011 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.011 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.011 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.011 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.011 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.011 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.011 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.011 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.011 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.011 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-21T03:58:02.011 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-21T03:58:02.011 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.011 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.011 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.012 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.012 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.012 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.012 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.012 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.012 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.012 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.012 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.012 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.012 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-21T03:58:02.012 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-21T03:58:02.012 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.012 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.012 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.013 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-21T03:58:02.013 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.013 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.013 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.013 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.013 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.013 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.013 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.013 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.013 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-21T03:58:02.013 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-21T03:58:02.013 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.013 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.013 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.013 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:58:02.014 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.014 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.014 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.014 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.014 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.014 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.014 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.014 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.014 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.014 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.014 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:58:02.014 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.014 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.015 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 24197, 2024-09-21T03:58:02.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "y", 2024-09-21T03:58:02.015 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540719139924082687, 2024-09-21T03:58:02.015 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-21T03:58:02.015 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-21T03:58:02.015 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.015 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.015 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.015 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:58:02.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:58:02.015 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.015 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.016 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.016 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:58:02.016 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.016 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.016 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.016 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:58:02.016 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.016 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.016 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.016 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.016 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.016 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.016 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.016 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.017 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.017 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.017 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.017 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.017 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.017 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.017 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.017 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.017 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.017 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.017 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.017 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.017 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.017 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.018 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.018 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.018 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.018 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.018 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.018 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.018 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.018 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.018 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.018 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.018 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.018 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.018 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.018 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.019 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.019 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.019 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.019 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.019 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.019 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.019 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.019 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.019 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.019 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.019 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.019 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.019 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.019 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.020 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.020 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.020 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.020 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.020 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.020 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.020 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.020 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.020 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.020 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.020 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.020 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.020 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.020 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.020 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.021 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.021 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.021 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.021 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.021 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.021 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.021 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.021 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.021 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.021 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-21T03:58:02.021 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-21T03:58:02.021 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.022 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.022 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.022 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.022 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.022 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.022 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.022 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:58:02.022 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.022 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.022 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.022 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.022 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-21T03:58:02.022 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-21T03:58:02.023 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.023 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.023 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.023 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-21T03:58:02.023 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.023 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.023 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.023 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-21T03:58:02.023 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.023 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.023 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.023 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.023 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-21T03:58:02.023 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-21T03:58:02.023 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.024 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.024 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.024 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.024 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.024 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.024 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.024 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-21T03:58:02.024 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.024 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.024 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.025 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.025 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-21T03:58:02.025 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-21T03:58:02.025 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.025 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.025 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.025 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.025 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.025 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.025 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.026 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-21T03:58:02.026 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.026 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.026 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.026 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.026 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-21T03:58:02.026 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-21T03:58:02.026 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.026 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.026 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.026 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-21T03:58:02.026 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.026 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.027 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.027 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-21T03:58:02.027 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.027 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.027 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.027 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-21T03:58:02.027 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-21T03:58:02.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.028 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.028 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.028 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.028 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.028 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.028 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.028 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:58:02.029 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.029 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.029 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.029 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-21T03:58:02.029 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-21T03:58:02.030 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.031 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.031 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.031 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.031 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.031 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.031 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.031 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:58:02.032 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.032 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.032 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.032 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.032 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-21T03:58:02.032 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-21T03:58:02.033 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.033 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.033 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.033 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.033 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.033 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.033 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.034 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-21T03:58:02.034 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.034 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.034 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.034 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.034 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.035 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.035 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.035 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.035 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.035 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.035 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.035 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.036 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.036 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.036 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.036 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.036 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.036 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.037 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.037 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.037 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.037 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-21T03:58:02.037 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.037 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.037 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.038 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-21T03:58:02.038 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-21T03:58:02.038 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.038 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.038 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.038 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.039 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.039 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.039 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.039 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:58:02.039 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.039 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.040 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.040 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.040 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-21T03:58:02.040 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-21T03:58:02.040 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.040 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.041 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.041 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-21T03:58:02.041 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.041 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.041 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.041 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:58:02.041 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:58:02.041 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.042 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.042 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.042 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-21T03:58:02.042 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-21T03:58:02.042 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:58:02.042 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.042 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.043 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.043 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-21T03:58:02.043 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-21T03:58:02.043 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.043 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:58:02.043 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:58:02.043 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.044 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.044 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.044 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-21T03:58:02.044 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:58:02.044 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:58:02.044 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.044 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.044 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-21T03:58:02.044 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:58:02.044 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-21T03:58:02.044 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.044 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:58:02.044 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.044 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.044 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.045 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.045 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-21T03:58:02.045 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-21T03:58:02.045 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.045 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.045 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.045 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:58:02.045 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.045 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.045 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.046 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:58:02.046 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:58:02.046 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.046 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.046 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.046 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-21T03:58:02.046 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-21T03:58:02.046 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:02.046 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.046 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.046 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-21T03:58:02.046 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-21T03:58:02.046 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-21T03:58:02.047 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.047 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:58:02.047 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:58:02.047 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.047 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.047 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-21T03:58:02.047 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-21T03:58:02.047 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.047 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.047 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.047 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-21T03:58:02.047 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.047 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.047 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.048 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:58:02.048 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:58:02.048 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.048 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.048 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.048 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-21T03:58:02.048 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-21T03:58:02.048 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:58:02.048 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.048 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.048 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.048 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-21T03:58:02.048 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-21T03:58:02.048 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.049 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:58:02.049 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:58:02.049 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.049 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.049 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.049 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.049 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.049 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.049 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.049 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.049 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.049 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.049 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.049 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.050 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.050 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.050 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.050 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.050 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.050 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.050 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.050 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.050 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.050 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.050 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.050 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.051 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.051 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.051 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.051 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.051 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.051 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.051 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.051 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.051 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.051 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.051 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.051 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.051 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.051 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.052 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.052 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.052 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.052 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.052 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.052 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.052 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.052 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.052 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.052 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.052 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.052 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.053 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.053 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.053 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.053 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.053 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.053 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.053 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.053 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.053 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.053 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.053 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.053 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.053 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.053 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.054 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.054 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.054 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.054 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.054 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.054 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.054 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.054 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-21T03:58:02.054 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-21T03:58:02.054 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:02.054 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.054 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.054 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.055 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.055 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.055 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.055 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:58:02.055 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.055 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.055 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.055 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.055 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-21T03:58:02.055 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-21T03:58:02.055 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.055 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-21T03:58:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-21T03:58:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-21T03:58:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-21T03:58:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-21T03:58:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-21T03:58:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-21T03:58:02.057 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.057 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.057 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.057 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.057 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-21T03:58:02.057 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-21T03:58:02.057 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.057 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.057 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.057 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.057 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.057 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.058 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.058 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:58:02.058 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.058 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.058 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.058 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.058 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:58:02.058 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:58:02.058 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.058 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.058 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.058 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:58:02.058 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.059 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.059 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.059 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:58:02.059 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.059 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.059 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.059 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.059 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.059 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.059 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.059 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.059 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.059 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.059 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.059 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.060 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.060 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.060 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.060 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.060 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.060 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-21T03:58:02.060 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-21T03:58:02.060 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.060 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.060 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.060 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:58:02.060 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:58:02.060 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.060 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.061 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:58:02.061 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:58:02.061 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.061 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.061 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-21T03:58:02.061 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:58:02.061 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:58:02.061 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.061 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.061 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:02.061 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.061 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.061 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.061 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:58:02.061 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.062 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.062 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.062 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.062 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.062 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.062 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-21T03:58:02.062 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.062 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.062 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.062 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-21T03:58:02.062 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-21T03:58:02.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:02.062 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.062 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.063 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-21T03:58:02.063 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.063 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.063 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.063 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:58:02.063 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.063 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.063 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.063 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-21T03:58:02.063 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-21T03:58:02.063 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.063 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.063 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.064 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-21T03:58:02.064 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.064 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.064 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.064 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:58:02.064 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.064 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.064 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.064 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.064 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-21T03:58:02.064 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-21T03:58:02.065 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.065 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.065 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.065 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-21T03:58:02.065 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.065 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.065 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.065 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:58:02.065 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.065 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.066 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.066 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.066 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-21T03:58:02.066 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-21T03:58:02.066 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.066 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.066 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.066 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.067 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.067 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.067 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.067 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:58:02.067 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:58:02.067 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.067 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.067 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-21T03:58:02.068 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-21T03:58:02.068 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.068 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.068 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.068 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:58:02.068 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.068 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.068 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.068 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:58:02.069 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.069 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.069 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.069 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:58:02.069 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:58:02.069 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:02.069 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.070 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.070 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-21T03:58:02.070 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.070 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.070 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.070 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:58:02.070 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.070 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.070 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.071 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.071 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-21T03:58:02.071 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-21T03:58:02.071 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.071 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.071 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.071 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.071 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.071 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.072 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.072 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-21T03:58:02.072 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-21T03:58:02.072 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-21T03:58:02.072 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.072 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:58:02.072 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.072 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.073 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.073 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.073 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-21T03:58:02.073 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-21T03:58:02.073 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.073 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.073 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.073 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.073 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.074 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.074 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.074 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:58:02.074 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.074 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.074 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.074 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-21T03:58:02.075 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-21T03:58:02.075 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.075 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.075 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.075 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.075 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.075 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.075 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.075 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:58:02.076 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.076 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.076 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.076 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.076 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-21T03:58:02.076 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-21T03:58:02.076 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.076 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.076 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.077 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.077 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.077 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.077 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.077 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:58:02.077 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.077 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.077 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.077 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.077 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-21T03:58:02.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-21T03:58:02.078 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.078 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.078 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.078 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:58:02.078 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.078 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.078 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.078 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:58:02.078 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.079 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.079 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.079 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.079 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-21T03:58:02.079 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-21T03:58:02.079 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.079 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.079 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.079 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:58:02.079 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.079 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.079 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.079 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:58:02.079 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.079 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.080 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.080 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-21T03:58:02.080 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:58:02.080 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.080 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.080 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:58:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-21T03:58:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-21T03:58:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-21T03:58:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:58:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:58:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-21T03:58:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-21T03:58:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:58:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-21T03:58:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-21T03:58:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:58:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:58:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:58:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-21T03:58:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:58:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:58:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:58:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-21T03:58:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:58:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:58:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-21T03:58:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-21T03:58:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-21T03:58:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:58:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-21T03:58:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-21T03:58:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-21T03:58:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:58:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-21T03:58:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nginx": { 2024-09-21T03:58:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nginx", 2024-09-21T03:58:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:58:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nginx container image", 2024-09-21T03:58:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-21T03:58:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-21T03:58:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:58:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:58:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-21T03:58:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-21T03:58:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:58:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-21T03:58:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:58:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:58:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:58:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:58:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-21T03:58:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-21T03:58:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:58:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:58:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-21T03:58:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-21T03:58:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:58:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-21T03:58:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-21T03:58:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-21T03:58:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:58:02.105 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:58:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba_metrics": { 2024-09-21T03:58:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:58:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:58:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:58:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-21T03:58:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:58:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:58:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:58:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-21T03:58:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-21T03:58:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:58:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-21T03:58:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:58:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:58:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:58:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:58:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:58:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-21T03:58:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:58:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-21T03:58:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-21T03:58:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-21T03:58:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:58:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-21T03:58:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-21T03:58:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-21T03:58:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:58:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-21T03:58:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-21T03:58:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:58:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-21T03:58:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-21T03:58:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:58:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:58:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-21T03:58:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:58:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:58:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:58:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-21T03:58:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-21T03:58:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:58:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:58:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-21T03:58:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-21T03:58:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:58:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-21T03:58:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-21T03:58:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:58:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-21T03:58:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-21T03:58:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:58:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:58:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:58:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:58:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:58:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:58:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:58:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-21T03:58:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:58:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-21T03:58:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-21T03:58:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:58:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-21T03:58:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-21T03:58:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:58:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-21T03:58:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-21T03:58:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-21T03:58:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-21T03:58:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-21T03:58:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-21T03:58:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-21T03:58:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-21T03:58:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:58:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-21T03:58:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-21T03:58:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-21T03:58:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:58:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-21T03:58:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:58:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:58:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:58:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-21T03:58:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-21T03:58:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:58:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-21T03:58:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-21T03:58:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:58:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-21T03:58:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-21T03:58:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:58:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-21T03:58:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-21T03:58:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:58:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-21T03:58:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:58:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:58:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-21T03:58:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-21T03:58:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-21T03:58:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:58:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-21T03:58:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-21T03:58:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:58:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:58:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:58:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-21T03:58:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:58:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_keepalive_interval": { 2024-09-21T03:58:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:58:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-21T03:58:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:58:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-21T03:58:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-21T03:58:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:58:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-21T03:58:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-21T03:58:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:58:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-21T03:58:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:58:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:58:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-21T03:58:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:58:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:58:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-21T03:58:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:58:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:58:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-21T03:58:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-21T03:58:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-21T03:58:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-21T03:58:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:58:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-21T03:58:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-21T03:58:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-21T03:58:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:58:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-21T03:58:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:58:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:58:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:58:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:58:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:58:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:58:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:58:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:58:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:58:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:58:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:58:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:58:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:58:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:58:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:58:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:58:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:58:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:58:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:58:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:58:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:58:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:58:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:58:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:58:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:58:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:58:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:58:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:58:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:58:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:58:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:58:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:58:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:58:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-21T03:58:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:58:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:58:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:58:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:58:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:58:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:58:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:58:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:58:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:58:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:58:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:58:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:58:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:58:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:58:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "[]", 2024-09-21T03:58:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:58:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:58:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "{}", 2024-09-21T03:58:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:58:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:58:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:58:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:58:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:58:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:58:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:58:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:58:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:58:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:58:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:58:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:58:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:58:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:58:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:58:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:58:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:58:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:58:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:58:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:58:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:58:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:58:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:58:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:58:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:58:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:58:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:58:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-21T03:58:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:58:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:58:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-21T03:58:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:58:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:58:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:58:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:58:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:58:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:58:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:58:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:58:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:58:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:58:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:58:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:58:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:58:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:58:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:58:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:58:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:58:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:58:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-21T03:58:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-21T03:58:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-21T03:58:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-21T03:58:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-21T03:58:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-21T03:58:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:58:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-21T03:58:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-21T03:58:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-21T03:58:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-21T03:58:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-21T03:58:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-21T03:58:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-21T03:58:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-21T03:58:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:58:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:58:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:58:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:58:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-21T03:58:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:58:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:58:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-21T03:58:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-21T03:58:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-21T03:58:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-21T03:58:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-21T03:58:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-21T03:58:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.218 INFO:teuthology.orchestra.run.smithi016.stdout: "sso_oauth2": { 2024-09-21T03:58:02.218 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sso_oauth2", 2024-09-21T03:58:02.218 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.218 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.218 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.218 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.218 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.218 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.218 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.219 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.219 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.219 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.219 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.219 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.219 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-21T03:58:02.219 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-21T03:58:02.219 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.219 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.219 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.219 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-21T03:58:02.219 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.219 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.219 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.220 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.220 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-21T03:58:02.220 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.220 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.220 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.220 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.220 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.220 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.220 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-21T03:58:02.220 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-21T03:58:02.220 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.220 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.220 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.220 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:58:02.221 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-21T03:58:02.221 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-21T03:58:02.221 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.221 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.221 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.221 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.221 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.221 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.221 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-21T03:58:02.221 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-21T03:58:02.221 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.221 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.221 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.221 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.222 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.222 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.222 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.222 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.222 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.222 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.222 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.222 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.222 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.222 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.222 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.222 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-21T03:58:02.222 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.222 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.223 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.223 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-21T03:58:02.223 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-21T03:58:02.223 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.223 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.223 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.223 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.223 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.223 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.223 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.223 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-21T03:58:02.223 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.223 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.224 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.225 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.226 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-21T03:58:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-21T03:58:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-21T03:58:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.229 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:58:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-21T03:58:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-21T03:58:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-21T03:58:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.230 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:58:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-21T03:58:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-21T03:58:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-21T03:58:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:58:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-21T03:58:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-21T03:58:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-21T03:58:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:58:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-21T03:58:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-21T03:58:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:58:02.233 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:58:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:58:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:58:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.234 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:58:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.235 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.236 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.236 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.236 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.236 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-21T03:58:02.236 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-21T03:58:02.236 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.236 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.236 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.236 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-21T03:58:02.236 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.236 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.236 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.236 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:58:02.237 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.237 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.237 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.237 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.237 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.237 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.237 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.237 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-21T03:58:02.237 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.237 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.237 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.237 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.237 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.237 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.237 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.238 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.238 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.238 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.238 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.238 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.238 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.238 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.238 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.238 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.238 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.238 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.238 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.238 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.238 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.239 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.239 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.239 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.239 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.239 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.239 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.239 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.239 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.239 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.239 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.239 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.239 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.239 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.239 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.239 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.240 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.240 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.240 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.240 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.240 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.240 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.240 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.240 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.240 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.240 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.240 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.240 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.240 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.240 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.240 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.241 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.241 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.241 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.241 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.241 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.241 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.241 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.241 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.241 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.241 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.241 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.241 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.241 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.241 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.241 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.241 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.242 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.242 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.242 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.242 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.242 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.242 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-21T03:58:02.242 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-21T03:58:02.242 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.242 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.242 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.242 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-21T03:58:02.242 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.242 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.242 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.242 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.243 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.243 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.243 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.243 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.243 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-21T03:58:02.243 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-21T03:58:02.243 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.243 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.243 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.243 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-21T03:58:02.243 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.243 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.243 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.243 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.243 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.244 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.244 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.244 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.244 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:58:02.244 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:58:02.244 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.244 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.244 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.244 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:58:02.244 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.244 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.244 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.244 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.244 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.245 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.245 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.245 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.245 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.245 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.245 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.245 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.245 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.245 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.245 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.245 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.245 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.245 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.245 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.246 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.246 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.246 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.246 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.246 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.246 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.246 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-21T03:58:02.246 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-21T03:58:02.246 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:58:02.246 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.246 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-21T03:58:02.246 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-21T03:58:02.247 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.247 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.247 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.247 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-21T03:58:02.247 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.247 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.247 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.247 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:58:02.247 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.247 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.247 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.247 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.247 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-21T03:58:02.248 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-21T03:58:02.248 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.248 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.248 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.248 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-21T03:58:02.248 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.248 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.248 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.248 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:58:02.248 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.248 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.248 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.248 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.248 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-21T03:58:02.249 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-21T03:58:02.249 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.249 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.249 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.249 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.249 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.249 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.249 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.249 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:58:02.249 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.249 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.249 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.249 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.249 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:58:02.250 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:58:02.250 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.250 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.250 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.250 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-21T03:58:02.250 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-21T03:58:02.250 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.250 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.250 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:58:02.250 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.250 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.250 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.250 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.251 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.251 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.251 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.251 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.251 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.251 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.251 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.251 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.251 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.251 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.251 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.251 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.252 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.252 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.252 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.252 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.252 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.252 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.252 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.252 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.252 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.252 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.252 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.252 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.252 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.252 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.253 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.253 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.253 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.253 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.253 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.253 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.253 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.253 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.253 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.253 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.253 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.253 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.253 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.253 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.253 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.254 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.254 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.254 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.254 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.254 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.254 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.254 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.254 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.254 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.254 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.254 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.254 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.254 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.254 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.254 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.255 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.255 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.255 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.255 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.255 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.255 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.255 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.255 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.255 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.255 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.255 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.255 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.255 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.255 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-21T03:58:02.255 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-21T03:58:02.256 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.256 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.256 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.256 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.256 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.256 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.256 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.256 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:58:02.256 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.256 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.256 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.256 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.256 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-21T03:58:02.256 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-21T03:58:02.256 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.257 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.257 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.257 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-21T03:58:02.257 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.257 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.257 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.257 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-21T03:58:02.257 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.257 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.257 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.257 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.257 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.257 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.257 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.257 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.258 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.258 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.258 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.258 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.258 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.258 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.258 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.258 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.258 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.258 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.258 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-21T03:58:02.258 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-21T03:58:02.259 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.259 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.259 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.259 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-21T03:58:02.259 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.259 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.259 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.259 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:58:02.259 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.259 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.259 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.259 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.259 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-21T03:58:02.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-21T03:58:02.260 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.260 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.260 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.260 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:58:02.260 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:58:02.260 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-21T03:58:02.260 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.260 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:58:02.260 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.260 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.260 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.260 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.260 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-21T03:58:02.260 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-21T03:58:02.261 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.261 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.261 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.261 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.261 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.261 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.261 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.261 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:58:02.261 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.261 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.261 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.261 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-21T03:58:02.262 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-21T03:58:02.262 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.262 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.262 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.262 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-21T03:58:02.262 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.262 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.262 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.262 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:58:02.262 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.262 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.262 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.262 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.262 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.263 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.263 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.263 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-21T03:58:02.263 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.263 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.263 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.263 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.263 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.263 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.263 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.263 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.263 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.263 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.263 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.264 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.264 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.264 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.264 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.264 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.264 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.264 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.264 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.264 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.264 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.264 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.264 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.264 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.265 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.265 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.265 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.265 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.265 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.265 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.265 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.265 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.265 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.265 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.265 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.265 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.265 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.265 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.265 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.266 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.266 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.266 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.266 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.266 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.266 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.266 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.266 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.266 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.266 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.266 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.266 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.266 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.267 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.267 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.267 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.267 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.267 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.267 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.267 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.267 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.267 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.267 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.267 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.267 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.267 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.267 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.268 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.268 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.268 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.268 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.268 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.268 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.268 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.268 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.268 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.268 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.268 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.268 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.268 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.269 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.269 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.269 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.269 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.269 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.269 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.269 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.269 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-21T03:58:02.269 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.269 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.269 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.269 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.269 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.270 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.270 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.270 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.270 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.270 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.270 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.270 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.270 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.270 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.270 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.270 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.270 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.270 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.271 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.271 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.271 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.271 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.271 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.271 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.271 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.271 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.271 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.271 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.271 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.271 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.271 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.271 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.272 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.272 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.272 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.272 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.272 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.272 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.272 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.272 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.272 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.272 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.272 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.273 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.273 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.273 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.273 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.273 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.273 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.273 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.273 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.273 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.273 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.273 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.273 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.273 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.274 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.274 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.274 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.274 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.274 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.274 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.274 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.274 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.274 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.274 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.274 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.274 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.275 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.275 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.275 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.275 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.275 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.275 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.275 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.275 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.275 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.275 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.275 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.275 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.275 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.276 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.276 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.276 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.276 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.276 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-21T03:58:02.276 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.276 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.276 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.276 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-21T03:58:02.276 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:58:02.276 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.276 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.276 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.277 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-21T03:58:02.277 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.277 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.277 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.277 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:58:02.277 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.277 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.277 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.277 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.277 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-21T03:58:02.277 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-21T03:58:02.277 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.277 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.277 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.278 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:02.278 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-21T03:58:02.278 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.278 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.278 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:58:02.278 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.278 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.278 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.278 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.278 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.278 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.278 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.278 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.279 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.279 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.279 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.279 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.279 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.279 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.279 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.279 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.279 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.279 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.279 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.279 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.279 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.279 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.280 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.280 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.280 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.280 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.280 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.280 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.280 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.280 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.280 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.280 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.280 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.280 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.280 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.280 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.280 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.281 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.281 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.281 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.281 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.281 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.281 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.281 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.281 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.281 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.281 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.281 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.281 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.281 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.282 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.282 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.282 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.282 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.282 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.282 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.282 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.282 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.282 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.282 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.282 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.282 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.283 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.283 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.283 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.283 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.283 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.283 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.283 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.283 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.283 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.283 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.283 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.283 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.283 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.284 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.284 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.284 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.284 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.284 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.284 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.284 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.284 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.284 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.284 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-21T03:58:02.284 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.284 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.284 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.285 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-21T03:58:02.285 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-21T03:58:02.285 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.285 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.285 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.285 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-21T03:58:02.285 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.285 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.285 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.285 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:58:02.285 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:58:02.285 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.285 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.285 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.286 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.286 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.286 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.286 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.286 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.286 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.286 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.286 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.286 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.286 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.286 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.286 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.286 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.287 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.287 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.287 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.287 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.287 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.287 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.287 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.287 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.287 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.287 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.287 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.287 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.287 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.288 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.288 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.288 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.288 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.288 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.288 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.288 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.288 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.288 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.288 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.288 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.288 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.288 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.288 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.288 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-21T03:58:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-21T03:58:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.291 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.291 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.291 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.291 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.291 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:58:02.291 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:58:02.291 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.291 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.291 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-21T03:58:02.291 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-21T03:58:02.291 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.291 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.291 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.292 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-21T03:58:02.292 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.292 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.292 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.292 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:58:02.292 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.292 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.292 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.292 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-21T03:58:02.292 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-21T03:58:02.292 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.292 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.292 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.292 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-21T03:58:02.293 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.293 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.293 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.293 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:58:02.293 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.293 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.293 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.293 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-21T03:58:02.293 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-21T03:58:02.293 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.293 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.293 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.294 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.294 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.294 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.294 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.294 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:58:02.294 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.294 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.294 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.294 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.294 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.294 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.294 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.294 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.295 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.295 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.295 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.295 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.295 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.295 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.295 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.295 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.295 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-21T03:58:02.295 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-21T03:58:02.295 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.295 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.296 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.296 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-21T03:58:02.296 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.296 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.296 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.296 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:58:02.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:58:02.296 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.296 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.296 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.296 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.296 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.296 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-21T03:58:02.296 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.297 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.297 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.297 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.297 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.297 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.297 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.297 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.297 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.297 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.297 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.297 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.297 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.297 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.298 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.298 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.298 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.298 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.298 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.298 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.298 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.298 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.298 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.298 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.298 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.298 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.298 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.299 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.299 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.299 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.299 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.299 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.299 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.299 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.299 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.299 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.299 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.299 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.299 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.299 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.300 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.300 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.300 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.300 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.300 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.300 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.300 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.300 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.300 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.300 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.300 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.300 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.300 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.300 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.301 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.301 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.301 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.301 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.301 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.301 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.301 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.301 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.301 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.301 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.301 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.301 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.301 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.301 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.302 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.302 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.302 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.302 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.302 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.302 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.302 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.302 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.302 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.302 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.302 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.302 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.303 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.303 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.303 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.303 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-21T03:58:02.303 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.303 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.303 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.303 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.303 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.303 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.304 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.304 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.304 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.304 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.304 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.304 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.304 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.304 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.304 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.304 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.304 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.304 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.304 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.304 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.305 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.305 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.305 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.305 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.305 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.305 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.305 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.305 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.305 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.305 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.305 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.305 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.305 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.305 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.306 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.306 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.306 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.306 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.306 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.306 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.306 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.306 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.306 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.306 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.306 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.306 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.306 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.307 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.307 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.307 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.307 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.307 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.307 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.307 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.307 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.307 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.307 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.307 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.307 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.307 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.308 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.308 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.308 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.308 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.308 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.308 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.308 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.308 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.308 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.308 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.308 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.308 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.308 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.308 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.309 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.309 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.309 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.309 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.309 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.309 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.309 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.309 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.309 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.309 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.309 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.309 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.309 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-21T03:58:02.309 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.310 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.310 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.310 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.310 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.310 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.310 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.310 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.310 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.310 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.310 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.310 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.310 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.310 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.310 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.310 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.311 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.311 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.311 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.311 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.311 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.311 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.311 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.311 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.311 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.311 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.311 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.311 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.311 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.312 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.312 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.312 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.312 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.312 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.312 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.312 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.312 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.312 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.312 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.312 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.312 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.312 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.313 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.313 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.313 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.313 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.313 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.313 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.313 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.313 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.313 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.313 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.313 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.313 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.313 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.313 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.314 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.314 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.314 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.314 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.314 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.314 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.314 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.314 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.314 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.314 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.314 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.314 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.314 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.314 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.315 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.315 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.315 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.315 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.315 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.315 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.315 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.315 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.315 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.315 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.315 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.315 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.316 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.316 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.316 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.316 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.316 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-21T03:58:02.316 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.316 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.316 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.316 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-21T03:58:02.316 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-21T03:58:02.316 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.316 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.316 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.316 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.317 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.317 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.317 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.317 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:58:02.317 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.317 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.317 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.317 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.317 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.317 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.317 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.317 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.317 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.317 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.317 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.318 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.318 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.318 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.318 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.318 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.318 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.318 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.318 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.318 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.318 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.318 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.318 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.318 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.318 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.319 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.319 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.319 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.319 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.319 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.319 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.319 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.319 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.320 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.320 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.320 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.320 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.320 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.320 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.320 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.320 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.320 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.320 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.320 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.320 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.320 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.320 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.320 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.321 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.321 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.321 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.321 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.321 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.321 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.321 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.321 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.321 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.321 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.321 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.321 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.321 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.321 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.322 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.322 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.322 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.322 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.322 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-21T03:58:02.322 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-21T03:58:02.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.322 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.322 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.322 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.322 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.323 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.323 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.323 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-21T03:58:02.323 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-21T03:58:02.323 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-21T03:58:02.323 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.323 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-21T03:58:02.323 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.323 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.323 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.323 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.324 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.324 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.324 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.324 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.324 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.324 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.324 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.324 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.324 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.324 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.324 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.324 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.324 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-21T03:58:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-21T03:58:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.337 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.337 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.337 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.337 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.337 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.337 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.337 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.337 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.337 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.337 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.337 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.337 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.337 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.338 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.338 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.338 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-21T03:58:02.338 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.338 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.338 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.338 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.338 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.338 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.338 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.338 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.338 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.338 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.338 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.338 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.339 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.339 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.339 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.339 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.339 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.339 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.339 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.339 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.339 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.339 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.339 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.339 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.339 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.339 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.339 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.339 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.340 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.340 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.340 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.340 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.340 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.340 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.340 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.340 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.340 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.340 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.340 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.340 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.340 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.340 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.341 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.341 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.341 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.341 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.341 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.341 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.341 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.341 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.341 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.341 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.341 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.341 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.341 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.341 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.341 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.341 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.342 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.342 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.342 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.342 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.342 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.342 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.342 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.342 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.342 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.342 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.342 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.342 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.342 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.342 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:58:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:58:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:58:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-21T03:58:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-21T03:58:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:02.345 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.345 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.345 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-21T03:58:02.345 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-21T03:58:02.345 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.345 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.345 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-21T03:58:02.345 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:58:02.345 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.345 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.345 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.345 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.345 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.345 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-21T03:58:02.346 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.346 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.346 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.346 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-21T03:58:02.346 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:58:02.346 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.346 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.346 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.346 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.346 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.346 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.346 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.346 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:58:02.346 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.347 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.347 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.347 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-21T03:58:02.347 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-21T03:58:02.347 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.347 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.347 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.347 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.347 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.347 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.347 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.348 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.348 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.348 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.348 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.348 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.348 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.348 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.348 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.348 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.348 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.348 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.348 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.349 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.349 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.349 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.349 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.349 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.349 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.349 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.349 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.349 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.349 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.349 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.349 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.349 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.349 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.350 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.350 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.350 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.350 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.350 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.350 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.350 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.350 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.350 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.350 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.350 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.350 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.350 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.351 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.351 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.351 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.351 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.351 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.351 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.351 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.351 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.351 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.351 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.351 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.351 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.351 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.351 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.352 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.352 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.352 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.352 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.352 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.352 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.352 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.352 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.352 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.352 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.352 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.352 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.352 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.353 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-21T03:58:02.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-21T03:58:02.353 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.353 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.353 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.353 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-21T03:58:02.353 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.353 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.353 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.353 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:58:02.353 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.353 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.354 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.354 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:58:02.354 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:58:02.354 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.354 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.354 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.354 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:58:02.354 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.354 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.354 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.354 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:58:02.354 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.354 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.355 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.355 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.355 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.355 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.355 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.355 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.355 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.355 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.355 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.355 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.355 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.355 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.355 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.356 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.356 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.356 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.356 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.356 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-21T03:58:02.356 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.356 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.356 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.356 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-21T03:58:02.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-21T03:58:02.356 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.357 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.357 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.357 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.357 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.357 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.357 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.357 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.357 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.357 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.357 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.357 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-21T03:58:02.358 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-21T03:58:02.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.358 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.358 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.358 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.358 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.358 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.358 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.358 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:58:02.358 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:58:02.358 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.358 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.359 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.359 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.359 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.359 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.359 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.359 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.359 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.359 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.359 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.359 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.359 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.359 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.360 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.360 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.360 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.360 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.360 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.360 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.360 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.360 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.360 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.360 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.360 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.360 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.361 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.361 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.361 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.361 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.361 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.361 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.361 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.361 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.361 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.362 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.362 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.362 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.362 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.362 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.362 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.362 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.362 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.362 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.362 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.362 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.362 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.362 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.362 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.363 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.363 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.363 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.363 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.363 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.363 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.363 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.363 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.363 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.363 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.363 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.363 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.363 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.364 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-21T03:58:02.364 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-21T03:58:02.364 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.364 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.364 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.364 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.364 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.364 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.364 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.364 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.364 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.364 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.364 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.365 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:58:02.365 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:58:02.365 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.365 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.365 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.365 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-21T03:58:02.365 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.365 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.365 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.365 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.365 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.365 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.365 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.365 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.365 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-21T03:58:02.366 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-21T03:58:02.366 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:02.366 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.366 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.366 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-21T03:58:02.366 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.366 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.366 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.366 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.366 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.366 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.366 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.367 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:58:02.367 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:58:02.367 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.367 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.367 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.367 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-21T03:58:02.367 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.367 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.367 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.367 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:58:02.367 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.367 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.367 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.368 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:58:02.368 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:58:02.368 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.368 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.368 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.368 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-21T03:58:02.368 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.368 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.368 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.368 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:58:02.368 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.368 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.368 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.368 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.369 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.369 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.369 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.369 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.369 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.369 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.369 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.369 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.369 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.369 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.369 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.369 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.369 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-21T03:58:02.370 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-21T03:58:02.370 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.370 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.370 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.370 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-21T03:58:02.370 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.370 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.370 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.370 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.370 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.370 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.370 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.370 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-21T03:58:02.370 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-21T03:58:02.371 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.371 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.371 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-21T03:58:02.371 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.371 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.371 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-21T03:58:02.371 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-21T03:58:02.371 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.371 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.371 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.371 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.371 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.372 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-21T03:58:02.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-21T03:58:02.372 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.372 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.372 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.372 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:58:02.372 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-21T03:58:02.372 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-21T03:58:02.372 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.372 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.372 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.372 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.372 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.372 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.373 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.373 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.373 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-21T03:58:02.373 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.373 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.373 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.373 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.373 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.373 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.373 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.373 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.373 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.373 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.373 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.374 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.374 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.374 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.374 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.374 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.374 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.374 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.374 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.374 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.374 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.374 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.374 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.374 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.375 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.375 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.375 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.375 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.375 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.375 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.375 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.375 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.375 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.375 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.375 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.375 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.375 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.376 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.376 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.376 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.376 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.376 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.376 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.376 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.376 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.376 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.376 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.376 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.376 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.377 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.377 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.377 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.377 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.377 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.377 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.377 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.377 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.377 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.377 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.377 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.377 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.377 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.378 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.378 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.378 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.378 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-21T03:58:02.378 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:58:02.378 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.378 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.378 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.378 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:02.378 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.378 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.378 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.378 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.379 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.379 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.379 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:58:02.379 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:58:02.379 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.379 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.379 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.379 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.379 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.379 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.379 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.379 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.379 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.380 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.380 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.380 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.380 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.380 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.380 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.380 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.380 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.380 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.380 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.380 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.381 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.381 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.381 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.381 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-21T03:58:02.381 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-21T03:58:02.381 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.381 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.381 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.381 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.381 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.381 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.381 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.381 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.381 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.382 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.382 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.382 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.382 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.382 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.382 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-21T03:58:02.382 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.382 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.382 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.382 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-21T03:58:02.382 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-21T03:58:02.382 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.382 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.382 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.383 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.383 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.383 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.383 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.383 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.383 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.383 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.383 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.383 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-21T03:58:02.383 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-21T03:58:02.383 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.383 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.383 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.384 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.384 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.384 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.384 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.384 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.384 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.384 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.384 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.384 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.384 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.384 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.384 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.384 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.384 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.384 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.385 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.385 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.385 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.385 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.385 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.385 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.385 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.385 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.385 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.385 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.385 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.385 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.385 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.385 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.386 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.386 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.386 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.386 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.386 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.386 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.386 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.386 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.386 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.386 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.386 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.386 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.386 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.387 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.387 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.387 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.387 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.387 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.387 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.387 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.387 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.387 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.387 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.387 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.387 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.387 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.387 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.388 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.388 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.388 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.388 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.388 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.388 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.388 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.388 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.388 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.388 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.388 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.388 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.388 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.388 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.389 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.389 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.389 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.389 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.389 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.389 INFO:teuthology.orchestra.run.smithi016.stdout: "max_requests": { 2024-09-21T03:58:02.389 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_requests", 2024-09-21T03:58:02.389 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.389 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.389 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.389 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:58:02.389 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.389 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.389 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.390 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:58:02.390 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.390 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.390 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.390 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:58:02.390 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:58:02.390 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.390 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.390 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.390 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.390 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.390 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.391 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.391 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.391 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.391 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.391 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.391 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:58:02.391 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:58:02.391 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.391 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.391 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.391 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.391 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.391 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.391 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.392 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.392 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.392 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.392 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.392 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.392 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.392 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.392 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.392 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.392 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.392 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.392 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.392 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.392 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.393 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.393 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.393 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.393 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.393 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.393 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-21T03:58:02.393 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.393 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.393 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.393 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.393 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.394 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.394 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.394 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.394 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.394 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.394 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.394 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.394 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.394 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.394 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.394 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.394 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.394 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.394 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.395 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.395 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.395 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.395 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.395 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.395 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.395 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.395 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.395 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.395 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.395 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.396 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.396 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.396 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.396 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.396 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.396 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.396 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.396 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.396 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.396 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.396 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.396 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.396 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.397 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.397 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.397 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.397 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.397 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.397 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.397 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.397 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.397 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.397 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.397 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.397 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.397 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.397 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.398 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.398 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.398 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.398 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.398 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.398 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.398 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.398 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.398 INFO:teuthology.orchestra.run.smithi016.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:58:02.398 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:58:02.398 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.398 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.398 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.399 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:58:02.399 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.399 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.399 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.399 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:58:02.399 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.399 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.399 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.399 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.399 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.399 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.399 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.400 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.400 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.400 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.400 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.400 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.400 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.400 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.400 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.400 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.400 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.400 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.400 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-21T03:58:02.400 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.400 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.401 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.401 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.401 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.401 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.401 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.401 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.401 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.401 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.401 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.401 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.401 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.401 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.401 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.401 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.402 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.402 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.402 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.402 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.402 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.402 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.402 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.402 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.402 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.402 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.402 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.402 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.402 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.402 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.403 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.403 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.403 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.403 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.403 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.403 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.403 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.403 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.403 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.403 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.403 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.403 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.403 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.403 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.404 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.404 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.404 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.404 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.404 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.404 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.404 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.404 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.404 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.404 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.404 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.404 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.404 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.405 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.405 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.405 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.405 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.405 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.405 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.405 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.405 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.405 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.405 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.405 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.405 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.405 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.406 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.406 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.406 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.406 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.406 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.406 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.406 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.406 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.406 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.406 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.406 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-21T03:58:02.406 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-21T03:58:02.406 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.407 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-21T03:58:02.407 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.407 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.407 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.407 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:58:02.407 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.407 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.407 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.407 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.407 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.407 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-21T03:58:02.408 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.408 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.408 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.408 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.408 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.408 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.408 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.408 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.408 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.408 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.408 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.408 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.408 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.408 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.408 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.408 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.409 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.409 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.409 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.409 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.409 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.409 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.409 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.409 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.409 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.409 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.409 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.409 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.410 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.410 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.410 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.410 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.410 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.410 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.410 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.410 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.410 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.410 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.411 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.411 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.411 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.411 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.411 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.411 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.411 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.411 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.411 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.411 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.411 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.411 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.411 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.412 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.412 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.412 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.412 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.412 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.412 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.412 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.412 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.412 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.412 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.412 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.413 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.413 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-21T03:58:02.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-21T03:58:02.413 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.413 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.413 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.413 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.413 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.413 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.413 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.413 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.413 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.413 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.414 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-21T03:58:02.414 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-21T03:58:02.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.414 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.414 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.414 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-21T03:58:02.414 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.414 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.414 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.414 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.414 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.414 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.414 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.415 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-21T03:58:02.415 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-21T03:58:02.415 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.415 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.415 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.415 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.415 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.415 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.415 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.415 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.415 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.415 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.416 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-21T03:58:02.416 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-21T03:58:02.416 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.416 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.416 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.416 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.416 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.416 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.416 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.416 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.416 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.416 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.417 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-21T03:58:02.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-21T03:58:02.417 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:02.417 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.417 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.417 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.417 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.417 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.417 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.417 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.417 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.417 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.417 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.418 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-21T03:58:02.418 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-21T03:58:02.418 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.418 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.418 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.418 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.418 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.418 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.418 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.418 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.418 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.418 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.418 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.418 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.419 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-21T03:58:02.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-21T03:58:02.419 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.419 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.419 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.419 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.419 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.419 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.419 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.419 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.419 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.420 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-21T03:58:02.420 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-21T03:58:02.420 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.420 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.420 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.420 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.420 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.420 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.420 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.420 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.420 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.420 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.420 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.421 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-21T03:58:02.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-21T03:58:02.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.421 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.421 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.421 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.421 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:58:02.421 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-21T03:58:02.421 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.421 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.421 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.421 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.422 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.422 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.422 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.422 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.422 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.422 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.422 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.422 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.422 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.422 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.422 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.422 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.423 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-21T03:58:02.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-21T03:58:02.423 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.423 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.423 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.423 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.423 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.423 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.423 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.423 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.423 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.424 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.424 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-21T03:58:02.424 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-21T03:58:02.424 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.424 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.424 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.424 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.424 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.424 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.424 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.424 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.424 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.425 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.425 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.425 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.425 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-21T03:58:02.425 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-21T03:58:02.425 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.425 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.425 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.425 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.425 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.425 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.425 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.425 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.425 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.425 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.426 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.426 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.426 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.426 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.426 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smb", 2024-09-21T03:58:02.426 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.426 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.426 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.426 INFO:teuthology.orchestra.run.smithi016.stdout: "internal_store_backend": { 2024-09-21T03:58:02.426 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "internal_store_backend", 2024-09-21T03:58:02.426 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.426 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.426 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.427 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.427 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.427 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.427 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.427 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:58:02.427 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.427 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.427 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.427 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.427 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.427 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.427 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.427 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.427 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.428 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.428 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.428 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.428 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.428 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.428 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.428 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.428 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.428 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.428 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.428 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.428 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.428 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.428 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.428 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.429 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.429 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.429 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.429 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.429 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.429 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.429 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.429 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.429 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.429 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.429 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.429 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.429 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.430 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.430 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.430 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.430 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.430 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.430 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.430 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.430 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.430 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.430 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.430 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.430 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.431 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.431 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.431 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.431 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.431 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.431 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.431 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.431 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.431 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.431 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.431 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.431 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.431 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.432 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.432 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.432 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.432 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.432 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.432 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.432 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.432 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.432 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.432 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.432 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.432 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.432 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: "update_orchestration": { 2024-09-21T03:58:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "update_orchestration", 2024-09-21T03:58:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:58:02.434 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.434 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.434 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.434 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.434 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.434 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.434 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-21T03:58:02.434 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.434 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.434 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.434 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-21T03:58:02.434 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-21T03:58:02.434 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.435 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.435 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.435 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.435 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.435 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.435 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.435 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:58:02.435 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.435 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.435 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.435 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.435 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-21T03:58:02.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-21T03:58:02.435 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.435 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.436 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.436 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.436 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.436 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.436 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.436 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:58:02.436 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.436 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.436 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.436 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.436 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.436 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.437 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.437 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.437 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.437 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.437 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.437 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.437 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.437 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.437 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.437 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.437 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.437 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.437 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.437 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.438 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.438 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.438 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.438 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.438 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.438 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.438 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.438 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.438 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.438 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.438 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.438 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.438 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.438 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.439 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.439 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.439 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.439 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.439 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.439 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.439 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.439 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.439 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.439 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.439 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.439 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.439 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.439 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.440 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.440 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.440 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.440 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.440 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.440 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.440 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.440 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.440 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.440 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.440 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.440 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.440 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.440 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.440 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.441 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.441 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.441 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.441 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.441 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.441 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.441 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.441 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.441 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.441 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.441 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.441 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.441 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.441 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.441 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.441 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.442 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.442 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.442 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.442 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.442 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.442 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.442 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.442 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.442 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.442 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-21T03:58:02.442 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.442 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.442 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.442 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.443 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.443 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.443 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.443 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.443 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.443 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.443 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.443 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.443 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.443 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.443 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.443 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.443 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.443 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.443 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.443 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.444 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.444 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.444 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.444 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.444 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.444 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.444 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.444 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.444 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.444 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.444 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.444 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.444 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.444 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.446 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.446 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.446 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.446 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.446 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.446 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.446 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.446 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.446 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.446 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.446 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.446 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.446 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.447 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.447 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.447 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.447 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.447 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.447 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.447 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.447 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.448 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.448 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.448 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.448 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.448 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.448 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.448 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.448 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.448 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.448 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.448 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-21T03:58:02.448 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.448 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.449 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.449 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.449 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.449 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.449 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.449 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.449 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.449 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.449 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.449 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.449 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.449 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.449 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.449 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.450 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.450 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.450 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.450 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.450 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.450 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.450 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.450 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.450 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.450 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.450 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.450 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.451 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.451 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.451 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.451 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.451 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.451 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.451 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.451 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.451 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.451 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.451 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.451 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.451 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.452 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.452 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.452 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.452 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.452 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.452 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.452 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.452 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.452 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.452 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.452 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.452 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.452 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.453 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.453 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.453 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.453 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.453 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.453 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.453 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.453 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.453 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.453 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.453 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.453 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.453 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.454 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.454 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.454 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.454 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.454 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.454 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.454 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.454 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.454 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.454 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.454 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.454 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.454 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.455 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.455 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.455 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.455 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.455 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-21T03:58:02.455 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.455 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.455 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.455 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-21T03:58:02.455 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-21T03:58:02.455 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.455 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.455 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.456 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:58:02.456 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.456 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.456 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.456 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.456 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.456 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.456 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.456 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:58:02.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:58:02.456 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.456 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.456 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.457 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-21T03:58:02.457 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.457 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.457 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.457 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.457 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.457 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.457 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.457 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.457 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.457 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.457 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.458 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.458 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.458 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.458 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.458 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.458 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.458 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.458 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.458 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.458 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.458 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.458 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.458 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.458 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.458 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.459 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.459 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.459 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.459 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.459 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.459 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.459 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.459 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.459 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.459 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.459 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.459 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.460 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.460 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.460 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.460 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.460 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.460 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.460 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.460 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.460 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.460 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.460 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.460 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.460 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.461 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.461 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.461 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.461 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.461 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.461 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.461 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.461 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.461 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.461 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.462 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.462 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.462 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.462 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.462 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.462 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.462 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.462 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.462 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.462 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.462 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.462 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.462 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.463 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.463 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.463 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.463 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.463 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.463 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.463 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.463 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.463 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.463 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.463 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.463 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-21T03:58:02.463 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.464 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.464 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.464 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-21T03:58:02.464 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-21T03:58:02.464 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.464 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.464 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.464 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.464 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.464 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.464 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.464 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:58:02.464 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.464 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.464 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.465 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-21T03:58:02.465 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-21T03:58:02.465 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.465 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.465 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.465 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.465 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.465 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.465 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.465 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:58:02.465 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.465 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.465 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.466 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-21T03:58:02.466 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-21T03:58:02.466 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.466 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.466 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.466 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.466 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.466 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.466 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.466 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:58:02.466 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.466 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.467 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.467 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-21T03:58:02.467 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-21T03:58:02.467 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.467 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.467 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.467 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.467 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.467 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.467 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.467 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:58:02.467 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.467 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.468 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.468 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-21T03:58:02.468 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-21T03:58:02.468 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.468 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.468 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.468 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.468 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.468 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.468 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.468 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:58:02.468 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.468 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.469 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.469 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-21T03:58:02.469 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-21T03:58:02.469 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.469 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.469 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.469 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.469 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.469 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.469 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.469 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.469 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.469 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.469 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.470 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-21T03:58:02.470 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-21T03:58:02.470 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.470 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.470 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.470 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.470 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.470 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.470 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.470 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.470 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.470 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.470 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.470 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-21T03:58:02.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-21T03:58:02.471 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.471 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.471 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.471 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:58:02.471 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.471 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.471 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.471 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.471 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.471 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.471 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.471 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-21T03:58:02.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-21T03:58:02.471 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.472 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.472 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.472 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.472 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.472 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.472 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.472 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.472 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.472 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.472 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.472 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:58:02.472 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:58:02.472 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.473 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.473 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.473 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-21T03:58:02.473 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-21T03:58:02.473 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.473 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.473 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.473 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.473 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.473 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.473 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-21T03:58:02.473 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-21T03:58:02.474 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.474 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.474 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.474 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-21T03:58:02.474 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.474 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.474 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.474 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.474 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.474 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.474 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.474 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-21T03:58:02.475 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-21T03:58:02.475 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.475 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.475 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.475 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.475 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.475 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.475 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.475 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.475 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.475 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.475 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-21T03:58:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-21T03:58:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.477 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.477 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.477 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.477 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.477 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.477 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.477 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.477 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.477 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.477 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.477 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.477 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.477 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.477 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.477 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.478 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.478 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.478 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.478 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.478 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.478 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.478 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.478 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.478 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.478 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.478 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.478 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.479 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.479 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.479 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.479 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.479 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.479 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.479 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.479 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.479 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.479 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.479 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.479 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.479 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.479 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.480 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.480 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.480 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.480 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.480 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.480 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.480 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.480 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.480 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.480 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.480 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.480 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.480 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-21T03:58:02.482 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-21T03:58:02.482 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.482 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.482 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.482 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.482 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.482 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.482 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.482 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.482 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.482 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.482 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.482 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-21T03:58:02.483 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-21T03:58:02.483 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.483 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.483 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.483 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.483 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.483 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.483 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.483 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.483 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.483 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.483 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.483 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.484 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.484 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.484 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.484 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.484 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.484 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.484 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.484 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.484 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.484 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.484 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.484 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.484 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-21T03:58:02.484 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-21T03:58:02.485 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.485 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.485 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.485 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:58:02.485 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.485 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.485 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.485 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.485 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.485 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.485 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.485 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.485 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.486 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.486 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-21T03:58:02.486 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.486 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.486 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.486 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.486 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.486 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.486 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.486 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.486 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.486 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.486 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.486 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.487 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.487 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.487 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.487 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.487 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.487 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.487 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.487 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.487 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.487 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.487 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.487 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.487 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.487 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.488 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.488 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.488 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.488 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.488 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.488 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.488 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.488 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.488 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.488 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.488 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.488 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.488 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.488 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.488 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.489 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.489 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.489 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.489 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.489 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.489 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.489 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.489 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.489 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.489 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.489 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.489 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.489 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.489 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.489 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.490 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.490 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.490 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.490 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.490 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.490 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.490 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.490 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.490 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.490 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.490 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.490 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.490 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.491 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.491 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.491 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.491 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.491 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.491 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.491 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.491 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.491 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.491 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.491 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.491 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.491 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.492 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.492 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.492 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.492 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-21T03:58:02.492 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.492 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.492 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.492 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.492 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.492 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.492 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.492 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.492 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.492 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.493 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.493 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.493 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.493 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.493 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.493 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.493 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.493 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.493 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.493 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.493 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.493 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.493 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.494 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.494 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.494 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.494 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.494 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.494 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.494 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.494 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.494 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.494 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.494 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.494 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.495 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.495 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.495 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.495 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.495 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.495 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.495 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.495 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.495 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.495 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.495 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.495 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.495 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.495 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.495 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.496 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.496 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.496 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.496 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.496 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.496 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.496 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.496 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.496 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.496 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.496 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.496 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.497 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.497 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.497 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.497 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.497 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-21T03:58:02.497 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-21T03:58:02.497 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.497 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.497 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.497 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-21T03:58:02.497 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.497 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.497 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.498 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:58:02.498 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.498 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.498 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.498 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-21T03:58:02.498 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-21T03:58:02.498 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.498 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.498 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.498 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.498 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.498 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.498 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.499 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-21T03:58:02.499 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.499 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.499 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.499 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-21T03:58:02.499 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:58:02.499 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.499 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.499 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.499 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.499 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.499 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.499 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.500 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:58:02.500 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.500 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.500 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.500 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:58:02.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:58:02.500 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.500 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.500 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.500 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.500 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.500 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:58:02.501 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.501 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.501 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.501 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.501 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.501 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.501 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.501 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.501 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.501 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.501 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.501 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.501 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.502 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.502 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.502 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.502 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.502 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.502 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.502 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-21T03:58:02.502 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.502 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.502 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.502 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-21T03:58:02.502 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-21T03:58:02.502 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:58:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-21T03:58:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-21T03:58:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-21T03:58:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.504 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.504 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.504 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.504 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.504 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.504 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.504 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.504 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.504 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.504 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:58:02.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:58:02.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.505 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:58:02.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.505 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.505 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.505 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.505 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.506 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.506 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.506 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.506 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.506 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.506 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.506 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.506 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.506 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.506 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.507 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.507 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.507 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.507 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.507 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.507 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.507 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.507 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.507 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.507 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.507 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.507 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.507 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.508 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.508 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.508 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.508 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.508 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.508 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.508 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.508 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.508 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.508 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.508 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.509 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.509 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.509 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.509 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.509 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.509 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.509 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.509 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.509 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.509 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.509 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.510 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.510 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.510 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.510 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.510 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.510 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.510 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.510 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.510 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.510 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.510 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.510 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.511 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.511 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.511 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.511 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.511 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.511 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.511 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.511 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.511 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-21T03:58:02.511 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-21T03:58:02.511 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.511 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.511 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.512 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.512 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.512 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.512 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.512 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.512 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.512 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.512 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-21T03:58:02.512 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-21T03:58:02.512 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.512 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.512 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.512 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-21T03:58:02.513 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.513 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.513 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.513 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.513 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.513 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.513 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.513 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-21T03:58:02.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-21T03:58:02.513 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.513 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.514 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:58:02.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.514 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.514 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.514 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.514 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.514 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.514 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.514 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.514 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.514 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:58:02.514 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.514 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.515 INFO:teuthology.orchestra.run.smithi016.stdout: "modules": [ 2024-09-21T03:58:02.515 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-21T03:58:02.515 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard", 2024-09-21T03:58:02.515 INFO:teuthology.orchestra.run.smithi016.stdout: "iostat", 2024-09-21T03:58:02.515 INFO:teuthology.orchestra.run.smithi016.stdout: "nfs", 2024-09-21T03:58:02.515 INFO:teuthology.orchestra.run.smithi016.stdout: "restful" 2024-09-21T03:58:02.515 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.515 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-21T03:58:02.515 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.515 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-21T03:58:02.515 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.516 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.516 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.516 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:58:02.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:58:02.516 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.517 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.517 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.517 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:58:02.517 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.517 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.518 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.518 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:58:02.518 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.518 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.518 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.520 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.520 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.520 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.520 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.520 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.520 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.521 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.521 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.521 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.521 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.521 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.521 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.522 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.522 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.522 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.522 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.523 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.523 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.523 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.523 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.523 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.523 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.524 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.524 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.524 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.524 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.524 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.524 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.525 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.525 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.525 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.525 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.525 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.525 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.526 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.526 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.526 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.526 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.526 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.526 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.527 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.527 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.527 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.527 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.527 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.527 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.527 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.528 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.528 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.528 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.528 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.528 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.528 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.529 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.529 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.529 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.529 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.529 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.529 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.530 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.530 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-21T03:58:02.530 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-21T03:58:02.530 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.530 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.530 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.531 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.531 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.531 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.531 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.531 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:58:02.531 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.531 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.532 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.532 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-21T03:58:02.532 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-21T03:58:02.532 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.532 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.532 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.533 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-21T03:58:02.533 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.533 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.533 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.533 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-21T03:58:02.533 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.533 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.534 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.534 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-21T03:58:02.534 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-21T03:58:02.534 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.534 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.534 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.535 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.535 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.535 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.535 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.535 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-21T03:58:02.535 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.536 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.536 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.536 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-21T03:58:02.536 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-21T03:58:02.536 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.536 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.537 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.537 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.537 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.537 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.537 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.537 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-21T03:58:02.537 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.538 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.538 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.538 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-21T03:58:02.538 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-21T03:58:02.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.538 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.539 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.539 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-21T03:58:02.539 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.539 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.539 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.539 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-21T03:58:02.539 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.540 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.540 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.540 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-21T03:58:02.540 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-21T03:58:02.540 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.540 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.541 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.541 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.541 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.541 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.541 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:58:02.541 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.542 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.542 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.542 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-21T03:58:02.542 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-21T03:58:02.542 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.542 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.543 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.543 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.543 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.543 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.543 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.543 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:58:02.543 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.544 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.544 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.544 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-21T03:58:02.544 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-21T03:58:02.544 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.544 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.545 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.545 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.545 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.545 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.545 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.545 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-21T03:58:02.546 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.546 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.546 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.546 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.546 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.546 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.546 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.547 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.547 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.547 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.547 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.547 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.547 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.548 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.548 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.548 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.548 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.548 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.548 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.548 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.549 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.549 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-21T03:58:02.549 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.549 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.549 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.549 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-21T03:58:02.549 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-21T03:58:02.550 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.550 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.550 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.550 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.550 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.550 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.551 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.551 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:58:02.551 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.551 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.551 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.551 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-21T03:58:02.552 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-21T03:58:02.552 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.552 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.552 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.552 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-21T03:58:02.552 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.553 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.553 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.553 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:58:02.553 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:58:02.553 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.553 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.554 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-21T03:58:02.554 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-21T03:58:02.554 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:58:02.554 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.554 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.554 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.554 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-21T03:58:02.555 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-21T03:58:02.555 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.555 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:58:02.555 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:58:02.555 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.555 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.556 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.556 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-21T03:58:02.556 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:58:02.556 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:58:02.556 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.556 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.556 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-21T03:58:02.557 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:58:02.557 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-21T03:58:02.557 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.557 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:58:02.557 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.557 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.558 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.558 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.558 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-21T03:58:02.558 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-21T03:58:02.558 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.558 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.558 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.559 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:58:02.559 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.559 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.559 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.559 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:58:02.559 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:58:02.559 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.560 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.560 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-21T03:58:02.560 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-21T03:58:02.560 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:02.560 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.560 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.561 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-21T03:58:02.561 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-21T03:58:02.561 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-21T03:58:02.561 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.561 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:58:02.561 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:58:02.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.562 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.562 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-21T03:58:02.562 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-21T03:58:02.562 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.563 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.563 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.563 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-21T03:58:02.563 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.564 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.564 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:58:02.564 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:58:02.564 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.564 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.565 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-21T03:58:02.565 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-21T03:58:02.565 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:58:02.565 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.566 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.566 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.566 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-21T03:58:02.566 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-21T03:58:02.566 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.567 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:58:02.567 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:58:02.567 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.567 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.567 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.568 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.568 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.568 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.568 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.568 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.569 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.569 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.569 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.569 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.569 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.570 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.570 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.570 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.570 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.570 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.571 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.571 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.571 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.571 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.572 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.572 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.572 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.572 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.573 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.573 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.573 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.573 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.573 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.574 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.574 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.574 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.574 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.574 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.575 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.575 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.575 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.575 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.575 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.575 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.576 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.576 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.576 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.576 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.576 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.577 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.577 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.577 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.577 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.578 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.578 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.578 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.578 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.578 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.579 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.579 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.579 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.579 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.580 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.580 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.580 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.580 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.580 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.581 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.581 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-21T03:58:02.581 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-21T03:58:02.581 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:02.582 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.582 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.582 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.582 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.582 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.582 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.582 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:58:02.582 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.582 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.582 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.582 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-21T03:58:02.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-21T03:58:02.582 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.583 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.583 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-21T03:58:02.583 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.583 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.583 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.583 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-21T03:58:02.583 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-21T03:58:02.583 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-21T03:58:02.583 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-21T03:58:02.583 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-21T03:58:02.583 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.583 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-21T03:58:02.583 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.584 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.584 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.584 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.584 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-21T03:58:02.584 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-21T03:58:02.584 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.584 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.584 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.584 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.584 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.584 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.584 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:58:02.584 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.584 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.585 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.585 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:58:02.585 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:58:02.585 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.585 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.585 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.585 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:58:02.585 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.585 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.585 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.585 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:58:02.585 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.585 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.586 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.586 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.586 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.586 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.586 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.586 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.586 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.586 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.586 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.586 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.586 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.586 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.586 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.586 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.587 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.587 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-21T03:58:02.587 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-21T03:58:02.587 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.587 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:58:02.587 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:58:02.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.587 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.587 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:58:02.587 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:58:02.587 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.587 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.588 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.588 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-21T03:58:02.588 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:58:02.588 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:58:02.588 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.588 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.588 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:02.588 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.588 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.588 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.588 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:58:02.588 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.588 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.588 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.588 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.589 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.589 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-21T03:58:02.589 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.589 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.589 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.589 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-21T03:58:02.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-21T03:58:02.589 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:02.589 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.589 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.589 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-21T03:58:02.589 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.589 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.590 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.590 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:58:02.590 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.590 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.590 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.590 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-21T03:58:02.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-21T03:58:02.590 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.590 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.590 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.590 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-21T03:58:02.590 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.590 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.591 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.591 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:58:02.591 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.591 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.591 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.591 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.591 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-21T03:58:02.591 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-21T03:58:02.591 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.591 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.591 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.591 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-21T03:58:02.591 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.591 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.592 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.592 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:58:02.592 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.592 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.592 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.592 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-21T03:58:02.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-21T03:58:02.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.592 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.592 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.592 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.593 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.593 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:58:02.593 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:58:02.593 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.593 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.593 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-21T03:58:02.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-21T03:58:02.593 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.593 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.593 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.593 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:58:02.593 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.593 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.593 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.594 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:58:02.594 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.594 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.594 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.594 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:58:02.594 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:58:02.594 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:02.594 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.594 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.594 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-21T03:58:02.594 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.595 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:58:02.595 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.595 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.595 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.595 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-21T03:58:02.595 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-21T03:58:02.595 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.595 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.595 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.595 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.595 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.595 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.595 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.596 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-21T03:58:02.596 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-21T03:58:02.596 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-21T03:58:02.596 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:58:02.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.596 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.596 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.596 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-21T03:58:02.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-21T03:58:02.596 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.596 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.596 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.597 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.597 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.597 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.597 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.597 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:58:02.597 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.597 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.597 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-21T03:58:02.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-21T03:58:02.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.597 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.597 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.597 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.598 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.598 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.598 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.598 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:58:02.598 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.598 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.598 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.598 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-21T03:58:02.598 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-21T03:58:02.598 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.598 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.598 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.598 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.599 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.599 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.599 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.599 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:58:02.599 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.599 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.599 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-21T03:58:02.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-21T03:58:02.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.599 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.599 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.599 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:58:02.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.600 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.600 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.600 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:58:02.600 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.600 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.600 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.600 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-21T03:58:02.600 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-21T03:58:02.600 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.600 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.600 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.600 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:58:02.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.601 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.601 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.601 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:58:02.601 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.601 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.601 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.601 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-21T03:58:02.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:58:02.601 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.601 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.601 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.601 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:58:02.601 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.601 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.602 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.602 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-21T03:58:02.602 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.602 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.602 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.602 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-21T03:58:02.602 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-21T03:58:02.602 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.602 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.602 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.602 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:58:02.602 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.602 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.603 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:58:02.603 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.603 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.603 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.603 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-21T03:58:02.603 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-21T03:58:02.603 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.603 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.603 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.603 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:58:02.603 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.603 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.603 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.604 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-21T03:58:02.604 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.604 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.604 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.604 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-21T03:58:02.604 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:58:02.604 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.604 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.604 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.604 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:58:02.604 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.604 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.604 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.604 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:58:02.605 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.605 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.605 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.605 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-21T03:58:02.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:58:02.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.605 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.605 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:58:02.605 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.605 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.605 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.605 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:58:02.605 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.606 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.606 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-21T03:58:02.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:58:02.606 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.606 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.606 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.606 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:58:02.606 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.606 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.606 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-21T03:58:02.606 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.607 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.607 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-21T03:58:02.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-21T03:58:02.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.607 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.607 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:58:02.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.607 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.607 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.607 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-21T03:58:02.607 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.607 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.607 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.608 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-21T03:58:02.608 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-21T03:58:02.608 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.608 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.608 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:58:02.608 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.608 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-21T03:58:02.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.608 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.609 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nginx": { 2024-09-21T03:58:02.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nginx", 2024-09-21T03:58:02.609 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.609 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.609 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.609 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:58:02.609 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.609 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nginx container image", 2024-09-21T03:58:02.609 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.609 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.609 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.609 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-21T03:58:02.610 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-21T03:58:02.610 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:58:02.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.610 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:58:02.610 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.610 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.610 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.610 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-21T03:58:02.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-21T03:58:02.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.611 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.611 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.611 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:58:02.611 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.611 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.611 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.611 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-21T03:58:02.611 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.611 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.611 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:58:02.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:58:02.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.612 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.612 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:58:02.612 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.612 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.612 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:58:02.612 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.612 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.612 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.612 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.613 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-21T03:58:02.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-21T03:58:02.613 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.613 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.613 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.613 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:58:02.613 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.613 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.613 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.613 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:58:02.613 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.613 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.613 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.613 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-21T03:58:02.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-21T03:58:02.614 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.614 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.614 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.614 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:58:02.614 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.614 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.614 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.614 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-21T03:58:02.614 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.614 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-21T03:58:02.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-21T03:58:02.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.615 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:58:02.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.615 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:58:02.615 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.615 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.615 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.615 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.615 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba_metrics": { 2024-09-21T03:58:02.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:58:02.616 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.616 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.616 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:58:02.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.616 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.616 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:58:02.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.616 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.616 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-21T03:58:02.616 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:58:02.617 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.617 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.617 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.617 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:58:02.617 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.617 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.617 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.617 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:58:02.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.617 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.617 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-21T03:58:02.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-21T03:58:02.617 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.618 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.618 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.618 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.618 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.618 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.618 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.618 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:58:02.618 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.618 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-21T03:58:02.618 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:58:02.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:58:02.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.619 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:58:02.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.619 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.619 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:58:02.619 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:58:02.619 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-21T03:58:02.620 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.620 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.620 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:58:02.620 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.620 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.620 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-21T03:58:02.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-21T03:58:02.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.621 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-21T03:58:02.621 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.621 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.621 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:58:02.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.621 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-21T03:58:02.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-21T03:58:02.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.621 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.621 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-21T03:58:02.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.622 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.622 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:58:02.622 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.622 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.622 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.622 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-21T03:58:02.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-21T03:58:02.622 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.622 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.623 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.623 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:58:02.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.623 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.623 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.623 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-21T03:58:02.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-21T03:58:02.623 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.623 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.623 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.623 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:58:02.623 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:58:02.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.624 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-21T03:58:02.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:58:02.624 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.624 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.624 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:58:02.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.625 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:58:02.625 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.625 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.625 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.625 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.625 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-21T03:58:02.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-21T03:58:02.625 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:58:02.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.626 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:58:02.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.626 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-21T03:58:02.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-21T03:58:02.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.626 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.626 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:58:02.627 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.627 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-21T03:58:02.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-21T03:58:02.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:58:02.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.628 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.628 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.628 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.628 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.628 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.628 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.628 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.628 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.629 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.629 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.629 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.629 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.629 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.629 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.629 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.629 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.629 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.629 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.629 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-21T03:58:02.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-21T03:58:02.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.630 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.630 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.630 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.630 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.630 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:58:02.630 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.630 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.630 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.630 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.631 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.631 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.631 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.631 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:58:02.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.631 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.632 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.632 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.632 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.632 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.632 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.632 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.632 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.632 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.632 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.633 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.633 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.633 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.633 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.633 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.634 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:58:02.634 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:58:02.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.634 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:58:02.634 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.635 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:58:02.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:58:02.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.635 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.635 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-21T03:58:02.635 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.635 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.635 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.635 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:58:02.635 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.635 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.635 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.635 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-21T03:58:02.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-21T03:58:02.636 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.636 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.636 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.636 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:02.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:58:02.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-21T03:58:02.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-21T03:58:02.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:02.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:58:02.637 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.637 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.637 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-21T03:58:02.637 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-21T03:58:02.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.638 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.638 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.638 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.638 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-21T03:58:02.638 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.638 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.638 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.638 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-21T03:58:02.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-21T03:58:02.638 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.639 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.639 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-21T03:58:02.639 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.639 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.639 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.639 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-21T03:58:02.639 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-21T03:58:02.639 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:58:02.639 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.639 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-21T03:58:02.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-21T03:58:02.640 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.640 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.640 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.640 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-21T03:58:02.640 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.640 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:58:02.640 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.640 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.640 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.640 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.640 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-21T03:58:02.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:58:02.641 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.641 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.641 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.641 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:58:02.641 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.641 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:58:02.641 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.641 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.641 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.641 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-21T03:58:02.641 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-21T03:58:02.642 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.642 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.642 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.642 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.642 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.642 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:58:02.642 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.642 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.642 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.642 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-21T03:58:02.642 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-21T03:58:02.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.643 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.643 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.643 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.643 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.643 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.643 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.643 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:58:02.643 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.643 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.643 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.643 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-21T03:58:02.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-21T03:58:02.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.644 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.644 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.644 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.644 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.644 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.644 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.644 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:58:02.644 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.644 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.644 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-21T03:58:02.644 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-21T03:58:02.644 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.644 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.645 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.645 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.645 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.645 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.645 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:58:02.645 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.645 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.645 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.645 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-21T03:58:02.645 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:58:02.645 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.645 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.645 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.646 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.646 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.646 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:58:02.646 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.646 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.646 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.646 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-21T03:58:02.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-21T03:58:02.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-21T03:58:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:58:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.648 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.648 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.648 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.648 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-21T03:58:02.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-21T03:58:02.648 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.648 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.648 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.648 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.649 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:58:02.649 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.649 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.649 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.649 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:58:02.649 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:58:02.649 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.649 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.649 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.649 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-21T03:58:02.649 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:58:02.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.650 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.650 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.650 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_keepalive_interval": { 2024-09-21T03:58:02.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:58:02.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-21T03:58:02.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.650 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.651 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:58:02.651 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.651 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.651 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.651 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-21T03:58:02.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-21T03:58:02.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.651 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.651 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.651 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.651 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.651 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.651 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.651 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:58:02.651 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.652 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.652 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.652 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.652 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-21T03:58:02.652 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-21T03:58:02.652 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.652 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.652 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.652 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.652 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.652 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.652 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.652 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:58:02.652 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.653 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.653 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.653 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-21T03:58:02.653 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:58:02.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.653 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.653 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.653 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:58:02.653 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.653 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-21T03:58:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:58:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:58:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.655 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-21T03:58:02.655 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:58:02.655 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.655 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.655 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.655 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.655 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.655 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:58:02.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.656 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.656 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.656 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-21T03:58:02.656 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.656 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.656 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.656 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.656 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.656 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.656 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.656 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.657 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.657 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.657 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.657 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.657 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.657 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.657 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.657 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.657 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.657 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.657 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.657 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.657 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.657 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.658 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.658 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.658 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.658 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.658 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.658 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.659 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.659 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.659 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.659 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.659 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.659 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.659 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.659 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.659 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.659 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.659 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.659 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.659 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.660 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.660 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.660 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.660 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.660 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.660 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.661 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.661 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.661 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.661 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-21T03:58:02.661 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-21T03:58:02.661 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.661 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.661 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.661 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-21T03:58:02.661 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.661 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.662 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:58:02.662 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.662 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.662 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.663 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.663 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.663 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-21T03:58:02.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-21T03:58:02.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.663 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.663 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-21T03:58:02.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.663 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.663 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:58:02.664 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.664 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.664 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.664 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.664 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-21T03:58:02.664 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.664 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.664 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.664 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:58:02.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:58:02.664 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.664 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.665 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:02.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.665 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:58:02.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:58:02.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.665 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.666 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.666 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.666 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.666 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.666 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:58:02.666 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:58:02.666 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.666 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.666 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.666 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.667 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.667 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.667 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.667 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.667 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.667 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.667 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:58:02.667 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:58:02.667 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.667 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.667 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.667 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.668 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.668 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.668 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.668 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.668 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.668 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.668 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:58:02.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:58:02.668 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.668 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.668 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.668 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.668 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.668 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.669 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.669 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.669 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:58:02.669 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:58:02.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.669 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.669 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.669 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.670 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.670 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.670 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:58:02.670 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:58:02.670 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.670 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.670 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.670 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.670 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.670 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.670 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.671 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.671 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.671 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.671 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:58:02.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:58:02.671 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.671 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.671 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.671 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.671 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.671 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.672 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.672 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.672 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.672 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:58:02.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:58:02.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.672 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.672 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.672 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.672 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.672 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.672 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.673 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.673 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.673 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:58:02.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:58:02.673 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.673 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.673 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.673 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.673 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.673 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.674 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.674 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:58:02.674 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:58:02.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.674 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.674 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.674 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.674 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.674 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.674 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.674 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.675 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:58:02.675 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:58:02.675 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.675 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.675 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.675 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.675 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.675 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.675 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.675 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.676 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.676 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:58:02.676 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:58:02.676 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.676 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.676 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.676 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.676 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.676 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.676 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.676 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.676 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.677 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:58:02.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:58:02.677 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.677 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.677 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.677 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.677 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.677 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.677 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.677 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.678 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:58:02.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:58:02.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.678 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.678 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.678 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:58:02.678 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.678 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.678 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.678 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.678 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.678 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.678 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.679 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:58:02.679 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:58:02.679 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.679 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.679 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.679 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.679 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.679 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.679 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.679 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.679 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.679 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.680 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-21T03:58:02.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:58:02.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.680 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.680 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.680 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.680 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.680 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.680 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.680 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.680 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:58:02.681 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:58:02.681 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.681 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.681 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.681 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:58:02.681 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.681 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.681 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.681 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.681 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.681 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:58:02.681 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:58:02.682 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.682 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.682 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.682 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.682 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.682 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.682 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.682 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.682 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.682 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.682 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:58:02.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:58:02.682 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.683 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.683 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.683 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.683 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.683 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.683 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.683 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.683 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.683 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:58:02.683 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:58:02.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.684 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.684 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.684 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.684 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.684 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.684 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.684 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.684 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.684 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.684 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:58:02.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:58:02.684 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.684 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.684 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.684 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.685 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.685 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.685 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.685 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.685 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.685 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.685 INFO:teuthology.orchestra.run.smithi016.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:58:02.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:58:02.685 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.685 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.685 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.685 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "[]", 2024-09-21T03:58:02.686 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.686 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.686 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.686 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.686 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.686 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.686 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.686 INFO:teuthology.orchestra.run.smithi016.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:58:02.686 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:58:02.686 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.686 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.686 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.686 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "{}", 2024-09-21T03:58:02.687 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.687 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.687 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.687 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.687 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.687 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:58:02.687 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:58:02.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.687 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.687 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.687 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.687 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.688 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.688 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.688 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.688 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.688 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.688 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.688 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:58:02.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:58:02.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.688 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.688 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.688 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.688 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.688 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.689 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.689 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.689 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.689 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.689 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.689 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:58:02.689 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:58:02.689 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.689 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.689 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.689 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.689 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.690 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.690 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.690 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.690 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.690 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.690 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:58:02.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:58:02.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.690 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.690 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.690 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.690 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.690 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.690 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.691 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.691 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.691 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.691 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:58:02.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:58:02.691 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.691 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.691 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.691 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.691 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.691 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.691 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.692 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.692 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.692 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.692 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.692 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:58:02.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:58:02.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.692 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.692 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.692 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.692 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.692 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.692 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.692 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.693 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.693 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.693 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.693 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:58:02.693 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:58:02.693 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.693 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.693 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.693 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.693 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.693 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.694 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.694 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.694 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.694 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:58:02.694 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:58:02.694 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.694 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.694 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.694 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.694 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.694 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.694 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.694 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.694 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.695 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.695 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.695 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:58:02.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:58:02.695 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.695 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.695 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.695 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.695 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.695 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.695 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.695 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.695 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.695 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.696 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.696 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:58:02.696 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:58:02.696 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.696 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.696 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.696 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.696 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.696 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.696 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.696 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.696 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.696 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.696 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.697 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:58:02.697 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:58:02.697 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.697 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.697 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.697 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:58:02.697 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.697 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.697 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.697 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.697 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.697 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.697 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.698 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:58:02.698 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:58:02.698 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.698 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.698 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.698 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:02.698 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.698 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.698 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.698 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.698 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.699 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:58:02.699 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:58:02.699 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.699 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.699 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.699 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-21T03:58:02.699 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.699 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.699 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.699 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.699 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.699 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.699 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.699 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:58:02.700 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:58:02.700 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.700 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.700 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.700 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-21T03:58:02.700 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.700 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.700 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.700 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.700 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.700 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.700 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:58:02.701 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:58:02.701 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.701 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.701 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.701 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.701 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.701 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.701 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.701 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.701 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.701 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:58:02.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:58:02.702 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.702 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.702 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.702 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:58:02.702 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.702 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.702 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.702 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.702 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.702 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:58:02.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:58:02.703 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.703 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.703 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.703 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.703 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.703 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.703 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.703 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.703 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.703 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.703 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.703 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:58:02.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:58:02.704 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.704 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.704 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.704 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.704 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.704 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.704 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.704 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.704 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.704 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:58:02.704 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:58:02.704 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.705 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.705 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.705 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.705 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.705 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.705 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.705 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.705 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.705 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:58:02.705 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:58:02.705 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.706 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.706 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.706 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.706 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.706 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.706 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.706 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.706 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.706 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:58:02.706 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:58:02.706 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.707 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.707 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:02.707 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.707 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.707 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.707 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.707 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.707 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.707 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.707 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:58:02.707 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:58:02.707 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.707 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.707 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.708 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:58:02.708 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.708 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.708 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.708 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.708 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.708 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.708 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.708 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-21T03:58:02.708 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-21T03:58:02.708 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.708 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.708 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.708 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.709 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.709 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.709 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.709 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.709 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.709 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.709 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.709 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-21T03:58:02.709 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-21T03:58:02.709 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.709 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.709 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.709 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.710 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.710 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.710 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.710 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.710 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.710 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.710 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-21T03:58:02.710 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-21T03:58:02.710 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.710 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.710 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.710 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.710 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.710 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.711 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.711 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:58:02.711 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.711 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.711 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.711 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-21T03:58:02.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-21T03:58:02.711 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.711 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.711 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.711 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-21T03:58:02.711 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.711 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.711 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-21T03:58:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-21T03:58:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.713 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.713 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.713 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.713 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.713 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.713 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.713 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.713 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.713 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.713 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.713 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.713 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.713 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.714 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.714 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.714 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.714 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.714 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.714 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.714 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.714 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.714 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.714 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.714 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.714 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.714 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.714 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.714 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.715 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.715 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.715 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.715 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.715 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.715 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.715 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.715 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.715 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.715 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.715 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.715 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.715 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.715 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.715 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.716 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.716 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.716 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.716 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.716 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.716 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.716 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.716 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.716 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.716 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.716 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.716 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.716 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.717 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.717 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.717 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.717 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.717 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.718 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.718 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.718 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.718 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.718 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.718 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.718 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.718 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.718 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.718 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.718 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-21T03:58:02.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-21T03:58:02.718 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.718 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.719 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.719 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.719 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.719 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.719 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.719 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-21T03:58:02.719 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.719 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.719 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.719 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:58:02.719 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:58:02.719 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.719 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.720 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.720 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.720 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.720 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.720 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.720 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.720 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.720 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.720 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.720 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:58:02.720 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:58:02.720 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.720 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.721 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-21T03:58:02.721 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.721 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.721 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.721 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.721 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.721 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.721 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.721 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:58:02.721 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:58:02.721 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.721 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.722 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.722 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-21T03:58:02.722 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.722 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.722 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.722 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.722 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.722 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.722 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.722 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.722 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.722 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.722 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.723 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.723 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.723 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.723 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.723 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.723 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.723 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.723 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.723 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-21T03:58:02.723 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-21T03:58:02.723 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.723 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.723 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.724 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.724 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.724 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.724 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.724 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.724 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.724 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.724 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.724 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-21T03:58:02.724 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-21T03:58:02.724 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.724 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.724 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.724 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-21T03:58:02.725 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.725 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.725 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.725 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.725 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.725 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.725 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.725 INFO:teuthology.orchestra.run.smithi016.stdout: "sso_oauth2": { 2024-09-21T03:58:02.725 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sso_oauth2", 2024-09-21T03:58:02.725 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.725 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.725 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.725 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.726 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.726 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.726 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.726 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.726 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.726 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.726 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.726 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-21T03:58:02.726 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-21T03:58:02.726 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.726 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.726 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.726 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-21T03:58:02.726 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.727 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.727 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.727 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.727 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-21T03:58:02.727 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.727 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.727 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.727 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.727 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.727 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-21T03:58:02.727 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-21T03:58:02.727 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.727 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.727 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.728 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:58:02.728 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-21T03:58:02.728 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-21T03:58:02.728 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.728 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.728 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.728 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.728 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.728 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-21T03:58:02.728 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-21T03:58:02.728 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.728 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.728 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.728 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.729 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.729 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.729 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.729 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.729 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.729 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.729 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.729 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.729 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.729 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-21T03:58:02.729 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.729 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.730 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.730 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-21T03:58:02.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-21T03:58:02.730 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.730 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.730 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.730 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.730 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.730 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.730 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.730 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-21T03:58:02.730 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.730 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.730 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.731 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.731 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.731 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.731 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.731 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.731 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.731 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.731 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.731 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.731 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.731 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.731 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.731 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.731 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.732 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.732 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.732 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.732 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.732 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.732 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.732 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.732 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.732 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.732 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.732 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.732 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.732 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.732 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.733 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.733 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.733 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.733 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.733 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.733 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.733 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.733 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.733 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.733 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.733 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.734 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.734 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.734 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.734 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.734 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.734 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.734 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.734 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.734 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.734 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.734 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.734 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.734 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.734 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.735 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.735 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.735 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.735 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.735 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.735 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.735 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.735 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.735 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.735 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.735 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.735 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.735 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.735 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-21T03:58:02.736 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-21T03:58:02.736 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.736 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.736 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.736 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-21T03:58:02.736 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.736 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.736 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.736 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:58:02.736 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.736 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.736 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.736 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.736 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-21T03:58:02.737 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-21T03:58:02.737 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.737 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.737 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.737 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-21T03:58:02.737 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.737 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.737 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.737 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:58:02.737 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.737 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.737 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.737 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.737 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-21T03:58:02.737 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-21T03:58:02.738 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.738 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.738 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.738 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-21T03:58:02.738 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.738 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.738 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.738 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:58:02.738 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.738 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.738 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.738 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.738 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-21T03:58:02.738 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-21T03:58:02.738 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.739 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.739 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.739 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-21T03:58:02.739 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.739 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.739 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.739 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:58:02.739 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.739 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.739 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.739 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-21T03:58:02.739 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-21T03:58:02.739 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.739 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.740 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.740 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.740 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.740 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.740 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.740 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:58:02.740 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.740 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.740 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.740 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.740 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:58:02.740 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:58:02.740 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.740 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.740 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.741 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:58:02.741 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.741 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.741 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.741 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:58:02.741 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.741 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.741 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.741 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.741 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.741 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.741 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.741 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.741 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.741 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.742 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.742 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.742 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.742 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.742 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.742 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.742 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.742 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.742 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-21T03:58:02.742 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-21T03:58:02.742 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.742 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.742 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.742 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-21T03:58:02.742 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:58:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-21T03:58:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.744 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.744 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.744 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.744 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.744 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.744 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.744 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.744 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.744 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.744 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.744 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.744 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.744 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.744 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.744 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.745 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.745 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.745 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.745 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.745 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.745 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.745 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.745 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.745 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.745 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.745 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.745 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.745 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.745 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.745 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.746 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.746 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.746 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.746 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.746 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.746 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.746 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.746 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.746 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.746 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.746 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.746 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.746 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.746 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.746 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.747 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.747 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.747 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.747 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.747 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.747 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.747 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.747 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.747 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.747 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.747 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.747 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.747 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.747 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.747 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-21T03:58:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-21T03:58:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-21T03:58:02.749 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.749 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.749 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.749 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.749 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.749 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.749 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.749 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-21T03:58:02.749 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-21T03:58:02.749 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.749 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.749 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.749 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-21T03:58:02.749 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.750 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.750 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.750 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.750 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.750 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.750 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.750 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.750 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:58:02.750 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:58:02.750 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.750 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.750 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.750 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:58:02.750 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.751 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.751 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.751 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.751 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.751 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.751 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.751 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.751 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.751 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.751 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.751 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.751 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.752 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.752 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.752 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.752 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.752 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.752 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.752 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.752 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.752 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.752 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-21T03:58:02.752 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-21T03:58:02.752 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:58:02.752 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.752 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-21T03:58:02.753 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-21T03:58:02.753 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.753 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.753 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.753 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-21T03:58:02.753 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.753 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.753 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.753 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:58:02.753 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.753 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.753 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.753 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-21T03:58:02.753 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-21T03:58:02.754 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.754 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.754 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.754 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-21T03:58:02.754 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.754 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.754 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.754 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:58:02.754 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.754 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.754 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.754 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-21T03:58:02.754 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-21T03:58:02.755 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.755 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.755 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.755 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.755 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.755 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.755 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.755 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:58:02.755 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.755 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.755 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.755 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.755 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:58:02.755 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:58:02.755 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.756 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.756 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.756 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-21T03:58:02.756 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-21T03:58:02.756 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.756 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.756 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:58:02.756 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.756 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.756 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.756 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.756 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.756 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.756 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.756 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.757 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.757 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.757 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.757 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.757 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.757 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.757 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.757 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.757 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.757 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.757 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.757 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.757 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.757 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.757 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.758 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.758 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.758 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.758 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.758 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.758 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.758 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.758 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.758 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.758 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.758 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.758 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.758 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.758 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.759 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.759 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.759 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.759 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.759 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.759 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.759 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.759 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.759 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.759 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.759 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.759 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.759 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.759 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.759 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.760 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.760 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.760 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.760 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.760 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.760 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.760 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.760 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.760 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.760 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.760 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.760 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.760 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.760 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.761 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.761 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.761 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.761 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.761 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.761 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-21T03:58:02.761 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-21T03:58:02.761 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.761 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.761 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.761 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.761 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.761 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.761 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.761 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:58:02.762 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.762 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.762 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.762 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.762 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-21T03:58:02.762 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-21T03:58:02.762 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.762 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.762 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.762 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-21T03:58:02.762 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.762 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.762 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.762 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-21T03:58:02.763 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.763 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.763 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.763 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.763 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.763 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.763 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.763 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.763 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.763 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.763 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.763 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.763 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.763 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.764 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.764 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.764 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-21T03:58:02.764 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-21T03:58:02.764 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.764 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.764 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.764 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-21T03:58:02.764 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.764 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.764 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.764 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:58:02.764 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.765 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.765 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.765 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.765 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-21T03:58:02.765 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-21T03:58:02.765 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.765 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.765 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.765 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:58:02.765 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:58:02.765 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-21T03:58:02.765 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.765 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:58:02.765 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.765 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.766 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.766 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-21T03:58:02.766 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-21T03:58:02.766 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.766 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.766 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.766 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.766 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.766 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.766 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.766 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:58:02.766 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.766 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.766 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.767 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-21T03:58:02.767 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-21T03:58:02.767 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.767 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.767 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.767 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-21T03:58:02.767 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.767 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.767 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.767 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:58:02.767 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.767 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.767 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.768 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.768 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.768 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.768 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.768 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-21T03:58:02.768 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.768 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.768 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.768 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.768 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.768 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.768 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.768 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.768 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.768 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.769 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.769 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.769 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.769 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.769 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.769 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.769 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.769 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.769 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.769 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.769 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.769 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.769 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.769 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.770 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.770 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.770 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.770 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.770 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.770 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.770 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.770 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.770 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.770 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.770 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.770 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.770 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.770 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.771 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.771 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.771 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.771 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.771 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.771 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.771 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.771 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.771 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.771 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.771 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.771 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.771 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.771 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.772 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.772 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.772 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.772 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.772 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.772 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.772 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.772 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.772 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.772 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.772 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.772 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.772 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.772 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.772 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.772 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.773 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.773 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.773 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.773 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.773 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.773 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.773 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.773 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.773 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.773 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.773 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.773 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.773 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.773 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.773 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.774 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.774 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.774 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.774 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-21T03:58:02.774 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.774 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.774 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.774 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.774 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.774 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.774 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.774 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.774 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.775 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.775 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.775 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.775 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.775 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.775 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.775 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.775 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.775 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.775 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.775 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.775 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.775 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.775 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.776 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.776 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.776 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.776 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.776 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.776 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.776 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.776 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.776 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.776 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.776 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.776 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.776 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.776 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.777 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.777 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.777 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.777 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.777 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.777 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.777 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.777 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.777 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.777 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.777 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.777 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.777 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.777 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.777 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.777 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.778 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.778 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.778 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.778 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.778 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.778 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.778 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.778 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.778 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.778 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.778 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.778 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.778 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.778 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.778 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-21T03:58:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-21T03:58:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:58:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-21T03:58:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:58:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-21T03:58:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-21T03:58:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-21T03:58:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:58:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-21T03:58:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-21T03:58:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-21T03:58:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-21T03:58:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:58:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:58:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.791 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.791 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.791 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.791 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.791 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.791 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.791 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.791 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.791 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.791 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.791 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.791 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.791 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.791 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.791 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.792 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.792 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.792 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.792 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.792 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.792 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.792 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.792 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.792 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.792 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.792 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.792 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.792 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.792 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.792 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.793 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.793 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.793 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.793 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.793 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.793 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.793 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.793 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.793 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.793 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.793 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.793 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.793 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.793 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.793 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.794 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.794 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.794 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.794 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.794 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.794 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.794 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.794 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.794 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.794 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.794 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.794 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-21T03:58:02.794 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-21T03:58:02.794 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.794 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.795 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.795 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.795 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.795 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.795 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.795 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:58:02.795 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:58:02.795 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.795 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.795 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.795 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-21T03:58:02.795 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-21T03:58:02.795 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.795 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.796 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.796 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-21T03:58:02.796 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.796 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.796 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.796 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:58:02.796 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.796 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.796 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.796 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.796 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-21T03:58:02.796 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-21T03:58:02.796 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.796 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.796 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.797 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-21T03:58:02.797 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.797 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.797 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.797 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:58:02.797 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.797 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.797 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.797 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-21T03:58:02.797 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-21T03:58:02.797 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.797 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.797 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.798 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.798 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.798 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.798 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.798 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:58:02.798 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.798 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.798 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.798 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.798 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.798 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.798 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.798 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.798 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.799 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.799 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.799 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.799 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.799 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.799 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.799 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.799 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.799 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.799 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-21T03:58:02.799 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-21T03:58:02.799 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.799 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.799 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.799 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-21T03:58:02.800 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.800 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.800 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.800 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:58:02.800 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:58:02.800 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.800 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.800 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.800 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.800 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.800 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.800 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-21T03:58:02.800 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.800 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.801 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.801 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.801 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.801 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.801 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.801 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.801 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.801 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.801 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.801 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.801 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.801 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.801 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.801 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.802 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.802 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.802 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.802 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.802 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.802 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.802 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.802 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.802 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.802 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.802 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.802 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.802 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.802 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.802 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.803 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.803 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.803 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.803 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.803 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.803 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.803 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.803 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.803 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.803 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.803 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.803 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.803 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.803 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.804 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.804 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.804 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.804 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.804 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.804 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.804 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.804 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.804 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.804 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.804 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.804 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.804 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.805 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.805 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.805 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.805 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.805 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.805 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.805 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.805 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.805 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.805 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.805 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.805 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.805 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.805 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.806 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.806 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.806 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.806 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.806 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.806 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.806 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.806 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.806 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.806 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.806 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.806 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.806 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.806 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.807 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-21T03:58:02.807 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.807 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.807 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.807 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.807 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.807 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.807 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.807 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.807 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.807 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.807 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.807 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.807 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.807 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.808 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.808 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.808 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.808 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.808 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.808 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.808 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.808 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.808 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.808 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.808 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.808 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.808 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.808 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.809 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.809 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.809 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.809 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.809 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.809 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.809 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.809 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.809 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.809 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.809 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.809 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.809 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.809 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.810 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.810 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.810 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.810 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.810 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.810 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.810 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.810 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.810 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.810 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.810 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.810 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.810 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.811 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.811 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.811 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.811 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.811 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.811 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.811 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.811 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.811 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.811 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.811 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.811 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.811 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.811 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.812 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.812 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.812 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.812 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.812 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.812 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.812 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.812 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.812 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.812 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.812 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.812 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.812 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.813 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.813 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.813 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.813 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.813 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-21T03:58:02.813 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.813 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.813 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.813 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.813 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.813 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.814 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.814 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.814 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.814 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.814 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.814 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.814 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.814 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.814 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.814 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.814 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.814 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.814 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.815 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.815 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.815 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.815 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.815 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.815 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.815 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.815 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.815 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.815 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.815 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.815 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.815 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.816 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.816 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.816 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.816 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.816 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.816 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.816 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.816 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.816 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.816 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.816 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.816 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.816 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.816 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.817 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.817 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.817 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.817 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.817 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.817 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.817 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.817 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.817 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.817 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.817 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.817 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.817 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.818 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.818 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.818 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.818 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.818 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.818 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.818 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.818 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.818 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.818 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.818 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.818 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.818 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.818 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.819 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.819 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.819 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.819 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.819 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.819 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.819 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.819 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.819 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.819 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.819 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.819 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.819 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-21T03:58:02.820 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.820 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.820 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.820 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-21T03:58:02.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-21T03:58:02.820 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.820 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.820 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.820 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.820 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.820 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.820 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.820 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:58:02.821 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.821 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.821 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.821 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.821 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.821 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.821 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.821 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.821 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.821 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.821 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.821 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.821 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.821 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.822 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.822 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.822 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.822 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.822 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.822 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.822 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.822 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.822 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.822 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.822 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.822 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.822 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.822 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.823 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.823 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.823 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.823 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.823 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.823 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.823 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.823 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.823 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.823 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.823 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.823 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.823 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.824 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.824 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.824 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.824 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.824 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.824 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.824 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.824 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.824 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.824 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.824 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.824 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.824 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.824 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.825 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.825 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.825 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.825 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.825 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.825 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.825 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.825 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.825 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.825 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.825 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.825 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.825 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.825 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-21T03:58:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-21T03:58:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-21T03:58:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-21T03:58:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-21T03:58:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-21T03:58:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-21T03:58:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-21T03:58:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.837 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.837 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.837 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.837 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.837 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.837 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.837 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.837 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.837 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.837 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.837 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.837 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.837 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.838 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.838 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.838 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.838 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.838 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.838 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.838 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.838 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.838 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.838 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.838 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.838 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.838 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.839 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.839 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.839 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.839 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.839 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.839 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.839 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.839 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.839 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.839 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.839 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.839 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.839 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.840 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.840 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.840 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.840 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.840 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.840 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.840 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.840 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.840 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.840 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.840 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.840 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.841 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.841 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.841 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.841 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.841 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.841 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.841 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.841 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.841 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.841 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-21T03:58:02.841 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.841 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.841 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.841 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.841 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.842 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.842 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.842 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.842 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.842 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.842 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.842 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.842 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.842 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.842 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.842 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.842 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.842 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.843 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.843 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.843 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.843 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.843 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.843 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.843 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.843 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.843 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.843 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.843 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.843 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.843 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.843 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.844 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.844 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.844 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.844 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.844 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.844 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.844 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.844 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.844 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.844 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.844 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.844 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.844 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.844 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.845 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.845 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.845 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.845 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.845 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.845 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.845 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.845 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.845 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.845 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.845 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.845 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.845 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.845 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.846 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.846 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.846 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.846 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.846 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.846 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.846 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.846 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.846 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.846 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.846 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.846 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.846 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:58:02.847 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:58:02.847 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.847 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.847 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.847 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:58:02.847 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.847 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.847 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.847 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.847 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.847 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.847 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.847 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.847 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.847 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.848 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.848 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.848 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.848 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.848 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.848 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.848 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.848 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.848 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.848 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.848 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.848 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.848 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-21T03:58:02.848 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-21T03:58:02.849 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:02.849 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.849 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.849 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-21T03:58:02.849 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-21T03:58:02.849 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.849 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.849 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-21T03:58:02.849 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:58:02.849 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.849 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.849 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.849 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.849 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.849 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.850 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-21T03:58:02.850 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.850 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.850 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.850 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-21T03:58:02.850 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:58:02.850 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.850 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.850 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.850 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.850 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.850 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.850 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.850 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:58:02.851 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.851 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.851 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.851 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.851 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-21T03:58:02.851 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-21T03:58:02.851 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.851 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.851 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.851 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.851 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.851 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.851 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.851 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.852 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.852 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.852 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.852 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.852 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.852 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.852 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.852 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.852 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.852 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.852 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.852 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.852 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.852 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.853 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.853 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.853 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.853 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.853 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.853 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.853 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.853 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.853 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.853 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.853 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.853 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.853 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.853 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.854 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.854 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.854 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.854 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.854 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.854 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.854 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.854 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.854 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.854 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.854 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.854 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.854 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.855 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.855 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.855 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.855 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.855 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.855 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.855 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.855 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.855 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.855 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.855 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.855 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.855 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.855 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.856 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.856 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.856 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.856 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.856 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.856 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.856 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.856 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.856 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.856 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.856 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.856 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.856 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.856 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.857 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.857 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.857 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.857 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-21T03:58:02.857 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-21T03:58:02.857 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.857 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.857 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.857 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-21T03:58:02.857 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.857 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.857 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.857 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:58:02.857 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.857 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.858 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.858 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.858 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:58:02.858 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:58:02.858 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.858 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.858 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.858 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:58:02.858 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.858 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.858 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.858 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:58:02.858 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.858 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.859 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.859 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.859 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.859 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.859 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.859 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.859 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.859 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.859 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.859 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.859 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.859 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.859 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.859 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.859 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.860 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.860 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.860 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.860 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.860 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-21T03:58:02.860 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.860 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.860 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.860 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-21T03:58:02.860 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-21T03:58:02.860 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.860 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.860 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.860 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.860 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.861 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.861 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.861 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.861 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.861 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.861 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.861 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.861 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-21T03:58:02.861 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-21T03:58:02.861 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.861 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.861 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.861 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.861 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.861 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.862 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.862 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:58:02.862 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:58:02.862 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.862 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.862 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.862 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.862 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.862 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.862 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.862 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.862 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.862 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.863 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.863 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.863 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.863 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.863 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.863 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.863 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.863 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.863 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.863 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.863 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.863 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.863 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.863 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.864 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.864 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.864 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.864 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.864 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.864 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.864 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.864 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.864 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.864 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.864 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.864 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.864 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.864 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.865 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.865 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.865 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.865 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.865 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.865 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.865 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.865 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.865 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.865 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.865 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.866 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.866 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.866 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.866 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.866 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.866 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.866 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.866 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.866 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.866 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.866 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.866 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.866 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.866 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.867 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.867 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.867 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.867 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.867 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.867 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.867 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.867 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.867 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.867 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.867 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-21T03:58:02.867 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-21T03:58:02.867 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.867 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.867 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.868 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.868 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.868 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.868 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.868 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.868 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.868 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.868 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.868 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.868 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:58:02.868 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:58:02.868 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.868 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.868 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.869 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-21T03:58:02.869 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.869 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.869 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.869 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.869 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.869 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.869 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.869 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.869 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-21T03:58:02.869 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-21T03:58:02.869 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:02.869 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.869 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.869 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-21T03:58:02.870 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.870 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.870 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.870 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.870 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.870 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.870 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.870 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.870 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:58:02.870 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:58:02.870 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.870 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.870 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.870 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-21T03:58:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:58:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:58:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:58:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-21T03:58:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:58:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.873 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.873 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.873 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.873 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.873 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.873 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.873 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.873 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-21T03:58:02.873 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-21T03:58:02.873 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.873 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.873 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.873 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-21T03:58:02.873 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.873 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.874 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.874 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.874 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.874 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.874 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.874 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.874 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-21T03:58:02.874 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-21T03:58:02.874 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.874 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.874 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.874 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-21T03:58:02.874 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.874 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.875 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.875 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-21T03:58:02.875 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-21T03:58:02.875 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.875 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.875 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.875 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.875 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.875 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-21T03:58:02.875 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-21T03:58:02.875 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.875 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.875 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.876 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:58:02.876 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-21T03:58:02.876 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-21T03:58:02.876 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.876 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.876 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.876 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.876 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.876 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.876 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.876 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.876 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-21T03:58:02.877 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.877 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.877 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.877 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.877 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.877 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.877 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.877 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.877 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.877 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.877 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.877 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.877 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.877 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.877 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.878 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.878 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.878 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.878 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.878 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.878 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.878 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.878 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.878 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.878 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.878 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.878 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.879 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.879 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.879 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.879 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.879 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.879 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.879 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.879 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.879 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.879 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.879 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.879 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.879 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.880 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.880 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.880 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.880 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.880 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.880 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.880 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.880 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.880 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.880 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.880 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.880 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.880 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.880 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.880 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.881 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.881 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.881 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.881 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.881 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.881 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.881 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.881 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.881 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.881 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.881 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.881 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.881 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.882 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-21T03:58:02.882 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:58:02.882 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.882 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.882 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.882 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:58:02.882 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.882 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.882 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.882 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.882 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.882 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.882 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.882 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.883 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:58:02.883 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:58:02.883 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.883 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.883 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.883 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.883 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.883 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.883 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.883 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.883 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.883 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.883 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.883 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.884 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.884 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.884 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.884 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.884 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.884 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.884 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.884 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.884 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.884 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.884 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.884 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.884 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.884 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.885 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-21T03:58:02.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-21T03:58:02.885 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.885 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.885 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.885 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.885 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.885 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.885 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.885 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.885 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.885 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.885 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.885 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.886 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.886 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.886 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-21T03:58:02.886 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.886 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.886 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.886 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-21T03:58:02.886 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-21T03:58:02.886 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.886 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.886 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.886 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.886 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.887 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.887 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.887 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.887 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.887 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.887 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-21T03:58:02.887 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-21T03:58:02.887 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.887 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.887 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.887 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.887 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.888 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.888 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.888 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.888 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.888 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.888 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.888 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.888 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.888 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.888 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.888 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.888 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.889 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.889 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.889 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.889 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.889 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.889 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.889 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.889 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.889 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.889 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.889 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.889 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.889 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.890 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.890 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.890 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.890 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.890 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.890 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.890 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.890 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.890 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.890 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.890 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.890 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.890 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.890 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.891 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.891 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.891 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.891 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.891 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.891 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.891 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.891 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.891 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.891 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.891 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.891 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.891 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.891 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.892 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.892 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.892 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.892 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.892 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.892 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.892 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.892 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.892 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.892 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.892 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.892 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.892 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: "max_requests": { 2024-09-21T03:58:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_requests", 2024-09-21T03:58:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:58:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:58:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:58:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:58:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:58:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:58:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-21T03:58:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:58:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:58:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:58:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:58:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-21T03:58:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.906 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.906 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.906 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.906 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.906 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.906 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.906 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.906 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.906 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.906 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.906 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.906 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.906 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.906 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.907 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.907 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.907 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.907 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.907 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.907 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.907 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.907 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.907 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.907 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.907 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.907 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.907 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.907 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.908 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.908 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.908 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.908 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.908 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.908 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.908 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.908 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.908 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.908 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.908 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.909 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.909 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.909 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.909 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.909 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.909 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.909 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.909 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.909 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.909 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.909 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.909 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.909 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.910 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.910 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.910 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.910 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.910 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.910 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.910 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.910 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.910 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.910 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.910 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.910 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.910 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.910 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.910 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.911 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-21T03:58:02.911 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-21T03:58:02.911 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.911 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.911 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.911 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-21T03:58:02.911 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.911 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.911 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.911 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:58:02.911 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.911 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.911 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.911 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.912 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.912 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.912 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.912 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-21T03:58:02.912 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.912 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.912 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.912 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.912 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.912 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.912 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.912 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.912 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.912 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.913 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.913 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.913 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.913 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.913 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.913 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.913 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.913 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.913 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.913 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.913 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.913 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.913 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.913 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.914 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.914 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.914 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.914 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.914 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.914 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.914 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.914 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.914 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.914 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.914 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.914 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.914 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.915 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.915 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.915 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.915 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.915 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.915 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.915 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.915 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.915 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.915 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.915 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.915 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.915 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.915 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.916 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.916 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.916 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.916 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.916 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.916 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.916 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.916 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.916 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.916 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.916 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.916 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.916 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.916 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.917 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.917 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.917 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.917 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.917 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.917 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-21T03:58:02.917 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-21T03:58:02.917 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.917 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.917 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.917 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.917 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.917 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.917 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.918 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.918 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.918 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.918 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.918 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-21T03:58:02.918 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-21T03:58:02.918 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.918 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.918 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.918 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-21T03:58:02.918 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.919 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.919 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.919 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.919 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.919 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.919 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.919 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-21T03:58:02.919 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-21T03:58:02.919 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.919 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.919 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.919 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.919 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.920 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.920 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.920 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.920 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.920 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-21T03:58:02.920 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-21T03:58:02.920 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.920 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.920 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.920 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.921 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.921 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.921 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.921 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.921 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.921 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-21T03:58:02.921 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-21T03:58:02.921 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:58:02.921 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.921 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.921 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.921 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.922 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.922 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.922 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.922 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.922 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.922 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.922 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-21T03:58:02.922 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-21T03:58:02.922 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.922 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.922 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.922 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.922 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.923 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.923 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.923 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.923 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.923 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.923 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.923 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-21T03:58:02.923 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-21T03:58:02.923 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.923 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.923 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.923 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.923 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.923 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.924 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.924 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.924 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.924 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.924 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.924 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-21T03:58:02.924 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-21T03:58:02.924 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.924 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.924 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.924 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.924 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.925 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.925 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.925 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.925 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.925 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.925 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.925 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-21T03:58:02.925 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-21T03:58:02.925 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.925 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.925 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.925 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.925 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:58:02.925 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-21T03:58:02.926 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.926 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.926 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.926 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.926 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.926 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.926 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.926 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.926 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.926 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.926 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.926 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.926 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.927 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.927 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.927 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.927 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.927 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.927 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-21T03:58:02.927 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-21T03:58:02.927 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.927 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.927 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.927 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.927 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.927 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.928 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.928 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.928 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.928 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.928 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.928 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-21T03:58:02.928 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-21T03:58:02.928 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.928 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.928 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.928 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.928 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.928 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.928 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.929 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.485+0000 7f64c5200640 1 -- 172.21.15.16:0/2153629282 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64a40b6510 msgr2=0x7f64a40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:02.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.485+0000 7f64c5200640 1 --2- 172.21.15.16:0/2153629282 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64a40b6510 0x7f64a40b8a20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f64b00095f0 tx=0x7f64b0005f90 comp rx=0 tx=0).stop 2024-09-21T03:58:02.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.485+0000 7f64c5200640 1 -- 172.21.15.16:0/2153629282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64c0106750 msgr2=0x7f64c019c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:02.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.485+0000 7f64c5200640 1 --2- 172.21.15.16:0/2153629282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64c0106750 0x7f64c019c580 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f64ac006010 tx=0x7f64ac0042f0 comp rx=0 tx=0).stop 2024-09-21T03:58:02.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.485+0000 7f64c5200640 1 -- 172.21.15.16:0/2153629282 shutdown_connections 2024-09-21T03:58:02.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.485+0000 7f64c5200640 1 --2- 172.21.15.16:0/2153629282 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f64a40b6510 0x7f64a40b8a20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:02.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.485+0000 7f64c5200640 1 --2- 172.21.15.16:0/2153629282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c01070f0 0x7f64c019d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:02.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.485+0000 7f64c5200640 1 --2- 172.21.15.16:0/2153629282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64c0106750 0x7f64c019c580 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:02.930 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.930 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.930 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.930 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-21T03:58:02.930 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-21T03:58:02.930 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.930 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.931 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.931 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.931 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.931 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.931 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.931 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.931 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.931 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.931 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.931 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.931 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.931 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.931 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smb", 2024-09-21T03:58:02.932 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.932 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.932 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.932 INFO:teuthology.orchestra.run.smithi016.stdout: "internal_store_backend": { 2024-09-21T03:58:02.932 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "internal_store_backend", 2024-09-21T03:58:02.932 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.932 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.932 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.932 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.932 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.932 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.932 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.932 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:58:02.932 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.933 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.933 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.933 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.933 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.933 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.933 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.933 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.933 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.933 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.933 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.933 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.933 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.933 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.934 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.934 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.934 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.934 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.934 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.934 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.934 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.934 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.934 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.934 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.934 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.934 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.934 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.934 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.935 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.935 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.935 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.935 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.935 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.935 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.935 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.935 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.935 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.935 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.935 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.935 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.935 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.935 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.936 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.936 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.936 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.936 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.936 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.936 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.936 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.936 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.936 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.936 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.936 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.936 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.936 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.936 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.937 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.937 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.937 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.937 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.937 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.937 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.937 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.937 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.937 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.937 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.937 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.937 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.937 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.937 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.938 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.938 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.938 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.938 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.938 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.938 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.938 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.938 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.938 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.938 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.938 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.938 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.938 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.938 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.939 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.939 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.939 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.939 INFO:teuthology.orchestra.run.smithi016.stdout: "update_orchestration": { 2024-09-21T03:58:02.939 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "update_orchestration", 2024-09-21T03:58:02.939 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.939 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.939 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.939 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.939 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.939 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.939 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.939 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:58:02.939 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.940 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.940 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.940 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.940 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.940 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.940 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.940 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-21T03:58:02.940 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.940 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.940 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.940 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-21T03:58:02.940 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-21T03:58:02.940 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.940 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.940 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.941 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.941 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.941 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.941 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.941 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:58:02.941 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.941 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.941 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.941 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.941 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-21T03:58:02.941 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-21T03:58:02.941 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.941 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.941 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.942 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.942 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.942 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.942 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.942 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:58:02.942 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.942 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.942 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.942 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.942 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.942 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.942 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.942 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.942 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.942 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.943 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.943 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.943 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.943 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.943 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.943 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.943 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.943 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.943 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.943 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.943 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.943 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.943 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.943 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.944 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.944 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.944 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.944 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.944 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.944 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.944 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.944 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.944 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.944 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.944 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.944 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.944 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.944 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.945 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.945 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.945 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.945 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.945 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.945 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.945 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.945 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.945 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.945 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.945 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.945 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.945 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.945 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.946 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.946 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.946 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.946 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.946 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.946 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.946 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.946 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.946 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.946 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.946 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.946 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.946 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.946 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.947 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.947 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.947 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.947 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.947 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.947 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.947 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.947 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.947 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.947 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.947 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.947 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.947 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.947 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.948 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.948 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.948 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.948 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.948 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.948 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.948 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.948 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.948 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.948 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.948 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-21T03:58:02.948 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.948 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.948 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.949 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.949 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.949 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.949 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.949 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.949 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.949 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.949 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.949 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.949 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.949 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.949 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.949 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.949 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.950 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.950 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.950 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.950 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.950 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.950 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.950 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.950 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.950 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.950 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.950 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.950 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.950 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.950 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.951 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.951 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.951 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.951 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.951 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.951 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.951 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.951 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.951 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.951 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.951 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.951 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.951 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.951 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.952 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.952 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.952 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.952 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.952 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.952 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.952 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.952 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.952 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.952 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.952 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.952 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.952 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.953 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.953 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.953 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.953 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.953 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.953 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.953 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.953 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.953 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.953 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.953 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.953 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.953 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.953 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.954 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.954 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.954 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.954 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.954 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.954 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.954 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.954 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.954 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.954 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.954 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.954 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.954 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.954 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.955 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.955 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-21T03:58:02.955 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.955 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.955 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.955 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.955 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.955 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.955 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.955 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.955 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.955 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.955 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.955 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.956 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.956 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.956 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.956 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.956 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.956 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.956 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.956 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.956 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.956 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.956 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.956 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.956 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.956 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.958 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.958 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.958 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.958 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.958 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.958 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.958 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.958 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.958 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.958 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.958 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.958 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.958 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.958 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.958 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.959 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.959 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.959 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.959 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.959 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.959 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.959 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.959 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.959 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.959 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.959 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.959 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.959 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.960 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.960 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.960 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.960 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.960 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.960 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.960 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.960 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.960 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.960 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.960 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.960 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.960 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.961 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.961 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.961 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.961 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.961 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-21T03:58:02.961 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.961 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.961 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.961 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-21T03:58:02.961 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-21T03:58:02.961 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.961 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.961 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.961 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:58:02.962 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.962 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.962 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.962 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.962 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.962 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.962 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.962 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.962 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:58:02.962 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:58:02.962 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:02.962 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.962 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.962 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-21T03:58:02.963 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.963 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.963 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.963 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.963 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.963 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.963 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.963 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.963 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.963 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.963 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.963 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.963 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.963 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.963 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.964 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.964 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.964 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.964 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.964 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.964 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.964 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.964 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.964 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.964 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.964 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.964 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.964 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.964 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.965 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.965 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.965 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.965 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.965 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.965 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.965 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.965 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.965 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.965 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.965 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.965 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.965 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.965 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.966 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.966 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.966 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.966 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.966 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.966 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.966 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.966 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.966 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.966 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.966 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.966 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.966 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.967 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.967 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.967 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.967 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.967 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.967 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.967 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.967 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.967 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.967 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.967 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.967 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.967 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.967 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.968 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.968 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.968 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.968 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.968 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.968 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.968 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.968 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.968 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.968 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.968 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.968 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.968 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.969 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.969 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.969 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.969 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.969 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.969 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.969 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.969 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-21T03:58:02.969 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.969 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.969 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.969 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-21T03:58:02.970 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-21T03:58:02.970 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.970 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.970 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.970 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.970 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.970 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.970 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.970 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:58:02.970 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.970 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.970 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.971 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-21T03:58:02.971 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-21T03:58:02.971 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.971 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.971 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.971 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.971 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.971 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.971 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.971 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:58:02.971 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.971 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.971 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.972 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-21T03:58:02.972 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-21T03:58:02.972 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.972 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.972 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.972 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:02.972 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.972 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.972 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.972 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:58:02.972 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.972 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.972 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.972 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.973 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-21T03:58:02.973 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-21T03:58:02.973 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.973 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.973 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.973 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.973 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.973 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.973 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.973 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:58:02.973 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.973 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.973 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.973 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.973 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-21T03:58:02.974 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-21T03:58:02.974 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.974 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.974 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.974 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.974 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.974 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.974 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.974 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:58:02.974 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.974 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.974 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.974 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.974 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-21T03:58:02.975 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-21T03:58:02.975 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.975 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.975 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.975 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.975 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.975 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.975 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.975 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.975 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.975 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.975 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.975 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.975 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-21T03:58:02.975 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-21T03:58:02.976 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.976 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.976 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.976 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.976 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.976 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.976 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.976 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.976 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.976 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.976 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.976 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.976 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-21T03:58:02.977 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-21T03:58:02.977 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.977 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.977 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.977 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:58:02.977 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.977 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.977 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.977 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.977 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.977 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.977 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.977 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.977 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-21T03:58:02.978 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-21T03:58:02.978 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.978 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.978 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.978 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.978 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.978 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.978 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.978 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.978 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.978 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.978 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.978 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:58:02.978 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:58:02.979 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.979 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.979 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.979 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-21T03:58:02.979 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-21T03:58:02.979 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.979 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.979 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.979 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.979 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.979 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.979 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-21T03:58:02.979 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-21T03:58:02.980 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.980 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.980 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.980 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-21T03:58:02.980 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.980 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.980 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.980 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.980 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.980 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.980 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.980 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-21T03:58:02.980 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-21T03:58:02.981 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.981 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.981 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.981 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.981 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.981 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.981 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.981 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.981 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.981 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.981 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.981 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-21T03:58:02.981 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-21T03:58:02.982 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.982 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.982 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.982 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.982 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.982 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.982 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.982 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.982 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.982 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.982 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.982 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.982 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.982 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.983 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.983 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.983 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.983 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.983 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.983 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.983 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.983 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.983 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.983 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.983 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.983 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.983 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.983 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.984 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.984 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.984 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.984 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.984 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.984 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.984 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.984 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.984 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.984 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.984 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.984 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.984 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.984 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.984 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.985 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.985 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.985 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.985 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.985 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.985 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.985 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.985 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.985 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.985 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.985 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.985 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.985 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.985 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.985 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.986 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.986 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.986 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.986 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.986 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.986 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.986 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.986 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.986 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.986 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.986 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.986 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.986 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.986 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.987 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.987 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.987 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.987 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.987 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.987 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.987 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.987 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.987 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.987 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.987 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-21T03:58:02.987 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-21T03:58:02.987 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.987 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.988 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.988 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.988 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.988 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.988 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.988 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.988 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.988 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.988 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.988 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.988 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-21T03:58:02.988 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-21T03:58:02.988 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.988 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.988 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.989 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.989 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.989 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.989 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.989 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.989 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.989 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.989 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.989 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.989 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.989 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.989 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.989 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.989 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.989 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-21T03:58:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-21T03:58:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:02.990 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:58:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-21T03:58:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.991 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:02.992 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:02.992 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:02.992 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:02.992 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.992 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.992 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.992 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:02.992 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.992 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.992 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.992 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.992 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.992 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.993 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.993 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.993 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.993 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.993 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.993 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.993 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.993 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.993 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.993 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:02.993 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:02.993 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.993 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.993 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.994 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.994 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.994 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.994 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.994 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.994 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.994 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.994 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.994 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:02.994 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:02.994 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:02.994 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.994 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.994 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:02.995 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.995 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.995 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:02.995 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:02.995 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:02.995 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:02.995 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:02.995 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:02.995 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:02.995 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:02.995 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.995 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.995 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.995 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.996 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:02.996 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:02.996 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:02.996 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:02.996 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.996 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:02.996 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.996 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.996 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.996 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.996 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.996 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.996 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.997 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.485+0000 7f64c5200640 1 --2- 172.21.15.16:0/2153629282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64c01056d0 0x7f64c019bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:02.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.485+0000 7f64c5200640 1 -- 172.21.15.16:0/2153629282 >> 172.21.15.16:0/2153629282 conn(0x7f64c0101050 msgr2=0x7f64c0109a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:02.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.485+0000 7f64c5200640 1 -- 172.21.15.16:0/2153629282 shutdown_connections 2024-09-21T03:58:02.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:01.485+0000 7f64c5200640 1 -- 172.21.15.16:0/2153629282 wait complete. 2024-09-21T03:58:02.998 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:02.998 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:02.998 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:02.998 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:02.998 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:02.998 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:02.998 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:02.999 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:02.999 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:02.999 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:02.999 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:02.999 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:02.999 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:02.999 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.999 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:02.999 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:02.999 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:02.999 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-21T03:58:02.999 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:02.999 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:03.000 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:03.000 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:03.000 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:03.000 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:03.000 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.000 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:03.000 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:03.000 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.000 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.000 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:03.000 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:03.000 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:03.000 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:03.000 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:03.001 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:03.001 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:03.001 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:03.001 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:03.001 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.001 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.001 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.001 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.001 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:03.001 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:03.001 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:03.001 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.001 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:03.001 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:03.001 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.002 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.002 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:03.002 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:03.002 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.002 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.002 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.002 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.002 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:03.002 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:03.002 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:03.002 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.002 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:03.002 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:03.002 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.003 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.003 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:03.003 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:03.003 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:03.003 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:03.003 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:03.003 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:03.003 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:03.003 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:03.003 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:03.003 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.003 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.003 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.003 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.004 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:03.004 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:03.004 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:03.004 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.004 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:03.004 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:03.004 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.004 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.004 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:03.004 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:03.004 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.004 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.004 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.004 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.005 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-21T03:58:03.005 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-21T03:58:03.005 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:03.005 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.005 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:03.005 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-21T03:58:03.005 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.005 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.005 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:03.005 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:58:03.005 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.005 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.005 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.005 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.006 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-21T03:58:03.006 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-21T03:58:03.006 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:03.006 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.006 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:03.006 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:03.006 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.006 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.006 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:03.006 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-21T03:58:03.006 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.006 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.006 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.006 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.006 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-21T03:58:03.007 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:58:03.007 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:03.007 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.007 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:03.007 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:03.007 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.007 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.007 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:03.007 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:58:03.007 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.007 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.007 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.007 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.008 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:58:03.008 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:58:03.008 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:03.008 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.008 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:03.008 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:58:03.008 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.008 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.008 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:03.008 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:58:03.008 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.008 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.009 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.009 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.009 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:03.009 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:03.009 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:03.009 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:03.009 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:03.009 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:03.009 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.009 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.009 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:03.009 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:03.009 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.010 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.010 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.010 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:03.010 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:03.010 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.010 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:03.010 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-21T03:58:03.010 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:58:03.010 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:58:03.010 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:58:03.010 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-21T03:58:03.010 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-21T03:58:03.010 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:58:03.010 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.011 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:03.011 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-21T03:58:03.011 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.011 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.011 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:03.011 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:03.011 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.011 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.011 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.011 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.011 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-21T03:58:03.011 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-21T03:58:03.011 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:03.011 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.011 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:03.012 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:03.012 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.012 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.012 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:03.012 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:03.012 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.012 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.012 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.012 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.012 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:58:03.012 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:58:03.012 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:58:03.012 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.012 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:03.012 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:58:03.013 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.013 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.013 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:03.013 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:03.013 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.013 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.013 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.013 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.013 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:58:03.013 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:58:03.013 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:03.013 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.013 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:03.013 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:03.014 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.014 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.014 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:03.014 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:03.014 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:03.014 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:03.014 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:03.014 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:03.014 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:03.014 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:03.014 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:03.014 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.014 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.014 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.015 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:58:03.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:58:03.015 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:03.015 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.015 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:03.015 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:03.015 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.015 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.015 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:03.015 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:03.015 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.015 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.015 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.016 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.016 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:58:03.016 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:58:03.016 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:03.016 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.016 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:03.016 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:58:03.016 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.016 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.016 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:58:03.016 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:58:03.016 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:58:03.016 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:58:03.016 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:58:03.017 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:58:03.017 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:58:03.017 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:03.017 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:03.017 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.017 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.017 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.017 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.017 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:58:03.017 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:58:03.017 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:58:03.017 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.017 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:03.017 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:58:03.017 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.018 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.018 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:03.018 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:03.018 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.018 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.018 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.018 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.018 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:58:03.018 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:58:03.018 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:03.018 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:58:03.018 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:58:03.018 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:58:03.018 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.019 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.019 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:03.019 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:03.019 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.019 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.019 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.019 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.019 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-21T03:58:03.019 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-21T03:58:03.019 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:03.019 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.019 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:03.019 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:58:03.019 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.019 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.020 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:03.020 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:03.020 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.020 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.020 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.020 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.020 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-21T03:58:03.020 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-21T03:58:03.020 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:58:03.020 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.020 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:03.020 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-21T03:58:03.020 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.020 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.021 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:03.021 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:03.021 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.021 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.021 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.021 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.021 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-21T03:58:03.021 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-21T03:58:03.021 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:58:03.021 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:58:03.021 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:58:03.021 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:58:03.021 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:58:03.021 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:58:03.021 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:58:03.022 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:58:03.022 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:58:03.022 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:58:03.022 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:58:03.022 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:03.022 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:03.022 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:03.022 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:03.022 INFO:teuthology.orchestra.run.smithi016.stdout: "services": { 2024-09-21T03:58:03.022 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard": "https://172.21.15.16:8443/" 2024-09-21T03:58:03.022 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.022 INFO:teuthology.orchestra.run.smithi016.stdout: "always_on_modules": { 2024-09-21T03:58:03.022 INFO:teuthology.orchestra.run.smithi016.stdout: "octopus": [ 2024-09-21T03:58:03.023 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-21T03:58:03.023 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-21T03:58:03.023 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-21T03:58:03.023 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-21T03:58:03.023 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-21T03:58:03.023 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-21T03:58:03.023 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-21T03:58:03.023 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-21T03:58:03.023 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-21T03:58:03.023 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-21T03:58:03.023 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:03.023 INFO:teuthology.orchestra.run.smithi016.stdout: "pacific": [ 2024-09-21T03:58:03.023 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-21T03:58:03.023 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-21T03:58:03.024 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-21T03:58:03.024 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-21T03:58:03.024 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-21T03:58:03.024 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-21T03:58:03.024 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-21T03:58:03.024 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-21T03:58:03.024 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-21T03:58:03.024 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-21T03:58:03.024 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:03.024 INFO:teuthology.orchestra.run.smithi016.stdout: "quincy": [ 2024-09-21T03:58:03.024 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-21T03:58:03.024 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-21T03:58:03.024 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-21T03:58:03.024 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-21T03:58:03.024 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-21T03:58:03.025 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-21T03:58:03.025 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-21T03:58:03.025 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-21T03:58:03.025 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-21T03:58:03.025 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-21T03:58:03.025 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:03.025 INFO:teuthology.orchestra.run.smithi016.stdout: "reef": [ 2024-09-21T03:58:03.025 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-21T03:58:03.025 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-21T03:58:03.025 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-21T03:58:03.025 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-21T03:58:03.025 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-21T03:58:03.025 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-21T03:58:03.025 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-21T03:58:03.026 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-21T03:58:03.026 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-21T03:58:03.026 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-21T03:58:03.026 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:58:03.026 INFO:teuthology.orchestra.run.smithi016.stdout: "squid": [ 2024-09-21T03:58:03.026 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-21T03:58:03.026 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-21T03:58:03.026 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-21T03:58:03.026 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-21T03:58:03.026 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-21T03:58:03.026 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-21T03:58:03.026 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-21T03:58:03.026 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-21T03:58:03.026 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-21T03:58:03.026 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-21T03:58:03.027 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:58:03.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.027 INFO:teuthology.orchestra.run.smithi016.stdout: "last_failure_osd_epoch": 3, 2024-09-21T03:58:03.027 INFO:teuthology.orchestra.run.smithi016.stdout: "active_clients": [ 2024-09-21T03:58:03.027 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:03.027 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-21T03:58:03.027 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-21T03:58:03.027 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:03.027 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-21T03:58:03.027 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-21T03:58:03.027 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2016398981 2024-09-21T03:58:03.027 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:03.027 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:58:03.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.028 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:03.028 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "libcephsqlite", 2024-09-21T03:58:03.028 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-21T03:58:03.028 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:03.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-21T03:58:03.028 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-21T03:58:03.028 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2041301734 2024-09-21T03:58:03.028 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:03.028 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:58:03.028 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.028 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:03.028 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-21T03:58:03.028 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-21T03:58:03.028 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:03.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-21T03:58:03.029 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-21T03:58:03.029 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 4171842414 2024-09-21T03:58:03.029 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:03.029 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:58:03.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:03.029 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:03.029 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-21T03:58:03.029 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-21T03:58:03.029 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:58:03.029 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-21T03:58:03.029 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-21T03:58:03.029 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 4135499316 2024-09-21T03:58:03.029 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:03.029 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:58:03.030 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:03.030 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:58:03.030 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:58:03.030 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:01 smithi123 ceph-mon[34673]: pgmap v769: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 111 MiB/s wr, 1.02k op/s 2024-09-21T03:58:03.030 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:01 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/441704293' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:58:03.030 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:01 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/626190644' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T03:58:03.030 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:02 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:03.030 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:02 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2153629282' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:58:03.030 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:02 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:03.030 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:02 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:58:03.031 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:02 smithi123 ceph-mon[34673]: from='mgr.14154 ' entity='mgr.x' 2024-09-21T03:58:03.031 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:01 smithi134 ceph-mon[35031]: pgmap v769: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 111 MiB/s wr, 1.02k op/s 2024-09-21T03:58:03.031 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:01 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/441704293' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:58:03.031 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:01 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/626190644' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T03:58:03.031 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:02 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:03.031 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:02 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2153629282' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:58:03.031 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:02 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:03.031 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:02 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:58:03.031 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:02 smithi134 ceph-mon[35031]: from='mgr.14154 ' entity='mgr.x' 2024-09-21T03:58:03.031 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:01 smithi016 ceph-mon[30375]: pgmap v769: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 111 MiB/s wr, 1.02k op/s 2024-09-21T03:58:03.031 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:01 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/441704293' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:58:03.031 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:01 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/626190644' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T03:58:03.081 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T03:58:03.082 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T03:58:03.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:02 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:03.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:02 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2153629282' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:58:03.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:02 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:03.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:02 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:58:03.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:02 smithi016 ceph-mon[30375]: from='mgr.14154 ' entity='mgr.x' 2024-09-21T03:58:03.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:03 smithi016 ceph-mon[30375]: pgmap v770: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 66 MiB/s wr, 664 op/s 2024-09-21T03:58:03.475 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:58:03.604 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:03 smithi134 ceph-mon[35031]: pgmap v770: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 66 MiB/s wr, 664 op/s 2024-09-21T03:58:03.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:03 smithi123 ceph-mon[34673]: pgmap v770: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 66 MiB/s wr, 664 op/s 2024-09-21T03:58:04.268 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:58:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:58:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1095, 2024-09-21T03:58:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 1088, 2024-09-21T03:58:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 850768, 2024-09-21T03:58:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 417802, 2024-09-21T03:58:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-09-21T03:58:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-21T03:58:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.275015839, 2024-09-21T03:58:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.134017493, 2024-09-21T03:58:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.074868459, 2024-09-21T03:58:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.054478466, 2024-09-21T03:58:04.269 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:04.269 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1088, 2024-09-21T03:58:04.269 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.412179305, 2024-09-21T03:58:04.269 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000378841 2024-09-21T03:58:04.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.269 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:04.269 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.269 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.269 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.269 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 570208, 2024-09-21T03:58:04.269 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 152272 2024-09-21T03:58:04.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.269 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:58:04.270 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 5, 2024-09-21T03:58:04.270 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 190, 2024-09-21T03:58:04.270 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1710, 2024-09-21T03:58:04.270 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 490814, 2024-09-21T03:58:04.270 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-09-21T03:58:04.270 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-09-21T03:58:04.270 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.035954297, 2024-09-21T03:58:04.270 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.028784435, 2024-09-21T03:58:04.270 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.003588357, 2024-09-21T03:58:04.270 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000069417, 2024-09-21T03:58:04.270 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:04.270 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 188, 2024-09-21T03:58:04.270 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.067084459, 2024-09-21T03:58:04.271 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000356832 2024-09-21T03:58:04.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.271 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:04.271 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.271 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.271 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.271 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:58:04.271 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 489792 2024-09-21T03:58:04.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.271 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:58:04.271 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1597, 2024-09-21T03:58:04.271 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 1599, 2024-09-21T03:58:04.271 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 407399, 2024-09-21T03:58:04.272 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 507847, 2024-09-21T03:58:04.272 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-09-21T03:58:04.272 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-09-21T03:58:04.272 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.282897342, 2024-09-21T03:58:04.272 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.114863045, 2024-09-21T03:58:04.272 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.048728448, 2024-09-21T03:58:04.272 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.100668377, 2024-09-21T03:58:04.272 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:04.272 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1597, 2024-09-21T03:58:04.272 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.376350539, 2024-09-21T03:58:04.272 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000235660 2024-09-21T03:58:04.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.272 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:04.272 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.273 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.273 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.273 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:58:04.273 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:58:04.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.273 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-21T03:58:04.273 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-09-21T03:58:04.273 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-21T03:58:04.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.273 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-09-21T03:58:04.273 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-09-21T03:58:04.273 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-09-21T03:58:04.273 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3810, 2024-09-21T03:58:04.274 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.221612321, 2024-09-21T03:58:04.274 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000058165 2024-09-21T03:58:04.274 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:04.274 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.274 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-09-21T03:58:04.274 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-09-21T03:58:04.274 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-09-21T03:58:04.274 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.274 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.274 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.274 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:04.274 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.274 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-09-21T03:58:04.274 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-09-21T03:58:04.275 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-09-21T03:58:04.275 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-09-21T03:58:04.275 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.275 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.275 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.275 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-09-21T03:58:04.275 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-09-21T03:58:04.275 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-09-21T03:58:04.275 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-09-21T03:58:04.275 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-09-21T03:58:04.275 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-09-21T03:58:04.275 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-09-21T03:58:04.276 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 0, 2024-09-21T03:58:04.276 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2024-09-21T03:58:04.276 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 0, 2024-09-21T03:58:04.276 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-09-21T03:58:04.276 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 0, 2024-09-21T03:58:04.276 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-09-21T03:58:04.276 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-09-21T03:58:04.276 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-09-21T03:58:04.276 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 1, 2024-09-21T03:58:04.276 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-09-21T03:58:04.276 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-09-21T03:58:04.276 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-09-21T03:58:04.276 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-09-21T03:58:04.276 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:58:04.277 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-09-21T03:58:04.277 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-09-21T03:58:04.277 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-09-21T03:58:04.277 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-09-21T03:58:04.277 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-09-21T03:58:04.277 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-09-21T03:58:04.277 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-09-21T03:58:04.277 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-09-21T03:58:04.277 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-09-21T03:58:04.277 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:58:04.277 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-09-21T03:58:04.277 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2024-09-21T03:58:04.277 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2024-09-21T03:58:04.277 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-09-21T03:58:04.278 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:58:04.278 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-09-21T03:58:04.278 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:58:04.278 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:58:04.278 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:58:04.278 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:58:04.278 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:58:04.278 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-09-21T03:58:04.278 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:58:04.278 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:58:04.278 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:58:04.278 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:58:04.278 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:58:04.278 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:58:04.279 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-09-21T03:58:04.279 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:58:04.279 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-09-21T03:58:04.279 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-09-21T03:58:04.279 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.279 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-09-21T03:58:04.279 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-09-21T03:58:04.279 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-09-21T03:58:04.279 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-09-21T03:58:04.279 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-09-21T03:58:04.279 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-09-21T03:58:04.279 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-09-21T03:58:04.279 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-09-21T03:58:04.279 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:58:04.280 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-09-21T03:58:04.280 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-09-21T03:58:04.280 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-09-21T03:58:04.280 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-09-21T03:58:04.280 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-09-21T03:58:04.280 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:58:04.280 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:58:04.280 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-09-21T03:58:04.280 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-09-21T03:58:04.280 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:58:04.280 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:58:04.280 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:58:04.280 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-09-21T03:58:04.280 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-09-21T03:58:04.281 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:58:04.281 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-09-21T03:58:04.281 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0, 2024-09-21T03:58:04.281 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:58:04.281 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_succeeded": 0, 2024-09-21T03:58:04.281 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_write_failed": 0 2024-09-21T03:58:04.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.281 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-09-21T03:58:04.281 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-09-21T03:58:04.281 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-09-21T03:58:04.281 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-09-21T03:58:04.281 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-09-21T03:58:04.281 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 4, 2024-09-21T03:58:04.281 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-09-21T03:58:04.282 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-09-21T03:58:04.282 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-09-21T03:58:04.282 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-09-21T03:58:04.282 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-09-21T03:58:04.282 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 2, 2024-09-21T03:58:04.282 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-09-21T03:58:04.282 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-09-21T03:58:04.282 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-09-21T03:58:04.282 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-09-21T03:58:04.282 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 4195424, 2024-09-21T03:58:04.282 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 4195283, 2024-09-21T03:58:04.282 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-09-21T03:58:04.282 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.282 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.283 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.283 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 4 2024-09-21T03:58:04.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.283 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-09-21T03:58:04.283 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 3, 2024-09-21T03:58:04.283 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 3, 2024-09-21T03:58:04.283 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 0, 2024-09-21T03:58:04.283 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 1, 2024-09-21T03:58:04.283 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 1, 2024-09-21T03:58:04.283 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 0, 2024-09-21T03:58:04.283 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 0, 2024-09-21T03:58:04.283 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 0, 2024-09-21T03:58:04.283 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 0, 2024-09-21T03:58:04.284 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-09-21T03:58:04.284 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-09-21T03:58:04.284 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-09-21T03:58:04.284 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 115896, 2024-09-21T03:58:04.284 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 316424192 2024-09-21T03:58:04.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.284 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-09-21T03:58:04.284 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-09-21T03:58:04.284 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-09-21T03:58:04.284 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-09-21T03:58:04.284 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-09-21T03:58:04.284 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-09-21T03:58:04.284 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-09-21T03:58:04.284 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.285 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.285 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.285 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-09-21T03:58:04.285 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.285 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.285 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.285 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-09-21T03:58:04.285 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.285 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.285 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.285 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-09-21T03:58:04.285 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.286 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.286 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.286 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-09-21T03:58:04.286 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.286 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.286 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.286 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-09-21T03:58:04.286 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.286 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.286 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.286 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-09-21T03:58:04.287 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.287 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.287 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.287 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-09-21T03:58:04.287 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.287 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.287 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.287 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-09-21T03:58:04.287 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.287 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.287 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.288 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-09-21T03:58:04.288 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.288 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.288 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.288 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-09-21T03:58:04.288 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.288 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.289 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.289 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-09-21T03:58:04.289 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.289 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.289 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.289 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-09-21T03:58:04.289 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.289 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.289 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.289 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-09-21T03:58:04.289 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.290 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.290 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.290 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-09-21T03:58:04.290 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.290 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.290 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.290 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-09-21T03:58:04.290 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.290 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.290 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.291 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-09-21T03:58:04.291 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.291 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.291 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.291 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-09-21T03:58:04.291 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.291 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.291 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.291 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-09-21T03:58:04.291 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.291 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.291 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.292 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-09-21T03:58:04.292 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.292 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.292 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.292 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-09-21T03:58:04.292 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.292 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.292 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.292 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-09-21T03:58:04.292 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.292 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.292 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.293 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-09-21T03:58:04.293 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.293 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.293 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.293 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-09-21T03:58:04.293 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.293 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.293 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.293 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-09-21T03:58:04.293 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.293 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.294 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.294 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-09-21T03:58:04.294 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.294 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.294 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.294 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-09-21T03:58:04.294 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.294 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.294 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.294 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-09-21T03:58:04.294 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.295 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.295 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.295 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-09-21T03:58:04.295 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.295 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.295 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.295 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:58:04.295 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:58:04.295 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-09-21T03:58:04.295 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.295 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.295 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.296 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:04.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.296 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-09-21T03:58:04.296 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 0, 2024-09-21T03:58:04.296 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 1, 2024-09-21T03:58:04.296 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 1, 2024-09-21T03:58:04.296 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 1, 2024-09-21T03:58:04.296 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-09-21T03:58:04.296 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-09-21T03:58:04.296 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-09-21T03:58:04.296 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 1202, 2024-09-21T03:58:04.296 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-09-21T03:58:04.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.296 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-21T03:58:04.297 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:58:04.297 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-21T03:58:04.297 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:58:04.297 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:58:04.297 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:58:04.297 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:58:04.297 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:58:04.297 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:58:04.297 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:58:04.297 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:58:04.297 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:58:04.297 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:58:04.297 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:58:04.297 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:58:04.298 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:58:04.298 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-21T03:58:04.298 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:58:04.298 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-21T03:58:04.298 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:58:04.298 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:58:04.298 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:58:04.298 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:58:04.298 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:58:04.298 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:58:04.298 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:58:04.298 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-21T03:58:04.298 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:58:04.299 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:58:04.299 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:58:04.299 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-21T03:58:04.299 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-21T03:58:04.299 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-21T03:58:04.299 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:58:04.299 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:58:04.299 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:58:04.299 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:58:04.299 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 198478, 2024-09-21T03:58:04.299 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 54, 2024-09-21T03:58:04.299 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:58:04.299 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-21T03:58:04.299 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-21T03:58:04.299 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-21T03:58:04.300 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:58:04.300 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-21T03:58:04.300 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:58:04.300 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-21T03:58:04.300 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 8952, 2024-09-21T03:58:04.300 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 168, 2024-09-21T03:58:04.300 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:58:04.300 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:58:04.300 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-21T03:58:04.300 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-21T03:58:04.300 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 8672, 2024-09-21T03:58:04.300 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 13, 2024-09-21T03:58:04.300 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-21T03:58:04.300 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-21T03:58:04.301 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-21T03:58:04.301 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-21T03:58:04.301 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.301 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-09-21T03:58:04.301 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-09-21T03:58:04.301 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-09-21T03:58:04.301 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 2435, 2024-09-21T03:58:04.301 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 21, 2024-09-21T03:58:04.301 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 3, 2024-09-21T03:58:04.301 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 2435, 2024-09-21T03:58:04.301 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-09-21T03:58:04.301 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2435, 2024-09-21T03:58:04.301 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 2.839955997, 2024-09-21T03:58:04.302 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001166306 2024-09-21T03:58:04.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.302 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-09-21T03:58:04.302 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-09-21T03:58:04.302 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2435, 2024-09-21T03:58:04.302 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 2436 2024-09-21T03:58:04.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.302 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 2435, 2024-09-21T03:58:04.302 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 2435, 2024-09-21T03:58:04.302 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-09-21T03:58:04.302 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-09-21T03:58:04.302 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-09-21T03:58:04.302 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 1592, 2024-09-21T03:58:04.302 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-09-21T03:58:04.303 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 841, 2024-09-21T03:58:04.303 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-09-21T03:58:04.303 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-09-21T03:58:04.303 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-09-21T03:58:04.303 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-09-21T03:58:04.303 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-09-21T03:58:04.303 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-09-21T03:58:04.303 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-09-21T03:58:04.303 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-09-21T03:58:04.303 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-09-21T03:58:04.303 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-09-21T03:58:04.303 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1, 2024-09-21T03:58:04.303 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-09-21T03:58:04.303 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:58:04.304 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-09-21T03:58:04.304 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:58:04.304 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-09-21T03:58:04.304 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-09-21T03:58:04.304 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-09-21T03:58:04.304 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:58:04.304 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-09-21T03:58:04.304 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:58:04.304 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-09-21T03:58:04.304 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-09-21T03:58:04.304 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-09-21T03:58:04.304 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-09-21T03:58:04.304 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-09-21T03:58:04.305 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-09-21T03:58:04.305 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-09-21T03:58:04.305 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-09-21T03:58:04.305 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-09-21T03:58:04.305 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-09-21T03:58:04.305 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-09-21T03:58:04.305 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-09-21T03:58:04.305 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-09-21T03:58:04.305 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-09-21T03:58:04.305 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-09-21T03:58:04.305 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-09-21T03:58:04.305 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-09-21T03:58:04.305 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 86, 2024-09-21T03:58:04.306 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-09-21T03:58:04.306 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 2, 2024-09-21T03:58:04.306 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 6, 2024-09-21T03:58:04.306 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 6, 2024-09-21T03:58:04.306 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-09-21T03:58:04.306 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-09-21T03:58:04.306 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-09-21T03:58:04.306 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-09-21T03:58:04.306 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-09-21T03:58:04.306 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.306 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-09-21T03:58:04.306 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-09-21T03:58:04.306 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:58:04.306 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-09-21T03:58:04.307 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-09-21T03:58:04.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.307 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-09-21T03:58:04.307 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-09-21T03:58:04.307 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:58:04.307 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-09-21T03:58:04.307 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-09-21T03:58:04.307 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-09-21T03:58:04.307 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-09-21T03:58:04.307 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-09-21T03:58:04.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.307 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:58:04.307 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-21T03:58:04.307 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-21T03:58:04.307 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-21T03:58:04.308 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 2693, 2024-09-21T03:58:04.308 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 1001383, 2024-09-21T03:58:04.308 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:04.308 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 2693, 2024-09-21T03:58:04.308 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-21T03:58:04.308 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-21T03:58:04.308 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2693, 2024-09-21T03:58:04.308 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1001383, 2024-09-21T03:58:04.308 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-21T03:58:04.308 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.308 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.308 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.308 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:04.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.309 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-09-21T03:58:04.309 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-21T03:58:04.309 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-21T03:58:04.309 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-21T03:58:04.309 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-21T03:58:04.309 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-21T03:58:04.309 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:04.309 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:04.309 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2435, 2024-09-21T03:58:04.309 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 1125, 2024-09-21T03:58:04.309 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2435, 2024-09-21T03:58:04.309 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1125, 2024-09-21T03:58:04.309 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-21T03:58:04.309 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.309 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.310 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.310 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:04.310 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.310 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-09-21T03:58:04.310 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-21T03:58:04.310 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-09-21T03:58:04.310 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-21T03:58:04.310 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-21T03:58:04.310 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-21T03:58:04.310 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:04.310 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:04.310 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2435, 2024-09-21T03:58:04.310 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 2435, 2024-09-21T03:58:04.310 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2435, 2024-09-21T03:58:04.311 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2435, 2024-09-21T03:58:04.311 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-21T03:58:04.311 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.311 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.311 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.311 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:04.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.311 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:58:04.311 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-21T03:58:04.311 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-09-21T03:58:04.311 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-21T03:58:04.311 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-21T03:58:04.311 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-21T03:58:04.311 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:04.312 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:04.312 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-21T03:58:04.312 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-21T03:58:04.312 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-09-21T03:58:04.312 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-09-21T03:58:04.312 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-21T03:58:04.312 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.312 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.312 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.312 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:04.312 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.312 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x55bd0a01d2b0": { 2024-09-21T03:58:04.312 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-21T03:58:04.312 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-09-21T03:58:04.312 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-21T03:58:04.313 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-21T03:58:04.313 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-21T03:58:04.313 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:04.313 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:04.313 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-21T03:58:04.313 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-21T03:58:04.313 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-09-21T03:58:04.313 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-09-21T03:58:04.313 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-21T03:58:04.313 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:04.313 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:04.313 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:04.313 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:04.313 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:04.314 INFO:teuthology.orchestra.run.smithi016.stdout: "trackedop": { 2024-09-21T03:58:04.314 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-09-21T03:58:04.314 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:04.314 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:58:05.273 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-21T03:58:05.273 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-21T03:58:05.273 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T03:58:05.273 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T03:58:05.322 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:05 smithi016 ceph-mon[30375]: pgmap v771: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 336 op/s 2024-09-21T03:58:05.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:05 smithi134 ceph-mon[35031]: pgmap v771: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 336 op/s 2024-09-21T03:58:05.654 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:58:05.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:05 smithi123 ceph-mon[34673]: pgmap v771: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 336 op/s 2024-09-21T03:58:06.421 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:58:06.421 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:58:06.421 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-09-21T03:58:06.421 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-09-21T03:58:06.421 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:58:06.421 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-09-21T03:58:06.422 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-21T03:58:06.422 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-21T03:58:06.422 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.004441443, 2024-09-21T03:58:06.422 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:58:06.422 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000798455, 2024-09-21T03:58:06.422 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:58:06.422 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:06.422 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:06.422 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:06.422 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:06.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:06.422 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:06.422 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:06.423 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:06.423 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:06.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:06.423 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:58:06.423 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:58:06.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:06.423 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:58:06.423 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 257, 2024-09-21T03:58:06.423 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 252, 2024-09-21T03:58:06.423 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 540976, 2024-09-21T03:58:06.423 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 154928, 2024-09-21T03:58:06.423 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-21T03:58:06.423 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-21T03:58:06.423 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.090828276, 2024-09-21T03:58:06.424 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.050965982, 2024-09-21T03:58:06.424 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.037275610, 2024-09-21T03:58:06.424 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:58:06.424 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:06.424 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 252, 2024-09-21T03:58:06.424 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.099656378, 2024-09-21T03:58:06.424 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000395461 2024-09-21T03:58:06.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:06.424 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:06.424 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:06.424 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:06.424 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:06.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:06.424 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 540976, 2024-09-21T03:58:06.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 154928 2024-09-21T03:58:06.425 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:06.425 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:58:06.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2, 2024-09-21T03:58:06.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 192, 2024-09-21T03:58:06.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 320, 2024-09-21T03:58:06.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 194528, 2024-09-21T03:58:06.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-21T03:58:06.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-21T03:58:06.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.051571440, 2024-09-21T03:58:06.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.045190826, 2024-09-21T03:58:06.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.003379728, 2024-09-21T03:58:06.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:58:06.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:06.425 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 190, 2024-09-21T03:58:06.426 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.069342744, 2024-09-21T03:58:06.426 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000364961 2024-09-21T03:58:06.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:06.426 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:06.426 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:06.426 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:06.426 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:06.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:06.426 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:58:06.426 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 194528 2024-09-21T03:58:06.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:06.426 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-21T03:58:06.426 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-21T03:58:06.426 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-21T03:58:06.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:06.427 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-21T03:58:06.427 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:58:06.427 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-21T03:58:06.427 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:58:06.427 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:58:06.427 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:58:06.427 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:58:06.427 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:58:06.427 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:58:06.427 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:58:06.427 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:58:06.427 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:58:06.427 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:58:06.428 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:58:06.428 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:58:06.428 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:58:06.428 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-21T03:58:06.428 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:58:06.428 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-21T03:58:06.428 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:58:06.428 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:58:06.428 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:58:06.428 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:58:06.428 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:58:06.428 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:58:06.428 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:58:06.429 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-21T03:58:06.429 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:58:06.429 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:58:06.429 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:58:06.429 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-21T03:58:06.429 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-21T03:58:06.429 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-21T03:58:06.429 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:58:06.429 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:58:06.429 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:58:06.429 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:58:06.429 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148951, 2024-09-21T03:58:06.429 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 28, 2024-09-21T03:58:06.430 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:58:06.430 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-21T03:58:06.430 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-21T03:58:06.430 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-21T03:58:06.430 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:58:06.430 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-21T03:58:06.430 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:58:06.430 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-21T03:58:06.430 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-21T03:58:06.430 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-21T03:58:06.430 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:58:06.430 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:58:06.430 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-21T03:58:06.430 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-21T03:58:06.431 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 968, 2024-09-21T03:58:06.431 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-21T03:58:06.431 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-21T03:58:06.431 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-21T03:58:06.431 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-21T03:58:06.431 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-21T03:58:06.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:06.431 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:58:06.431 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-21T03:58:06.431 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-21T03:58:06.431 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-21T03:58:06.431 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 255, 2024-09-21T03:58:06.431 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 502306, 2024-09-21T03:58:06.432 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:06.432 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 255, 2024-09-21T03:58:06.432 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-21T03:58:06.432 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-21T03:58:06.432 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 255, 2024-09-21T03:58:06.432 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 502306, 2024-09-21T03:58:06.432 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-21T03:58:06.432 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:06.432 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:06.432 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:06.432 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:06.432 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:06.432 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:58:07.252 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-21T03:58:07.253 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-21T03:58:07.253 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T03:58:07.253 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T03:58:07.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:07 smithi134 ceph-mon[35031]: pgmap v772: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 237 op/s 2024-09-21T03:58:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:58:07.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:07 smithi123 ceph-mon[34673]: pgmap v772: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 237 op/s 2024-09-21T03:58:07.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:07 smithi016 ceph-mon[30375]: pgmap v772: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 237 op/s 2024-09-21T03:58:08.355 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:58:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:58:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 836, 2024-09-21T03:58:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 838, 2024-09-21T03:58:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 280841, 2024-09-21T03:58:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 265849, 2024-09-21T03:58:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-09-21T03:58:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-21T03:58:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.246263141, 2024-09-21T03:58:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.129614208, 2024-09-21T03:58:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.040728583, 2024-09-21T03:58:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.046285683, 2024-09-21T03:58:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 836, 2024-09-21T03:58:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.281674343, 2024-09-21T03:58:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000336931 2024-09-21T03:58:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:58:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:58:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:58:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1805, 2024-09-21T03:58:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 1800, 2024-09-21T03:58:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 962693, 2024-09-21T03:58:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 645107, 2024-09-21T03:58:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-21T03:58:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-09-21T03:58:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.329754508, 2024-09-21T03:58:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.149072833, 2024-09-21T03:58:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.078127242, 2024-09-21T03:58:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.093003120, 2024-09-21T03:58:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1799, 2024-09-21T03:58:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.330874957, 2024-09-21T03:58:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000183921 2024-09-21T03:58:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 567808, 2024-09-21T03:58:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 153104 2024-09-21T03:58:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:58:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 9, 2024-09-21T03:58:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 194, 2024-09-21T03:58:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 4612, 2024-09-21T03:58:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 493205, 2024-09-21T03:58:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-09-21T03:58:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-09-21T03:58:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.037054203, 2024-09-21T03:58:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.027546000, 2024-09-21T03:58:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.006922160, 2024-09-21T03:58:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000082095, 2024-09-21T03:58:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 193, 2024-09-21T03:58:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.064003646, 2024-09-21T03:58:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000331625 2024-09-21T03:58:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-21T03:58:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 492240 2024-09-21T03:58:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-21T03:58:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-09-21T03:58:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-21T03:58:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-09-21T03:58:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-09-21T03:58:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-09-21T03:58:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3782, 2024-09-21T03:58:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.190691635, 2024-09-21T03:58:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000050420 2024-09-21T03:58:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-09-21T03:58:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-09-21T03:58:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-09-21T03:58:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-09-21T03:58:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-09-21T03:58:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-09-21T03:58:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-09-21T03:58:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-09-21T03:58:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-09-21T03:58:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-09-21T03:58:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-09-21T03:58:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-09-21T03:58:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-09-21T03:58:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-09-21T03:58:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 0, 2024-09-21T03:58:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2024-09-21T03:58:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 0, 2024-09-21T03:58:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-09-21T03:58:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 0, 2024-09-21T03:58:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-09-21T03:58:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-09-21T03:58:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-09-21T03:58:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 1, 2024-09-21T03:58:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-09-21T03:58:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-09-21T03:58:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-09-21T03:58:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-09-21T03:58:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:58:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-09-21T03:58:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-09-21T03:58:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-09-21T03:58:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-09-21T03:58:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-09-21T03:58:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-09-21T03:58:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-09-21T03:58:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-09-21T03:58:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-09-21T03:58:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:58:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-09-21T03:58:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2024-09-21T03:58:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2024-09-21T03:58:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-09-21T03:58:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:58:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-09-21T03:58:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:58:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:58:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:58:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:58:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:58:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-09-21T03:58:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:58:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:58:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:58:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:58:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:58:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:58:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-09-21T03:58:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:58:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-09-21T03:58:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-09-21T03:58:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-09-21T03:58:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-09-21T03:58:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-09-21T03:58:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-09-21T03:58:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-09-21T03:58:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-09-21T03:58:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-09-21T03:58:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-09-21T03:58:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:58:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-09-21T03:58:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-09-21T03:58:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-09-21T03:58:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-09-21T03:58:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-09-21T03:58:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:58:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:58:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-09-21T03:58:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-09-21T03:58:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:58:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:58:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:58:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-09-21T03:58:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-09-21T03:58:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:58:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-09-21T03:58:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0, 2024-09-21T03:58:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:58:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_succeeded": 0, 2024-09-21T03:58:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_write_failed": 0 2024-09-21T03:58:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-09-21T03:58:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-09-21T03:58:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-09-21T03:58:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-09-21T03:58:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-09-21T03:58:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 3, 2024-09-21T03:58:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-09-21T03:58:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-09-21T03:58:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-09-21T03:58:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-09-21T03:58:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-09-21T03:58:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 2, 2024-09-21T03:58:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-09-21T03:58:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-09-21T03:58:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-09-21T03:58:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-09-21T03:58:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 4195203, 2024-09-21T03:58:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 4194892, 2024-09-21T03:58:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-09-21T03:58:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 3 2024-09-21T03:58:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-09-21T03:58:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 3, 2024-09-21T03:58:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 3, 2024-09-21T03:58:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 0, 2024-09-21T03:58:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 1, 2024-09-21T03:58:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 1, 2024-09-21T03:58:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 0, 2024-09-21T03:58:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 0, 2024-09-21T03:58:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 0, 2024-09-21T03:58:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 0, 2024-09-21T03:58:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-09-21T03:58:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-09-21T03:58:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-09-21T03:58:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 115276, 2024-09-21T03:58:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 316428288 2024-09-21T03:58:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-09-21T03:58:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-09-21T03:58:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-09-21T03:58:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-09-21T03:58:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-09-21T03:58:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-09-21T03:58:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-09-21T03:58:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-09-21T03:58:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-09-21T03:58:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-09-21T03:58:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-09-21T03:58:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-09-21T03:58:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-09-21T03:58:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-09-21T03:58:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-09-21T03:58:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-09-21T03:58:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-09-21T03:58:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-09-21T03:58:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-09-21T03:58:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-09-21T03:58:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-09-21T03:58:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-09-21T03:58:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-09-21T03:58:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-09-21T03:58:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-09-21T03:58:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-09-21T03:58:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-09-21T03:58:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-09-21T03:58:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-09-21T03:58:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-09-21T03:58:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-09-21T03:58:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-09-21T03:58:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-09-21T03:58:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-09-21T03:58:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-09-21T03:58:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:58:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:58:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-09-21T03:58:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-09-21T03:58:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2024-09-21T03:58:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 1, 2024-09-21T03:58:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 0, 2024-09-21T03:58:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 1, 2024-09-21T03:58:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-09-21T03:58:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-09-21T03:58:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-09-21T03:58:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 1967, 2024-09-21T03:58:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-09-21T03:58:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-21T03:58:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:58:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-21T03:58:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:58:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:58:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:58:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:58:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:58:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:58:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:58:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:58:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:58:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:58:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:58:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:58:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:58:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-21T03:58:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:58:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-21T03:58:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:58:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:58:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:58:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:58:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:58:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:58:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:58:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-21T03:58:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:58:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:58:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:58:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-21T03:58:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-21T03:58:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-21T03:58:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:58:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:58:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:58:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:58:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 197447, 2024-09-21T03:58:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 53, 2024-09-21T03:58:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:58:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-21T03:58:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-21T03:58:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-21T03:58:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:58:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-21T03:58:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:58:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-21T03:58:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 8952, 2024-09-21T03:58:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 168, 2024-09-21T03:58:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:58:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:58:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-21T03:58:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-21T03:58:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 8672, 2024-09-21T03:58:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 13, 2024-09-21T03:58:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-21T03:58:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-21T03:58:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-21T03:58:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-21T03:58:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-09-21T03:58:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-09-21T03:58:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-09-21T03:58:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 2386, 2024-09-21T03:58:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 21, 2024-09-21T03:58:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 3, 2024-09-21T03:58:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 2386, 2024-09-21T03:58:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-09-21T03:58:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2386, 2024-09-21T03:58:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 7.232887882, 2024-09-21T03:58:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.003031386 2024-09-21T03:58:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-09-21T03:58:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-09-21T03:58:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2386, 2024-09-21T03:58:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 2387 2024-09-21T03:58:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 2386, 2024-09-21T03:58:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 2386, 2024-09-21T03:58:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-09-21T03:58:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-09-21T03:58:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-09-21T03:58:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 1544, 2024-09-21T03:58:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-09-21T03:58:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 840, 2024-09-21T03:58:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-09-21T03:58:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-09-21T03:58:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-09-21T03:58:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-09-21T03:58:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-09-21T03:58:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-09-21T03:58:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-09-21T03:58:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-09-21T03:58:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-09-21T03:58:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-09-21T03:58:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1, 2024-09-21T03:58:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-09-21T03:58:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:58:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-09-21T03:58:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:58:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-09-21T03:58:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-09-21T03:58:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-09-21T03:58:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:58:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-09-21T03:58:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:58:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-09-21T03:58:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-09-21T03:58:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-09-21T03:58:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-09-21T03:58:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-09-21T03:58:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-09-21T03:58:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-09-21T03:58:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-09-21T03:58:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-09-21T03:58:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-09-21T03:58:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-09-21T03:58:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-09-21T03:58:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-09-21T03:58:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-09-21T03:58:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-09-21T03:58:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-09-21T03:58:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-09-21T03:58:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 86, 2024-09-21T03:58:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-09-21T03:58:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 2, 2024-09-21T03:58:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 6, 2024-09-21T03:58:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 6, 2024-09-21T03:58:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-09-21T03:58:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-09-21T03:58:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-09-21T03:58:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-09-21T03:58:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-09-21T03:58:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-09-21T03:58:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-09-21T03:58:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:58:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-09-21T03:58:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-09-21T03:58:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-09-21T03:58:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-09-21T03:58:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:58:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-09-21T03:58:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-09-21T03:58:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-09-21T03:58:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-09-21T03:58:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-09-21T03:58:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:58:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-21T03:58:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-21T03:58:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-21T03:58:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 2646, 2024-09-21T03:58:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 993778, 2024-09-21T03:58:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 2646, 2024-09-21T03:58:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-21T03:58:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-21T03:58:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2646, 2024-09-21T03:58:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 993778, 2024-09-21T03:58:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-21T03:58:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-09-21T03:58:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-21T03:58:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-21T03:58:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-21T03:58:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-21T03:58:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-21T03:58:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2386, 2024-09-21T03:58:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 904, 2024-09-21T03:58:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2386, 2024-09-21T03:58:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 904, 2024-09-21T03:58:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-21T03:58:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-09-21T03:58:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-21T03:58:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-09-21T03:58:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-21T03:58:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-21T03:58:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-21T03:58:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2386, 2024-09-21T03:58:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 2386, 2024-09-21T03:58:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2386, 2024-09-21T03:58:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2386, 2024-09-21T03:58:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-21T03:58:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:58:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-21T03:58:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-09-21T03:58:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-21T03:58:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-21T03:58:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-21T03:58:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-21T03:58:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-21T03:58:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-09-21T03:58:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-09-21T03:58:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-21T03:58:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x55f847f436c0": { 2024-09-21T03:58:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-21T03:58:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-09-21T03:58:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-21T03:58:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-21T03:58:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-21T03:58:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-21T03:58:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-21T03:58:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-09-21T03:58:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-09-21T03:58:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-21T03:58:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "trackedop": { 2024-09-21T03:58:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-09-21T03:58:08.401 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:08.401 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:58:09.347 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-21T03:58:09.347 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-21T03:58:09.347 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T03:58:09.347 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T03:58:09.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:09 smithi134 ceph-mon[35031]: pgmap v773: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 127 KiB/s wr, 18 op/s 2024-09-21T03:58:09.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:09 smithi123 ceph-mon[34673]: pgmap v773: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 127 KiB/s wr, 18 op/s 2024-09-21T03:58:09.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:09 smithi016 ceph-mon[30375]: pgmap v773: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 127 KiB/s wr, 18 op/s 2024-09-21T03:58:09.730 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:58:10.516 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:58:10.516 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:58:10.516 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-09-21T03:58:10.516 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4, 2024-09-21T03:58:10.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 2944, 2024-09-21T03:58:10.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 448, 2024-09-21T03:58:10.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-21T03:58:10.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-21T03:58:10.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.006224744, 2024-09-21T03:58:10.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000056958, 2024-09-21T03:58:10.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.003206753, 2024-09-21T03:58:10.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:58:10.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:10.517 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-09-21T03:58:10.517 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.005465680, 2024-09-21T03:58:10.517 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001821893 2024-09-21T03:58:10.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:10.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:10.518 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:10.518 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:10.518 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:10.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:10.518 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:58:10.518 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:58:10.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:10.518 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:58:10.518 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2, 2024-09-21T03:58:10.518 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 193, 2024-09-21T03:58:10.518 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 320, 2024-09-21T03:58:10.518 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 194704, 2024-09-21T03:58:10.518 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-21T03:58:10.518 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-21T03:58:10.519 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.031976824, 2024-09-21T03:58:10.519 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.026667517, 2024-09-21T03:58:10.519 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002502790, 2024-09-21T03:58:10.519 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:58:10.519 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:10.519 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 191, 2024-09-21T03:58:10.519 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.057667203, 2024-09-21T03:58:10.519 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000301922 2024-09-21T03:58:10.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:10.519 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:10.519 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:10.519 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:10.519 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:10.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:10.520 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:58:10.520 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 194704 2024-09-21T03:58:10.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:10.520 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:58:10.520 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 255, 2024-09-21T03:58:10.520 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 252, 2024-09-21T03:58:10.520 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 538576, 2024-09-21T03:58:10.520 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 155760, 2024-09-21T03:58:10.520 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-21T03:58:10.520 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-21T03:58:10.520 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.071978299, 2024-09-21T03:58:10.520 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.040624114, 2024-09-21T03:58:10.520 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.036821278, 2024-09-21T03:58:10.520 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:58:10.520 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:10.521 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 251, 2024-09-21T03:58:10.521 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.080260803, 2024-09-21T03:58:10.521 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000319764 2024-09-21T03:58:10.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:10.521 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:10.521 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:10.521 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:10.521 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:10.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:10.521 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 538576, 2024-09-21T03:58:10.521 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 155760 2024-09-21T03:58:10.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:10.521 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-21T03:58:10.521 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-21T03:58:10.521 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-21T03:58:10.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:10.522 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-21T03:58:10.522 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:58:10.522 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-21T03:58:10.522 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:58:10.522 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:58:10.522 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:58:10.522 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:58:10.522 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:58:10.522 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:58:10.522 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:58:10.522 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:58:10.522 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:58:10.522 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:58:10.523 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:58:10.523 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:58:10.523 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:58:10.523 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-21T03:58:10.523 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:58:10.523 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-21T03:58:10.523 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:58:10.523 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:58:10.523 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:58:10.523 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:58:10.523 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:58:10.523 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:58:10.523 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:58:10.523 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-21T03:58:10.523 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:58:10.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:58:10.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:58:10.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-21T03:58:10.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-21T03:58:10.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-21T03:58:10.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:58:10.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:58:10.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:58:10.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:58:10.524 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148704, 2024-09-21T03:58:10.524 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 29, 2024-09-21T03:58:10.524 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:58:10.524 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-21T03:58:10.524 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-21T03:58:10.524 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-21T03:58:10.525 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:58:10.525 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-21T03:58:10.525 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:58:10.525 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-21T03:58:10.525 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-21T03:58:10.525 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-21T03:58:10.525 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:58:10.525 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:58:10.525 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-21T03:58:10.525 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-21T03:58:10.525 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 968, 2024-09-21T03:58:10.525 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-21T03:58:10.525 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-21T03:58:10.525 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-21T03:58:10.526 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-21T03:58:10.526 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-21T03:58:10.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:58:10.526 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:58:10.526 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-21T03:58:10.526 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-21T03:58:10.526 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-21T03:58:10.526 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 257, 2024-09-21T03:58:10.526 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 502566, 2024-09-21T03:58:10.526 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:10.526 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 257, 2024-09-21T03:58:10.526 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-21T03:58:10.526 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-21T03:58:10.526 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 257, 2024-09-21T03:58:10.527 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 502566, 2024-09-21T03:58:10.527 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-21T03:58:10.527 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:58:10.527 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:58:10.527 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:58:10.527 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:10.527 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:58:10.527 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:58:11.329 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-21T03:58:11.329 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-21T03:58:11.329 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T03:58:11.329 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T03:58:11.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:11 smithi134 ceph-mon[35031]: pgmap v774: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-09-21T03:58:11.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:11 smithi123 ceph-mon[34673]: pgmap v774: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-09-21T03:58:11.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:11 smithi016 ceph-mon[30375]: pgmap v774: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-09-21T03:58:11.716 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:58:12.347 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-21T03:58:12.347 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:58:12.347 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 3645, 2024-09-21T03:58:12.347 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 3199, 2024-09-21T03:58:12.347 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 1651589, 2024-09-21T03:58:12.347 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 2610911, 2024-09-21T03:58:12.347 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 16, 2024-09-21T03:58:12.347 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 5, 2024-09-21T03:58:12.348 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.376989679, 2024-09-21T03:58:12.348 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.142982678, 2024-09-21T03:58:12.348 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.200451507, 2024-09-21T03:58:12.348 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.057181464, 2024-09-21T03:58:12.348 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:12.348 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3198, 2024-09-21T03:58:12.348 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.961838607, 2024-09-21T03:58:12.348 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000300762 2024-09-21T03:58:12.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.348 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:12.348 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2543, 2024-09-21T03:58:12.348 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000874947, 2024-09-21T03:58:12.349 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000344 2024-09-21T03:58:12.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.349 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 660976, 2024-09-21T03:58:12.349 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 192336 2024-09-21T03:58:12.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.349 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:58:12.349 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1240, 2024-09-21T03:58:12.349 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 1421, 2024-09-21T03:58:12.349 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 300365, 2024-09-21T03:58:12.349 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 1124561, 2024-09-21T03:58:12.349 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 10, 2024-09-21T03:58:12.349 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 7, 2024-09-21T03:58:12.349 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.132693025, 2024-09-21T03:58:12.349 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.059815899, 2024-09-21T03:58:12.350 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.057419390, 2024-09-21T03:58:12.350 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.017864429, 2024-09-21T03:58:12.350 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:12.350 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1420, 2024-09-21T03:58:12.350 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.497211165, 2024-09-21T03:58:12.350 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000350148 2024-09-21T03:58:12.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.350 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:12.350 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1355, 2024-09-21T03:58:12.350 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000435793, 2024-09-21T03:58:12.350 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000321 2024-09-21T03:58:12.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.350 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-09-21T03:58:12.351 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 422720 2024-09-21T03:58:12.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.351 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:58:12.351 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 2907, 2024-09-21T03:58:12.351 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 3191, 2024-09-21T03:58:12.351 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 859548, 2024-09-21T03:58:12.351 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 7088279, 2024-09-21T03:58:12.351 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 161, 2024-09-21T03:58:12.351 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 9, 2024-09-21T03:58:12.351 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.356070920, 2024-09-21T03:58:12.351 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.156198188, 2024-09-21T03:58:12.351 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.175021355, 2024-09-21T03:58:12.352 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.037294328, 2024-09-21T03:58:12.352 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:12.352 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3189, 2024-09-21T03:58:12.352 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.761023731, 2024-09-21T03:58:12.352 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000238640 2024-09-21T03:58:12.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.352 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:12.352 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3375, 2024-09-21T03:58:12.352 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000768373, 2024-09-21T03:58:12.352 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000227 2024-09-21T03:58:12.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.352 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:58:12.352 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:58:12.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.352 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-09-21T03:58:12.353 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-09-21T03:58:12.353 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-09-21T03:58:12.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.353 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-09-21T03:58:12.353 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-21T03:58:12.353 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-21T03:58:12.353 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2061, 2024-09-21T03:58:12.353 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.922057046, 2024-09-21T03:58:12.353 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000447383 2024-09-21T03:58:12.353 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:12.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.353 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-09-21T03:58:12.353 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-21T03:58:12.354 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-21T03:58:12.354 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 45, 2024-09-21T03:58:12.354 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.002554994, 2024-09-21T03:58:12.354 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000056777 2024-09-21T03:58:12.354 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:12.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.354 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-09-21T03:58:12.354 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 265, 2024-09-21T03:58:12.354 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 265, 2024-09-21T03:58:12.354 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-09-21T03:58:12.354 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 265, 2024-09-21T03:58:12.354 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.105058906, 2024-09-21T03:58:12.354 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.004170033 2024-09-21T03:58:12.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.355 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-09-21T03:58:12.355 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 0, 2024-09-21T03:58:12.355 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-09-21T03:58:12.355 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-09-21T03:58:12.355 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 390, 2024-09-21T03:58:12.355 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2024-09-21T03:58:12.355 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-09-21T03:58:12.355 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 45, 2024-09-21T03:58:12.355 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 5, 2024-09-21T03:58:12.355 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 40, 2024-09-21T03:58:12.355 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 0, 2024-09-21T03:58:12.355 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 37, 2024-09-21T03:58:12.355 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 0, 2024-09-21T03:58:12.356 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 7, 2024-09-21T03:58:12.356 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 7, 2024-09-21T03:58:12.356 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 2, 2024-09-21T03:58:12.356 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 478, 2024-09-21T03:58:12.356 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 469, 2024-09-21T03:58:12.356 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 0, 2024-09-21T03:58:12.356 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 0, 2024-09-21T03:58:12.356 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:58:12.356 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-09-21T03:58:12.356 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2024-09-21T03:58:12.356 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 660, 2024-09-21T03:58:12.357 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-09-21T03:58:12.357 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2024-09-21T03:58:12.357 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2024-09-21T03:58:12.357 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2024-09-21T03:58:12.357 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2024-09-21T03:58:12.357 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-09-21T03:58:12.357 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:58:12.357 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-09-21T03:58:12.357 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 4, 2024-09-21T03:58:12.357 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 234, 2024-09-21T03:58:12.357 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-09-21T03:58:12.357 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:58:12.357 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-09-21T03:58:12.357 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:58:12.358 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:58:12.358 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:58:12.358 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:58:12.358 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:58:12.358 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-09-21T03:58:12.358 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 3, 2024-09-21T03:58:12.358 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 219, 2024-09-21T03:58:12.358 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 428, 2024-09-21T03:58:12.358 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:58:12.358 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:58:12.358 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:58:12.358 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 1250, 2024-09-21T03:58:12.358 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 1051, 2024-09-21T03:58:12.358 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 6, 2024-09-21T03:58:12.359 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 51 2024-09-21T03:58:12.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.359 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-09-21T03:58:12.359 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2024-09-21T03:58:12.359 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2024-09-21T03:58:12.359 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2024-09-21T03:58:12.359 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 0, 2024-09-21T03:58:12.359 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 6, 2024-09-21T03:58:12.359 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 5, 2024-09-21T03:58:12.359 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-09-21T03:58:12.359 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:58:12.359 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 22, 2024-09-21T03:58:12.359 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 17, 2024-09-21T03:58:12.359 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-09-21T03:58:12.360 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-09-21T03:58:12.360 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-09-21T03:58:12.360 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:58:12.360 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:58:12.360 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-09-21T03:58:12.360 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-09-21T03:58:12.360 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 11, 2024-09-21T03:58:12.360 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 342, 2024-09-21T03:58:12.360 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:58:12.360 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2024-09-21T03:58:12.360 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-09-21T03:58:12.360 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:58:12.360 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-09-21T03:58:12.360 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0, 2024-09-21T03:58:12.361 INFO:teuthology.orchestra.run.smithi123.stdout: "uninline_started": 0, 2024-09-21T03:58:12.361 INFO:teuthology.orchestra.run.smithi123.stdout: "uninline_succeeded": 0, 2024-09-21T03:58:12.361 INFO:teuthology.orchestra.run.smithi123.stdout: "uninline_write_failed": 0 2024-09-21T03:58:12.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.361 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-09-21T03:58:12.361 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-09-21T03:58:12.361 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 1695, 2024-09-21T03:58:12.361 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 1670, 2024-09-21T03:58:12.361 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 1670, 2024-09-21T03:58:12.361 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 25, 2024-09-21T03:58:12.361 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-09-21T03:58:12.361 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 39, 2024-09-21T03:58:12.361 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 54, 2024-09-21T03:58:12.361 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 53, 2024-09-21T03:58:12.362 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 53, 2024-09-21T03:58:12.362 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 1, 2024-09-21T03:58:12.362 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 1, 2024-09-21T03:58:12.362 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-09-21T03:58:12.362 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 1, 2024-09-21T03:58:12.362 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 9659672, 2024-09-21T03:58:12.362 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 9692971, 2024-09-21T03:58:12.362 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-09-21T03:58:12.362 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-09-21T03:58:12.362 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 835, 2024-09-21T03:58:12.362 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 3.179762632, 2024-09-21T03:58:12.362 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.003808098 2024-09-21T03:58:12.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.363 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-09-21T03:58:12.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.363 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-09-21T03:58:12.363 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 28, 2024-09-21T03:58:12.363 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 45, 2024-09-21T03:58:12.363 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 17, 2024-09-21T03:58:12.363 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 18, 2024-09-21T03:58:12.363 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 22, 2024-09-21T03:58:12.363 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 4, 2024-09-21T03:58:12.363 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 45, 2024-09-21T03:58:12.363 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 67, 2024-09-21T03:58:12.363 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 22, 2024-09-21T03:58:12.363 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 7, 2024-09-21T03:58:12.363 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 49, 2024-09-21T03:58:12.364 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 42, 2024-09-21T03:58:12.364 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 126676, 2024-09-21T03:58:12.364 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 339484672 2024-09-21T03:58:12.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.364 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-09-21T03:58:12.364 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 344, 2024-09-21T03:58:12.364 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2024-09-21T03:58:12.364 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 265, 2024-09-21T03:58:12.364 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 20, 2024-09-21T03:58:12.364 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 0, 2024-09-21T03:58:12.364 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-09-21T03:58:12.364 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 22, 2024-09-21T03:58:12.364 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.018946359, 2024-09-21T03:58:12.364 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000861198 2024-09-21T03:58:12.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.364 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-09-21T03:58:12.365 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 66, 2024-09-21T03:58:12.365 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.450291185, 2024-09-21T03:58:12.365 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.006822593 2024-09-21T03:58:12.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.365 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-09-21T03:58:12.365 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.365 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.365 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.365 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-09-21T03:58:12.365 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.365 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.365 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.366 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-09-21T03:58:12.366 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 124, 2024-09-21T03:58:12.366 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.516173269, 2024-09-21T03:58:12.366 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.004162687 2024-09-21T03:58:12.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.366 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-09-21T03:58:12.366 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.366 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.366 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.366 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-09-21T03:58:12.366 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-09-21T03:58:12.366 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000327837, 2024-09-21T03:58:12.367 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000327837 2024-09-21T03:58:12.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.367 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-09-21T03:58:12.367 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.367 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.367 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.367 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-09-21T03:58:12.367 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.367 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.367 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.367 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-09-21T03:58:12.368 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.368 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.368 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.368 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-09-21T03:58:12.368 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.369 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.369 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.369 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-09-21T03:58:12.369 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 10, 2024-09-21T03:58:12.369 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.016703606, 2024-09-21T03:58:12.369 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001670360 2024-09-21T03:58:12.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.369 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-09-21T03:58:12.369 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.369 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.369 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.369 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-09-21T03:58:12.370 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.370 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.370 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.370 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-09-21T03:58:12.370 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-09-21T03:58:12.370 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000178416, 2024-09-21T03:58:12.370 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000178416 2024-09-21T03:58:12.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.370 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-09-21T03:58:12.370 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 5, 2024-09-21T03:58:12.370 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.011419862, 2024-09-21T03:58:12.370 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002283972 2024-09-21T03:58:12.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.371 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-09-21T03:58:12.371 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2024-09-21T03:58:12.371 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.002925864, 2024-09-21T03:58:12.371 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001462932 2024-09-21T03:58:12.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.371 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-09-21T03:58:12.371 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.371 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.371 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.371 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-09-21T03:58:12.371 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.371 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.372 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.372 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-09-21T03:58:12.372 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 4, 2024-09-21T03:58:12.372 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.037045985, 2024-09-21T03:58:12.372 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.009261496 2024-09-21T03:58:12.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.372 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-09-21T03:58:12.372 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.372 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.372 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.372 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-09-21T03:58:12.372 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.372 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.373 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.373 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-09-21T03:58:12.373 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2024-09-21T03:58:12.373 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.002090189, 2024-09-21T03:58:12.373 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001045094 2024-09-21T03:58:12.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.373 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-09-21T03:58:12.373 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.373 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.373 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.373 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-09-21T03:58:12.373 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.374 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.374 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.374 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-09-21T03:58:12.374 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.374 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.374 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.374 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-09-21T03:58:12.374 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 10, 2024-09-21T03:58:12.374 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.026371846, 2024-09-21T03:58:12.374 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002637184 2024-09-21T03:58:12.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.374 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-09-21T03:58:12.375 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.375 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.375 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.375 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-09-21T03:58:12.375 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 18, 2024-09-21T03:58:12.375 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.022584488, 2024-09-21T03:58:12.375 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001254693 2024-09-21T03:58:12.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.375 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:58:12.375 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:58:12.375 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-09-21T03:58:12.375 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.376 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.376 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.376 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:12.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.376 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-09-21T03:58:12.376 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-09-21T03:58:12.376 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 3, 2024-09-21T03:58:12.376 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 2, 2024-09-21T03:58:12.376 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2024-09-21T03:58:12.376 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-09-21T03:58:12.376 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 55, 2024-09-21T03:58:12.376 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 55, 2024-09-21T03:58:12.376 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 715, 2024-09-21T03:58:12.376 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-09-21T03:58:12.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.377 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-09-21T03:58:12.377 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:58:12.377 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-09-21T03:58:12.377 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:58:12.377 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:58:12.377 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:58:12.377 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:58:12.377 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:58:12.377 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:58:12.377 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:58:12.377 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:58:12.377 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:58:12.377 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:58:12.378 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:58:12.378 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:58:12.378 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:58:12.378 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-09-21T03:58:12.378 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:58:12.378 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-09-21T03:58:12.378 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:58:12.378 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:58:12.378 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:58:12.378 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:58:12.378 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:58:12.378 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:58:12.378 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:58:12.379 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-09-21T03:58:12.379 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:58:12.379 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:58:12.379 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:58:12.379 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-09-21T03:58:12.379 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-09-21T03:58:12.379 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-09-21T03:58:12.379 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:58:12.379 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:58:12.379 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:58:12.379 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:58:12.379 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 311294, 2024-09-21T03:58:12.379 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 118, 2024-09-21T03:58:12.379 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:58:12.379 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-09-21T03:58:12.380 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-09-21T03:58:12.380 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-09-21T03:58:12.380 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:58:12.380 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-09-21T03:58:12.380 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:58:12.380 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-09-21T03:58:12.380 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8952, 2024-09-21T03:58:12.380 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 168, 2024-09-21T03:58:12.380 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:58:12.380 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:58:12.380 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-09-21T03:58:12.380 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-09-21T03:58:12.380 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 124829, 2024-09-21T03:58:12.380 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 1512, 2024-09-21T03:58:12.381 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-09-21T03:58:12.381 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-09-21T03:58:12.381 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-09-21T03:58:12.381 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-09-21T03:58:12.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.381 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-09-21T03:58:12.381 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-09-21T03:58:12.381 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-09-21T03:58:12.381 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 2107, 2024-09-21T03:58:12.381 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 6074290, 2024-09-21T03:58:12.381 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 37, 2024-09-21T03:58:12.381 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 2107, 2024-09-21T03:58:12.381 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-09-21T03:58:12.381 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2107, 2024-09-21T03:58:12.382 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 38.854039161, 2024-09-21T03:58:12.382 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.018440455 2024-09-21T03:58:12.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.382 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-09-21T03:58:12.382 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-09-21T03:58:12.382 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2107, 2024-09-21T03:58:12.382 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 3040 2024-09-21T03:58:12.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.382 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 2107, 2024-09-21T03:58:12.382 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 2, 2024-09-21T03:58:12.382 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 2105, 2024-09-21T03:58:12.382 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-09-21T03:58:12.382 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-09-21T03:58:12.382 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 368, 2024-09-21T03:58:12.383 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 88, 2024-09-21T03:58:12.383 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 2, 2024-09-21T03:58:12.383 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 837, 2024-09-21T03:58:12.383 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 120, 2024-09-21T03:58:12.383 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-09-21T03:58:12.383 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-09-21T03:58:12.383 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-09-21T03:58:12.383 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-09-21T03:58:12.383 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 544, 2024-09-21T03:58:12.383 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-09-21T03:58:12.383 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-09-21T03:58:12.383 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-09-21T03:58:12.383 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 0, 2024-09-21T03:58:12.383 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 154, 2024-09-21T03:58:12.384 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:58:12.384 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-09-21T03:58:12.384 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:58:12.384 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-09-21T03:58:12.384 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-09-21T03:58:12.384 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-09-21T03:58:12.384 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:58:12.384 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-09-21T03:58:12.384 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:58:12.384 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-09-21T03:58:12.384 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-09-21T03:58:12.384 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-09-21T03:58:12.384 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-09-21T03:58:12.384 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-09-21T03:58:12.385 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-09-21T03:58:12.385 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-09-21T03:58:12.385 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-09-21T03:58:12.385 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-09-21T03:58:12.385 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-09-21T03:58:12.385 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-09-21T03:58:12.385 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-09-21T03:58:12.385 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-09-21T03:58:12.385 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-09-21T03:58:12.385 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-09-21T03:58:12.385 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-09-21T03:58:12.385 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-09-21T03:58:12.385 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 86, 2024-09-21T03:58:12.385 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-09-21T03:58:12.385 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 11, 2024-09-21T03:58:12.386 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 12, 2024-09-21T03:58:12.386 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 12, 2024-09-21T03:58:12.386 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-09-21T03:58:12.386 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-09-21T03:58:12.386 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 907, 2024-09-21T03:58:12.386 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 0, 2024-09-21T03:58:12.386 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 20 2024-09-21T03:58:12.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.386 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-09-21T03:58:12.386 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-09-21T03:58:12.386 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:58:12.386 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-09-21T03:58:12.386 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-09-21T03:58:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-09-21T03:58:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-09-21T03:58:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 10, 2024-09-21T03:58:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-09-21T03:58:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 5, 2024-09-21T03:58:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 30, 2024-09-21T03:58:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 17, 2024-09-21T03:58:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-09-21T03:58:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:58:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-21T03:58:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 7788, 2024-09-21T03:58:12.388 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 2121207, 2024-09-21T03:58:12.388 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:12.388 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 7788, 2024-09-21T03:58:12.388 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-21T03:58:12.388 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-21T03:58:12.388 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 7788, 2024-09-21T03:58:12.388 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 2121207, 2024-09-21T03:58:12.388 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:12.388 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.388 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.388 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.388 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:12.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.388 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-09-21T03:58:12.389 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:12.389 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-21T03:58:12.389 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:12.389 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-21T03:58:12.389 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-21T03:58:12.389 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:12.389 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:12.389 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 2107, 2024-09-21T03:58:12.389 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 6076111, 2024-09-21T03:58:12.389 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2107, 2024-09-21T03:58:12.389 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 6076111, 2024-09-21T03:58:12.389 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:12.389 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.390 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.390 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.390 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:12.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.390 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-09-21T03:58:12.390 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:12.390 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-09-21T03:58:12.390 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:12.390 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-21T03:58:12.390 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-21T03:58:12.390 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:12.390 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:12.390 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 2107, 2024-09-21T03:58:12.391 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 2107, 2024-09-21T03:58:12.391 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2107, 2024-09-21T03:58:12.391 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 2107, 2024-09-21T03:58:12.391 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:12.391 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.391 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.391 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.391 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:12.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.391 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:58:12.391 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:12.391 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-21T03:58:12.391 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:12.391 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 17, 2024-09-21T03:58:12.391 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1821, 2024-09-21T03:58:12.392 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:12.392 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 17, 2024-09-21T03:58:12.392 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-21T03:58:12.392 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-21T03:58:12.392 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2, 2024-09-21T03:58:12.392 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1821, 2024-09-21T03:58:12.392 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:12.392 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.392 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.392 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.392 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:12.392 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.392 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x556e52359450": { 2024-09-21T03:58:12.392 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:12.392 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-21T03:58:12.393 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:12.393 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 1695, 2024-09-21T03:58:12.393 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 5498667, 2024-09-21T03:58:12.393 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:12.393 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 1695, 2024-09-21T03:58:12.393 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-21T03:58:12.393 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-21T03:58:12.393 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 835, 2024-09-21T03:58:12.393 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 5498667, 2024-09-21T03:58:12.393 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:12.393 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:12.393 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:12.393 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:12.393 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:12.394 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:12.394 INFO:teuthology.orchestra.run.smithi123.stdout: "trackedop": { 2024-09-21T03:58:12.394 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 319 2024-09-21T03:58:12.394 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:12.394 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-21T03:58:13.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:13 smithi134 ceph-mon[35031]: pgmap v775: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-21T03:58:13.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:13 smithi123 ceph-mon[34673]: pgmap v775: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-21T03:58:13.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:13 smithi016 ceph-mon[30375]: pgmap v775: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-21T03:58:13.747 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-21T03:58:13.747 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-21T03:58:13.747 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T03:58:13.748 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T03:58:14.000 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:58:14.564 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-21T03:58:14.565 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:58:14.565 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 99, 2024-09-21T03:58:14.565 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 115, 2024-09-21T03:58:14.565 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 34657, 2024-09-21T03:58:14.565 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 113864, 2024-09-21T03:58:14.565 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 16, 2024-09-21T03:58:14.565 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-09-21T03:58:14.565 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.021015863, 2024-09-21T03:58:14.566 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.006980172, 2024-09-21T03:58:14.566 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.011842321, 2024-09-21T03:58:14.566 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000359858, 2024-09-21T03:58:14.566 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:14.566 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 111, 2024-09-21T03:58:14.566 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.021082092, 2024-09-21T03:58:14.566 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000189928 2024-09-21T03:58:14.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.566 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:14.566 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 142, 2024-09-21T03:58:14.566 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000078049, 2024-09-21T03:58:14.567 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000549 2024-09-21T03:58:14.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.567 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:58:14.567 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:58:14.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.567 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:58:14.567 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1423, 2024-09-21T03:58:14.567 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 1191, 2024-09-21T03:58:14.567 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 353462, 2024-09-21T03:58:14.567 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 734680, 2024-09-21T03:58:14.567 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 9, 2024-09-21T03:58:14.567 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-09-21T03:58:14.567 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.128956807, 2024-09-21T03:58:14.568 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.069620734, 2024-09-21T03:58:14.568 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.066247054, 2024-09-21T03:58:14.568 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.004072417, 2024-09-21T03:58:14.568 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:14.568 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1189, 2024-09-21T03:58:14.568 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.063449200, 2024-09-21T03:58:14.568 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000053363 2024-09-21T03:58:14.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.568 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:14.568 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2271, 2024-09-21T03:58:14.568 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000549495, 2024-09-21T03:58:14.568 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000241 2024-09-21T03:58:14.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.569 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:58:14.569 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 403792 2024-09-21T03:58:14.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.569 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:58:14.569 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 374, 2024-09-21T03:58:14.569 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 363, 2024-09-21T03:58:14.569 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 612344, 2024-09-21T03:58:14.569 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 193846, 2024-09-21T03:58:14.569 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 8, 2024-09-21T03:58:14.569 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 5, 2024-09-21T03:58:14.569 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.051082290, 2024-09-21T03:58:14.570 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.023277197, 2024-09-21T03:58:14.570 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.033566373, 2024-09-21T03:58:14.570 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000635429, 2024-09-21T03:58:14.570 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:14.570 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 365, 2024-09-21T03:58:14.570 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.043215372, 2024-09-21T03:58:14.570 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000118398 2024-09-21T03:58:14.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.570 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:14.571 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 105, 2024-09-21T03:58:14.571 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000020900, 2024-09-21T03:58:14.571 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000199 2024-09-21T03:58:14.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.571 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 584848, 2024-09-21T03:58:14.571 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 162192 2024-09-21T03:58:14.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.571 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-09-21T03:58:14.571 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-09-21T03:58:14.571 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-09-21T03:58:14.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.572 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-09-21T03:58:14.572 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-21T03:58:14.572 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-21T03:58:14.572 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 38, 2024-09-21T03:58:14.572 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.001716554, 2024-09-21T03:58:14.572 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000045172 2024-09-21T03:58:14.572 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:14.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.572 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-09-21T03:58:14.572 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-21T03:58:14.572 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-21T03:58:14.573 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2024-09-21T03:58:14.573 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000041105, 2024-09-21T03:58:14.573 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000020552 2024-09-21T03:58:14.573 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:14.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.573 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-09-21T03:58:14.573 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 2, 2024-09-21T03:58:14.573 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 1, 2024-09-21T03:58:14.573 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-09-21T03:58:14.573 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-09-21T03:58:14.574 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000881166, 2024-09-21T03:58:14.574 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000881166 2024-09-21T03:58:14.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.574 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-09-21T03:58:14.574 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 1, 2024-09-21T03:58:14.574 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-09-21T03:58:14.574 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-09-21T03:58:14.574 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 11, 2024-09-21T03:58:14.574 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2024-09-21T03:58:14.574 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-09-21T03:58:14.574 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 14, 2024-09-21T03:58:14.574 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 0, 2024-09-21T03:58:14.575 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 14, 2024-09-21T03:58:14.575 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 0, 2024-09-21T03:58:14.575 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 14, 2024-09-21T03:58:14.575 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 0, 2024-09-21T03:58:14.575 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 1, 2024-09-21T03:58:14.575 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 1, 2024-09-21T03:58:14.575 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 2, 2024-09-21T03:58:14.575 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 25, 2024-09-21T03:58:14.575 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 23, 2024-09-21T03:58:14.575 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 0, 2024-09-21T03:58:14.575 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 1, 2024-09-21T03:58:14.575 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:58:14.575 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-09-21T03:58:14.575 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2024-09-21T03:58:14.576 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-09-21T03:58:14.576 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-09-21T03:58:14.576 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2024-09-21T03:58:14.576 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2024-09-21T03:58:14.576 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2024-09-21T03:58:14.576 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2024-09-21T03:58:14.576 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-09-21T03:58:14.576 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:58:14.576 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-09-21T03:58:14.576 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 22, 2024-09-21T03:58:14.576 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 10738143907, 2024-09-21T03:58:14.576 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-09-21T03:58:14.576 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:58:14.576 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-09-21T03:58:14.577 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:58:14.577 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:58:14.577 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:58:14.577 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:58:14.577 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:58:14.577 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-09-21T03:58:14.577 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:58:14.577 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 12, 2024-09-21T03:58:14.577 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 13, 2024-09-21T03:58:14.577 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:58:14.578 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:58:14.578 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:58:14.578 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 13, 2024-09-21T03:58:14.578 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:58:14.578 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 0, 2024-09-21T03:58:14.578 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 1 2024-09-21T03:58:14.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.578 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-09-21T03:58:14.578 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2024-09-21T03:58:14.579 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2024-09-21T03:58:14.579 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2024-09-21T03:58:14.579 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 2, 2024-09-21T03:58:14.579 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 0, 2024-09-21T03:58:14.579 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-09-21T03:58:14.579 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-09-21T03:58:14.579 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:58:14.579 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 0, 2024-09-21T03:58:14.579 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 0, 2024-09-21T03:58:14.579 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-09-21T03:58:14.579 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-09-21T03:58:14.579 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-09-21T03:58:14.580 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:58:14.580 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:58:14.580 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-09-21T03:58:14.580 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-09-21T03:58:14.580 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 11, 2024-09-21T03:58:14.580 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 55, 2024-09-21T03:58:14.580 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:58:14.580 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2024-09-21T03:58:14.580 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-09-21T03:58:14.580 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:58:14.580 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-09-21T03:58:14.581 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0, 2024-09-21T03:58:14.581 INFO:teuthology.orchestra.run.smithi123.stdout: "uninline_started": 0, 2024-09-21T03:58:14.581 INFO:teuthology.orchestra.run.smithi123.stdout: "uninline_succeeded": 0, 2024-09-21T03:58:14.581 INFO:teuthology.orchestra.run.smithi123.stdout: "uninline_write_failed": 0 2024-09-21T03:58:14.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.581 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-09-21T03:58:14.581 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-09-21T03:58:14.581 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 3, 2024-09-21T03:58:14.581 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-09-21T03:58:14.581 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-09-21T03:58:14.581 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 3, 2024-09-21T03:58:14.581 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-09-21T03:58:14.581 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-09-21T03:58:14.581 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 2, 2024-09-21T03:58:14.582 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-09-21T03:58:14.582 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-09-21T03:58:14.582 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 2, 2024-09-21T03:58:14.582 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 2, 2024-09-21T03:58:14.582 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-09-21T03:58:14.582 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-09-21T03:58:14.582 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-09-21T03:58:14.582 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 4195203, 2024-09-21T03:58:14.582 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-09-21T03:58:14.582 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-09-21T03:58:14.583 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2024-09-21T03:58:14.583 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.003452634, 2024-09-21T03:58:14.583 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001726317 2024-09-21T03:58:14.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.583 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-09-21T03:58:14.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.583 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-09-21T03:58:14.583 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 17, 2024-09-21T03:58:14.583 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 17, 2024-09-21T03:58:14.583 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 0, 2024-09-21T03:58:14.583 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 15, 2024-09-21T03:58:14.583 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 16, 2024-09-21T03:58:14.583 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 1, 2024-09-21T03:58:14.584 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 14, 2024-09-21T03:58:14.584 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 14, 2024-09-21T03:58:14.584 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 0, 2024-09-21T03:58:14.584 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 1, 2024-09-21T03:58:14.584 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 1, 2024-09-21T03:58:14.584 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 0, 2024-09-21T03:58:14.584 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 119176, 2024-09-21T03:58:14.584 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 325861376 2024-09-21T03:58:14.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.584 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-09-21T03:58:14.585 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 3, 2024-09-21T03:58:14.585 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2024-09-21T03:58:14.585 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 2, 2024-09-21T03:58:14.585 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 10, 2024-09-21T03:58:14.585 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 0, 2024-09-21T03:58:14.585 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-09-21T03:58:14.585 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.585 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.586 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.586 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-09-21T03:58:14.586 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-09-21T03:58:14.586 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000881166, 2024-09-21T03:58:14.586 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000881166 2024-09-21T03:58:14.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.586 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-09-21T03:58:14.586 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.586 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.586 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.587 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-09-21T03:58:14.587 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.587 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.587 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.587 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-09-21T03:58:14.587 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.587 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.587 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.587 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-09-21T03:58:14.588 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.588 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.588 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.588 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-09-21T03:58:14.588 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.588 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.588 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.589 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-09-21T03:58:14.589 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.589 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.589 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.589 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-09-21T03:58:14.589 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.589 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.589 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.589 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-09-21T03:58:14.590 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.590 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.590 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.590 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-09-21T03:58:14.590 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.591 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.591 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.591 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-09-21T03:58:14.591 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.591 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.591 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.592 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-09-21T03:58:14.592 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.592 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.593 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.594 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-09-21T03:58:14.594 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.594 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.594 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.594 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-09-21T03:58:14.594 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.594 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.594 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.594 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-09-21T03:58:14.595 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.595 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.595 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.595 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-09-21T03:58:14.595 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.595 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.595 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.595 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-09-21T03:58:14.595 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.595 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.595 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.596 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-09-21T03:58:14.596 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.596 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.596 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.596 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-09-21T03:58:14.596 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.596 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.596 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.596 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-09-21T03:58:14.596 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.596 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.596 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.597 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-09-21T03:58:14.597 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.597 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.597 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.597 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-09-21T03:58:14.597 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.597 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.597 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.597 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-09-21T03:58:14.597 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.597 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.597 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.598 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-09-21T03:58:14.598 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.598 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.598 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.598 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-09-21T03:58:14.598 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.598 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.598 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.598 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-09-21T03:58:14.598 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.599 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.599 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.599 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-09-21T03:58:14.599 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.599 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.599 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.599 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-09-21T03:58:14.599 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.599 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.599 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.599 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:58:14.599 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:58:14.600 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-09-21T03:58:14.600 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.600 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.600 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.600 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:14.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.600 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-09-21T03:58:14.600 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-09-21T03:58:14.600 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 1, 2024-09-21T03:58:14.600 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 0, 2024-09-21T03:58:14.600 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2024-09-21T03:58:14.600 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-09-21T03:58:14.600 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 1, 2024-09-21T03:58:14.600 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 1, 2024-09-21T03:58:14.601 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 719, 2024-09-21T03:58:14.601 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-09-21T03:58:14.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.601 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-09-21T03:58:14.601 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:58:14.601 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-09-21T03:58:14.601 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:58:14.601 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:58:14.601 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:58:14.601 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:58:14.601 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:58:14.601 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:58:14.601 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:58:14.601 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:58:14.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:58:14.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:58:14.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:58:14.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:58:14.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:58:14.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-09-21T03:58:14.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:58:14.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-09-21T03:58:14.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:58:14.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:58:14.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:58:14.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:58:14.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:58:14.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:58:14.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:58:14.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-09-21T03:58:14.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:58:14.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:58:14.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:58:14.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-09-21T03:58:14.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-09-21T03:58:14.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-09-21T03:58:14.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:58:14.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:58:14.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:58:14.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:58:14.603 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 271237, 2024-09-21T03:58:14.603 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 92, 2024-09-21T03:58:14.604 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:58:14.604 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-09-21T03:58:14.604 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-09-21T03:58:14.604 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-09-21T03:58:14.604 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:58:14.604 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-09-21T03:58:14.604 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:58:14.604 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-09-21T03:58:14.604 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8952, 2024-09-21T03:58:14.604 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 168, 2024-09-21T03:58:14.604 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:58:14.604 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:58:14.604 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-09-21T03:58:14.604 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-09-21T03:58:14.605 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 68593, 2024-09-21T03:58:14.605 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 318, 2024-09-21T03:58:14.605 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-09-21T03:58:14.605 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-09-21T03:58:14.605 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-09-21T03:58:14.605 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-09-21T03:58:14.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.605 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-09-21T03:58:14.605 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-09-21T03:58:14.605 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-09-21T03:58:14.605 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 41, 2024-09-21T03:58:14.605 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 11713, 2024-09-21T03:58:14.605 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 33, 2024-09-21T03:58:14.605 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 41, 2024-09-21T03:58:14.605 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-09-21T03:58:14.605 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 41, 2024-09-21T03:58:14.606 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.274993176, 2024-09-21T03:58:14.606 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.006707150 2024-09-21T03:58:14.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.606 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-09-21T03:58:14.606 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-09-21T03:58:14.606 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 41, 2024-09-21T03:58:14.606 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 67 2024-09-21T03:58:14.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.606 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 41, 2024-09-21T03:58:14.606 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 0, 2024-09-21T03:58:14.606 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 41, 2024-09-21T03:58:14.606 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-09-21T03:58:14.606 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-09-21T03:58:14.606 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2024-09-21T03:58:14.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 10, 2024-09-21T03:58:14.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 0, 2024-09-21T03:58:14.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 2, 2024-09-21T03:58:14.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 6, 2024-09-21T03:58:14.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-09-21T03:58:14.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-09-21T03:58:14.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-09-21T03:58:14.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-09-21T03:58:14.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 6, 2024-09-21T03:58:14.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-09-21T03:58:14.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-09-21T03:58:14.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-09-21T03:58:14.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 0, 2024-09-21T03:58:14.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 20, 2024-09-21T03:58:14.608 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:58:14.608 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-09-21T03:58:14.608 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:58:14.608 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-09-21T03:58:14.608 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-09-21T03:58:14.608 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-09-21T03:58:14.608 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:58:14.608 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-09-21T03:58:14.608 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:58:14.608 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-09-21T03:58:14.608 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-09-21T03:58:14.608 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-09-21T03:58:14.608 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-09-21T03:58:14.608 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-09-21T03:58:14.608 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-09-21T03:58:14.609 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-09-21T03:58:14.609 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-09-21T03:58:14.609 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-09-21T03:58:14.609 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-09-21T03:58:14.609 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-09-21T03:58:14.609 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-09-21T03:58:14.609 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-09-21T03:58:14.609 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-09-21T03:58:14.609 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-09-21T03:58:14.609 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-09-21T03:58:14.609 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-09-21T03:58:14.609 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 86, 2024-09-21T03:58:14.609 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-09-21T03:58:14.609 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 2, 2024-09-21T03:58:14.609 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 10, 2024-09-21T03:58:14.610 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 10, 2024-09-21T03:58:14.610 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-09-21T03:58:14.610 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-09-21T03:58:14.610 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 21, 2024-09-21T03:58:14.610 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 0, 2024-09-21T03:58:14.610 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 2 2024-09-21T03:58:14.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.610 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-09-21T03:58:14.610 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-09-21T03:58:14.610 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:58:14.610 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-09-21T03:58:14.610 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-09-21T03:58:14.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.610 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-09-21T03:58:14.611 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-09-21T03:58:14.611 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:58:14.611 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-09-21T03:58:14.611 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 0, 2024-09-21T03:58:14.611 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 0, 2024-09-21T03:58:14.611 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 0, 2024-09-21T03:58:14.611 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-09-21T03:58:14.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.611 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:58:14.611 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:14.611 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-21T03:58:14.611 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:14.611 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 1892, 2024-09-21T03:58:14.611 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 812614, 2024-09-21T03:58:14.612 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:14.612 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 1892, 2024-09-21T03:58:14.612 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-21T03:58:14.612 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-21T03:58:14.612 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 1892, 2024-09-21T03:58:14.612 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 812614, 2024-09-21T03:58:14.612 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:14.612 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.612 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.612 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.612 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:14.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.612 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-09-21T03:58:14.613 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:14.613 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-21T03:58:14.613 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:14.613 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-21T03:58:14.613 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-21T03:58:14.613 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:14.613 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:14.613 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 41, 2024-09-21T03:58:14.613 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 11713, 2024-09-21T03:58:14.613 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 41, 2024-09-21T03:58:14.613 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 11713, 2024-09-21T03:58:14.613 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:14.613 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.613 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.613 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.614 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:14.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.614 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-09-21T03:58:14.614 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:14.614 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-09-21T03:58:14.614 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:14.614 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-21T03:58:14.614 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-21T03:58:14.614 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:14.614 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:14.614 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 41, 2024-09-21T03:58:14.614 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 41, 2024-09-21T03:58:14.614 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 41, 2024-09-21T03:58:14.614 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 41, 2024-09-21T03:58:14.615 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:14.615 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.615 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.615 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.615 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:14.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.615 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:58:14.615 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:14.615 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-21T03:58:14.615 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:14.615 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-21T03:58:14.615 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-21T03:58:14.615 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:14.615 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:14.615 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-21T03:58:14.616 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-21T03:58:14.616 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-09-21T03:58:14.616 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-09-21T03:58:14.616 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:14.616 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.616 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.616 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.616 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:14.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.616 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x55617ee2b450": { 2024-09-21T03:58:14.616 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:14.616 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-21T03:58:14.616 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:14.616 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 3, 2024-09-21T03:58:14.616 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 899, 2024-09-21T03:58:14.617 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:14.617 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 3, 2024-09-21T03:58:14.617 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-21T03:58:14.617 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-21T03:58:14.617 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2, 2024-09-21T03:58:14.617 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 899, 2024-09-21T03:58:14.617 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:14.617 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:14.617 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:14.617 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:14.617 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:14.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:14.617 INFO:teuthology.orchestra.run.smithi123.stdout: "trackedop": { 2024-09-21T03:58:14.617 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 60 2024-09-21T03:58:14.617 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:14.618 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-21T03:58:15.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:15 smithi123 ceph-mon[34673]: pgmap v776: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:58:15.477 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-21T03:58:15.477 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-21T03:58:15.477 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T03:58:15.477 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T03:58:15.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:15 smithi134 ceph-mon[35031]: pgmap v776: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:58:15.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:15 smithi016 ceph-mon[30375]: pgmap v776: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T03:58:15.722 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:58:16.262 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-21T03:58:16.262 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:58:16.262 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 382, 2024-09-21T03:58:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 371, 2024-09-21T03:58:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 628638, 2024-09-21T03:58:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 240546, 2024-09-21T03:58:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 15, 2024-09-21T03:58:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-09-21T03:58:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.058096047, 2024-09-21T03:58:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.024934358, 2024-09-21T03:58:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.039890162, 2024-09-21T03:58:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000301447, 2024-09-21T03:58:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 367, 2024-09-21T03:58:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.041501441, 2024-09-21T03:58:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000113082 2024-09-21T03:58:16.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 114, 2024-09-21T03:58:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000037086, 2024-09-21T03:58:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000325 2024-09-21T03:58:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 599136, 2024-09-21T03:58:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 160112 2024-09-21T03:58:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:58:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 552, 2024-09-21T03:58:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 876, 2024-09-21T03:58:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 133848, 2024-09-21T03:58:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 657167, 2024-09-21T03:58:16.264 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 8, 2024-09-21T03:58:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 4, 2024-09-21T03:58:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.103216723, 2024-09-21T03:58:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.052910736, 2024-09-21T03:58:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.045508083, 2024-09-21T03:58:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.004409575, 2024-09-21T03:58:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 878, 2024-09-21T03:58:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.052488132, 2024-09-21T03:58:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000059781 2024-09-21T03:58:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1185, 2024-09-21T03:58:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000283611, 2024-09-21T03:58:16.265 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000239 2024-09-21T03:58:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:58:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 405264 2024-09-21T03:58:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:58:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 71, 2024-09-21T03:58:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 73, 2024-09-21T03:58:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 26404, 2024-09-21T03:58:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 59269, 2024-09-21T03:58:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 10, 2024-09-21T03:58:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 7, 2024-09-21T03:58:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.009097457, 2024-09-21T03:58:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.002841679, 2024-09-21T03:58:16.266 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.006024639, 2024-09-21T03:58:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000318504, 2024-09-21T03:58:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 71, 2024-09-21T03:58:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.019888736, 2024-09-21T03:58:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000280123 2024-09-21T03:58:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 96, 2024-09-21T03:58:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000018314, 2024-09-21T03:58:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000190 2024-09-21T03:58:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:58:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:58:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.267 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-09-21T03:58:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-09-21T03:58:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-09-21T03:58:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-09-21T03:58:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-21T03:58:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-21T03:58:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 45, 2024-09-21T03:58:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.001359852, 2024-09-21T03:58:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000030218 2024-09-21T03:58:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-09-21T03:58:16.268 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-21T03:58:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-21T03:58:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2024-09-21T03:58:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000013785, 2024-09-21T03:58:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000006892 2024-09-21T03:58:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-09-21T03:58:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 1, 2024-09-21T03:58:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 1, 2024-09-21T03:58:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-09-21T03:58:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-09-21T03:58:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000116401, 2024-09-21T03:58:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000116401 2024-09-21T03:58:16.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-09-21T03:58:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 0, 2024-09-21T03:58:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-09-21T03:58:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-09-21T03:58:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 11, 2024-09-21T03:58:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2024-09-21T03:58:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-09-21T03:58:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 10, 2024-09-21T03:58:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 0, 2024-09-21T03:58:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 10, 2024-09-21T03:58:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 0, 2024-09-21T03:58:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 10, 2024-09-21T03:58:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 0, 2024-09-21T03:58:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 1, 2024-09-21T03:58:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 1, 2024-09-21T03:58:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 1, 2024-09-21T03:58:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 22, 2024-09-21T03:58:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 22, 2024-09-21T03:58:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 0, 2024-09-21T03:58:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 0, 2024-09-21T03:58:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:58:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-09-21T03:58:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2024-09-21T03:58:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-09-21T03:58:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-09-21T03:58:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2024-09-21T03:58:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2024-09-21T03:58:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2024-09-21T03:58:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2024-09-21T03:58:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-09-21T03:58:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:58:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-09-21T03:58:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 22, 2024-09-21T03:58:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 10738143907, 2024-09-21T03:58:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-09-21T03:58:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:58:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-09-21T03:58:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:58:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:58:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:58:16.272 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:58:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:58:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-09-21T03:58:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:58:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:58:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:58:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:58:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:58:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:58:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 0, 2024-09-21T03:58:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:58:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 0, 2024-09-21T03:58:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 0 2024-09-21T03:58:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.273 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-09-21T03:58:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2024-09-21T03:58:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2024-09-21T03:58:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2024-09-21T03:58:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 1, 2024-09-21T03:58:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 0, 2024-09-21T03:58:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-09-21T03:58:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-09-21T03:58:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:58:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 0, 2024-09-21T03:58:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 0, 2024-09-21T03:58:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-09-21T03:58:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-09-21T03:58:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-09-21T03:58:16.274 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:58:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:58:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-09-21T03:58:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-09-21T03:58:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 11, 2024-09-21T03:58:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 11, 2024-09-21T03:58:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:58:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2024-09-21T03:58:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-09-21T03:58:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:58:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-09-21T03:58:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0, 2024-09-21T03:58:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "uninline_started": 0, 2024-09-21T03:58:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "uninline_succeeded": 0, 2024-09-21T03:58:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: "uninline_write_failed": 0 2024-09-21T03:58:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-09-21T03:58:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-09-21T03:58:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 3, 2024-09-21T03:58:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-09-21T03:58:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-09-21T03:58:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 3, 2024-09-21T03:58:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-09-21T03:58:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-09-21T03:58:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 2, 2024-09-21T03:58:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-09-21T03:58:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-09-21T03:58:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 2, 2024-09-21T03:58:16.276 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 2, 2024-09-21T03:58:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-09-21T03:58:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-09-21T03:58:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-09-21T03:58:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 4195203, 2024-09-21T03:58:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-09-21T03:58:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-09-21T03:58:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2024-09-21T03:58:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.003609880, 2024-09-21T03:58:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001804940 2024-09-21T03:58:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-09-21T03:58:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-09-21T03:58:16.277 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 13, 2024-09-21T03:58:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 13, 2024-09-21T03:58:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 0, 2024-09-21T03:58:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 11, 2024-09-21T03:58:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 12, 2024-09-21T03:58:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 1, 2024-09-21T03:58:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 10, 2024-09-21T03:58:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 10, 2024-09-21T03:58:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 0, 2024-09-21T03:58:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 1, 2024-09-21T03:58:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 1, 2024-09-21T03:58:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 0, 2024-09-21T03:58:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 120456, 2024-09-21T03:58:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 324808704 2024-09-21T03:58:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.278 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-09-21T03:58:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 1, 2024-09-21T03:58:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2024-09-21T03:58:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 1, 2024-09-21T03:58:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 10, 2024-09-21T03:58:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 0, 2024-09-21T03:58:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-09-21T03:58:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-09-21T03:58:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-09-21T03:58:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-09-21T03:58:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-09-21T03:58:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-09-21T03:58:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-09-21T03:58:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-09-21T03:58:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000116401, 2024-09-21T03:58:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000116401 2024-09-21T03:58:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-09-21T03:58:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-09-21T03:58:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-09-21T03:58:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-09-21T03:58:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.282 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.283 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.283 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.283 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-09-21T03:58:16.283 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.283 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.283 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-09-21T03:58:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-09-21T03:58:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-09-21T03:58:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.284 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-09-21T03:58:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-09-21T03:58:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-09-21T03:58:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.285 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.286 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.286 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-09-21T03:58:16.286 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.286 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.286 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.286 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-09-21T03:58:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-09-21T03:58:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-09-21T03:58:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.287 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-09-21T03:58:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-09-21T03:58:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-09-21T03:58:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.288 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-09-21T03:58:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-09-21T03:58:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-09-21T03:58:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.289 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-09-21T03:58:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:58:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:58:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-09-21T03:58:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:16.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-09-21T03:58:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-09-21T03:58:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 1, 2024-09-21T03:58:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 0, 2024-09-21T03:58:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2024-09-21T03:58:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-09-21T03:58:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 1, 2024-09-21T03:58:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 1, 2024-09-21T03:58:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 705, 2024-09-21T03:58:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-09-21T03:58:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-09-21T03:58:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:58:16.291 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-09-21T03:58:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:58:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:58:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:58:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:58:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:58:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:58:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:58:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:58:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:58:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:58:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:58:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:58:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:58:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-09-21T03:58:16.292 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:58:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-09-21T03:58:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:58:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:58:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:58:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:58:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:58:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:58:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:58:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-09-21T03:58:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:58:16.293 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:58:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:58:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-09-21T03:58:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-09-21T03:58:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-09-21T03:58:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:58:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:58:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:58:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:58:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 238421, 2024-09-21T03:58:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 81, 2024-09-21T03:58:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:58:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-09-21T03:58:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-09-21T03:58:16.294 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-09-21T03:58:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:58:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-09-21T03:58:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:58:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-09-21T03:58:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8952, 2024-09-21T03:58:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 168, 2024-09-21T03:58:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:58:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:58:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-09-21T03:58:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-09-21T03:58:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 51512, 2024-09-21T03:58:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 200, 2024-09-21T03:58:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-09-21T03:58:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-09-21T03:58:16.295 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-09-21T03:58:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-09-21T03:58:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-09-21T03:58:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-09-21T03:58:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-09-21T03:58:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 37, 2024-09-21T03:58:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 11111, 2024-09-21T03:58:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 28, 2024-09-21T03:58:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 37, 2024-09-21T03:58:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-09-21T03:58:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 37, 2024-09-21T03:58:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.231994226, 2024-09-21T03:58:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.006270114 2024-09-21T03:58:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-09-21T03:58:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-09-21T03:58:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 37, 2024-09-21T03:58:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 59 2024-09-21T03:58:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 37, 2024-09-21T03:58:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 0, 2024-09-21T03:58:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 37, 2024-09-21T03:58:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-09-21T03:58:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-09-21T03:58:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2024-09-21T03:58:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 10, 2024-09-21T03:58:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 0, 2024-09-21T03:58:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 2, 2024-09-21T03:58:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 6, 2024-09-21T03:58:16.297 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-09-21T03:58:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-09-21T03:58:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-09-21T03:58:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-09-21T03:58:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 6, 2024-09-21T03:58:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-09-21T03:58:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-09-21T03:58:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-09-21T03:58:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 0, 2024-09-21T03:58:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 20, 2024-09-21T03:58:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:58:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-09-21T03:58:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:58:16.298 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-09-21T03:58:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-09-21T03:58:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-09-21T03:58:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:58:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-09-21T03:58:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:58:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-09-21T03:58:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-09-21T03:58:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-09-21T03:58:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-09-21T03:58:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-09-21T03:58:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-09-21T03:58:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-09-21T03:58:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-09-21T03:58:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-09-21T03:58:16.299 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-09-21T03:58:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-09-21T03:58:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-09-21T03:58:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-09-21T03:58:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-09-21T03:58:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-09-21T03:58:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-09-21T03:58:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-09-21T03:58:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 86, 2024-09-21T03:58:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-09-21T03:58:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 2, 2024-09-21T03:58:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 9, 2024-09-21T03:58:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 9, 2024-09-21T03:58:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-09-21T03:58:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-09-21T03:58:16.300 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 14, 2024-09-21T03:58:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 0, 2024-09-21T03:58:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 1 2024-09-21T03:58:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-09-21T03:58:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-09-21T03:58:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:58:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-09-21T03:58:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-09-21T03:58:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-09-21T03:58:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-09-21T03:58:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:58:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-09-21T03:58:16.301 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 0, 2024-09-21T03:58:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 0, 2024-09-21T03:58:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 0, 2024-09-21T03:58:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-09-21T03:58:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:58:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-21T03:58:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 1001, 2024-09-21T03:58:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 681069, 2024-09-21T03:58:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 1001, 2024-09-21T03:58:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-21T03:58:16.302 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-21T03:58:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 1001, 2024-09-21T03:58:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 681069, 2024-09-21T03:58:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-09-21T03:58:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-21T03:58:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-21T03:58:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-21T03:58:16.303 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 37, 2024-09-21T03:58:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 11111, 2024-09-21T03:58:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 37, 2024-09-21T03:58:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 11111, 2024-09-21T03:58:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-09-21T03:58:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-09-21T03:58:16.304 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-21T03:58:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-21T03:58:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 37, 2024-09-21T03:58:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 37, 2024-09-21T03:58:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 37, 2024-09-21T03:58:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 37, 2024-09-21T03:58:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:16.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:58:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-21T03:58:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-21T03:58:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-21T03:58:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-21T03:58:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-21T03:58:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-09-21T03:58:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-09-21T03:58:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.306 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x55873cd75380": { 2024-09-21T03:58:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-21T03:58:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 3, 2024-09-21T03:58:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 899, 2024-09-21T03:58:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 3, 2024-09-21T03:58:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-21T03:58:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-21T03:58:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2, 2024-09-21T03:58:16.307 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 899, 2024-09-21T03:58:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: "trackedop": { 2024-09-21T03:58:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 20 2024-09-21T03:58:16.308 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:16.308 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-21T03:58:17.079 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-21T03:58:17.079 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-21T03:58:17.079 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T03:58:17.079 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T03:58:17.326 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:58:17.578 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:17 smithi123 ceph-mon[34673]: pgmap v777: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T03:58:17.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:17 smithi134 ceph-mon[35031]: pgmap v777: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T03:58:17.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:17 smithi016 ceph-mon[30375]: pgmap v777: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T03:58:18.054 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-21T03:58:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:58:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 2418, 2024-09-21T03:58:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2418, 2024-09-21T03:58:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 688516, 2024-09-21T03:58:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 768107, 2024-09-21T03:58:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 6, 2024-09-21T03:58:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2024-09-21T03:58:18.054 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.208781202, 2024-09-21T03:58:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.090558842, 2024-09-21T03:58:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.044799529, 2024-09-21T03:58:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.060856981, 2024-09-21T03:58:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2417, 2024-09-21T03:58:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.074046081, 2024-09-21T03:58:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000030635 2024-09-21T03:58:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:58:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:58:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:58:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 4, 2024-09-21T03:58:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 198, 2024-09-21T03:58:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 876, 2024-09-21T03:58:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 497018, 2024-09-21T03:58:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 5, 2024-09-21T03:58:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2024-09-21T03:58:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.013822357, 2024-09-21T03:58:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.010319376, 2024-09-21T03:58:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.001709032, 2024-09-21T03:58:18.056 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000062984, 2024-09-21T03:58:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 196, 2024-09-21T03:58:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.013974866, 2024-09-21T03:58:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000071300 2024-09-21T03:58:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:58:18.057 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 496320 2024-09-21T03:58:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:58:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 268, 2024-09-21T03:58:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 263, 2024-09-21T03:58:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 570524, 2024-09-21T03:58:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 159182, 2024-09-21T03:58:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 3, 2024-09-21T03:58:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2024-09-21T03:58:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.038678999, 2024-09-21T03:58:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.019420691, 2024-09-21T03:58:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.025495854, 2024-09-21T03:58:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000026290, 2024-09-21T03:58:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 262, 2024-09-21T03:58:18.058 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.015660288, 2024-09-21T03:58:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000059772 2024-09-21T03:58:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 570256, 2024-09-21T03:58:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 158864 2024-09-21T03:58:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-09-21T03:58:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-09-21T03:58:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-09-21T03:58:18.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-09-21T03:58:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-21T03:58:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-21T03:58:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 4116, 2024-09-21T03:58:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.103995436, 2024-09-21T03:58:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000025266 2024-09-21T03:58:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-09-21T03:58:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-21T03:58:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-21T03:58:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.060 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-09-21T03:58:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 0, 2024-09-21T03:58:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 0, 2024-09-21T03:58:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-09-21T03:58:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-09-21T03:58:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 0, 2024-09-21T03:58:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-09-21T03:58:18.061 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-09-21T03:58:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 0, 2024-09-21T03:58:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2024-09-21T03:58:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-09-21T03:58:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 0, 2024-09-21T03:58:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 0, 2024-09-21T03:58:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 0, 2024-09-21T03:58:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 0, 2024-09-21T03:58:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 0, 2024-09-21T03:58:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 0, 2024-09-21T03:58:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 0, 2024-09-21T03:58:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 0, 2024-09-21T03:58:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 1, 2024-09-21T03:58:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 0, 2024-09-21T03:58:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 0, 2024-09-21T03:58:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 0, 2024-09-21T03:58:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 0, 2024-09-21T03:58:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:58:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-09-21T03:58:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2024-09-21T03:58:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-09-21T03:58:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-09-21T03:58:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2024-09-21T03:58:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2024-09-21T03:58:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2024-09-21T03:58:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2024-09-21T03:58:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-09-21T03:58:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:58:18.063 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-09-21T03:58:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 0, 2024-09-21T03:58:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 0, 2024-09-21T03:58:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-09-21T03:58:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:58:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-09-21T03:58:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:58:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:58:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:58:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:58:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:58:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-09-21T03:58:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:58:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:58:18.064 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:58:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:58:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:58:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:58:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 0, 2024-09-21T03:58:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:58:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 0, 2024-09-21T03:58:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 0 2024-09-21T03:58:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-09-21T03:58:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2024-09-21T03:58:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2024-09-21T03:58:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2024-09-21T03:58:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 0, 2024-09-21T03:58:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 0, 2024-09-21T03:58:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-09-21T03:58:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-09-21T03:58:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:58:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 0, 2024-09-21T03:58:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 0, 2024-09-21T03:58:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-09-21T03:58:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-09-21T03:58:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-09-21T03:58:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:58:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:58:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-09-21T03:58:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-09-21T03:58:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:58:18.066 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:58:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:58:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2024-09-21T03:58:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-09-21T03:58:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:58:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-09-21T03:58:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0, 2024-09-21T03:58:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "uninline_started": 0, 2024-09-21T03:58:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "uninline_succeeded": 0, 2024-09-21T03:58:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "uninline_write_failed": 0 2024-09-21T03:58:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-09-21T03:58:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-09-21T03:58:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 0, 2024-09-21T03:58:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-09-21T03:58:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-09-21T03:58:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 3, 2024-09-21T03:58:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-09-21T03:58:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-09-21T03:58:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 0, 2024-09-21T03:58:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-09-21T03:58:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-09-21T03:58:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 2, 2024-09-21T03:58:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 2, 2024-09-21T03:58:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-09-21T03:58:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-09-21T03:58:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-09-21T03:58:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 4195203, 2024-09-21T03:58:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 4194892, 2024-09-21T03:58:18.068 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-09-21T03:58:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 3 2024-09-21T03:58:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-09-21T03:58:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 3, 2024-09-21T03:58:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 3, 2024-09-21T03:58:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 0, 2024-09-21T03:58:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 1, 2024-09-21T03:58:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 1, 2024-09-21T03:58:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 0, 2024-09-21T03:58:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 0, 2024-09-21T03:58:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 0, 2024-09-21T03:58:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 0, 2024-09-21T03:58:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 0, 2024-09-21T03:58:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 0, 2024-09-21T03:58:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 0, 2024-09-21T03:58:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 116372, 2024-09-21T03:58:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 316420096 2024-09-21T03:58:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-09-21T03:58:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 0, 2024-09-21T03:58:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2024-09-21T03:58:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 0, 2024-09-21T03:58:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 0, 2024-09-21T03:58:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 0, 2024-09-21T03:58:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-09-21T03:58:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.070 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-09-21T03:58:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-09-21T03:58:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-09-21T03:58:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-09-21T03:58:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-09-21T03:58:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-09-21T03:58:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.072 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-09-21T03:58:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-09-21T03:58:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-09-21T03:58:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-09-21T03:58:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.074 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-09-21T03:58:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-09-21T03:58:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-09-21T03:58:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.075 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-09-21T03:58:18.076 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.076 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.076 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.076 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-09-21T03:58:18.076 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.076 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.076 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.076 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-09-21T03:58:18.076 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.076 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.076 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.077 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-09-21T03:58:18.077 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.077 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.077 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.077 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-09-21T03:58:18.077 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.077 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.077 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.077 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-09-21T03:58:18.077 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.077 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.077 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.078 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-09-21T03:58:18.078 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.078 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.078 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.078 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-09-21T03:58:18.078 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.078 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.078 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.078 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-09-21T03:58:18.078 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.078 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.078 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.079 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-09-21T03:58:18.079 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.079 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.079 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.079 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-09-21T03:58:18.079 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.079 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.079 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.079 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-09-21T03:58:18.079 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.079 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.079 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.080 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-09-21T03:58:18.080 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.080 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.080 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.080 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-09-21T03:58:18.080 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.080 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.080 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.080 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-09-21T03:58:18.080 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.080 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.080 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.081 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:58:18.081 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:58:18.081 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-09-21T03:58:18.081 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.081 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.081 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.081 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:18.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.081 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-09-21T03:58:18.081 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-09-21T03:58:18.081 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 1, 2024-09-21T03:58:18.081 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 0, 2024-09-21T03:58:18.081 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2024-09-21T03:58:18.082 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-09-21T03:58:18.082 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 0, 2024-09-21T03:58:18.082 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 0, 2024-09-21T03:58:18.082 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 2231, 2024-09-21T03:58:18.082 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-09-21T03:58:18.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.082 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-09-21T03:58:18.082 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:58:18.082 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-09-21T03:58:18.082 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:58:18.082 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:58:18.082 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:58:18.082 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:58:18.082 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:58:18.083 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:58:18.083 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:58:18.083 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:58:18.083 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:58:18.083 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:58:18.083 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:58:18.083 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:58:18.083 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:58:18.083 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-09-21T03:58:18.083 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:58:18.083 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-09-21T03:58:18.083 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:58:18.083 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:58:18.083 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:58:18.084 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:58:18.084 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:58:18.084 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:58:18.084 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:58:18.084 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-09-21T03:58:18.084 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:58:18.084 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:58:18.084 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:58:18.084 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-09-21T03:58:18.084 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-09-21T03:58:18.084 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-09-21T03:58:18.084 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:58:18.084 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:58:18.084 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:58:18.084 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:58:18.085 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 181287, 2024-09-21T03:58:18.085 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 45, 2024-09-21T03:58:18.085 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:58:18.085 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-09-21T03:58:18.085 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-09-21T03:58:18.085 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-09-21T03:58:18.085 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:58:18.085 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-09-21T03:58:18.085 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:58:18.085 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-09-21T03:58:18.085 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8952, 2024-09-21T03:58:18.085 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 168, 2024-09-21T03:58:18.085 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:58:18.085 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:58:18.086 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-09-21T03:58:18.086 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-09-21T03:58:18.086 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 8672, 2024-09-21T03:58:18.086 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 13, 2024-09-21T03:58:18.086 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-09-21T03:58:18.086 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-09-21T03:58:18.086 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-09-21T03:58:18.086 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-09-21T03:58:18.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.086 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-09-21T03:58:18.086 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-09-21T03:58:18.086 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-09-21T03:58:18.086 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 2417, 2024-09-21T03:58:18.086 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 21, 2024-09-21T03:58:18.087 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 3, 2024-09-21T03:58:18.087 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 2417, 2024-09-21T03:58:18.087 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-09-21T03:58:18.087 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2417, 2024-09-21T03:58:18.087 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.369966461, 2024-09-21T03:58:18.087 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000566804 2024-09-21T03:58:18.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.087 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-09-21T03:58:18.087 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-09-21T03:58:18.087 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2417, 2024-09-21T03:58:18.087 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 2418 2024-09-21T03:58:18.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.087 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 2417, 2024-09-21T03:58:18.087 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 2417, 2024-09-21T03:58:18.088 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 0, 2024-09-21T03:58:18.088 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-09-21T03:58:18.088 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-09-21T03:58:18.088 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 1559, 2024-09-21T03:58:18.088 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 0, 2024-09-21T03:58:18.088 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 856, 2024-09-21T03:58:18.088 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 0, 2024-09-21T03:58:18.088 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 0, 2024-09-21T03:58:18.088 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-09-21T03:58:18.088 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-09-21T03:58:18.088 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-09-21T03:58:18.088 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-09-21T03:58:18.088 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 0, 2024-09-21T03:58:18.088 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-09-21T03:58:18.089 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-09-21T03:58:18.089 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-09-21T03:58:18.089 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 1, 2024-09-21T03:58:18.089 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 0, 2024-09-21T03:58:18.089 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:58:18.089 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-09-21T03:58:18.089 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:58:18.089 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-09-21T03:58:18.089 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-09-21T03:58:18.089 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-09-21T03:58:18.089 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:58:18.089 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-09-21T03:58:18.089 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:58:18.090 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-09-21T03:58:18.090 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-09-21T03:58:18.090 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-09-21T03:58:18.090 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-09-21T03:58:18.090 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-09-21T03:58:18.090 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-09-21T03:58:18.090 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-09-21T03:58:18.090 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-09-21T03:58:18.090 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-09-21T03:58:18.090 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-09-21T03:58:18.090 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-09-21T03:58:18.090 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-09-21T03:58:18.090 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-09-21T03:58:18.090 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-09-21T03:58:18.091 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-09-21T03:58:18.091 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-09-21T03:58:18.091 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-09-21T03:58:18.091 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 86, 2024-09-21T03:58:18.091 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-09-21T03:58:18.091 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 2, 2024-09-21T03:58:18.091 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 4, 2024-09-21T03:58:18.091 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 4, 2024-09-21T03:58:18.091 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-09-21T03:58:18.091 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-09-21T03:58:18.091 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 0, 2024-09-21T03:58:18.091 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 2, 2024-09-21T03:58:18.091 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 0 2024-09-21T03:58:18.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.092 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-09-21T03:58:18.092 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 0, 2024-09-21T03:58:18.092 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:58:18.092 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-09-21T03:58:18.092 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-09-21T03:58:18.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.092 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-09-21T03:58:18.092 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-09-21T03:58:18.092 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:58:18.092 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-09-21T03:58:18.092 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 0, 2024-09-21T03:58:18.092 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 0, 2024-09-21T03:58:18.092 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 0, 2024-09-21T03:58:18.092 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-09-21T03:58:18.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.093 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:58:18.093 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:18.093 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-21T03:58:18.093 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:18.093 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 2686, 2024-09-21T03:58:18.093 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1001480, 2024-09-21T03:58:18.093 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:18.093 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 2686, 2024-09-21T03:58:18.093 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-21T03:58:18.093 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-21T03:58:18.093 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2686, 2024-09-21T03:58:18.093 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1001480, 2024-09-21T03:58:18.093 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:18.094 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.094 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.094 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.094 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:18.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.094 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-09-21T03:58:18.094 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:18.094 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-21T03:58:18.094 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:18.094 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-21T03:58:18.094 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-21T03:58:18.094 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:18.094 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:18.094 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 2417, 2024-09-21T03:58:18.095 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 904, 2024-09-21T03:58:18.095 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2417, 2024-09-21T03:58:18.095 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 904, 2024-09-21T03:58:18.095 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:18.095 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.095 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.095 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.095 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:18.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.095 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-09-21T03:58:18.095 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:18.095 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-09-21T03:58:18.095 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:18.095 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-21T03:58:18.096 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-21T03:58:18.096 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:18.096 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:18.096 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 2417, 2024-09-21T03:58:18.096 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 2417, 2024-09-21T03:58:18.096 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2417, 2024-09-21T03:58:18.096 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 2417, 2024-09-21T03:58:18.096 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:18.096 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.096 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.096 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.096 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:18.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.096 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:58:18.096 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:18.097 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-21T03:58:18.097 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:18.097 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-21T03:58:18.097 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-21T03:58:18.097 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:18.097 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:18.097 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-21T03:58:18.097 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-21T03:58:18.097 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-09-21T03:58:18.097 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-09-21T03:58:18.097 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:18.097 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.097 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.097 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.098 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:18.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.098 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x5653dd269380": { 2024-09-21T03:58:18.098 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:58:18.098 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-21T03:58:18.098 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:58:18.098 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-21T03:58:18.098 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-21T03:58:18.098 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:18.098 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:18.098 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-21T03:58:18.098 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-21T03:58:18.098 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-09-21T03:58:18.098 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-09-21T03:58:18.099 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:58:18.099 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:58:18.099 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:58:18.099 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:58:18.099 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:18.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:58:18.099 INFO:teuthology.orchestra.run.smithi123.stdout: "trackedop": { 2024-09-21T03:58:18.099 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 0 2024-09-21T03:58:18.099 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:58:18.099 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-21T03:58:19.480 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-21T03:58:19.480 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-21T03:58:19.480 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T03:58:19.480 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T03:58:19.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:19 smithi134 ceph-mon[35031]: pgmap v778: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:58:19.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:19 smithi123 ceph-mon[34673]: pgmap v778: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:58:19.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:19 smithi016 ceph-mon[30375]: pgmap v778: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:58:19.841 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:58:20.484 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-21T03:58:20.484 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:58:20.484 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 92, 2024-09-21T03:58:20.484 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 92, 2024-09-21T03:58:20.484 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 33308, 2024-09-21T03:58:20.484 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 109378, 2024-09-21T03:58:20.484 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 16, 2024-09-21T03:58:20.484 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-09-21T03:58:20.485 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.015773484, 2024-09-21T03:58:20.485 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.004246306, 2024-09-21T03:58:20.485 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.009516499, 2024-09-21T03:58:20.485 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000541171, 2024-09-21T03:58:20.485 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:20.485 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 91, 2024-09-21T03:58:20.485 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.026477631, 2024-09-21T03:58:20.485 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000290962 2024-09-21T03:58:20.485 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.485 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:20.485 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 121, 2024-09-21T03:58:20.485 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000039069, 2024-09-21T03:58:20.485 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000322 2024-09-21T03:58:20.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.486 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 4528, 2024-09-21T03:58:20.486 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 1888 2024-09-21T03:58:20.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.486 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:58:20.486 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 365, 2024-09-21T03:58:20.486 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 356, 2024-09-21T03:58:20.486 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 613890, 2024-09-21T03:58:20.486 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 189310, 2024-09-21T03:58:20.486 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 7, 2024-09-21T03:58:20.486 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-09-21T03:58:20.486 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.046977690, 2024-09-21T03:58:20.486 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.022584182, 2024-09-21T03:58:20.487 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.031981001, 2024-09-21T03:58:20.487 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000377781, 2024-09-21T03:58:20.487 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:20.487 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 355, 2024-09-21T03:58:20.487 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.029735910, 2024-09-21T03:58:20.487 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000083763 2024-09-21T03:58:20.487 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.487 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:20.487 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 105, 2024-09-21T03:58:20.487 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000022005, 2024-09-21T03:58:20.487 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000209 2024-09-21T03:58:20.487 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.487 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 588064, 2024-09-21T03:58:20.488 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 161104 2024-09-21T03:58:20.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.488 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:58:20.488 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 752, 2024-09-21T03:58:20.488 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 949, 2024-09-21T03:58:20.488 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 181834, 2024-09-21T03:58:20.488 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 576028, 2024-09-21T03:58:20.488 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 8, 2024-09-21T03:58:20.488 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-09-21T03:58:20.488 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.092836099, 2024-09-21T03:58:20.488 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.047020202, 2024-09-21T03:58:20.488 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.042113142, 2024-09-21T03:58:20.488 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.005051607, 2024-09-21T03:58:20.489 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:20.489 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 947, 2024-09-21T03:58:20.489 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.046712685, 2024-09-21T03:58:20.489 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000049327 2024-09-21T03:58:20.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.489 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:20.489 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1464, 2024-09-21T03:58:20.489 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000342912, 2024-09-21T03:58:20.489 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000234 2024-09-21T03:58:20.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.489 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:58:20.489 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 405440 2024-09-21T03:58:20.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.490 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-21T03:58:20.490 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-21T03:58:20.490 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-21T03:58:20.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.490 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-21T03:58:20.490 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-21T03:58:20.490 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-21T03:58:20.490 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 44, 2024-09-21T03:58:20.490 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001790237, 2024-09-21T03:58:20.490 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000040687 2024-09-21T03:58:20.490 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:20.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.490 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-21T03:58:20.491 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-21T03:58:20.491 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-21T03:58:20.491 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-09-21T03:58:20.491 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000024532, 2024-09-21T03:58:20.491 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000012266 2024-09-21T03:58:20.491 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:20.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.491 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-21T03:58:20.491 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 2, 2024-09-21T03:58:20.491 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 1, 2024-09-21T03:58:20.491 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-21T03:58:20.491 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-09-21T03:58:20.491 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000340365, 2024-09-21T03:58:20.492 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000340365 2024-09-21T03:58:20.492 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.492 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-21T03:58:20.492 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 1, 2024-09-21T03:58:20.492 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-09-21T03:58:20.492 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-21T03:58:20.492 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 11, 2024-09-21T03:58:20.492 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-09-21T03:58:20.492 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-21T03:58:20.492 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 11, 2024-09-21T03:58:20.492 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-09-21T03:58:20.492 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 11, 2024-09-21T03:58:20.492 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-09-21T03:58:20.493 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 10, 2024-09-21T03:58:20.493 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-09-21T03:58:20.493 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-09-21T03:58:20.493 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-09-21T03:58:20.493 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 2, 2024-09-21T03:58:20.493 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 26, 2024-09-21T03:58:20.493 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 24, 2024-09-21T03:58:20.493 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-09-21T03:58:20.493 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 1, 2024-09-21T03:58:20.493 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:58:20.494 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-21T03:58:20.494 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-21T03:58:20.494 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 1000, 2024-09-21T03:58:20.494 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 1, 2024-09-21T03:58:20.494 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-09-21T03:58:20.494 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-09-21T03:58:20.494 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-09-21T03:58:20.494 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-09-21T03:58:20.494 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-21T03:58:20.494 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:58:20.494 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-09-21T03:58:20.494 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 22, 2024-09-21T03:58:20.494 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 10738143907, 2024-09-21T03:58:20.495 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-09-21T03:58:20.495 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:58:20.495 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-21T03:58:20.495 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:58:20.495 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:58:20.495 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:58:20.495 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:58:20.495 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:58:20.495 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-09-21T03:58:20.495 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:58:20.495 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:58:20.495 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:58:20.495 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:58:20.496 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:58:20.496 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:58:20.496 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-09-21T03:58:20.496 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:58:20.496 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-09-21T03:58:20.496 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 1 2024-09-21T03:58:20.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.496 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-21T03:58:20.496 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-21T03:58:20.496 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-21T03:58:20.496 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-21T03:58:20.496 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 2, 2024-09-21T03:58:20.496 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-09-21T03:58:20.496 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-09-21T03:58:20.497 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-21T03:58:20.497 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:58:20.497 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-09-21T03:58:20.497 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-09-21T03:58:20.497 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-21T03:58:20.497 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-21T03:58:20.497 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-21T03:58:20.497 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:58:20.497 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:58:20.497 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-21T03:58:20.497 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-21T03:58:20.497 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 11, 2024-09-21T03:58:20.497 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 22, 2024-09-21T03:58:20.497 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:58:20.498 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-09-21T03:58:20.498 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-21T03:58:20.498 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:58:20.498 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-21T03:58:20.498 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0, 2024-09-21T03:58:20.498 INFO:teuthology.orchestra.run.smithi134.stdout: "uninline_started": 0, 2024-09-21T03:58:20.498 INFO:teuthology.orchestra.run.smithi134.stdout: "uninline_succeeded": 0, 2024-09-21T03:58:20.498 INFO:teuthology.orchestra.run.smithi134.stdout: "uninline_write_failed": 0 2024-09-21T03:58:20.498 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.498 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-21T03:58:20.498 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-09-21T03:58:20.499 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 4, 2024-09-21T03:58:20.499 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-21T03:58:20.499 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-21T03:58:20.499 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 4, 2024-09-21T03:58:20.499 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-21T03:58:20.499 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-21T03:58:20.499 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 2, 2024-09-21T03:58:20.499 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-21T03:58:20.499 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-21T03:58:20.499 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 2, 2024-09-21T03:58:20.499 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-09-21T03:58:20.499 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-21T03:58:20.499 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-21T03:58:20.499 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-09-21T03:58:20.500 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 4195424, 2024-09-21T03:58:20.500 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-09-21T03:58:20.500 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-21T03:58:20.500 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3, 2024-09-21T03:58:20.500 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.005492853, 2024-09-21T03:58:20.500 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001830951 2024-09-21T03:58:20.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.500 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-09-21T03:58:20.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.500 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-21T03:58:20.500 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 14, 2024-09-21T03:58:20.500 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 14, 2024-09-21T03:58:20.501 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 0, 2024-09-21T03:58:20.501 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 12, 2024-09-21T03:58:20.501 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 13, 2024-09-21T03:58:20.501 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 1, 2024-09-21T03:58:20.501 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 11, 2024-09-21T03:58:20.501 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 11, 2024-09-21T03:58:20.501 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 0, 2024-09-21T03:58:20.501 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-09-21T03:58:20.501 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 1, 2024-09-21T03:58:20.501 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 1, 2024-09-21T03:58:20.501 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 118192, 2024-09-21T03:58:20.501 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 324816896 2024-09-21T03:58:20.501 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.501 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-21T03:58:20.501 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 3, 2024-09-21T03:58:20.502 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-09-21T03:58:20.502 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 2, 2024-09-21T03:58:20.502 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 4, 2024-09-21T03:58:20.502 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-09-21T03:58:20.502 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-21T03:58:20.502 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.502 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.502 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.502 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.502 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-21T03:58:20.502 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-09-21T03:58:20.502 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000340365, 2024-09-21T03:58:20.503 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000340365 2024-09-21T03:58:20.503 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.503 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-21T03:58:20.503 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.503 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.503 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.503 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.503 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-21T03:58:20.503 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.503 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.503 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.503 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.503 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-21T03:58:20.503 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.504 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.504 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.504 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.504 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-21T03:58:20.504 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.504 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.504 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.504 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.504 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-21T03:58:20.504 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.504 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.504 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.504 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.504 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-21T03:58:20.505 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.505 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.505 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.505 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.505 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-21T03:58:20.505 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.505 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.505 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.505 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.505 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-21T03:58:20.505 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.505 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.505 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.505 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.506 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-21T03:58:20.506 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.506 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.506 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.506 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.506 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-21T03:58:20.506 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.506 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.506 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.506 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.506 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-21T03:58:20.506 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.507 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.507 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.507 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.507 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-21T03:58:20.507 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.507 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.507 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.507 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.507 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-21T03:58:20.507 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.507 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.507 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.507 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.507 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-21T03:58:20.508 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.508 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.508 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.508 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.508 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-21T03:58:20.508 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.508 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.508 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.508 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.508 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-21T03:58:20.508 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.508 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.508 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.509 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.509 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-21T03:58:20.509 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.509 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.509 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.509 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.509 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-21T03:58:20.509 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.509 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.509 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.509 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.509 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-21T03:58:20.509 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.510 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.510 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.510 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.510 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-21T03:58:20.510 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.510 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.510 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.510 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.510 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-21T03:58:20.510 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.510 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.511 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.511 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.511 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-21T03:58:20.511 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.511 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.511 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.511 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.511 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-21T03:58:20.511 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.511 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.511 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.511 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.511 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-21T03:58:20.511 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.512 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.512 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.512 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-21T03:58:20.512 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.512 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.512 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.512 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-21T03:58:20.512 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.512 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.512 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.512 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-21T03:58:20.513 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.513 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.513 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.513 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.513 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:58:20.513 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:58:20.513 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-21T03:58:20.513 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.513 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.513 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.513 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:20.514 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.514 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-21T03:58:20.514 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 0, 2024-09-21T03:58:20.514 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 1, 2024-09-21T03:58:20.514 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 1, 2024-09-21T03:58:20.514 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 0, 2024-09-21T03:58:20.515 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-21T03:58:20.515 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 1, 2024-09-21T03:58:20.515 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 1, 2024-09-21T03:58:20.515 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 71, 2024-09-21T03:58:20.515 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-21T03:58:20.515 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.515 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-21T03:58:20.516 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:58:20.516 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-21T03:58:20.516 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:58:20.516 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:58:20.516 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:58:20.516 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:58:20.516 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:58:20.516 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:58:20.517 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:58:20.517 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:58:20.517 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:58:20.517 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:58:20.517 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:58:20.517 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:58:20.517 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:58:20.517 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-21T03:58:20.518 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:58:20.518 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-21T03:58:20.518 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:58:20.518 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:58:20.518 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:58:20.518 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:58:20.519 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:58:20.519 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:58:20.519 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:58:20.519 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-21T03:58:20.519 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:58:20.519 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:58:20.519 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:58:20.520 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-21T03:58:20.520 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-21T03:58:20.520 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-21T03:58:20.520 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:58:20.520 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:58:20.520 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:58:20.520 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:58:20.520 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 262999, 2024-09-21T03:58:20.521 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 87, 2024-09-21T03:58:20.521 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:58:20.521 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-21T03:58:20.521 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-21T03:58:20.521 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-21T03:58:20.521 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:58:20.521 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-21T03:58:20.521 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:58:20.521 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-21T03:58:20.521 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 8952, 2024-09-21T03:58:20.521 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 168, 2024-09-21T03:58:20.521 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:58:20.521 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:58:20.522 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-21T03:58:20.522 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-21T03:58:20.522 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 55416, 2024-09-21T03:58:20.522 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 213, 2024-09-21T03:58:20.522 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-21T03:58:20.522 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-21T03:58:20.522 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-21T03:58:20.522 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-21T03:58:20.522 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.522 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-21T03:58:20.522 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-21T03:58:20.522 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-21T03:58:20.522 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 38, 2024-09-21T03:58:20.523 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 11332, 2024-09-21T03:58:20.523 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 33, 2024-09-21T03:58:20.523 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 38, 2024-09-21T03:58:20.523 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-21T03:58:20.523 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 38, 2024-09-21T03:58:20.523 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.277993197, 2024-09-21T03:58:20.523 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.007315610 2024-09-21T03:58:20.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.523 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-21T03:58:20.523 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-21T03:58:20.524 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 38, 2024-09-21T03:58:20.524 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 60 2024-09-21T03:58:20.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.524 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 38, 2024-09-21T03:58:20.524 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 0, 2024-09-21T03:58:20.524 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 38, 2024-09-21T03:58:20.524 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-21T03:58:20.524 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-21T03:58:20.524 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2024-09-21T03:58:20.524 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 10, 2024-09-21T03:58:20.524 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 0, 2024-09-21T03:58:20.524 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 3, 2024-09-21T03:58:20.525 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 6, 2024-09-21T03:58:20.525 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-21T03:58:20.525 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-21T03:58:20.525 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-21T03:58:20.525 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-21T03:58:20.525 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 6, 2024-09-21T03:58:20.525 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-21T03:58:20.525 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-21T03:58:20.525 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-21T03:58:20.525 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 0, 2024-09-21T03:58:20.525 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 20, 2024-09-21T03:58:20.525 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:58:20.525 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-21T03:58:20.525 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:58:20.526 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-21T03:58:20.526 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-21T03:58:20.526 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-21T03:58:20.526 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:58:20.526 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-21T03:58:20.526 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:58:20.526 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-21T03:58:20.526 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-21T03:58:20.526 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-21T03:58:20.526 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-21T03:58:20.526 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-21T03:58:20.526 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-21T03:58:20.526 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-21T03:58:20.527 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-21T03:58:20.527 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-21T03:58:20.527 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-21T03:58:20.527 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-21T03:58:20.527 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-21T03:58:20.527 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-21T03:58:20.527 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-21T03:58:20.527 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-21T03:58:20.527 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-21T03:58:20.527 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-21T03:58:20.527 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 86, 2024-09-21T03:58:20.527 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-21T03:58:20.527 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 2, 2024-09-21T03:58:20.527 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 8, 2024-09-21T03:58:20.528 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 8, 2024-09-21T03:58:20.528 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-09-21T03:58:20.528 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-21T03:58:20.528 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 14, 2024-09-21T03:58:20.528 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 0, 2024-09-21T03:58:20.528 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 1 2024-09-21T03:58:20.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.528 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-21T03:58:20.528 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-09-21T03:58:20.528 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:58:20.528 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-21T03:58:20.528 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-21T03:58:20.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.529 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-21T03:58:20.529 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-21T03:58:20.529 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:58:20.529 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-21T03:58:20.529 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-09-21T03:58:20.529 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-09-21T03:58:20.529 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-09-21T03:58:20.529 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-21T03:58:20.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.529 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:58:20.529 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:20.529 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-21T03:58:20.529 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:20.530 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 1205, 2024-09-21T03:58:20.530 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 702318, 2024-09-21T03:58:20.530 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:20.530 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 1205, 2024-09-21T03:58:20.530 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-21T03:58:20.530 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-21T03:58:20.530 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1205, 2024-09-21T03:58:20.530 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 702318, 2024-09-21T03:58:20.530 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:20.530 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.530 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.531 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.531 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:20.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.531 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-21T03:58:20.531 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:20.531 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-21T03:58:20.531 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:20.531 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-21T03:58:20.531 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-21T03:58:20.531 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:20.531 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:20.531 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 38, 2024-09-21T03:58:20.531 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 11332, 2024-09-21T03:58:20.532 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 38, 2024-09-21T03:58:20.532 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 11332, 2024-09-21T03:58:20.532 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:20.532 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.532 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.532 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.532 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:20.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.532 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-21T03:58:20.532 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:20.532 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-21T03:58:20.532 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:20.533 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-21T03:58:20.533 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-21T03:58:20.533 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:20.533 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:20.533 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 38, 2024-09-21T03:58:20.533 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 38, 2024-09-21T03:58:20.533 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 38, 2024-09-21T03:58:20.533 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 38, 2024-09-21T03:58:20.533 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:20.533 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.533 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.533 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.533 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:20.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.534 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:58:20.534 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:20.534 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-21T03:58:20.534 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:20.534 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-21T03:58:20.534 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-21T03:58:20.534 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:20.534 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:20.534 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-21T03:58:20.534 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-21T03:58:20.534 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-21T03:58:20.534 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-21T03:58:20.535 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:20.535 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.535 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.535 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.535 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:20.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.535 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x564b5e595380": { 2024-09-21T03:58:20.535 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:20.535 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-21T03:58:20.535 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:20.535 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 4, 2024-09-21T03:58:20.535 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1120, 2024-09-21T03:58:20.535 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:20.536 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 4, 2024-09-21T03:58:20.536 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-21T03:58:20.536 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-21T03:58:20.536 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 3, 2024-09-21T03:58:20.536 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1120, 2024-09-21T03:58:20.536 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:20.536 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:20.536 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:20.536 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:20.536 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:20.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:20.536 INFO:teuthology.orchestra.run.smithi134.stdout: "trackedop": { 2024-09-21T03:58:20.536 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 30 2024-09-21T03:58:20.537 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:20.537 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-21T03:58:21.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:21 smithi134 ceph-mon[35031]: pgmap v779: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-21T03:58:21.601 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-21T03:58:21.601 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-21T03:58:21.601 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T03:58:21.601 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T03:58:21.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:21 smithi123 ceph-mon[34673]: pgmap v779: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-21T03:58:21.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:21 smithi016 ceph-mon[30375]: pgmap v779: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-21T03:58:21.864 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:58:22.385 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-21T03:58:22.385 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:58:22.385 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1693, 2024-09-21T03:58:22.385 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1695, 2024-09-21T03:58:22.385 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 4702149, 2024-09-21T03:58:22.386 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 538384, 2024-09-21T03:58:22.386 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2024-09-21T03:58:22.386 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 2, 2024-09-21T03:58:22.386 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.178512074, 2024-09-21T03:58:22.386 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.072645330, 2024-09-21T03:58:22.386 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.045092376, 2024-09-21T03:58:22.386 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.047900453, 2024-09-21T03:58:22.386 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:22.386 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1693, 2024-09-21T03:58:22.386 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.052747024, 2024-09-21T03:58:22.386 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000031155 2024-09-21T03:58:22.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.386 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:22.387 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.387 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.387 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.387 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:58:22.387 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:58:22.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.387 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:58:22.387 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 433, 2024-09-21T03:58:22.387 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 428, 2024-09-21T03:58:22.387 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 611600, 2024-09-21T03:58:22.387 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 210594, 2024-09-21T03:58:22.387 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 4, 2024-09-21T03:58:22.387 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-09-21T03:58:22.387 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.055882513, 2024-09-21T03:58:22.388 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.026101012, 2024-09-21T03:58:22.388 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.029324876, 2024-09-21T03:58:22.388 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.005554620, 2024-09-21T03:58:22.388 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:22.388 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 427, 2024-09-21T03:58:22.388 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.020307874, 2024-09-21T03:58:22.388 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000047559 2024-09-21T03:58:22.388 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.388 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:22.388 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.388 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.388 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.388 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.388 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 569600, 2024-09-21T03:58:22.388 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 157104 2024-09-21T03:58:22.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.389 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:58:22.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 898, 2024-09-21T03:58:22.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1088, 2024-09-21T03:58:22.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1529984, 2024-09-21T03:58:22.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 690039, 2024-09-21T03:58:22.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 4, 2024-09-21T03:58:22.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-09-21T03:58:22.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.107886788, 2024-09-21T03:58:22.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.052210490, 2024-09-21T03:58:22.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.020774805, 2024-09-21T03:58:22.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.026426770, 2024-09-21T03:58:22.389 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:22.389 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1087, 2024-09-21T03:58:22.389 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.039407389, 2024-09-21T03:58:22.390 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000036253 2024-09-21T03:58:22.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.390 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:22.390 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.390 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.390 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.390 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-09-21T03:58:22.390 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 406064 2024-09-21T03:58:22.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.390 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-21T03:58:22.390 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-21T03:58:22.390 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-21T03:58:22.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.391 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-21T03:58:22.391 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-21T03:58:22.391 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-21T03:58:22.391 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4564, 2024-09-21T03:58:22.391 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.132470551, 2024-09-21T03:58:22.391 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000029025 2024-09-21T03:58:22.391 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:22.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.391 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-21T03:58:22.391 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-21T03:58:22.391 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-21T03:58:22.391 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.391 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.391 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.392 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:22.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.392 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-21T03:58:22.392 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2024-09-21T03:58:22.392 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-09-21T03:58:22.392 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-21T03:58:22.392 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.392 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.392 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.392 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-21T03:58:22.392 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-09-21T03:58:22.392 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-09-21T03:58:22.392 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-21T03:58:22.392 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2024-09-21T03:58:22.392 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-09-21T03:58:22.393 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-21T03:58:22.393 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 37, 2024-09-21T03:58:22.393 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 8, 2024-09-21T03:58:22.393 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 29, 2024-09-21T03:58:22.393 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-09-21T03:58:22.393 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 25, 2024-09-21T03:58:22.393 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-09-21T03:58:22.393 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-09-21T03:58:22.393 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-09-21T03:58:22.393 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 2, 2024-09-21T03:58:22.393 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2024-09-21T03:58:22.393 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2024-09-21T03:58:22.393 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-09-21T03:58:22.393 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-09-21T03:58:22.393 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:58:22.393 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-21T03:58:22.394 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-21T03:58:22.394 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-09-21T03:58:22.394 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-21T03:58:22.394 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-09-21T03:58:22.394 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-09-21T03:58:22.394 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-09-21T03:58:22.394 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-09-21T03:58:22.394 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-21T03:58:22.394 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:58:22.394 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-09-21T03:58:22.394 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 4, 2024-09-21T03:58:22.394 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 234, 2024-09-21T03:58:22.394 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-09-21T03:58:22.395 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:58:22.395 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-21T03:58:22.395 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:58:22.395 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:58:22.395 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:58:22.395 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:58:22.395 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:58:22.395 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-09-21T03:58:22.395 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:58:22.395 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:58:22.395 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:58:22.395 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:58:22.395 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:58:22.395 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:58:22.395 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-09-21T03:58:22.396 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:58:22.396 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-09-21T03:58:22.396 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-09-21T03:58:22.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.396 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-21T03:58:22.396 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-21T03:58:22.396 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-21T03:58:22.396 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-21T03:58:22.396 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2024-09-21T03:58:22.396 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-09-21T03:58:22.396 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-09-21T03:58:22.396 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-21T03:58:22.396 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:58:22.396 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-09-21T03:58:22.397 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-09-21T03:58:22.397 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-21T03:58:22.397 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-21T03:58:22.397 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-21T03:58:22.397 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:58:22.397 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:58:22.397 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-21T03:58:22.397 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-21T03:58:22.397 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:58:22.397 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:58:22.397 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:58:22.397 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-09-21T03:58:22.397 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-21T03:58:22.397 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:58:22.398 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-21T03:58:22.398 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0, 2024-09-21T03:58:22.398 INFO:teuthology.orchestra.run.smithi134.stdout: "uninline_started": 0, 2024-09-21T03:58:22.398 INFO:teuthology.orchestra.run.smithi134.stdout: "uninline_succeeded": 0, 2024-09-21T03:58:22.398 INFO:teuthology.orchestra.run.smithi134.stdout: "uninline_write_failed": 0 2024-09-21T03:58:22.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.398 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-21T03:58:22.398 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-09-21T03:58:22.398 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2024-09-21T03:58:22.398 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-21T03:58:22.398 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-21T03:58:22.398 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 1695, 2024-09-21T03:58:22.398 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-21T03:58:22.398 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-21T03:58:22.399 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2024-09-21T03:58:22.399 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-21T03:58:22.399 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-21T03:58:22.399 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 2, 2024-09-21T03:58:22.399 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 53, 2024-09-21T03:58:22.399 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-21T03:58:22.399 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-21T03:58:22.399 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 9659672, 2024-09-21T03:58:22.399 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 9692971, 2024-09-21T03:58:22.399 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 9692372, 2024-09-21T03:58:22.399 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-21T03:58:22.399 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.399 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.399 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.400 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 1695 2024-09-21T03:58:22.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.400 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-21T03:58:22.400 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 20, 2024-09-21T03:58:22.400 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 37, 2024-09-21T03:58:22.400 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 17, 2024-09-21T03:58:22.400 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 10, 2024-09-21T03:58:22.400 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 14, 2024-09-21T03:58:22.400 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 4, 2024-09-21T03:58:22.400 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 37, 2024-09-21T03:58:22.400 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 58, 2024-09-21T03:58:22.400 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 21, 2024-09-21T03:58:22.400 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-09-21T03:58:22.400 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-09-21T03:58:22.401 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-09-21T03:58:22.401 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 118404, 2024-09-21T03:58:22.401 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 317468672 2024-09-21T03:58:22.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.401 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-21T03:58:22.401 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2024-09-21T03:58:22.401 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-09-21T03:58:22.401 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2024-09-21T03:58:22.401 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2024-09-21T03:58:22.401 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-09-21T03:58:22.401 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-21T03:58:22.401 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.401 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.401 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.402 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-21T03:58:22.402 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.402 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.402 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.402 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-21T03:58:22.402 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.402 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.402 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.402 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-21T03:58:22.402 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.402 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.402 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.403 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-21T03:58:22.403 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.403 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.403 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.403 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-21T03:58:22.403 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.403 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.403 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.403 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-21T03:58:22.403 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.403 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.404 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.404 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-21T03:58:22.404 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.404 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.404 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.404 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-21T03:58:22.404 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.404 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.404 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.404 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-21T03:58:22.404 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.404 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.405 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.405 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-21T03:58:22.405 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.405 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.405 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.406 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-21T03:58:22.406 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.406 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.406 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.406 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-21T03:58:22.406 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.406 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.406 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.406 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-21T03:58:22.406 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.406 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.407 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.407 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-21T03:58:22.407 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.407 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.407 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.407 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-21T03:58:22.407 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.407 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.407 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.407 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-21T03:58:22.407 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.408 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.408 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.408 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-21T03:58:22.408 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.408 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.408 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.408 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-21T03:58:22.408 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.408 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.408 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.409 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-21T03:58:22.409 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.409 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.409 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.409 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-21T03:58:22.409 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.409 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.409 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.409 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-21T03:58:22.409 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.409 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.409 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.410 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-21T03:58:22.410 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.410 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.410 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.410 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-21T03:58:22.410 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.410 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.410 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.410 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-21T03:58:22.410 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.410 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.411 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.411 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-21T03:58:22.411 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.411 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.411 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.411 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-21T03:58:22.411 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.411 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.411 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.411 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-21T03:58:22.411 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.412 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.412 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.412 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-21T03:58:22.412 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.412 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.412 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.412 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:58:22.412 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:58:22.412 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-21T03:58:22.412 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.412 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.413 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.413 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:22.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.413 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-21T03:58:22.413 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2024-09-21T03:58:22.413 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 3, 2024-09-21T03:58:22.413 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 2, 2024-09-21T03:58:22.413 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 1, 2024-09-21T03:58:22.413 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-21T03:58:22.413 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-09-21T03:58:22.413 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-09-21T03:58:22.413 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 2230, 2024-09-21T03:58:22.413 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-21T03:58:22.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.414 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-21T03:58:22.414 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:58:22.414 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-21T03:58:22.414 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:58:22.414 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:58:22.414 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:58:22.414 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:58:22.414 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:58:22.414 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:58:22.414 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:58:22.414 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:58:22.414 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:58:22.414 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:58:22.414 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:58:22.415 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:58:22.415 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:58:22.415 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-21T03:58:22.415 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:58:22.415 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-21T03:58:22.415 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:58:22.415 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:58:22.415 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:58:22.415 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:58:22.415 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:58:22.415 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:58:22.415 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:58:22.415 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-21T03:58:22.415 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:58:22.416 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:58:22.416 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:58:22.416 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-21T03:58:22.416 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-21T03:58:22.416 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-21T03:58:22.416 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:58:22.416 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:58:22.416 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:58:22.416 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:58:22.416 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 189546, 2024-09-21T03:58:22.416 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 55, 2024-09-21T03:58:22.416 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:58:22.416 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-21T03:58:22.416 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-21T03:58:22.416 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-21T03:58:22.417 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:58:22.417 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-21T03:58:22.417 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:58:22.417 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-21T03:58:22.417 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 8952, 2024-09-21T03:58:22.417 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 168, 2024-09-21T03:58:22.417 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:58:22.417 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:58:22.417 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-21T03:58:22.417 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-21T03:58:22.417 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 85389, 2024-09-21T03:58:22.417 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 1047, 2024-09-21T03:58:22.417 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-21T03:58:22.417 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-21T03:58:22.418 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-21T03:58:22.418 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-21T03:58:22.418 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.418 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-21T03:58:22.418 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-21T03:58:22.418 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-21T03:58:22.418 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 2754, 2024-09-21T03:58:22.418 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 26, 2024-09-21T03:58:22.418 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 4, 2024-09-21T03:58:22.418 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 2754, 2024-09-21T03:58:22.418 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-21T03:58:22.418 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2754, 2024-09-21T03:58:22.418 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.996952175, 2024-09-21T03:58:22.418 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000725109 2024-09-21T03:58:22.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.419 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-21T03:58:22.419 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-21T03:58:22.419 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2754, 2024-09-21T03:58:22.419 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2755 2024-09-21T03:58:22.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.419 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 2754, 2024-09-21T03:58:22.419 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 2754, 2024-09-21T03:58:22.419 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2024-09-21T03:58:22.419 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-21T03:58:22.419 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-21T03:58:22.419 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 1708, 2024-09-21T03:58:22.419 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2024-09-21T03:58:22.419 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 1043, 2024-09-21T03:58:22.420 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2024-09-21T03:58:22.420 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2024-09-21T03:58:22.420 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-21T03:58:22.420 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-21T03:58:22.420 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-21T03:58:22.420 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-21T03:58:22.420 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2024-09-21T03:58:22.420 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-21T03:58:22.420 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-21T03:58:22.420 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-21T03:58:22.420 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 2, 2024-09-21T03:58:22.420 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2024-09-21T03:58:22.420 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:58:22.420 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-21T03:58:22.421 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:58:22.421 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-21T03:58:22.421 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-21T03:58:22.421 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-21T03:58:22.421 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:58:22.421 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-21T03:58:22.421 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:58:22.421 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-21T03:58:22.421 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-21T03:58:22.421 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-21T03:58:22.421 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-21T03:58:22.421 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-21T03:58:22.421 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-21T03:58:22.421 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-21T03:58:22.421 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-21T03:58:22.422 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-21T03:58:22.422 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-21T03:58:22.422 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-21T03:58:22.422 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-21T03:58:22.422 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-21T03:58:22.422 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-21T03:58:22.422 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-21T03:58:22.422 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-21T03:58:22.422 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-21T03:58:22.422 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 86, 2024-09-21T03:58:22.422 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-21T03:58:22.422 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 2, 2024-09-21T03:58:22.422 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 5, 2024-09-21T03:58:22.423 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 5, 2024-09-21T03:58:22.423 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-09-21T03:58:22.423 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-21T03:58:22.423 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2024-09-21T03:58:22.423 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2024-09-21T03:58:22.423 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2024-09-21T03:58:22.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.423 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-21T03:58:22.423 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2024-09-21T03:58:22.423 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:58:22.423 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-21T03:58:22.423 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-21T03:58:22.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.423 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-21T03:58:22.423 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-21T03:58:22.423 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:58:22.424 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-21T03:58:22.424 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-09-21T03:58:22.424 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-09-21T03:58:22.424 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-09-21T03:58:22.424 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-21T03:58:22.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.424 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:58:22.424 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:22.424 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-21T03:58:22.424 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:22.424 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 3020, 2024-09-21T03:58:22.424 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 6555390, 2024-09-21T03:58:22.424 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:22.424 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 3020, 2024-09-21T03:58:22.425 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-21T03:58:22.425 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-21T03:58:22.425 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 3020, 2024-09-21T03:58:22.425 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 6555390, 2024-09-21T03:58:22.425 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:22.425 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.425 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.425 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.425 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:22.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.425 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-21T03:58:22.425 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:22.425 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-21T03:58:22.425 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:22.425 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-21T03:58:22.425 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-21T03:58:22.426 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:22.426 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:22.426 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2754, 2024-09-21T03:58:22.426 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 5501523, 2024-09-21T03:58:22.426 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2754, 2024-09-21T03:58:22.426 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 5501523, 2024-09-21T03:58:22.426 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:22.426 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.426 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.426 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.426 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:22.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.426 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-21T03:58:22.426 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:22.426 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-21T03:58:22.427 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:22.427 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-21T03:58:22.427 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-21T03:58:22.427 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:22.427 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:22.427 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2754, 2024-09-21T03:58:22.427 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 2754, 2024-09-21T03:58:22.427 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2754, 2024-09-21T03:58:22.427 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2754, 2024-09-21T03:58:22.427 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:22.427 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.427 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.427 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.427 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:22.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.428 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:58:22.428 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:22.428 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-21T03:58:22.428 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:22.428 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-21T03:58:22.428 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-21T03:58:22.428 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:22.428 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:22.428 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-21T03:58:22.428 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-21T03:58:22.428 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-21T03:58:22.428 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-21T03:58:22.428 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:22.428 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.428 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.429 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.429 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:22.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.429 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x56506a4df450": { 2024-09-21T03:58:22.429 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:22.429 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-21T03:58:22.429 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:22.429 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-21T03:58:22.429 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-21T03:58:22.429 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:22.429 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:22.429 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-21T03:58:22.429 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-21T03:58:22.429 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-21T03:58:22.429 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-21T03:58:22.429 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:22.430 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:22.430 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:22.430 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:22.430 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:22.430 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:22.430 INFO:teuthology.orchestra.run.smithi134.stdout: "trackedop": { 2024-09-21T03:58:22.430 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-09-21T03:58:22.430 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:22.430 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-21T03:58:23.155 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-21T03:58:23.155 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-21T03:58:23.155 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T03:58:23.155 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T03:58:23.398 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:23 smithi134 ceph-mon[35031]: pgmap v780: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:58:23.416 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:58:23.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:23 smithi123 ceph-mon[34673]: pgmap v780: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:58:23.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:23 smithi016 ceph-mon[30375]: pgmap v780: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:58:23.954 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-21T03:58:23.954 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:58:23.955 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 508, 2024-09-21T03:58:23.955 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 648, 2024-09-21T03:58:23.955 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 116954, 2024-09-21T03:58:23.955 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 139508, 2024-09-21T03:58:23.955 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 11, 2024-09-21T03:58:23.955 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-09-21T03:58:23.955 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.074978678, 2024-09-21T03:58:23.955 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.031895036, 2024-09-21T03:58:23.955 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.036578854, 2024-09-21T03:58:23.955 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.005216757, 2024-09-21T03:58:23.955 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:23.955 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 647, 2024-09-21T03:58:23.955 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.044679529, 2024-09-21T03:58:23.956 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000069056 2024-09-21T03:58:23.956 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.956 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:23.956 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1099, 2024-09-21T03:58:23.956 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000273557, 2024-09-21T03:58:23.956 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000248 2024-09-21T03:58:23.956 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.956 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:58:23.956 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:58:23.956 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.956 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:58:23.956 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 370, 2024-09-21T03:58:23.956 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 359, 2024-09-21T03:58:23.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 619877, 2024-09-21T03:58:23.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 196383, 2024-09-21T03:58:23.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 7, 2024-09-21T03:58:23.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-09-21T03:58:23.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.048520974, 2024-09-21T03:58:23.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.023367978, 2024-09-21T03:58:23.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.032429079, 2024-09-21T03:58:23.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000501332, 2024-09-21T03:58:23.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:23.957 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 358, 2024-09-21T03:58:23.957 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.027733912, 2024-09-21T03:58:23.957 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000077469 2024-09-21T03:58:23.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:23.958 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 104, 2024-09-21T03:58:23.958 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000056974, 2024-09-21T03:58:23.958 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000547 2024-09-21T03:58:23.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 593552, 2024-09-21T03:58:23.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 161744 2024-09-21T03:58:23.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.958 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:58:23.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 123, 2024-09-21T03:58:23.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 314, 2024-09-21T03:58:23.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 47360, 2024-09-21T03:58:23.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 594962, 2024-09-21T03:58:23.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 14, 2024-09-21T03:58:23.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-09-21T03:58:23.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.027403597, 2024-09-21T03:58:23.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.016512744, 2024-09-21T03:58:23.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.009658159, 2024-09-21T03:58:23.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000145904, 2024-09-21T03:58:23.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:23.959 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 312, 2024-09-21T03:58:23.959 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.023800573, 2024-09-21T03:58:23.959 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000076283 2024-09-21T03:58:23.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:23.959 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 214, 2024-09-21T03:58:23.959 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000045181, 2024-09-21T03:58:23.959 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000211 2024-09-21T03:58:23.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.960 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:58:23.960 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 497456 2024-09-21T03:58:23.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.960 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-21T03:58:23.960 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-21T03:58:23.960 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-21T03:58:23.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.960 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-21T03:58:23.960 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-21T03:58:23.960 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-21T03:58:23.960 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 44, 2024-09-21T03:58:23.960 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001167988, 2024-09-21T03:58:23.960 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000026545 2024-09-21T03:58:23.961 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:23.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.961 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-21T03:58:23.961 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-21T03:58:23.961 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-21T03:58:23.961 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-09-21T03:58:23.961 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000023856, 2024-09-21T03:58:23.961 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000011928 2024-09-21T03:58:23.961 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:23.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.961 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-21T03:58:23.961 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2024-09-21T03:58:23.961 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-09-21T03:58:23.962 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-21T03:58:23.962 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.962 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.962 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.962 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-21T03:58:23.962 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-09-21T03:58:23.962 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-09-21T03:58:23.962 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-21T03:58:23.962 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 11, 2024-09-21T03:58:23.962 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-09-21T03:58:23.962 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-21T03:58:23.962 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 10, 2024-09-21T03:58:23.962 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-09-21T03:58:23.963 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 10, 2024-09-21T03:58:23.963 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-09-21T03:58:23.963 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 10, 2024-09-21T03:58:23.963 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-09-21T03:58:23.963 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-09-21T03:58:23.963 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-09-21T03:58:23.963 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 1, 2024-09-21T03:58:23.963 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 22, 2024-09-21T03:58:23.963 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 22, 2024-09-21T03:58:23.963 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-09-21T03:58:23.963 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-09-21T03:58:23.963 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:58:23.963 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-21T03:58:23.964 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-21T03:58:23.964 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-09-21T03:58:23.964 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-21T03:58:23.964 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-09-21T03:58:23.964 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-09-21T03:58:23.964 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-09-21T03:58:23.964 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-09-21T03:58:23.964 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-21T03:58:23.964 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:58:23.964 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-09-21T03:58:23.964 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 22, 2024-09-21T03:58:23.964 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 10738143907, 2024-09-21T03:58:23.964 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-09-21T03:58:23.965 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:58:23.965 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-21T03:58:23.965 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:58:23.965 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:58:23.965 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:58:23.965 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:58:23.965 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:58:23.965 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-09-21T03:58:23.965 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:58:23.965 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:58:23.965 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:58:23.965 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:58:23.965 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:58:23.965 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:58:23.966 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-09-21T03:58:23.966 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:58:23.966 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-09-21T03:58:23.966 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-09-21T03:58:23.966 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.966 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-21T03:58:23.966 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-21T03:58:23.966 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-21T03:58:23.966 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-21T03:58:23.966 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 1, 2024-09-21T03:58:23.967 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-09-21T03:58:23.967 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-09-21T03:58:23.967 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-21T03:58:23.967 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:58:23.967 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-09-21T03:58:23.967 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-09-21T03:58:23.967 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-21T03:58:23.967 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-21T03:58:23.967 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-21T03:58:23.967 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:58:23.967 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:58:23.967 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-21T03:58:23.967 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-21T03:58:23.967 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 11, 2024-09-21T03:58:23.967 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 11, 2024-09-21T03:58:23.968 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:58:23.968 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-09-21T03:58:23.968 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-21T03:58:23.968 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:58:23.968 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-21T03:58:23.968 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0, 2024-09-21T03:58:23.968 INFO:teuthology.orchestra.run.smithi134.stdout: "uninline_started": 0, 2024-09-21T03:58:23.968 INFO:teuthology.orchestra.run.smithi134.stdout: "uninline_succeeded": 0, 2024-09-21T03:58:23.968 INFO:teuthology.orchestra.run.smithi134.stdout: "uninline_write_failed": 0 2024-09-21T03:58:23.968 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.968 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-21T03:58:23.968 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-09-21T03:58:23.968 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 2, 2024-09-21T03:58:23.968 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-21T03:58:23.969 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-21T03:58:23.969 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 2, 2024-09-21T03:58:23.969 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-21T03:58:23.969 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-21T03:58:23.969 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 2, 2024-09-21T03:58:23.969 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-21T03:58:23.969 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-21T03:58:23.969 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 2, 2024-09-21T03:58:23.969 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-09-21T03:58:23.969 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-21T03:58:23.969 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-21T03:58:23.969 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-09-21T03:58:23.969 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 4194892, 2024-09-21T03:58:23.969 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-09-21T03:58:23.969 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-21T03:58:23.970 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-09-21T03:58:23.970 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001567926, 2024-09-21T03:58:23.970 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001567926 2024-09-21T03:58:23.970 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.970 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-09-21T03:58:23.970 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.970 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-21T03:58:23.970 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 13, 2024-09-21T03:58:23.970 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 13, 2024-09-21T03:58:23.970 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 0, 2024-09-21T03:58:23.970 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 11, 2024-09-21T03:58:23.970 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 12, 2024-09-21T03:58:23.970 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 1, 2024-09-21T03:58:23.970 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 10, 2024-09-21T03:58:23.971 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 10, 2024-09-21T03:58:23.971 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 0, 2024-09-21T03:58:23.971 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-09-21T03:58:23.971 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-09-21T03:58:23.971 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-09-21T03:58:23.971 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 122164, 2024-09-21T03:58:23.971 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 324808704 2024-09-21T03:58:23.971 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.971 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-21T03:58:23.971 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2024-09-21T03:58:23.971 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-09-21T03:58:23.971 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2024-09-21T03:58:23.971 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2024-09-21T03:58:23.972 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-09-21T03:58:23.972 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-21T03:58:23.972 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.972 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.972 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.972 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-21T03:58:23.972 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.972 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.972 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.972 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-21T03:58:23.972 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.972 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.972 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.973 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-21T03:58:23.973 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.973 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.973 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.973 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-21T03:58:23.973 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.973 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.973 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.973 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-21T03:58:23.973 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.973 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.974 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.974 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-21T03:58:23.974 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.974 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.974 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.974 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-21T03:58:23.974 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.974 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.974 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.974 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-21T03:58:23.974 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.975 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.975 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.975 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-21T03:58:23.975 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.975 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.975 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.975 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-21T03:58:23.975 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.976 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.976 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.976 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.976 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-21T03:58:23.976 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.976 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.976 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.977 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-21T03:58:23.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.977 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.977 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-21T03:58:23.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.977 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.977 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-21T03:58:23.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.978 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.978 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.978 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.978 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-21T03:58:23.978 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.978 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.978 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.978 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.978 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-21T03:58:23.978 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.978 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.978 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.978 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.979 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-21T03:58:23.979 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.979 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.979 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.979 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.979 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-21T03:58:23.979 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.979 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.979 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.979 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.979 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-21T03:58:23.979 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.979 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.980 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.980 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-21T03:58:23.980 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.980 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.980 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.980 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-21T03:58:23.980 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.980 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.980 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.980 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-21T03:58:23.981 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.981 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.981 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.981 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.981 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-21T03:58:23.981 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.981 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.981 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.981 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.981 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-21T03:58:23.981 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.981 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.981 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.981 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.982 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-21T03:58:23.982 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.982 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.982 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.982 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-21T03:58:23.982 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.982 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.982 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.982 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-21T03:58:23.982 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.982 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.983 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.983 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.983 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-21T03:58:23.983 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.983 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.983 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.983 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.983 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:58:23.983 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:58:23.983 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-21T03:58:23.983 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.983 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.984 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.984 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:23.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.984 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-21T03:58:23.984 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 0, 2024-09-21T03:58:23.984 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 0, 2024-09-21T03:58:23.984 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 0, 2024-09-21T03:58:23.984 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 0, 2024-09-21T03:58:23.984 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-21T03:58:23.984 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-09-21T03:58:23.984 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-09-21T03:58:23.984 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 0, 2024-09-21T03:58:23.984 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-21T03:58:23.985 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.985 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-21T03:58:23.985 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:58:23.985 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-21T03:58:23.985 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:58:23.985 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:58:23.985 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:58:23.985 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:58:23.985 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:58:23.985 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:58:23.985 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:58:23.985 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:58:23.985 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:58:23.985 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:58:23.986 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:58:23.986 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:58:23.986 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:58:23.986 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-21T03:58:23.986 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:58:23.986 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-21T03:58:23.986 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:58:23.986 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:58:23.986 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:58:23.986 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:58:23.986 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:58:23.986 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:58:23.986 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:58:23.986 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-21T03:58:23.987 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:58:23.987 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:58:23.987 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:58:23.987 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-21T03:58:23.987 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-21T03:58:23.987 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-21T03:58:23.987 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:58:23.987 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:58:23.987 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:58:23.987 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:58:23.987 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 230823, 2024-09-21T03:58:23.987 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 75, 2024-09-21T03:58:23.987 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:58:23.987 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-21T03:58:23.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-21T03:58:23.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-21T03:58:23.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:58:23.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-21T03:58:23.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:58:23.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-21T03:58:23.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 8952, 2024-09-21T03:58:23.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 168, 2024-09-21T03:58:23.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:58:23.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:58:23.988 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-21T03:58:23.988 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-21T03:58:23.988 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 51192, 2024-09-21T03:58:23.989 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 198, 2024-09-21T03:58:23.989 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-21T03:58:23.989 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-21T03:58:23.989 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-21T03:58:23.989 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-21T03:58:23.989 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.989 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-21T03:58:23.989 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-21T03:58:23.989 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-21T03:58:23.989 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 35, 2024-09-21T03:58:23.989 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 10710, 2024-09-21T03:58:23.989 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 33, 2024-09-21T03:58:23.989 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 35, 2024-09-21T03:58:23.989 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-21T03:58:23.990 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 35, 2024-09-21T03:58:23.990 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.260993598, 2024-09-21T03:58:23.990 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.007456959 2024-09-21T03:58:23.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.990 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-21T03:58:23.990 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-21T03:58:23.990 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 35, 2024-09-21T03:58:23.990 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 57 2024-09-21T03:58:23.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.990 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 35, 2024-09-21T03:58:23.990 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 0, 2024-09-21T03:58:23.990 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 35, 2024-09-21T03:58:23.990 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-21T03:58:23.990 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-21T03:58:23.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2024-09-21T03:58:23.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 10, 2024-09-21T03:58:23.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 0, 2024-09-21T03:58:23.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 1, 2024-09-21T03:58:23.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 5, 2024-09-21T03:58:23.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-21T03:58:23.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-21T03:58:23.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-21T03:58:23.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-21T03:58:23.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 6, 2024-09-21T03:58:23.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-21T03:58:23.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-21T03:58:23.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-21T03:58:23.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 0, 2024-09-21T03:58:23.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 20, 2024-09-21T03:58:23.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:58:23.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-21T03:58:23.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:58:23.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-21T03:58:23.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-21T03:58:23.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-21T03:58:23.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:58:23.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-21T03:58:23.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:58:23.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-21T03:58:23.992 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-21T03:58:23.992 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-21T03:58:23.992 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-21T03:58:23.993 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-21T03:58:23.993 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-21T03:58:23.993 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-21T03:58:23.993 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-21T03:58:23.993 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-21T03:58:23.993 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-21T03:58:23.993 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-21T03:58:23.993 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-21T03:58:23.993 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-21T03:58:23.993 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-21T03:58:23.993 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-21T03:58:23.993 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-21T03:58:23.993 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-21T03:58:23.993 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 86, 2024-09-21T03:58:23.994 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-21T03:58:23.994 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 2, 2024-09-21T03:58:23.994 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 8, 2024-09-21T03:58:23.994 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 8, 2024-09-21T03:58:23.994 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-09-21T03:58:23.994 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-21T03:58:23.994 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 14, 2024-09-21T03:58:23.994 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 0, 2024-09-21T03:58:23.994 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 1 2024-09-21T03:58:23.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.994 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-21T03:58:23.994 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-09-21T03:58:23.994 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:58:23.994 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-21T03:58:23.994 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-21T03:58:23.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.995 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-21T03:58:23.995 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-21T03:58:23.995 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:58:23.995 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-21T03:58:23.995 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-09-21T03:58:23.995 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-09-21T03:58:23.995 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-09-21T03:58:23.995 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-21T03:58:23.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.995 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:58:23.995 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:23.995 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-21T03:58:23.995 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:23.996 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 997, 2024-09-21T03:58:23.996 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 676267, 2024-09-21T03:58:23.996 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:23.996 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 997, 2024-09-21T03:58:23.996 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-21T03:58:23.996 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-21T03:58:23.996 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 997, 2024-09-21T03:58:23.996 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 676267, 2024-09-21T03:58:23.996 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:23.996 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.996 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.996 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.996 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:23.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.996 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-21T03:58:23.997 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:23.997 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-21T03:58:23.997 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:23.997 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-21T03:58:23.997 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-21T03:58:23.997 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:23.997 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:23.997 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 35, 2024-09-21T03:58:23.997 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 10710, 2024-09-21T03:58:23.997 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 35, 2024-09-21T03:58:23.997 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 10710, 2024-09-21T03:58:23.997 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:23.997 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.997 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.998 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.998 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:23.998 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.998 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-21T03:58:23.998 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:23.998 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-21T03:58:23.998 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:23.998 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-21T03:58:23.998 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-21T03:58:23.998 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:23.998 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:23.998 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 35, 2024-09-21T03:58:23.998 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 35, 2024-09-21T03:58:23.998 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 35, 2024-09-21T03:58:23.999 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 35, 2024-09-21T03:58:23.999 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:23.999 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:23.999 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:23.999 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:23.999 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:23.999 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:23.999 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:58:23.999 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:23.999 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-21T03:58:23.999 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:23.999 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-21T03:58:23.999 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-21T03:58:23.999 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:23.999 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:24.000 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-21T03:58:24.000 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-21T03:58:24.000 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-21T03:58:24.000 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-21T03:58:24.000 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:24.000 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:24.000 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:24.000 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:24.000 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:24.000 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:24.000 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x56288012d380": { 2024-09-21T03:58:24.000 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:24.000 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-21T03:58:24.000 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:24.001 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 2, 2024-09-21T03:58:24.001 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 588, 2024-09-21T03:58:24.001 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:24.001 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 2, 2024-09-21T03:58:24.001 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-21T03:58:24.001 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-21T03:58:24.001 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1, 2024-09-21T03:58:24.001 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 588, 2024-09-21T03:58:24.001 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:24.001 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:24.001 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:24.001 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:24.001 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:24.001 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:24.001 INFO:teuthology.orchestra.run.smithi134.stdout: "trackedop": { 2024-09-21T03:58:24.002 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 20 2024-09-21T03:58:24.002 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:24.002 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-21T03:58:24.827 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-21T03:58:24.827 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-21T03:58:24.827 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T03:58:24.827 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T03:58:25.203 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:58:25.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:25 smithi134 ceph-mon[35031]: pgmap v781: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T03:58:25.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:25 smithi123 ceph-mon[34673]: pgmap v781: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T03:58:25.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:25 smithi016 ceph-mon[30375]: pgmap v781: 129 pgs: 129 active+clean; 10 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T03:58:25.749 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-21T03:58:25.749 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:58:25.749 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 275, 2024-09-21T03:58:25.749 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 268, 2024-09-21T03:58:25.749 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 574914, 2024-09-21T03:58:25.749 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 161076, 2024-09-21T03:58:25.749 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 5, 2024-09-21T03:58:25.749 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 1, 2024-09-21T03:58:25.749 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.044623250, 2024-09-21T03:58:25.750 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.020261455, 2024-09-21T03:58:25.750 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.029369165, 2024-09-21T03:58:25.750 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000029713, 2024-09-21T03:58:25.750 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:25.750 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 268, 2024-09-21T03:58:25.750 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.016386603, 2024-09-21T03:58:25.750 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000061144 2024-09-21T03:58:25.750 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.750 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:25.750 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.750 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.750 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.750 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.750 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 574080, 2024-09-21T03:58:25.751 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 160752 2024-09-21T03:58:25.751 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.751 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:58:25.751 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 4, 2024-09-21T03:58:25.751 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 200, 2024-09-21T03:58:25.751 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 834, 2024-09-21T03:58:25.751 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 498977, 2024-09-21T03:58:25.751 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 5, 2024-09-21T03:58:25.751 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-09-21T03:58:25.751 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.014747905, 2024-09-21T03:58:25.751 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.011233305, 2024-09-21T03:58:25.751 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.001657509, 2024-09-21T03:58:25.751 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000081785, 2024-09-21T03:58:25.752 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:25.752 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 198, 2024-09-21T03:58:25.752 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.016425691, 2024-09-21T03:58:25.752 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000082958 2024-09-21T03:58:25.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.752 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:25.752 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.752 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.752 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.752 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:58:25.752 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 498336 2024-09-21T03:58:25.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.752 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:58:25.752 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1713, 2024-09-21T03:58:25.753 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1715, 2024-09-21T03:58:25.753 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 512715, 2024-09-21T03:58:25.753 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 544792, 2024-09-21T03:58:25.753 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 4, 2024-09-21T03:58:25.753 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-09-21T03:58:25.753 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.185769344, 2024-09-21T03:58:25.753 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.077502151, 2024-09-21T03:58:25.753 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.043353436, 2024-09-21T03:58:25.753 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.052057257, 2024-09-21T03:58:25.753 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:58:25.753 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1713, 2024-09-21T03:58:25.753 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.057643289, 2024-09-21T03:58:25.753 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000033650 2024-09-21T03:58:25.753 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.753 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:58:25.754 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.754 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.754 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.754 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.754 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:58:25.754 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:58:25.754 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.754 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-21T03:58:25.754 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-21T03:58:25.754 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-21T03:58:25.754 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.754 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-21T03:58:25.754 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-21T03:58:25.754 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-21T03:58:25.755 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3425, 2024-09-21T03:58:25.755 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.105618010, 2024-09-21T03:58:25.755 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000030837 2024-09-21T03:58:25.755 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:25.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.755 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-21T03:58:25.755 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-21T03:58:25.755 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-21T03:58:25.755 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.755 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.755 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.755 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:25.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.755 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-21T03:58:25.755 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2024-09-21T03:58:25.755 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-09-21T03:58:25.756 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-21T03:58:25.756 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.756 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.756 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.756 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.756 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-21T03:58:25.756 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-09-21T03:58:25.756 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-09-21T03:58:25.756 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-21T03:58:25.756 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2024-09-21T03:58:25.756 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-09-21T03:58:25.756 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-21T03:58:25.756 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 0, 2024-09-21T03:58:25.756 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-09-21T03:58:25.757 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 0, 2024-09-21T03:58:25.757 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-09-21T03:58:25.757 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 0, 2024-09-21T03:58:25.757 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-09-21T03:58:25.757 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-09-21T03:58:25.757 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-09-21T03:58:25.757 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 1, 2024-09-21T03:58:25.757 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2024-09-21T03:58:25.757 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2024-09-21T03:58:25.757 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-09-21T03:58:25.757 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-09-21T03:58:25.757 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:58:25.757 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-21T03:58:25.757 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-21T03:58:25.757 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-09-21T03:58:25.757 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-21T03:58:25.758 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-09-21T03:58:25.758 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-09-21T03:58:25.758 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-09-21T03:58:25.758 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-09-21T03:58:25.758 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-21T03:58:25.758 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:58:25.758 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-09-21T03:58:25.758 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 0, 2024-09-21T03:58:25.758 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 0, 2024-09-21T03:58:25.758 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-09-21T03:58:25.758 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:58:25.758 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-21T03:58:25.758 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:58:25.758 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:58:25.759 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:58:25.759 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:58:25.759 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:58:25.759 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-09-21T03:58:25.759 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:58:25.759 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:58:25.759 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:58:25.759 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:58:25.759 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:58:25.759 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:58:25.759 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-09-21T03:58:25.759 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:58:25.759 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-09-21T03:58:25.759 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-09-21T03:58:25.760 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.760 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-21T03:58:25.760 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-21T03:58:25.760 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-21T03:58:25.760 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-21T03:58:25.760 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2024-09-21T03:58:25.760 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-09-21T03:58:25.760 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-09-21T03:58:25.760 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-21T03:58:25.760 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:58:25.760 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-09-21T03:58:25.760 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-09-21T03:58:25.760 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-21T03:58:25.760 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-21T03:58:25.760 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-21T03:58:25.761 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:58:25.761 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:58:25.761 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-21T03:58:25.761 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-21T03:58:25.761 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:58:25.761 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:58:25.761 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:58:25.761 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-09-21T03:58:25.761 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-21T03:58:25.761 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:58:25.761 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-21T03:58:25.761 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0, 2024-09-21T03:58:25.761 INFO:teuthology.orchestra.run.smithi134.stdout: "uninline_started": 0, 2024-09-21T03:58:25.761 INFO:teuthology.orchestra.run.smithi134.stdout: "uninline_succeeded": 0, 2024-09-21T03:58:25.762 INFO:teuthology.orchestra.run.smithi134.stdout: "uninline_write_failed": 0 2024-09-21T03:58:25.762 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.762 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-21T03:58:25.762 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-09-21T03:58:25.762 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2024-09-21T03:58:25.762 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-21T03:58:25.762 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-21T03:58:25.762 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 2, 2024-09-21T03:58:25.762 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-21T03:58:25.762 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-21T03:58:25.762 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2024-09-21T03:58:25.762 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-21T03:58:25.762 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-21T03:58:25.762 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 2, 2024-09-21T03:58:25.762 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-09-21T03:58:25.763 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-21T03:58:25.763 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-21T03:58:25.763 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-09-21T03:58:25.763 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 4194892, 2024-09-21T03:58:25.763 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 4194352, 2024-09-21T03:58:25.763 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-21T03:58:25.763 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.763 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.763 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.763 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.763 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 2 2024-09-21T03:58:25.763 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.763 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-21T03:58:25.763 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 3, 2024-09-21T03:58:25.763 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 3, 2024-09-21T03:58:25.764 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 0, 2024-09-21T03:58:25.764 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 1, 2024-09-21T03:58:25.764 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 1, 2024-09-21T03:58:25.764 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 0, 2024-09-21T03:58:25.764 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 0, 2024-09-21T03:58:25.764 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 0, 2024-09-21T03:58:25.764 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 0, 2024-09-21T03:58:25.764 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-09-21T03:58:25.764 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-09-21T03:58:25.764 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-09-21T03:58:25.764 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 116680, 2024-09-21T03:58:25.764 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 315367424 2024-09-21T03:58:25.764 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.764 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-21T03:58:25.765 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2024-09-21T03:58:25.765 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-09-21T03:58:25.765 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2024-09-21T03:58:25.765 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2024-09-21T03:58:25.765 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-09-21T03:58:25.765 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-21T03:58:25.765 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.765 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.765 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.765 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.765 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-21T03:58:25.765 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.765 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.765 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.765 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.766 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-21T03:58:25.766 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.766 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.766 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.766 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-21T03:58:25.766 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.766 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.766 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.766 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-21T03:58:25.766 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.766 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.766 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.767 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-21T03:58:25.767 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.767 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.767 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.767 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-21T03:58:25.767 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.767 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.767 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.767 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-21T03:58:25.768 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.768 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.768 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.768 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.768 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-21T03:58:25.768 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.768 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.768 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.768 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.768 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-21T03:58:25.769 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.769 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.769 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.769 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-21T03:58:25.769 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.769 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.770 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.770 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.770 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-21T03:58:25.770 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.770 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.770 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.770 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.770 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-21T03:58:25.770 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.771 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.771 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.771 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.771 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-21T03:58:25.771 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.771 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.771 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.771 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.771 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-21T03:58:25.771 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.771 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.771 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.771 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.772 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-21T03:58:25.772 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.772 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.772 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.772 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.772 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-21T03:58:25.772 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.772 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.772 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.772 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.772 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-21T03:58:25.772 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.772 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.772 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.773 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.773 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-21T03:58:25.773 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.773 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.773 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.773 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.773 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-21T03:58:25.773 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.773 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.773 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.773 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.773 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-21T03:58:25.773 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.774 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.774 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.774 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.774 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-21T03:58:25.774 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.774 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.774 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.774 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.774 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-21T03:58:25.774 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.774 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.774 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.774 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.775 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-21T03:58:25.775 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.775 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.775 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.775 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.775 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-21T03:58:25.775 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.775 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.775 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.775 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.775 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-21T03:58:25.775 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.775 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.775 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.776 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.776 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-21T03:58:25.776 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.776 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.776 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.776 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.776 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-21T03:58:25.776 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.776 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.776 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.776 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.776 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-21T03:58:25.777 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.777 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.777 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.777 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.777 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:58:25.777 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:58:25.777 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-21T03:58:25.777 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.777 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.777 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.777 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:25.777 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.777 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-21T03:58:25.778 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 0, 2024-09-21T03:58:25.778 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 0, 2024-09-21T03:58:25.778 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 0, 2024-09-21T03:58:25.778 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 0, 2024-09-21T03:58:25.778 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-21T03:58:25.778 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-09-21T03:58:25.778 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-09-21T03:58:25.778 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 0, 2024-09-21T03:58:25.778 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-21T03:58:25.778 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.778 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-21T03:58:25.778 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:58:25.778 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-21T03:58:25.779 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:58:25.779 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:58:25.779 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:58:25.779 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:58:25.779 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:58:25.779 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:58:25.779 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:58:25.779 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:58:25.779 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:58:25.779 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:58:25.779 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:58:25.779 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:58:25.779 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:58:25.779 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-21T03:58:25.780 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:58:25.780 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-21T03:58:25.780 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:58:25.780 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:58:25.780 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:58:25.780 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:58:25.780 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:58:25.780 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:58:25.780 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:58:25.780 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-21T03:58:25.780 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:58:25.780 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:58:25.780 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:58:25.781 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-21T03:58:25.781 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-21T03:58:25.781 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-21T03:58:25.781 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:58:25.781 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:58:25.781 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:58:25.781 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:58:25.781 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 182066, 2024-09-21T03:58:25.781 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 48, 2024-09-21T03:58:25.781 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:58:25.781 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-21T03:58:25.781 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-21T03:58:25.781 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-21T03:58:25.781 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:58:25.781 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-21T03:58:25.782 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:58:25.782 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-21T03:58:25.782 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 8952, 2024-09-21T03:58:25.782 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 168, 2024-09-21T03:58:25.782 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:58:25.782 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:58:25.782 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-21T03:58:25.782 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-21T03:58:25.782 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 8672, 2024-09-21T03:58:25.782 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 13, 2024-09-21T03:58:25.782 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-21T03:58:25.782 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-21T03:58:25.782 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-21T03:58:25.782 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-21T03:58:25.783 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.783 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-21T03:58:25.783 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-21T03:58:25.783 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-21T03:58:25.783 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 1716, 2024-09-21T03:58:25.783 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 21, 2024-09-21T03:58:25.783 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 3, 2024-09-21T03:58:25.783 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 1716, 2024-09-21T03:58:25.783 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-21T03:58:25.783 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1716, 2024-09-21T03:58:25.783 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 4.010904057, 2024-09-21T03:58:25.783 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002337356 2024-09-21T03:58:25.783 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.783 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-21T03:58:25.784 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-21T03:58:25.784 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1716, 2024-09-21T03:58:25.784 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1717 2024-09-21T03:58:25.784 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.784 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 1716, 2024-09-21T03:58:25.784 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 1716, 2024-09-21T03:58:25.784 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2024-09-21T03:58:25.784 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-21T03:58:25.784 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-21T03:58:25.784 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 855, 2024-09-21T03:58:25.784 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2024-09-21T03:58:25.784 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 859, 2024-09-21T03:58:25.784 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2024-09-21T03:58:25.784 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2024-09-21T03:58:25.784 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-21T03:58:25.785 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-21T03:58:25.785 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-21T03:58:25.785 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-21T03:58:25.785 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2024-09-21T03:58:25.785 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-21T03:58:25.785 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-21T03:58:25.785 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-21T03:58:25.785 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 1, 2024-09-21T03:58:25.785 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2024-09-21T03:58:25.785 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:58:25.785 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-21T03:58:25.785 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:58:25.785 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-21T03:58:25.785 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-21T03:58:25.786 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-21T03:58:25.786 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:58:25.786 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-21T03:58:25.786 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:58:25.786 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-21T03:58:25.786 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-21T03:58:25.786 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-21T03:58:25.786 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-21T03:58:25.786 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-21T03:58:25.786 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-21T03:58:25.786 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-21T03:58:25.786 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-21T03:58:25.786 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-21T03:58:25.786 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-21T03:58:25.787 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-21T03:58:25.787 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-21T03:58:25.787 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-21T03:58:25.787 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-21T03:58:25.787 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-21T03:58:25.787 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-21T03:58:25.787 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-21T03:58:25.787 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 86, 2024-09-21T03:58:25.787 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-21T03:58:25.787 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 2, 2024-09-21T03:58:25.787 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 4, 2024-09-21T03:58:25.787 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 4, 2024-09-21T03:58:25.787 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-09-21T03:58:25.787 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-21T03:58:25.788 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2024-09-21T03:58:25.788 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2024-09-21T03:58:25.788 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2024-09-21T03:58:25.788 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.788 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-21T03:58:25.788 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2024-09-21T03:58:25.788 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:58:25.788 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-21T03:58:25.788 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-21T03:58:25.788 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.788 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-21T03:58:25.788 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-21T03:58:25.788 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:58:25.788 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-21T03:58:25.788 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-09-21T03:58:25.789 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-09-21T03:58:25.789 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-09-21T03:58:25.789 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-21T03:58:25.789 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.789 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:58:25.789 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:25.789 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-21T03:58:25.789 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:25.789 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 1988, 2024-09-21T03:58:25.789 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 892691, 2024-09-21T03:58:25.789 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:25.789 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 1988, 2024-09-21T03:58:25.789 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-21T03:58:25.789 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-21T03:58:25.790 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1988, 2024-09-21T03:58:25.790 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 892691, 2024-09-21T03:58:25.790 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:25.790 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.790 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.790 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.790 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:25.790 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.790 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-21T03:58:25.790 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:25.790 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-21T03:58:25.790 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:25.790 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-21T03:58:25.790 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-21T03:58:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 1716, 2024-09-21T03:58:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 593, 2024-09-21T03:58:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1716, 2024-09-21T03:58:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 593, 2024-09-21T03:58:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-21T03:58:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-21T03:58:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-21T03:58:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-21T03:58:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 1716, 2024-09-21T03:58:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 1716, 2024-09-21T03:58:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1716, 2024-09-21T03:58:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1716, 2024-09-21T03:58:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:58:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-21T03:58:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-21T03:58:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-21T03:58:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-21T03:58:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-21T03:58:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-21T03:58:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-21T03:58:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x559eb87092b0": { 2024-09-21T03:58:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-21T03:58:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-21T03:58:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-21T03:58:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-21T03:58:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-21T03:58:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-21T03:58:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-21T03:58:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-21T03:58:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-21T03:58:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-21T03:58:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-21T03:58:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-21T03:58:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-21T03:58:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-21T03:58:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-21T03:58:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-21T03:58:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "trackedop": { 2024-09-21T03:58:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-09-21T03:58:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-21T03:58:25.795 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-21T03:58:26.586 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-21T03:58:26.586 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-21T03:58:26.586 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T03:58:26.596 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T03:58:26.596 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:58:26.596 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:58:26.658 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:58:26.658 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:58:26.738 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T03:58:27.162 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:58:27.539 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:27 smithi016 ceph-mon[30375]: pgmap v782: 129 pgs: 129 active+clean; 9.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 75 KiB/s wr, 26 op/s 2024-09-21T03:58:27.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:27 smithi134 ceph-mon[35031]: pgmap v782: 129 pgs: 129 active+clean; 9.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 75 KiB/s wr, 26 op/s 2024-09-21T03:58:27.637 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:27 smithi123 ceph-mon[34673]: pgmap v782: 129 pgs: 129 active+clean; 9.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 75 KiB/s wr, 26 op/s 2024-09-21T03:58:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.861+0000 7f3406e00640 1 -- 172.21.15.16:0/2973854082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3400109c40 msgr2=0x7f3400110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.861+0000 7f3406e00640 1 --2- 172.21.15.16:0/2973854082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3400109c40 0x7f3400110470 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009ca0 tx=0x7f33f802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:58:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.861+0000 7f3406e00640 1 -- 172.21.15.16:0/2973854082 shutdown_connections 2024-09-21T03:58:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.861+0000 7f3406e00640 1 --2- 172.21.15.16:0/2973854082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3400109c40 0x7f3400110470 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.861+0000 7f3406e00640 1 --2- 172.21.15.16:0/2973854082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34001092f0 0x7f3400109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.861+0000 7f3406e00640 1 --2- 172.21.15.16:0/2973854082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34001039b0 0x7f3400103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.861+0000 7f3406e00640 1 -- 172.21.15.16:0/2973854082 >> 172.21.15.16:0/2973854082 conn(0x7f34000ff400 msgr2=0x7f3400101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.861+0000 7f3406e00640 1 -- 172.21.15.16:0/2973854082 shutdown_connections 2024-09-21T03:58:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.861+0000 7f3406e00640 1 -- 172.21.15.16:0/2973854082 wait complete. 2024-09-21T03:58:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.862+0000 7f3406e00640 1 Processor -- start 2024-09-21T03:58:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.862+0000 7f3406e00640 1 -- start start 2024-09-21T03:58:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.863+0000 7f3406e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34001039b0 0x7f340019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.863+0000 7f3406e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34001092f0 0x7f340019c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.863+0000 7f3406e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3400109c40 0x7f34001a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.863+0000 7f3406e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3400113900 con 0x7f34001039b0 2024-09-21T03:58:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.863+0000 7f3406e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3400113780 con 0x7f3400109c40 2024-09-21T03:58:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.863+0000 7f3406e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3400113a80 con 0x7f34001092f0 2024-09-21T03:58:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.863+0000 7f3405a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34001039b0 0x7f340019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.863+0000 7f3406400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3400109c40 0x7f34001a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.863+0000 7f3405a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34001039b0 0x7f340019c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35694/0 (socket says 172.21.15.16:35694) 2024-09-21T03:58:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.863+0000 7f3405a00640 1 -- 172.21.15.16:0/1028572132 learned_addr learned my addr 172.21.15.16:0/1028572132 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:58:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.863+0000 7f3405000640 1 --2- 172.21.15.16:0/1028572132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34001092f0 0x7f340019c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.864+0000 7f3406400640 1 -- 172.21.15.16:0/1028572132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34001092f0 msgr2=0x7f340019c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.864+0000 7f3406400640 1 --2- 172.21.15.16:0/1028572132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34001092f0 0x7f340019c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.864+0000 7f3406400640 1 -- 172.21.15.16:0/1028572132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34001039b0 msgr2=0x7f340019c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.864+0000 7f3406400640 1 --2- 172.21.15.16:0/1028572132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34001039b0 0x7f340019c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.864+0000 7f3406400640 1 -- 172.21.15.16:0/1028572132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34001a23c0 con 0x7f3400109c40 2024-09-21T03:58:27.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.864+0000 7f3406400640 1 --2- 172.21.15.16:0/1028572132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3400109c40 0x7f34001a1c90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f33f802ade0 tx=0x7f33f8031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.864+0000 7f33fe000640 1 -- 172.21.15.16:0/1028572132 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8003b70 con 0x7f3400109c40 2024-09-21T03:58:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.864+0000 7f33fe000640 1 -- 172.21.15.16:0/1028572132 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33f8031910 con 0x7f3400109c40 2024-09-21T03:58:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.864+0000 7f33fe000640 1 -- 172.21.15.16:0/1028572132 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f802f950 con 0x7f3400109c40 2024-09-21T03:58:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.864+0000 7f3406e00640 1 -- 172.21.15.16:0/1028572132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34001a2650 con 0x7f3400109c40 2024-09-21T03:58:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.864+0000 7f3406e00640 1 -- 172.21.15.16:0/1028572132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34001a8e10 con 0x7f3400109c40 2024-09-21T03:58:27.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.865+0000 7f3406e00640 1 -- 172.21.15.16:0/1028572132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3400103dc0 con 0x7f3400109c40 2024-09-21T03:58:27.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.868+0000 7f33fe000640 1 -- 172.21.15.16:0/1028572132 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f33f8004b40 con 0x7f3400109c40 2024-09-21T03:58:27.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.869+0000 7f33fe000640 1 --2- 172.21.15.16:0/1028572132 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f33e80b6590 0x7f33e80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:27.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.869+0000 7f33fe000640 1 -- 172.21.15.16:0/1028572132 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f33f8108840 con 0x7f3400109c40 2024-09-21T03:58:27.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.869+0000 7f3405a00640 1 --2- 172.21.15.16:0/1028572132 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f33e80b6590 0x7f33e80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.870+0000 7f3405a00640 1 --2- 172.21.15.16:0/1028572132 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f33e80b6590 0x7f33e80b8aa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009cd0 tx=0x7f33f4005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:27.871+0000 7f33fe000640 1 -- 172.21.15.16:0/1028572132 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f33f80cecc0 con 0x7f3400109c40 2024-09-21T03:58:28.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:28.083+0000 7f3406e00640 1 -- 172.21.15.16:0/1028572132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f340019d580 con 0x7f3400109c40 2024-09-21T03:58:28.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:28.084+0000 7f33fe000640 1 -- 172.21.15.16:0/1028572132 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) ==== 81+0+586 (secure 0 0 0) 0x7f340019d580 con 0x7f3400109c40 2024-09-21T03:58:28.087 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/557742783 2024-09-22T03:34:53.217208+0000 2024-09-21T03:58:28.087 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3238676548 2024-09-22T03:34:53.217208+0000 2024-09-21T03:58:28.087 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/500178737 2024-09-22T03:34:53.217208+0000 2024-09-21T03:58:28.087 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2357336233 2024-09-22T03:34:22.549156+0000 2024-09-21T03:58:28.088 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/428542610 2024-09-22T03:34:22.549156+0000 2024-09-21T03:58:28.088 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/428542610 2024-09-22T03:34:22.549156+0000 2024-09-21T03:58:28.088 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3806844703 2024-09-22T03:34:22.549156+0000 2024-09-21T03:58:28.088 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2351581621 2024-09-22T03:34:53.217208+0000 2024-09-21T03:58:28.088 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/500178737 2024-09-22T03:34:53.217208+0000 2024-09-21T03:58:28.088 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/355731724 2024-09-22T03:34:22.549156+0000 2024-09-21T03:58:28.088 INFO:teuthology.orchestra.run.smithi016.stderr:listed 10 entries 2024-09-21T03:58:28.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:28.086+0000 7f3406e00640 1 -- 172.21.15.16:0/1028572132 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f33e80b6590 msgr2=0x7f33e80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:28.086+0000 7f3406e00640 1 --2- 172.21.15.16:0/1028572132 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f33e80b6590 0x7f33e80b8aa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009cd0 tx=0x7f33f4005f90 comp rx=0 tx=0).stop 2024-09-21T03:58:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:28.086+0000 7f3406e00640 1 -- 172.21.15.16:0/1028572132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3400109c40 msgr2=0x7f34001a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:28.086+0000 7f3406e00640 1 --2- 172.21.15.16:0/1028572132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3400109c40 0x7f34001a1c90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f33f802ade0 tx=0x7f33f8031660 comp rx=0 tx=0).stop 2024-09-21T03:58:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:28.087+0000 7f3406e00640 1 -- 172.21.15.16:0/1028572132 shutdown_connections 2024-09-21T03:58:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:28.087+0000 7f3406e00640 1 --2- 172.21.15.16:0/1028572132 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f33e80b6590 0x7f33e80b8aa0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:28.087+0000 7f3406e00640 1 --2- 172.21.15.16:0/1028572132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3400109c40 0x7f34001a1c90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:28.087+0000 7f3406e00640 1 --2- 172.21.15.16:0/1028572132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34001092f0 0x7f340019c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:28.087+0000 7f3406e00640 1 --2- 172.21.15.16:0/1028572132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34001039b0 0x7f340019c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:28.087+0000 7f3406e00640 1 -- 172.21.15.16:0/1028572132 >> 172.21.15.16:0/1028572132 conn(0x7f34000ff400 msgr2=0x7f3400108900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:28.087+0000 7f3406e00640 1 -- 172.21.15.16:0/1028572132 shutdown_connections 2024-09-21T03:58:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:28.087+0000 7f3406e00640 1 -- 172.21.15.16:0/1028572132 wait complete. 2024-09-21T03:58:28.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:28 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1028572132' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:58:28.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:28 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1028572132' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:58:28.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:28 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1028572132' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:58:28.890 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:58:28.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:58:28.890 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T03:58:28.975 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-21T03:58:28.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:58:28.976 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:58:29.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:58:29.049 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-21T03:58:29.131 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:58:29.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:58:29.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:58:29.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:58:29.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-21T03:58:29.465 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:29 smithi016 ceph-mon[30375]: pgmap v783: 129 pgs: 129 active+clean; 7.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 123 KiB/s wr, 57 op/s 2024-09-21T03:58:29.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:29 smithi123 ceph-mon[34673]: pgmap v783: 129 pgs: 129 active+clean; 7.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 123 KiB/s wr, 57 op/s 2024-09-21T03:58:29.658 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:29 smithi134 ceph-mon[35031]: pgmap v783: 129 pgs: 129 active+clean; 7.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 123 KiB/s wr, 57 op/s 2024-09-21T03:58:29.658 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T03:58:30.141 INFO:tasks.cephadm:Teardown begin 2024-09-21T03:58:30.141 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T03:58:30.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:58:30.804 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:58:31.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:31 smithi134 ceph-mon[35031]: pgmap v784: 129 pgs: 129 active+clean; 5.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 125 KiB/s wr, 103 op/s 2024-09-21T03:58:31.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.651+0000 7f001f000640 1 -- 172.21.15.16:0/2954413166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0018105280 msgr2=0x7f001810d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:31.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.651+0000 7f001f000640 1 --2- 172.21.15.16:0/2954413166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0018105280 0x7f001810d910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0000009900 tx=0x7f000002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:58:31.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.651+0000 7f001f000640 1 -- 172.21.15.16:0/2954413166 shutdown_connections 2024-09-21T03:58:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.651+0000 7f001f000640 1 --2- 172.21.15.16:0/2954413166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f001810de50 0x7f0018110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.651+0000 7f001f000640 1 --2- 172.21.15.16:0/2954413166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0018105280 0x7f001810d910 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.651+0000 7f001f000640 1 --2- 172.21.15.16:0/2954413166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00181048a0 0x7f0018104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.651+0000 7f001f000640 1 -- 172.21.15.16:0/2954413166 >> 172.21.15.16:0/2954413166 conn(0x7f0018079720 msgr2=0x7f0018079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.651+0000 7f001f000640 1 -- 172.21.15.16:0/2954413166 shutdown_connections 2024-09-21T03:58:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.651+0000 7f001f000640 1 -- 172.21.15.16:0/2954413166 wait complete. 2024-09-21T03:58:31.656 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:31 smithi016 ceph-mon[30375]: pgmap v784: 129 pgs: 129 active+clean; 5.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 125 KiB/s wr, 103 op/s 2024-09-21T03:58:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.653+0000 7f001f000640 1 Processor -- start 2024-09-21T03:58:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.653+0000 7f001f000640 1 -- start start 2024-09-21T03:58:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.653+0000 7f001f000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00181048a0 0x7f00181a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.653+0000 7f001f000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0018105280 0x7f00181a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.653+0000 7f001f000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f001810de50 0x7f00181ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.653+0000 7f001f000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0018113a90 con 0x7f00181048a0 2024-09-21T03:58:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.653+0000 7f001f000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0018113910 con 0x7f001810de50 2024-09-21T03:58:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.653+0000 7f001f000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0018113c10 con 0x7f0018105280 2024-09-21T03:58:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.654+0000 7f001d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f001810de50 0x7f00181ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.654+0000 7f0017e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00181048a0 0x7f00181a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.654+0000 7f001d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f001810de50 0x7f00181ac7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:59078/0 (socket says 172.21.15.16:59078) 2024-09-21T03:58:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.654+0000 7f0017e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00181048a0 0x7f00181a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35722/0 (socket says 172.21.15.16:35722) 2024-09-21T03:58:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.654+0000 7f001d000640 1 -- 172.21.15.16:0/3514625359 learned_addr learned my addr 172.21.15.16:0/3514625359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:58:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.654+0000 7f000fe00640 1 --2- 172.21.15.16:0/3514625359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0018105280 0x7f00181a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.654+0000 7f001d000640 1 -- 172.21.15.16:0/3514625359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0018105280 msgr2=0x7f00181a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.654+0000 7f001d000640 1 --2- 172.21.15.16:0/3514625359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0018105280 0x7f00181a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.654+0000 7f001d000640 1 -- 172.21.15.16:0/3514625359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00181048a0 msgr2=0x7f00181a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.654+0000 7f001d000640 1 --2- 172.21.15.16:0/3514625359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00181048a0 0x7f00181a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.654+0000 7f001d000640 1 -- 172.21.15.16:0/3514625359 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00181acd10 con 0x7f001810de50 2024-09-21T03:58:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.655+0000 7f001d000640 1 --2- 172.21.15.16:0/3514625359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f001810de50 0x7f00181ac7d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f001000eb10 tx=0x7f001000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.655+0000 7f0015600640 1 -- 172.21.15.16:0/3514625359 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010004b20 con 0x7f001810de50 2024-09-21T03:58:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.655+0000 7f0015600640 1 -- 172.21.15.16:0/3514625359 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0010004cc0 con 0x7f001810de50 2024-09-21T03:58:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.655+0000 7f001f000640 1 -- 172.21.15.16:0/3514625359 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00181acee0 con 0x7f001810de50 2024-09-21T03:58:31.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:31 smithi123 ceph-mon[34673]: pgmap v784: 129 pgs: 129 active+clean; 5.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 125 KiB/s wr, 103 op/s 2024-09-21T03:58:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.655+0000 7f0015600640 1 -- 172.21.15.16:0/3514625359 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010010670 con 0x7f001810de50 2024-09-21T03:58:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.655+0000 7f001f000640 1 -- 172.21.15.16:0/3514625359 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00181ad330 con 0x7f001810de50 2024-09-21T03:58:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.657+0000 7f001f000640 1 -- 172.21.15.16:0/3514625359 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0018104cb0 con 0x7f001810de50 2024-09-21T03:58:31.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.659+0000 7f0015600640 1 -- 172.21.15.16:0/3514625359 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0010005020 con 0x7f001810de50 2024-09-21T03:58:31.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.660+0000 7f0015600640 1 --2- 172.21.15.16:0/3514625359 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efffc0b6590 0x7efffc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:31.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.660+0000 7f0015600640 1 -- 172.21.15.16:0/3514625359 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f00100e7c00 con 0x7f001810de50 2024-09-21T03:58:31.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.660+0000 7f0017e00640 1 --2- 172.21.15.16:0/3514625359 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efffc0b6590 0x7efffc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:31.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.661+0000 7f0017e00640 1 --2- 172.21.15.16:0/3514625359 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efffc0b6590 0x7efffc0b8aa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0018109860 tx=0x7f0008005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:31.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.664+0000 7f0015600640 1 -- 172.21.15.16:0/3514625359 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f00100ae080 con 0x7f001810de50 2024-09-21T03:58:31.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.907+0000 7f001f000640 1 -- 172.21.15.16:0/3514625359 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00181ad7e0 con 0x7f001810de50 2024-09-21T03:58:31.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.908+0000 7f0015600640 1 -- 172.21.15.16:0/3514625359 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f00100b2460 con 0x7f001810de50 2024-09-21T03:58:31.911 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:58:31.911 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:58:31.912 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:58:31.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.912+0000 7f001f000640 1 -- 172.21.15.16:0/3514625359 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efffc0b6590 msgr2=0x7efffc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:31.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.912+0000 7f001f000640 1 --2- 172.21.15.16:0/3514625359 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efffc0b6590 0x7efffc0b8aa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0018109860 tx=0x7f0008005f90 comp rx=0 tx=0).stop 2024-09-21T03:58:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.912+0000 7f001f000640 1 -- 172.21.15.16:0/3514625359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f001810de50 msgr2=0x7f00181ac7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.912+0000 7f001f000640 1 --2- 172.21.15.16:0/3514625359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f001810de50 0x7f00181ac7d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f001000eb10 tx=0x7f001000c2d0 comp rx=0 tx=0).stop 2024-09-21T03:58:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.912+0000 7f001f000640 1 -- 172.21.15.16:0/3514625359 shutdown_connections 2024-09-21T03:58:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.912+0000 7f001f000640 1 --2- 172.21.15.16:0/3514625359 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7efffc0b6590 0x7efffc0b8aa0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.912+0000 7f001f000640 1 --2- 172.21.15.16:0/3514625359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f001810de50 0x7f00181ac7d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.912+0000 7f001f000640 1 --2- 172.21.15.16:0/3514625359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0018105280 0x7f00181a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.912+0000 7f001f000640 1 --2- 172.21.15.16:0/3514625359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00181048a0 0x7f00181a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.912+0000 7f001f000640 1 -- 172.21.15.16:0/3514625359 >> 172.21.15.16:0/3514625359 conn(0x7f0018079720 msgr2=0x7f00180ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.913+0000 7f001f000640 1 -- 172.21.15.16:0/3514625359 shutdown_connections 2024-09-21T03:58:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:31.913+0000 7f001f000640 1 -- 172.21.15.16:0/3514625359 wait complete. 2024-09-21T03:58:32.598 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:32 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3514625359' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:58:32.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:32 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3514625359' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:58:32.635 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:58:32.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:32 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3514625359' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:58:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:58:33.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:33 smithi016 ceph-mon[30375]: pgmap v785: 129 pgs: 129 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 125 KiB/s wr, 140 op/s 2024-09-21T03:58:33.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:33 smithi134 ceph-mon[35031]: pgmap v785: 129 pgs: 129 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 125 KiB/s wr, 140 op/s 2024-09-21T03:58:33.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.644+0000 7f2787e00640 1 -- 172.21.15.16:0/4018050053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f278810dc30 msgr2=0x7f2788110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:33.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.644+0000 7f2787e00640 1 --2- 172.21.15.16:0/4018050053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f278810dc30 0x7f2788110120 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2780009ca0 tx=0x7f278002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:58:33.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.645+0000 7f2787e00640 1 -- 172.21.15.16:0/4018050053 shutdown_connections 2024-09-21T03:58:33.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.645+0000 7f2787e00640 1 --2- 172.21.15.16:0/4018050053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f278810dc30 0x7f2788110120 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:33.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.645+0000 7f2787e00640 1 --2- 172.21.15.16:0/4018050053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2788105120 0x7f278810d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:33.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.645+0000 7f2787e00640 1 --2- 172.21.15.16:0/4018050053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27881047d0 0x7f2788104be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:33.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.645+0000 7f2787e00640 1 -- 172.21.15.16:0/4018050053 >> 172.21.15.16:0/4018050053 conn(0x7f27880fe2d0 msgr2=0x7f2788100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:33.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.645+0000 7f2787e00640 1 -- 172.21.15.16:0/4018050053 shutdown_connections 2024-09-21T03:58:33.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.645+0000 7f2787e00640 1 -- 172.21.15.16:0/4018050053 wait complete. 2024-09-21T03:58:33.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.646+0000 7f2787e00640 1 Processor -- start 2024-09-21T03:58:33.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.646+0000 7f2787e00640 1 -- start start 2024-09-21T03:58:33.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.647+0000 7f2787e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27881047d0 0x7f2788102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:33.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.647+0000 7f2787e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2788105120 0x7f2788102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:33.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.647+0000 7f2787e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f278810dc30 0x7f2788102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:33.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.647+0000 7f2787e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27881138b0 con 0x7f27881047d0 2024-09-21T03:58:33.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.647+0000 7f2787e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2788113730 con 0x7f278810dc30 2024-09-21T03:58:33.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.647+0000 7f2787e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2788113a30 con 0x7f2788105120 2024-09-21T03:58:33.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.647+0000 7f2786a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27881047d0 0x7f2788102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:33.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.647+0000 7f2786000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2788105120 0x7f2788102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:33.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.647+0000 7f2786a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27881047d0 0x7f2788102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42640/0 (socket says 172.21.15.16:42640) 2024-09-21T03:58:33.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.647+0000 7f2786a00640 1 -- 172.21.15.16:0/1296217011 learned_addr learned my addr 172.21.15.16:0/1296217011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:58:33.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.647+0000 7f2787400640 1 --2- 172.21.15.16:0/1296217011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f278810dc30 0x7f2788102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:33.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.648+0000 7f2786a00640 1 -- 172.21.15.16:0/1296217011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2788105120 msgr2=0x7f2788102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:33.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.648+0000 7f2786a00640 1 --2- 172.21.15.16:0/1296217011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2788105120 0x7f2788102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:33.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.648+0000 7f2786a00640 1 -- 172.21.15.16:0/1296217011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f278810dc30 msgr2=0x7f2788102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:33.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.648+0000 7f2786a00640 1 --2- 172.21.15.16:0/1296217011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f278810dc30 0x7f2788102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:33.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.648+0000 7f2786a00640 1 -- 172.21.15.16:0/1296217011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27881a8500 con 0x7f27881047d0 2024-09-21T03:58:33.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.648+0000 7f2786a00640 1 --2- 172.21.15.16:0/1296217011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27881047d0 0x7f2788102510 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f277800e9f0 tx=0x7f277800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:33.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.648+0000 7f278ca00640 1 -- 172.21.15.16:0/1296217011 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277800cd80 con 0x7f27881047d0 2024-09-21T03:58:33.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.649+0000 7f2787e00640 1 -- 172.21.15.16:0/1296217011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27881a87f0 con 0x7f27881047d0 2024-09-21T03:58:33.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.649+0000 7f278ca00640 1 -- 172.21.15.16:0/1296217011 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27780108e0 con 0x7f27881047d0 2024-09-21T03:58:33.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.649+0000 7f2787e00640 1 -- 172.21.15.16:0/1296217011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27881a8d30 con 0x7f27881047d0 2024-09-21T03:58:33.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.649+0000 7f278ca00640 1 -- 172.21.15.16:0/1296217011 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778018980 con 0x7f27881047d0 2024-09-21T03:58:33.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.650+0000 7f2787e00640 1 -- 172.21.15.16:0/1296217011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2788104be0 con 0x7f27881047d0 2024-09-21T03:58:33.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.652+0000 7f278ca00640 1 -- 172.21.15.16:0/1296217011 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f2778010430 con 0x7f27881047d0 2024-09-21T03:58:33.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:33 smithi123 ceph-mon[34673]: pgmap v785: 129 pgs: 129 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 125 KiB/s wr, 140 op/s 2024-09-21T03:58:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.654+0000 7f278ca00640 1 --2- 172.21.15.16:0/1296217011 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f276c0b6510 0x7f276c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.654+0000 7f278ca00640 1 -- 172.21.15.16:0/1296217011 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f27780e7eb0 con 0x7f27881047d0 2024-09-21T03:58:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.654+0000 7f2786000640 1 --2- 172.21.15.16:0/1296217011 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f276c0b6510 0x7f276c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.655+0000 7f2786000640 1 --2- 172.21.15.16:0/1296217011 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f276c0b6510 0x7f276c0b8a20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f27881100a0 tx=0x7f2774005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.656+0000 7f278ca00640 1 -- 172.21.15.16:0/1296217011 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f27780b2710 con 0x7f27881047d0 2024-09-21T03:58:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.896+0000 7f2787e00640 1 -- 172.21.15.16:0/1296217011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27881a9140 con 0x7f27881047d0 2024-09-21T03:58:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.897+0000 7f278ca00640 1 -- 172.21.15.16:0/1296217011 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11971 (secure 0 0 0) 0x7f27780b7650 con 0x7f27881047d0 2024-09-21T03:58:33.900 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:58:33.900 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-09-21T03:46:14:584115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2280740148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2280740148},{"type":"v1","addr":"172.21.15.16:6835","nonce":2280740148}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1667623391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1667623391},{"type":"v1","addr":"172.21.15.16:6837","nonce":1667623391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:43:09.332305+0000","modified":"2024-09-21T03:46:13.605821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24731,"mds_1":24689,"mds_2":24682,"mds_3":24670,"mds_4":24668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14700":{"gid":14700,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/89613887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":89613887},{"type":"v1","addr":"172.21.15.16:6839","nonce":89613887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14712":{"gid":14712,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/1546097467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":1546097467},{"type":"v1","addr":"172.21.15.16:6841","nonce":1546097467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2239407349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2239407349},{"type":"v1","addr":"172.21.15.134:6833","nonce":2239407349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/3581513383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3581513383},{"type":"v1","addr":"172.21.15.123:6833","nonce":3581513383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/135285598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":135285598},{"type":"v1","addr":"172.21.15.134:6835","nonce":135285598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.123:6835/86959152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":86959152},{"type":"v1","addr":"172.21.15.123:6835","nonce":86959152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.134:6837/2624530453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":2624530453},{"type":"v1","addr":"172.21.15.134:6837","nonce":2624530453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24682":{"gid":24682,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.134:6839/606193011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":606193011},{"type":"v1","addr":"172.21.15.134:6839","nonce":606193011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24689":{"gid":24689,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.123:6837/2001424876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2001424876},{"type":"v1","addr":"172.21.15.123:6837","nonce":2001424876}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/459522816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":459522816},{"type":"v1","addr":"172.21.15.123:6839","nonce":459522816}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24731,"qdb_cluster":[24731,24689,24682,24670,24668]},"id":1}]} 2024-09-21T03:58:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-09-21T03:58:33.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.900+0000 7f2787e00640 1 -- 172.21.15.16:0/1296217011 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f276c0b6510 msgr2=0x7f276c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:33.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.900+0000 7f2787e00640 1 --2- 172.21.15.16:0/1296217011 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f276c0b6510 0x7f276c0b8a20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f27881100a0 tx=0x7f2774005f90 comp rx=0 tx=0).stop 2024-09-21T03:58:33.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.900+0000 7f2787e00640 1 -- 172.21.15.16:0/1296217011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27881047d0 msgr2=0x7f2788102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:33.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.900+0000 7f2787e00640 1 --2- 172.21.15.16:0/1296217011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27881047d0 0x7f2788102510 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f277800e9f0 tx=0x7f277800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:58:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.900+0000 7f2787e00640 1 -- 172.21.15.16:0/1296217011 shutdown_connections 2024-09-21T03:58:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.900+0000 7f2787e00640 1 --2- 172.21.15.16:0/1296217011 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f276c0b6510 0x7f276c0b8a20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.900+0000 7f2787e00640 1 --2- 172.21.15.16:0/1296217011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f278810dc30 0x7f2788102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.900+0000 7f2787e00640 1 --2- 172.21.15.16:0/1296217011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2788105120 0x7f2788102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.900+0000 7f2787e00640 1 --2- 172.21.15.16:0/1296217011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27881047d0 0x7f2788102510 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.900+0000 7f2787e00640 1 -- 172.21.15.16:0/1296217011 >> 172.21.15.16:0/1296217011 conn(0x7f27880fe2d0 msgr2=0x7f2788107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.901+0000 7f2787e00640 1 -- 172.21.15.16:0/1296217011 shutdown_connections 2024-09-21T03:58:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:33.901+0000 7f2787e00640 1 -- 172.21.15.16:0/1296217011 wait complete. 2024-09-21T03:58:34.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:34 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1296217011' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:58:34.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:34 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1296217011' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:58:34.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:34 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1296217011' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:58:35.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:35 smithi016 ceph-mon[30375]: pgmap v786: 129 pgs: 129 active+clean; 3.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 125 KiB/s wr, 158 op/s 2024-09-21T03:58:35.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:58:35.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:35 smithi134 ceph-mon[35031]: pgmap v786: 129 pgs: 129 active+clean; 3.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 125 KiB/s wr, 158 op/s 2024-09-21T03:58:35.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:35 smithi123 ceph-mon[34673]: pgmap v786: 129 pgs: 129 active+clean; 3.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 125 KiB/s wr, 158 op/s 2024-09-21T03:58:35.943 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:58:36.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.593+0000 7f33f3c00640 1 -- 172.21.15.16:0/1741462792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec105d80 msgr2=0x7f33ec106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:36.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.594+0000 7f33f3c00640 1 --2- 172.21.15.16:0/1741462792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec105d80 0x7f33ec106190 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009900 tx=0x7f33dc02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:58:36.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.594+0000 7f33f3c00640 1 -- 172.21.15.16:0/1741462792 shutdown_connections 2024-09-21T03:58:36.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.594+0000 7f33f3c00640 1 --2- 172.21.15.16:0/1741462792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33ec10c010 0x7f33ec112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:36.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.594+0000 7f33f3c00640 1 --2- 172.21.15.16:0/1741462792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33ec10b6c0 0x7f33ec10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:36.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.594+0000 7f33f3c00640 1 --2- 172.21.15.16:0/1741462792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec105d80 0x7f33ec106190 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:36.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.594+0000 7f33f3c00640 1 -- 172.21.15.16:0/1741462792 >> 172.21.15.16:0/1741462792 conn(0x7f33ec101810 msgr2=0x7f33ec103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:36.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.594+0000 7f33f3c00640 1 -- 172.21.15.16:0/1741462792 shutdown_connections 2024-09-21T03:58:36.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.594+0000 7f33f3c00640 1 -- 172.21.15.16:0/1741462792 wait complete. 2024-09-21T03:58:36.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.595+0000 7f33f3c00640 1 Processor -- start 2024-09-21T03:58:36.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.595+0000 7f33f3c00640 1 -- start start 2024-09-21T03:58:36.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.595+0000 7f33f3c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33ec105d80 0x7f33ec19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:36.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.595+0000 7f33f3c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec10b6c0 0x7f33ec19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.595+0000 7f33f3c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33ec10c010 0x7f33ec1a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.595+0000 7f33f3c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec115cd0 con 0x7f33ec10b6c0 2024-09-21T03:58:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.595+0000 7f33f3c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec115b50 con 0x7f33ec10c010 2024-09-21T03:58:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.595+0000 7f33f3c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec115e50 con 0x7f33ec105d80 2024-09-21T03:58:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.596+0000 7f33e3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec10b6c0 0x7f33ec19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.596+0000 7f33e3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec10b6c0 0x7f33ec19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42644/0 (socket says 172.21.15.16:42644) 2024-09-21T03:58:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.596+0000 7f33e3e00640 1 -- 172.21.15.16:0/3335728542 learned_addr learned my addr 172.21.15.16:0/3335728542 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:58:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.596+0000 7f33f1200640 1 --2- 172.21.15.16:0/3335728542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33ec105d80 0x7f33ec19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.596+0000 7f33f1c00640 1 --2- 172.21.15.16:0/3335728542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33ec10c010 0x7f33ec1a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.596+0000 7f33f1200640 1 -- 172.21.15.16:0/3335728542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33ec10c010 msgr2=0x7f33ec1a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.596+0000 7f33f1200640 1 --2- 172.21.15.16:0/3335728542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33ec10c010 0x7f33ec1a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.596+0000 7f33f1200640 1 -- 172.21.15.16:0/3335728542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec10b6c0 msgr2=0x7f33ec19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.596+0000 7f33f1200640 1 --2- 172.21.15.16:0/3335728542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec10b6c0 0x7f33ec19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.596+0000 7f33f1200640 1 -- 172.21.15.16:0/3335728542 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ec1a2610 con 0x7f33ec105d80 2024-09-21T03:58:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.596+0000 7f33e3e00640 1 --2- 172.21.15.16:0/3335728542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec10b6c0 0x7f33ec19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:58:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.597+0000 7f33f1200640 1 --2- 172.21.15.16:0/3335728542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33ec105d80 0x7f33ec19c6a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f33dc02a910 tx=0x7f33dc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.597+0000 7f33ea000640 1 -- 172.21.15.16:0/3335728542 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc03f030 con 0x7f33ec105d80 2024-09-21T03:58:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.597+0000 7f33ea000640 1 -- 172.21.15.16:0/3335728542 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33dc003aa0 con 0x7f33ec105d80 2024-09-21T03:58:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.597+0000 7f33f3c00640 1 -- 172.21.15.16:0/3335728542 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ec1a28a0 con 0x7f33ec105d80 2024-09-21T03:58:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.597+0000 7f33ea000640 1 -- 172.21.15.16:0/3335728542 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc035070 con 0x7f33ec105d80 2024-09-21T03:58:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.597+0000 7f33f3c00640 1 -- 172.21.15.16:0/3335728542 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ec1a9060 con 0x7f33ec105d80 2024-09-21T03:58:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.598+0000 7f33f3c00640 1 -- 172.21.15.16:0/3335728542 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33ec106190 con 0x7f33ec105d80 2024-09-21T03:58:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.601+0000 7f33ea000640 1 -- 172.21.15.16:0/3335728542 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f33dc042030 con 0x7f33ec105d80 2024-09-21T03:58:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.602+0000 7f33ea000640 1 --2- 172.21.15.16:0/3335728542 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f33d00b6510 0x7f33d00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.602+0000 7f33ea000640 1 -- 172.21.15.16:0/3335728542 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7f33dc107f40 con 0x7f33ec105d80 2024-09-21T03:58:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.603+0000 7f33e3e00640 1 --2- 172.21.15.16:0/3335728542 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f33d00b6510 0x7f33d00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:36.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.603+0000 7f33e3e00640 1 --2- 172.21.15.16:0/3335728542 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f33d00b6510 0x7f33d00b8a20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f33ec10b1e0 tx=0x7f33d400b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.604+0000 7f33ea000640 1 -- 172.21.15.16:0/3335728542 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f33dc0d27a0 con 0x7f33ec105d80 2024-09-21T03:58:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.802+0000 7f33f3c00640 1 -- 172.21.15.16:0/3335728542 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33ec02e640 con 0x7f33ec105d80 2024-09-21T03:58:36.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.804+0000 7f33ea000640 1 -- 172.21.15.16:0/3335728542 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24902 (secure 0 0 0) 0x7f33dc0d76e0 con 0x7f33ec105d80 2024-09-21T03:58:36.807 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:58:36.807 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"fsid":"2b324840-77ca-11ef-baf5-efdc52797490","created":"2024-09-21T03:33:53.220358+0000","modified":"2024-09-21T03:44:54.658912+0000","last_up_change":"2024-09-21T03:42:06.614247+0000","last_in_change":"2024-09-21T03:41:43.379404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:37:55.708915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:43:03.567522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:43:06.048083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:43:16.061356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6a7e4b9e-078f-4ae9-ab9e-37be995e4deb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6803","nonce":311974551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6805","nonce":311974551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6809","nonce":311974551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":311974551},{"type":"v1","addr":"172.21.15.16:6807","nonce":311974551}]},"public_addr":"172.21.15.16:6803/311974551","cluster_addr":"172.21.15.16:6805/311974551","heartbeat_back_addr":"172.21.15.16:6809/311974551","heartbeat_front_addr":"172.21.15.16:6807/311974551","state":["exists","up"]},{"osd":1,"uuid":"3a497a64-e1a8-43ef-8509-0a79515da0aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6801","nonce":2152486571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6803","nonce":2152486571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6807","nonce":2152486571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2152486571},{"type":"v1","addr":"172.21.15.123:6805","nonce":2152486571}]},"public_addr":"172.21.15.123:6801/2152486571","cluster_addr":"172.21.15.123:6803/2152486571","heartbeat_back_addr":"172.21.15.123:6807/2152486571","heartbeat_front_addr":"172.21.15.123:6805/2152486571","state":["exists","up"]},{"osd":2,"uuid":"1c39518e-6cce-4a0f-9ee8-55c60ad325f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6801","nonce":2812351279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6803","nonce":2812351279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6807","nonce":2812351279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2812351279},{"type":"v1","addr":"172.21.15.134:6805","nonce":2812351279}]},"public_addr":"172.21.15.134:6801/2812351279","cluster_addr":"172.21.15.134:6803/2812351279","heartbeat_back_addr":"172.21.15.134:6807/2812351279","heartbeat_front_addr":"172.21.15.134:6805/2812351279","state":["exists","up"]},{"osd":3,"uuid":"0c515f8c-c41c-4b72-bc07-5b0a1ffd1ecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6811","nonce":2366216026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6813","nonce":2366216026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6817","nonce":2366216026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2366216026},{"type":"v1","addr":"172.21.15.16:6815","nonce":2366216026}]},"public_addr":"172.21.15.16:6811/2366216026","cluster_addr":"172.21.15.16:6813/2366216026","heartbeat_back_addr":"172.21.15.16:6817/2366216026","heartbeat_front_addr":"172.21.15.16:6815/2366216026","state":["exists","up"]},{"osd":4,"uuid":"ad5e6d0e-31e8-4779-9e74-f215032777ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6809","nonce":2354279116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6811","nonce":2354279116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6815","nonce":2354279116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2354279116},{"type":"v1","addr":"172.21.15.123:6813","nonce":2354279116}]},"public_addr":"172.21.15.123:6809/2354279116","cluster_addr":"172.21.15.123:6811/2354279116","heartbeat_back_addr":"172.21.15.123:6815/2354279116","heartbeat_front_addr":"172.21.15.123:6813/2354279116","state":["exists","up"]},{"osd":5,"uuid":"ca09bf88-1772-4dc0-8711-cbf9a5b31977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6809","nonce":2678416182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6811","nonce":2678416182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6815","nonce":2678416182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2678416182},{"type":"v1","addr":"172.21.15.134:6813","nonce":2678416182}]},"public_addr":"172.21.15.134:6809/2678416182","cluster_addr":"172.21.15.134:6811/2678416182","heartbeat_back_addr":"172.21.15.134:6815/2678416182","heartbeat_front_addr":"172.21.15.134:6813/2678416182","state":["exists","up"]},{"osd":6,"uuid":"0d41ed4e-8699-4d88-9254-12d3b84941e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6819","nonce":3377807744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6821","nonce":3377807744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6825","nonce":3377807744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3377807744},{"type":"v1","addr":"172.21.15.16:6823","nonce":3377807744}]},"public_addr":"172.21.15.16:6819/3377807744","cluster_addr":"172.21.15.16:6821/3377807744","heartbeat_back_addr":"172.21.15.16:6825/3377807744","heartbeat_front_addr":"172.21.15.16:6823/3377807744","state":["exists","up"]},{"osd":7,"uuid":"1541a272-c524-4f12-9b94-763f69bb1c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6817","nonce":1463412739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6819","nonce":1463412739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6823","nonce":1463412739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1463412739},{"type":"v1","addr":"172.21.15.123:6821","nonce":1463412739}]},"public_addr":"172.21.15.123:6817/1463412739","cluster_addr":"172.21.15.123:6819/1463412739","heartbeat_back_addr":"172.21.15.123:6823/1463412739","heartbeat_front_addr":"172.21.15.123:6821/1463412739","state":["exists","up"]},{"osd":8,"uuid":"cc00014c-eb99-49ab-984c-0bc26ef85130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6817","nonce":2644021450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6819","nonce":2644021450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6823","nonce":2644021450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2644021450},{"type":"v1","addr":"172.21.15.134:6821","nonce":2644021450}]},"public_addr":"172.21.15.134:6817/2644021450","cluster_addr":"172.21.15.134:6819/2644021450","heartbeat_back_addr":"172.21.15.134:6823/2644021450","heartbeat_front_addr":"172.21.15.134:6821/2644021450","state":["exists","up"]},{"osd":9,"uuid":"3cbfff74-31ff-4ffd-844b-54ec8563ba32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6827","nonce":1653164371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6829","nonce":1653164371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6833","nonce":1653164371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1653164371},{"type":"v1","addr":"172.21.15.16:6831","nonce":1653164371}]},"public_addr":"172.21.15.16:6827/1653164371","cluster_addr":"172.21.15.16:6829/1653164371","heartbeat_back_addr":"172.21.15.16:6833/1653164371","heartbeat_front_addr":"172.21.15.16:6831/1653164371","state":["exists","up"]},{"osd":10,"uuid":"0a1caaad-70cd-4c2c-88d3-e9a5cc885655","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6825","nonce":3682234496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6827","nonce":3682234496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6831","nonce":3682234496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3682234496},{"type":"v1","addr":"172.21.15.123:6829","nonce":3682234496}]},"public_addr":"172.21.15.123:6825/3682234496","cluster_addr":"172.21.15.123:6827/3682234496","heartbeat_back_addr":"172.21.15.123:6831/3682234496","heartbeat_front_addr":"172.21.15.123:6829/3682234496","state":["exists","up"]},{"osd":11,"uuid":"3aff3258-5a38-41ff-aac3-4a082feffdae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6825","nonce":584758685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6827","nonce":584758685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6831","nonce":584758685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":584758685},{"type":"v1","addr":"172.21.15.134:6829","nonce":584758685}]},"public_addr":"172.21.15.134:6825/584758685","cluster_addr":"172.21.15.134:6827/584758685","heartbeat_back_addr":"172.21.15.134:6831/584758685","heartbeat_front_addr":"172.21.15.134:6829/584758685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:02.177731+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:28.847562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:37:54.272384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:24.338596+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:38:52.430371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:17.832737+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:39:50.397890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:16.842245+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:40:42.102933+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:13.074732+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:41:40.057905+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:42:04.898190+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/557742783":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/3238676548":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6801/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/2357336233":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6801/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:6800/428542610":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/3806844703":"2024-09-22T03:34:22.549156+0000","172.21.15.16:0/2351581621":"2024-09-22T03:34:53.217208+0000","172.21.15.16:6800/500178737":"2024-09-22T03:34:53.217208+0000","172.21.15.16:0/355731724":"2024-09-22T03:34:22.549156+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:58:36.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.807+0000 7f33f3c00640 1 -- 172.21.15.16:0/3335728542 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f33d00b6510 msgr2=0x7f33d00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:36.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.807+0000 7f33f3c00640 1 --2- 172.21.15.16:0/3335728542 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f33d00b6510 0x7f33d00b8a20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f33ec10b1e0 tx=0x7f33d400b410 comp rx=0 tx=0).stop 2024-09-21T03:58:36.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.807+0000 7f33f3c00640 1 -- 172.21.15.16:0/3335728542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33ec105d80 msgr2=0x7f33ec19c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:36.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.807+0000 7f33f3c00640 1 --2- 172.21.15.16:0/3335728542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33ec105d80 0x7f33ec19c6a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f33dc02a910 tx=0x7f33dc004270 comp rx=0 tx=0).stop 2024-09-21T03:58:36.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.807+0000 7f33f3c00640 1 -- 172.21.15.16:0/3335728542 shutdown_connections 2024-09-21T03:58:36.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.807+0000 7f33f3c00640 1 --2- 172.21.15.16:0/3335728542 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f33d00b6510 0x7f33d00b8a20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:36.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.807+0000 7f33f3c00640 1 --2- 172.21.15.16:0/3335728542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33ec10c010 0x7f33ec1a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:36.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.807+0000 7f33f3c00640 1 --2- 172.21.15.16:0/3335728542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec10b6c0 0x7f33ec19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:36.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.807+0000 7f33f3c00640 1 --2- 172.21.15.16:0/3335728542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33ec105d80 0x7f33ec19c6a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:36.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.807+0000 7f33f3c00640 1 -- 172.21.15.16:0/3335728542 >> 172.21.15.16:0/3335728542 conn(0x7f33ec101810 msgr2=0x7f33ec103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:36.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.807+0000 7f33f3c00640 1 -- 172.21.15.16:0/3335728542 shutdown_connections 2024-09-21T03:58:36.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:36.807+0000 7f33f3c00640 1 -- 172.21.15.16:0/3335728542 wait complete. 2024-09-21T03:58:37.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:37 smithi016 ceph-mon[30375]: pgmap v787: 129 pgs: 129 active+clean; 186 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 125 KiB/s wr, 225 op/s 2024-09-21T03:58:37.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:37 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3335728542' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:58:37.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:37 smithi134 ceph-mon[35031]: pgmap v787: 129 pgs: 129 active+clean; 186 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 125 KiB/s wr, 225 op/s 2024-09-21T03:58:37.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:37 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3335728542' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:58:37.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:37 smithi123 ceph-mon[34673]: pgmap v787: 129 pgs: 129 active+clean; 186 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 125 KiB/s wr, 225 op/s 2024-09-21T03:58:37.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:37 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3335728542' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:58:37.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T03:58:38.081 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:58:38.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.716+0000 7fb685400640 1 -- 172.21.15.16:0/1143992170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb680103b60 msgr2=0x7fb680103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:38.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.716+0000 7fb685400640 1 --2- 172.21.15.16:0/1143992170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb680103b60 0x7fb680103f70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb670009900 tx=0x7fb67002a410 comp rx=0 tx=0).stop 2024-09-21T03:58:38.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.716+0000 7fb685400640 1 -- 172.21.15.16:0/1143992170 shutdown_connections 2024-09-21T03:58:38.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.716+0000 7fb685400640 1 --2- 172.21.15.16:0/1143992170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb680109df0 0x7fb680110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:38.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.716+0000 7fb685400640 1 --2- 172.21.15.16:0/1143992170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6801094a0 0x7fb6801098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:38.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.716+0000 7fb685400640 1 --2- 172.21.15.16:0/1143992170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb680103b60 0x7fb680103f70 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:38.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.716+0000 7fb685400640 1 -- 172.21.15.16:0/1143992170 >> 172.21.15.16:0/1143992170 conn(0x7fb68007b850 msgr2=0x7fb68007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:38.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.716+0000 7fb685400640 1 -- 172.21.15.16:0/1143992170 shutdown_connections 2024-09-21T03:58:38.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.716+0000 7fb685400640 1 -- 172.21.15.16:0/1143992170 wait complete. 2024-09-21T03:58:38.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.717+0000 7fb685400640 1 Processor -- start 2024-09-21T03:58:38.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.717+0000 7fb685400640 1 -- start start 2024-09-21T03:58:38.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.718+0000 7fb685400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb680103b60 0x7fb68019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:38.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.718+0000 7fb685400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6801094a0 0x7fb68019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:38.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.718+0000 7fb685400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb680109df0 0x7fb6801a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:38.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.718+0000 7fb685400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb680113ab0 con 0x7fb680109df0 2024-09-21T03:58:38.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.718+0000 7fb685400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb680113930 con 0x7fb6801094a0 2024-09-21T03:58:38.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.718+0000 7fb685400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb680113c30 con 0x7fb680103b60 2024-09-21T03:58:38.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.718+0000 7fb67e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb680103b60 0x7fb68019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:38.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.718+0000 7fb67f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb680109df0 0x7fb6801a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:38.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.718+0000 7fb67e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb680103b60 0x7fb68019c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:59926/0 (socket says 172.21.15.16:59926) 2024-09-21T03:58:38.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.718+0000 7fb67e800640 1 -- 172.21.15.16:0/1385968774 learned_addr learned my addr 172.21.15.16:0/1385968774 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:58:38.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.718+0000 7fb675e00640 1 --2- 172.21.15.16:0/1385968774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6801094a0 0x7fb68019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:38.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.719+0000 7fb67e800640 1 -- 172.21.15.16:0/1385968774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6801094a0 msgr2=0x7fb68019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:38.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.719+0000 7fb67e800640 1 --2- 172.21.15.16:0/1385968774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6801094a0 0x7fb68019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:38.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.719+0000 7fb67e800640 1 -- 172.21.15.16:0/1385968774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb680109df0 msgr2=0x7fb6801a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:38.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.719+0000 7fb67e800640 1 --2- 172.21.15.16:0/1385968774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb680109df0 0x7fb6801a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:38.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.719+0000 7fb67e800640 1 -- 172.21.15.16:0/1385968774 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6801a2630 con 0x7fb680103b60 2024-09-21T03:58:38.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.719+0000 7fb675e00640 1 --2- 172.21.15.16:0/1385968774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6801094a0 0x7fb68019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:58:38.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.719+0000 7fb67e800640 1 --2- 172.21.15.16:0/1385968774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb680103b60 0x7fb68019c6c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb670005940 tx=0x7fb6700040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:38.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.719+0000 7fb677e00640 1 -- 172.21.15.16:0/1385968774 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67003f030 con 0x7fb680103b60 2024-09-21T03:58:38.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.719+0000 7fb685400640 1 -- 172.21.15.16:0/1385968774 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6801a28c0 con 0x7fb680103b60 2024-09-21T03:58:38.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.719+0000 7fb677e00640 1 -- 172.21.15.16:0/1385968774 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6700044c0 con 0x7fb680103b60 2024-09-21T03:58:38.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.719+0000 7fb677e00640 1 -- 172.21.15.16:0/1385968774 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670035070 con 0x7fb680103b60 2024-09-21T03:58:38.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.719+0000 7fb685400640 1 -- 172.21.15.16:0/1385968774 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6801a9080 con 0x7fb680103b60 2024-09-21T03:58:38.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.720+0000 7fb685400640 1 -- 172.21.15.16:0/1385968774 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb680103f70 con 0x7fb680103b60 2024-09-21T03:58:38.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.723+0000 7fb677e00640 1 -- 172.21.15.16:0/1385968774 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fb670042030 con 0x7fb680103b60 2024-09-21T03:58:38.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.725+0000 7fb677e00640 1 --2- 172.21.15.16:0/1385968774 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6640b6590 0x7fb6640b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:38.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.725+0000 7fb677e00640 1 -- 172.21.15.16:0/1385968774 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9179+0+0 (secure 0 0 0) 0x7fb670107c10 con 0x7fb680103b60 2024-09-21T03:58:38.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.725+0000 7fb675e00640 1 --2- 172.21.15.16:0/1385968774 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6640b6590 0x7fb6640b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:38.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.726+0000 7fb675e00640 1 --2- 172.21.15.16:0/1385968774 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6640b6590 0x7fb6640b8aa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb680108f50 tx=0x7fb668005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:38.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.727+0000 7fb677e00640 1 -- 172.21.15.16:0/1385968774 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb6700d2470 con 0x7fb680103b60 2024-09-21T03:58:38.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.968+0000 7fb685400640 1 -- 172.21.15.16:0/1385968774 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb6801a92b0 con 0x7fb680103b60 2024-09-21T03:58:38.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.977+0000 7fb677e00640 1 -- 172.21.15.16:0/1385968774 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) ==== 161+0+0 (secure 0 0 0) 0x7fb6700d73b0 con 0x7fb680103b60 2024-09-21T03:58:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:58:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.981+0000 7fb685400640 1 -- 172.21.15.16:0/1385968774 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6640b6590 msgr2=0x7fb6640b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.981+0000 7fb685400640 1 --2- 172.21.15.16:0/1385968774 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6640b6590 0x7fb6640b8aa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb680108f50 tx=0x7fb668005f90 comp rx=0 tx=0).stop 2024-09-21T03:58:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.981+0000 7fb685400640 1 -- 172.21.15.16:0/1385968774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb680103b60 msgr2=0x7fb68019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.981+0000 7fb685400640 1 --2- 172.21.15.16:0/1385968774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb680103b60 0x7fb68019c6c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb670005940 tx=0x7fb6700040c0 comp rx=0 tx=0).stop 2024-09-21T03:58:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.981+0000 7fb685400640 1 -- 172.21.15.16:0/1385968774 shutdown_connections 2024-09-21T03:58:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.981+0000 7fb685400640 1 --2- 172.21.15.16:0/1385968774 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fb6640b6590 0x7fb6640b8aa0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.981+0000 7fb685400640 1 --2- 172.21.15.16:0/1385968774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb680109df0 0x7fb6801a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.981+0000 7fb685400640 1 --2- 172.21.15.16:0/1385968774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6801094a0 0x7fb68019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.981+0000 7fb685400640 1 --2- 172.21.15.16:0/1385968774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb680103b60 0x7fb68019c6c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:38.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.981+0000 7fb685400640 1 -- 172.21.15.16:0/1385968774 >> 172.21.15.16:0/1385968774 conn(0x7fb68007b850 msgr2=0x7fb680107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:38.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.981+0000 7fb685400640 1 -- 172.21.15.16:0/1385968774 shutdown_connections 2024-09-21T03:58:38.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:38.981+0000 7fb685400640 1 -- 172.21.15.16:0/1385968774 wait complete. 2024-09-21T03:58:39.242 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:38 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-a[30371]: 2024-09-21T03:58:38.969+0000 7fb90a800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:58:39.320 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-e[61920]: ignoring --setuser ceph since I am not root 2024-09-21T03:58:39.320 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-e[61920]: ignoring --setgroup ceph since I am not root 2024-09-21T03:58:39.320 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-mds[61924]: -- 172.21.15.123:0/1902465620 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557be4671860 con 0x557be8819a80 2024-09-21T03:58:39.320 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-e[61920]: starting mds.e at 2024-09-21T03:58:39.321 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-i[63682]: ignoring --setuser ceph since I am not root 2024-09-21T03:58:39.321 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-i[63682]: ignoring --setgroup ceph since I am not root 2024-09-21T03:58:39.321 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-mds[63686]: -- 172.21.15.134:0/551713609 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560dd5175860 con 0x560dd931d180 2024-09-21T03:58:39.321 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-i[63682]: starting mds.i at 2024-09-21T03:58:39.321 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-h[62190]: ignoring --setuser ceph since I am not root 2024-09-21T03:58:39.321 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-h[62190]: ignoring --setgroup ceph since I am not root 2024-09-21T03:58:39.321 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-mds[62194]: -- 172.21.15.123:0/4104531975 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a781bfdc20 con 0x55a785da5600 2024-09-21T03:58:39.321 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-h[62190]: starting mds.h at 2024-09-21T03:58:39.322 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-c[63402]: ignoring --setuser ceph since I am not root 2024-09-21T03:58:39.322 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-c[63402]: ignoring --setgroup ceph since I am not root 2024-09-21T03:58:39.322 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-mds[63406]: -- 172.21.15.134:0/4013326849 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55cadaf04ea0 con 0x55cadf0a3a80 2024-09-21T03:58:39.322 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-mds[63406]: -- 172.21.15.134:0/4013326849 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cadaefb860 con 0x55cadf0a3a80 2024-09-21T03:58:39.322 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-c[63402]: starting mds.c at 2024-09-21T03:58:39.322 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-k[61641]: ignoring --setuser ceph since I am not root 2024-09-21T03:58:39.322 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-k[61641]: ignoring --setgroup ceph since I am not root 2024-09-21T03:58:39.322 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-mds[61645]: -- 172.21.15.123:0/3358838706 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a74d53860 con 0x559a78efba80 2024-09-21T03:58:39.322 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-k[61641]: starting mds.k at 2024-09-21T03:58:39.323 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-f[63132]: ignoring --setuser ceph since I am not root 2024-09-21T03:58:39.323 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-f[63132]: ignoring --setgroup ceph since I am not root 2024-09-21T03:58:39.323 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-mds[63136]: -- 172.21.15.134:0/3610585927 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b6c8cbc20 con 0x564b70a73a80 2024-09-21T03:58:39.323 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-f[63132]: starting mds.f at 2024-09-21T03:58:39.323 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-b[62470]: ignoring --setuser ceph since I am not root 2024-09-21T03:58:39.323 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-b[62470]: ignoring --setgroup ceph since I am not root 2024-09-21T03:58:39.323 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-mds[62474]: -- 172.21.15.123:0/944120851 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5605af099860 con 0x5605b3241a80 2024-09-21T03:58:39.323 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-b[62470]: starting mds.b at 2024-09-21T03:58:39.323 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-l[62864]: ignoring --setuser ceph since I am not root 2024-09-21T03:58:39.323 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-l[62864]: ignoring --setgroup ceph since I am not root 2024-09-21T03:58:39.324 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-mds[62868]: -- 172.21.15.134:0/3167334911 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55927aba5c20 con 0x55927ed4d180 2024-09-21T03:58:39.324 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-l[62864]: starting mds.l at 2024-09-21T03:58:39.602 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-mon[35031]: pgmap v788: 129 pgs: 129 active+clean; 1.8 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 50 KiB/s wr, 211 op/s 2024-09-21T03:58:39.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/1385968774' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:58:39.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:58:39.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-mon[35031]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:58:39.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-mon[35031]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:58:39.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:58:39.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-mon[35031]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:58:39.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:39 smithi134 ceph-mon[35031]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T03:58:39.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-mon[34673]: pgmap v788: 129 pgs: 129 active+clean; 1.8 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 50 KiB/s wr, 211 op/s 2024-09-21T03:58:39.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/1385968774' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:58:39.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:58:39.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-mon[34673]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:58:39.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-mon[34673]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:58:39.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:58:39.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-mon[34673]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:58:39.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:39 smithi123 ceph-mon[34673]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T03:58:39.662 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:58:39 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-a[71991]: ignoring --setuser ceph since I am not root 2024-09-21T03:58:39.663 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:58:39 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-a[71991]: ignoring --setgroup ceph since I am not root 2024-09-21T03:58:39.663 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:58:39 smithi016 ceph-mds[71998]: -- 172.21.15.16:0/2748539160 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559654fe7860 con 0x55965918f600 2024-09-21T03:58:39.663 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:58:39 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-a[71991]: starting mds.a at 2024-09-21T03:58:39.663 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:58:39 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-g[71616]: ignoring --setuser ceph since I am not root 2024-09-21T03:58:39.663 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:58:39 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-g[71616]: ignoring --setgroup ceph since I am not root 2024-09-21T03:58:39.663 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:58:39 smithi016 ceph-mds[71621]: -- 172.21.15.16:0/427478852 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5646c269b860 con 0x5646c6843600 2024-09-21T03:58:39.663 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:58:39 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-g[71616]: starting mds.g at 2024-09-21T03:58:39.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:39 smithi016 ceph-mon[30375]: pgmap v788: 129 pgs: 129 active+clean; 1.8 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 50 KiB/s wr, 211 op/s 2024-09-21T03:58:39.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:39 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/1385968774' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:58:39.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:39 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:58:39.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:39 smithi016 ceph-mon[30375]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:58:39.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:39 smithi016 ceph-mon[30375]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:58:39.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:39 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:58:39.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:39 smithi016 ceph-mon[30375]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:58:39.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:39 smithi016 ceph-mon[30375]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T03:58:39.765 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T03:58:40.155 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:58:40.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.800+0000 7f8fbce00640 1 -- 172.21.15.16:0/528442582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb81092d0 msgr2=0x7f8fb81096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:40.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.800+0000 7f8fbce00640 1 --2- 172.21.15.16:0/528442582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb81092d0 0x7f8fb81096e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009860 tx=0x7f8fa002a340 comp rx=0 tx=0).stop 2024-09-21T03:58:40.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.800+0000 7f8fbce00640 1 -- 172.21.15.16:0/528442582 shutdown_connections 2024-09-21T03:58:40.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.800+0000 7f8fbce00640 1 --2- 172.21.15.16:0/528442582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb8109c20 0x7f8fb8110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:40.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.800+0000 7f8fbce00640 1 --2- 172.21.15.16:0/528442582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb81092d0 0x7f8fb81096e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:40.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.800+0000 7f8fbce00640 1 --2- 172.21.15.16:0/528442582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fb8103990 0x7f8fb8103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:40.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.800+0000 7f8fbce00640 1 -- 172.21.15.16:0/528442582 >> 172.21.15.16:0/528442582 conn(0x7f8fb80ff440 msgr2=0x7f8fb8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:40.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.800+0000 7f8fbce00640 1 -- 172.21.15.16:0/528442582 shutdown_connections 2024-09-21T03:58:40.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.800+0000 7f8fbce00640 1 -- 172.21.15.16:0/528442582 wait complete. 2024-09-21T03:58:40.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.801+0000 7f8fbce00640 1 Processor -- start 2024-09-21T03:58:40.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.801+0000 7f8fbce00640 1 -- start start 2024-09-21T03:58:40.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.802+0000 7f8fbce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb8103990 0x7f8fb819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:40.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.802+0000 7f8fbce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb81092d0 0x7f8fb819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:40.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.802+0000 7f8fbce00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fb8109c20 0x7f8fb819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:40.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.802+0000 7f8fbce00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb81138e0 con 0x7f8fb81092d0 2024-09-21T03:58:40.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.802+0000 7f8fbce00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb8113760 con 0x7f8fb8103990 2024-09-21T03:58:40.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.802+0000 7f8fbce00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb8113a60 con 0x7f8fb8109c20 2024-09-21T03:58:40.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.802+0000 7f8fb7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb8103990 0x7f8fb819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:40.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.802+0000 7f8faea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb81092d0 0x7f8fb819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.802+0000 7f8fb7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb8103990 0x7f8fb819a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:54584/0 (socket says 172.21.15.16:54584) 2024-09-21T03:58:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.802+0000 7f8fb7400640 1 -- 172.21.15.16:0/4080333473 learned_addr learned my addr 172.21.15.16:0/4080333473 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:58:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.802+0000 7f8fb7e00640 1 --2- 172.21.15.16:0/4080333473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fb8109c20 0x7f8fb819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.803+0000 7f8fb7400640 1 -- 172.21.15.16:0/4080333473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fb8109c20 msgr2=0x7f8fb819fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.803+0000 7f8fb7400640 1 --2- 172.21.15.16:0/4080333473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fb8109c20 0x7f8fb819fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.803+0000 7f8fb7400640 1 -- 172.21.15.16:0/4080333473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb81092d0 msgr2=0x7f8fb819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.803+0000 7f8fb7400640 1 --2- 172.21.15.16:0/4080333473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb81092d0 0x7f8fb819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.803+0000 7f8fb7400640 1 -- 172.21.15.16:0/4080333473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb81a0260 con 0x7f8fb8103990 2024-09-21T03:58:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.803+0000 7f8fb7400640 1 --2- 172.21.15.16:0/4080333473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb8103990 0x7f8fb819a2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800e9f0 tx=0x7f8fa800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.803+0000 7f8fb4c00640 1 -- 172.21.15.16:0/4080333473 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa800cd80 con 0x7f8fb8103990 2024-09-21T03:58:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.803+0000 7f8fbce00640 1 -- 172.21.15.16:0/4080333473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb81a0550 con 0x7f8fb8103990 2024-09-21T03:58:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.803+0000 7f8fb4c00640 1 -- 172.21.15.16:0/4080333473 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fa80054d0 con 0x7f8fb8103990 2024-09-21T03:58:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.803+0000 7f8fb4c00640 1 -- 172.21.15.16:0/4080333473 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8010690 con 0x7f8fb8103990 2024-09-21T03:58:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.803+0000 7f8fbce00640 1 -- 172.21.15.16:0/4080333473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb81a6cb0 con 0x7f8fb8103990 2024-09-21T03:58:40.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.804+0000 7f8fbce00640 1 -- 172.21.15.16:0/4080333473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb8103da0 con 0x7f8fb8103990 2024-09-21T03:58:40.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.807+0000 7f8fb4c00640 1 -- 172.21.15.16:0/4080333473 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f8fa8004e90 con 0x7f8fb8103990 2024-09-21T03:58:40.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.808+0000 7f8fb4c00640 1 --2- 172.21.15.16:0/4080333473 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8f9c0b6590 0x7f8f9c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:40.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.809+0000 7f8fb4c00640 1 -- 172.21.15.16:0/4080333473 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9609+0+0 (secure 0 0 0) 0x7f8fa80e7f60 con 0x7f8fb8103990 2024-09-21T03:58:40.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.809+0000 7f8faea00640 1 --2- 172.21.15.16:0/4080333473 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8f9c0b6590 0x7f8f9c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:40.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.809+0000 7f8faea00640 1 --2- 172.21.15.16:0/4080333473 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8f9c0b6590 0x7f8f9c0b8aa0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0035040 tx=0x7f8fa0030340 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:40.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:40.810+0000 7f8fb4c00640 1 -- 172.21.15.16:0/4080333473 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8fa80ae230 con 0x7f8fb8103990 2024-09-21T03:58:41.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:41.054+0000 7f8fbce00640 1 -- 172.21.15.16:0/4080333473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8fb802e640 con 0x7f8fb8103990 2024-09-21T03:58:41.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: pgmap v790: 129 pgs: 129 active+clean; 1.8 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 818 B/s wr, 161 op/s 2024-09-21T03:58:41.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.123:6832/2822220123,v1:172.21.15.123:6834/2822220123] up:boot 2024-09-21T03:58:41.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.123:6833/547287795,v1:172.21.15.123:6835/547287795] up:boot 2024-09-21T03:58:41.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.134:6833/2898260870,v1:172.21.15.134:6835/2898260870] up:boot 2024-09-21T03:58:41.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.134:6832/3786482281,v1:172.21.15.134:6834/3786482281] up:boot 2024-09-21T03:58:41.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.123:6836/2352807219,v1:172.21.15.123:6837/2352807219] up:boot 2024-09-21T03:58:41.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.134:6836/1571477773,v1:172.21.15.134:6837/1571477773] up:boot 2024-09-21T03:58:41.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.123:6838/950103228,v1:172.21.15.123:6839/950103228] up:boot 2024-09-21T03:58:41.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.134:6838/2538985929,v1:172.21.15.134:6839/2538985929] up:boot 2024-09-21T03:58:41.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.16:6838/2450213787,v1:172.21.15.16:6840/2450213787] up:boot 2024-09-21T03:58:41.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: mds.? [v2:172.21.15.16:6839/4249851835,v1:172.21.15.16:6841/4249851835] up:boot 2024-09-21T03:58:41.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T03:58:41.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:58:41.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:58:41.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:58:41.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:58:41.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:58:41.355 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:58:41.355 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:58:41.355 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:58:41.355 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:58:41.355 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:40 smithi134 ceph-mon[35031]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:58:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: pgmap v790: 129 pgs: 129 active+clean; 1.8 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 818 B/s wr, 161 op/s 2024-09-21T03:58:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.123:6832/2822220123,v1:172.21.15.123:6834/2822220123] up:boot 2024-09-21T03:58:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.123:6833/547287795,v1:172.21.15.123:6835/547287795] up:boot 2024-09-21T03:58:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.134:6833/2898260870,v1:172.21.15.134:6835/2898260870] up:boot 2024-09-21T03:58:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.134:6832/3786482281,v1:172.21.15.134:6834/3786482281] up:boot 2024-09-21T03:58:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.123:6836/2352807219,v1:172.21.15.123:6837/2352807219] up:boot 2024-09-21T03:58:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.134:6836/1571477773,v1:172.21.15.134:6837/1571477773] up:boot 2024-09-21T03:58:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.123:6838/950103228,v1:172.21.15.123:6839/950103228] up:boot 2024-09-21T03:58:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.134:6838/2538985929,v1:172.21.15.134:6839/2538985929] up:boot 2024-09-21T03:58:41.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.16:6838/2450213787,v1:172.21.15.16:6840/2450213787] up:boot 2024-09-21T03:58:41.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: mds.? [v2:172.21.15.16:6839/4249851835,v1:172.21.15.16:6841/4249851835] up:boot 2024-09-21T03:58:41.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T03:58:41.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:58:41.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:58:41.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:58:41.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:58:41.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:58:41.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:58:41.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:58:41.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:58:41.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:58:41.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:40 smithi123 ceph-mon[34673]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:58:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: pgmap v790: 129 pgs: 129 active+clean; 1.8 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 818 B/s wr, 161 op/s 2024-09-21T03:58:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.123:6832/2822220123,v1:172.21.15.123:6834/2822220123] up:boot 2024-09-21T03:58:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.123:6833/547287795,v1:172.21.15.123:6835/547287795] up:boot 2024-09-21T03:58:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.134:6833/2898260870,v1:172.21.15.134:6835/2898260870] up:boot 2024-09-21T03:58:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.134:6832/3786482281,v1:172.21.15.134:6834/3786482281] up:boot 2024-09-21T03:58:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.123:6836/2352807219,v1:172.21.15.123:6837/2352807219] up:boot 2024-09-21T03:58:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.134:6836/1571477773,v1:172.21.15.134:6837/1571477773] up:boot 2024-09-21T03:58:41.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.123:6838/950103228,v1:172.21.15.123:6839/950103228] up:boot 2024-09-21T03:58:41.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.134:6838/2538985929,v1:172.21.15.134:6839/2538985929] up:boot 2024-09-21T03:58:41.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.16:6838/2450213787,v1:172.21.15.16:6840/2450213787] up:boot 2024-09-21T03:58:41.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: mds.? [v2:172.21.15.16:6839/4249851835,v1:172.21.15.16:6841/4249851835] up:boot 2024-09-21T03:58:41.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T03:58:41.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:58:41.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:58:41.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:58:41.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:58:41.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:58:41.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:58:41.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:58:41.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:58:41.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:58:41.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:40 smithi016 ceph-mon[30375]: from='mgr.14154 172.21.15.16:0/1221384287' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:58:41.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:41.982+0000 7f8fb4c00640 1 -- 172.21.15.16:0/4080333473 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) ==== 104+0+0 (secure 0 0 0) 0x7f8fa80b2610 con 0x7f8fb8103990 2024-09-21T03:58:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:41.985+0000 7f8fbce00640 1 -- 172.21.15.16:0/4080333473 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8f9c0b6590 msgr2=0x7f8f9c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:41.985+0000 7f8fbce00640 1 --2- 172.21.15.16:0/4080333473 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8f9c0b6590 0x7f8f9c0b8aa0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0035040 tx=0x7f8fa0030340 comp rx=0 tx=0).stop 2024-09-21T03:58:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:41.985+0000 7f8fbce00640 1 -- 172.21.15.16:0/4080333473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb8103990 msgr2=0x7f8fb819a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:41.985+0000 7f8fbce00640 1 --2- 172.21.15.16:0/4080333473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb8103990 0x7f8fb819a2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800e9f0 tx=0x7f8fa800edf0 comp rx=0 tx=0).stop 2024-09-21T03:58:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:41.985+0000 7f8fbce00640 1 -- 172.21.15.16:0/4080333473 shutdown_connections 2024-09-21T03:58:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:41.985+0000 7f8fbce00640 1 --2- 172.21.15.16:0/4080333473 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f8f9c0b6590 0x7f8f9c0b8aa0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:41.985+0000 7f8fbce00640 1 --2- 172.21.15.16:0/4080333473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fb8109c20 0x7f8fb819fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:41.985+0000 7f8fbce00640 1 --2- 172.21.15.16:0/4080333473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb81092d0 0x7f8fb819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:41.985+0000 7f8fbce00640 1 --2- 172.21.15.16:0/4080333473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb8103990 0x7f8fb819a2f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:41.985+0000 7f8fbce00640 1 -- 172.21.15.16:0/4080333473 >> 172.21.15.16:0/4080333473 conn(0x7f8fb80ff440 msgr2=0x7f8fb8101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:41.986+0000 7f8fbce00640 1 -- 172.21.15.16:0/4080333473 shutdown_connections 2024-09-21T03:58:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:41.986+0000 7f8fbce00640 1 -- 172.21.15.16:0/4080333473 wait complete. 2024-09-21T03:58:42.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:41 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/4080333473' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:58:42.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:41 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:58:42.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:41 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/4080333473' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:58:42.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:41 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:58:42.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:41 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/4080333473' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:58:42.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:41 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:58:43.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T03:58:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:42 smithi134 ceph-mon[35031]: pgmap v791: 129 pgs: 129 active+clean; 1.8 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 114 op/s 2024-09-21T03:58:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:42 smithi134 ceph-mon[35031]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:58:43.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:42 smithi134 ceph-mon[35031]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:58:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:42 smithi134 ceph-mon[35031]: Cluster is now healthy 2024-09-21T03:58:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:42 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:58:43.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:42 smithi134 ceph-mon[35031]: fsmap 12 up:standby 2024-09-21T03:58:43.399 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:58:43.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:42 smithi123 ceph-mon[34673]: pgmap v791: 129 pgs: 129 active+clean; 1.8 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 114 op/s 2024-09-21T03:58:43.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:42 smithi123 ceph-mon[34673]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:58:43.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:42 smithi123 ceph-mon[34673]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:58:43.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:42 smithi123 ceph-mon[34673]: Cluster is now healthy 2024-09-21T03:58:43.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:42 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:58:43.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:42 smithi123 ceph-mon[34673]: fsmap 12 up:standby 2024-09-21T03:58:43.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:42 smithi016 ceph-mon[30375]: pgmap v791: 129 pgs: 129 active+clean; 1.8 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 114 op/s 2024-09-21T03:58:43.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:42 smithi016 ceph-mon[30375]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:58:43.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:42 smithi016 ceph-mon[30375]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:58:43.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:42 smithi016 ceph-mon[30375]: Cluster is now healthy 2024-09-21T03:58:43.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:42 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:58:43.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:42 smithi016 ceph-mon[30375]: fsmap 12 up:standby 2024-09-21T03:58:43.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.965+0000 7f7de4400640 1 -- 172.21.15.16:0/4257201343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ddc105d80 msgr2=0x7f7ddc106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:43.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.965+0000 7f7de4400640 1 --2- 172.21.15.16:0/4257201343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ddc105d80 0x7f7ddc106190 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009900 tx=0x7f7dd002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:58:43.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.965+0000 7f7de4400640 1 -- 172.21.15.16:0/4257201343 shutdown_connections 2024-09-21T03:58:43.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.965+0000 7f7de4400640 1 --2- 172.21.15.16:0/4257201343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ddc10c010 0x7f7ddc112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:43.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.965+0000 7f7de4400640 1 --2- 172.21.15.16:0/4257201343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ddc10b6c0 0x7f7ddc10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:43.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.965+0000 7f7de4400640 1 --2- 172.21.15.16:0/4257201343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ddc105d80 0x7f7ddc106190 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:43.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.965+0000 7f7de4400640 1 -- 172.21.15.16:0/4257201343 >> 172.21.15.16:0/4257201343 conn(0x7f7ddc101810 msgr2=0x7f7ddc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:43.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.965+0000 7f7de4400640 1 -- 172.21.15.16:0/4257201343 shutdown_connections 2024-09-21T03:58:43.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.965+0000 7f7de4400640 1 -- 172.21.15.16:0/4257201343 wait complete. 2024-09-21T03:58:43.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.966+0000 7f7de4400640 1 Processor -- start 2024-09-21T03:58:43.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.966+0000 7f7de4400640 1 -- start start 2024-09-21T03:58:43.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.966+0000 7f7de4400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ddc105d80 0x7f7ddc19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:43.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.967+0000 7f7de4400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ddc10b6c0 0x7f7ddc19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:43.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.967+0000 7f7de4400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ddc10c010 0x7f7ddc1a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:43.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.967+0000 7f7de4400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc115cd0 con 0x7f7ddc10b6c0 2024-09-21T03:58:43.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.967+0000 7f7de4400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc115b50 con 0x7f7ddc10c010 2024-09-21T03:58:43.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.967+0000 7f7de4400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc115e50 con 0x7f7ddc105d80 2024-09-21T03:58:43.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.967+0000 7f7de1a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ddc105d80 0x7f7ddc19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:43.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.967+0000 7f7de1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ddc10b6c0 0x7f7ddc19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:43.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.967+0000 7f7de1a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ddc105d80 0x7f7ddc19c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.16:32872/0 (socket says 172.21.15.16:32872) 2024-09-21T03:58:43.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.967+0000 7f7de1a00640 1 -- 172.21.15.16:0/2490709939 learned_addr learned my addr 172.21.15.16:0/2490709939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:58:43.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.967+0000 7f7de2400640 1 --2- 172.21.15.16:0/2490709939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ddc10c010 0x7f7ddc1a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:43.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.967+0000 7f7de1a00640 1 -- 172.21.15.16:0/2490709939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ddc10c010 msgr2=0x7f7ddc1a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:43.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.967+0000 7f7de1a00640 1 --2- 172.21.15.16:0/2490709939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ddc10c010 0x7f7ddc1a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:43.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.967+0000 7f7de1a00640 1 -- 172.21.15.16:0/2490709939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ddc10b6c0 msgr2=0x7f7ddc19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:43.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.967+0000 7f7de1a00640 1 --2- 172.21.15.16:0/2490709939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ddc10b6c0 0x7f7ddc19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:43.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.967+0000 7f7de1a00640 1 -- 172.21.15.16:0/2490709939 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ddc1a2650 con 0x7f7ddc105d80 2024-09-21T03:58:43.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.968+0000 7f7de1a00640 1 --2- 172.21.15.16:0/2490709939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ddc105d80 0x7f7ddc19c6e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0005940 tx=0x7f7dd00042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:43.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.968+0000 7f7dda000640 1 -- 172.21.15.16:0/2490709939 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd003f030 con 0x7f7ddc105d80 2024-09-21T03:58:43.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.968+0000 7f7de4400640 1 -- 172.21.15.16:0/2490709939 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ddc1a28e0 con 0x7f7ddc105d80 2024-09-21T03:58:43.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.968+0000 7f7de4400640 1 -- 172.21.15.16:0/2490709939 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ddc1a90a0 con 0x7f7ddc105d80 2024-09-21T03:58:43.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.968+0000 7f7dda000640 1 -- 172.21.15.16:0/2490709939 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dd0003cb0 con 0x7f7ddc105d80 2024-09-21T03:58:43.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.968+0000 7f7dda000640 1 -- 172.21.15.16:0/2490709939 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0049060 con 0x7f7ddc105d80 2024-09-21T03:58:43.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.969+0000 7f7de4400640 1 -- 172.21.15.16:0/2490709939 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ddc106190 con 0x7f7ddc105d80 2024-09-21T03:58:43.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.972+0000 7f7dda000640 1 -- 172.21.15.16:0/2490709939 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f7dd0042030 con 0x7f7ddc105d80 2024-09-21T03:58:43.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.973+0000 7f7dda000640 1 --2- 172.21.15.16:0/2490709939 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7dc40b6590 0x7f7dc40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:43.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.973+0000 7f7dda000640 1 -- 172.21.15.16:0/2490709939 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9609+0+0 (secure 0 0 0) 0x7f7dd0107f90 con 0x7f7ddc105d80 2024-09-21T03:58:43.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.974+0000 7f7de1000640 1 --2- 172.21.15.16:0/2490709939 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7dc40b6590 0x7f7dc40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:43.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.974+0000 7f7de1000640 1 --2- 172.21.15.16:0/2490709939 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7dc40b6590 0x7f7dc40b8aa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc10b1d0 tx=0x7f7dc800b390 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:43.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:43.975+0000 7f7dda000640 1 -- 172.21.15.16:0/2490709939 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7dd00d2640 con 0x7f7ddc105d80 2024-09-21T03:58:44.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:44.178+0000 7f7de4400640 1 -- 172.21.15.16:0/2490709939 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7ddc02e640 con 0x7f7ddc105d80 2024-09-21T03:58:45.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:44.999+0000 7f7dda000640 1 -- 172.21.15.16:0/2490709939 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v88) ==== 181+0+0 (secure 0 0 0) 0x7f7dd00d7580 con 0x7f7ddc105d80 2024-09-21T03:58:45.002 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-21T03:58:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:45.004+0000 7f7de4400640 1 -- 172.21.15.16:0/2490709939 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7dc40b6590 msgr2=0x7f7dc40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:45.004+0000 7f7de4400640 1 --2- 172.21.15.16:0/2490709939 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7dc40b6590 0x7f7dc40b8aa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc10b1d0 tx=0x7f7dc800b390 comp rx=0 tx=0).stop 2024-09-21T03:58:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:45.004+0000 7f7de4400640 1 -- 172.21.15.16:0/2490709939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ddc105d80 msgr2=0x7f7ddc19c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:45.004+0000 7f7de4400640 1 --2- 172.21.15.16:0/2490709939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ddc105d80 0x7f7ddc19c6e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0005940 tx=0x7f7dd00042f0 comp rx=0 tx=0).stop 2024-09-21T03:58:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:45.005+0000 7f7de4400640 1 -- 172.21.15.16:0/2490709939 shutdown_connections 2024-09-21T03:58:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:45.005+0000 7f7de4400640 1 --2- 172.21.15.16:0/2490709939 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f7dc40b6590 0x7f7dc40b8aa0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:45.005+0000 7f7de4400640 1 --2- 172.21.15.16:0/2490709939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ddc10c010 0x7f7ddc1a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:45.005+0000 7f7de4400640 1 --2- 172.21.15.16:0/2490709939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ddc10b6c0 0x7f7ddc19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:45.005+0000 7f7de4400640 1 --2- 172.21.15.16:0/2490709939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ddc105d80 0x7f7ddc19c6e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:45.005+0000 7f7de4400640 1 -- 172.21.15.16:0/2490709939 >> 172.21.15.16:0/2490709939 conn(0x7f7ddc101810 msgr2=0x7f7ddc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:45.006+0000 7f7de4400640 1 -- 172.21.15.16:0/2490709939 shutdown_connections 2024-09-21T03:58:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:45.006+0000 7f7de4400640 1 -- 172.21.15.16:0/2490709939 wait complete. 2024-09-21T03:58:45.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:44 smithi134 ceph-mon[35031]: pgmap v792: 129 pgs: 129 active+clean; 1.8 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 92 op/s 2024-09-21T03:58:45.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:44 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2490709939' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:58:45.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:44 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:58:45.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:44 smithi123 ceph-mon[34673]: pgmap v792: 129 pgs: 129 active+clean; 1.8 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 92 op/s 2024-09-21T03:58:45.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:44 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2490709939' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:58:45.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:44 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:58:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:44 smithi016 ceph-mon[30375]: pgmap v792: 129 pgs: 129 active+clean; 1.8 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 92 op/s 2024-09-21T03:58:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:44 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2490709939' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:58:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:44 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:58:46.272 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:46 smithi016 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:58:46.272 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:46 smithi016 ceph-mon[30375]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:58:46.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T03:58:46.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:46 smithi134 ceph-mon[35031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:58:46.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:46 smithi134 ceph-mon[35031]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:58:46.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:46 smithi123 ceph-mon[34673]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:58:46.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:46 smithi123 ceph-mon[34673]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:58:46.698 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:58:47.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:47 smithi016 ceph-mon[30375]: pgmap v794: 97 pgs: 97 active+clean; 577 KiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:58:47.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:47 smithi134 ceph-mon[35031]: pgmap v794: 97 pgs: 97 active+clean; 577 KiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:58:47.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:47 smithi123 ceph-mon[34673]: pgmap v794: 97 pgs: 97 active+clean; 577 KiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:58:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.484+0000 7f0e18a00640 1 -- 172.21.15.16:0/1989998435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e14105120 msgr2=0x7f0e1410d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.484+0000 7f0e18a00640 1 --2- 172.21.15.16:0/1989998435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e14105120 0x7f0e1410d670 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009900 tx=0x7f0dfc02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:58:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.484+0000 7f0e18a00640 1 -- 172.21.15.16:0/1989998435 shutdown_connections 2024-09-21T03:58:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.484+0000 7f0e18a00640 1 --2- 172.21.15.16:0/1989998435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e1410dc30 0x7f0e14110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.484+0000 7f0e18a00640 1 --2- 172.21.15.16:0/1989998435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e14105120 0x7f0e1410d670 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.484+0000 7f0e18a00640 1 --2- 172.21.15.16:0/1989998435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e141047d0 0x7f0e14104be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.484+0000 7f0e18a00640 1 -- 172.21.15.16:0/1989998435 >> 172.21.15.16:0/1989998435 conn(0x7f0e140fe2d0 msgr2=0x7f0e14100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.484+0000 7f0e18a00640 1 -- 172.21.15.16:0/1989998435 shutdown_connections 2024-09-21T03:58:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.484+0000 7f0e18a00640 1 -- 172.21.15.16:0/1989998435 wait complete. 2024-09-21T03:58:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.486+0000 7f0e18a00640 1 Processor -- start 2024-09-21T03:58:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.486+0000 7f0e18a00640 1 -- start start 2024-09-21T03:58:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.486+0000 7f0e18a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e141047d0 0x7f0e14102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.486+0000 7f0e18a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e14105120 0x7f0e14102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.486+0000 7f0e18a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e1410dc30 0x7f0e14102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.486+0000 7f0e18a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e14113890 con 0x7f0e141047d0 2024-09-21T03:58:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.486+0000 7f0e18a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e14113710 con 0x7f0e1410dc30 2024-09-21T03:58:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.486+0000 7f0e18a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0e14113a10 con 0x7f0e14105120 2024-09-21T03:58:47.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.487+0000 7f0e12800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e141047d0 0x7f0e14102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:47.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.487+0000 7f0e12800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e141047d0 0x7f0e14102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42242/0 (socket says 172.21.15.16:42242) 2024-09-21T03:58:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.487+0000 7f0e12800640 1 -- 172.21.15.16:0/2388334590 learned_addr learned my addr 172.21.15.16:0/2388334590 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:58:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.487+0000 7f0e13200640 1 --2- 172.21.15.16:0/2388334590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e1410dc30 0x7f0e14102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.487+0000 7f0e0be00640 1 --2- 172.21.15.16:0/2388334590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e14105120 0x7f0e14102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.487+0000 7f0e12800640 1 -- 172.21.15.16:0/2388334590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e14105120 msgr2=0x7f0e14102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.487+0000 7f0e12800640 1 --2- 172.21.15.16:0/2388334590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e14105120 0x7f0e14102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.487+0000 7f0e12800640 1 -- 172.21.15.16:0/2388334590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e1410dc30 msgr2=0x7f0e14102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.487+0000 7f0e12800640 1 --2- 172.21.15.16:0/2388334590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e1410dc30 0x7f0e14102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.487+0000 7f0e12800640 1 -- 172.21.15.16:0/2388334590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e141a8580 con 0x7f0e141047d0 2024-09-21T03:58:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.487+0000 7f0e12800640 1 --2- 172.21.15.16:0/2388334590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e141047d0 0x7f0e14102510 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400ea60 tx=0x7f0e0400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.488+0000 7f0e0b400640 1 -- 172.21.15.16:0/2388334590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0400cde0 con 0x7f0e141047d0 2024-09-21T03:58:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.488+0000 7f0e0b400640 1 -- 172.21.15.16:0/2388334590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e040035f0 con 0x7f0e141047d0 2024-09-21T03:58:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.488+0000 7f0e18a00640 1 -- 172.21.15.16:0/2388334590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e141a8870 con 0x7f0e141047d0 2024-09-21T03:58:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.488+0000 7f0e0b400640 1 -- 172.21.15.16:0/2388334590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04010690 con 0x7f0e141047d0 2024-09-21T03:58:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.488+0000 7f0e18a00640 1 -- 172.21.15.16:0/2388334590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e141a8d80 con 0x7f0e141047d0 2024-09-21T03:58:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.489+0000 7f0e18a00640 1 -- 172.21.15.16:0/2388334590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e14104be0 con 0x7f0e141047d0 2024-09-21T03:58:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.491+0000 7f0e0b400640 1 -- 172.21.15.16:0/2388334590 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f0e040037a0 con 0x7f0e141047d0 2024-09-21T03:58:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.493+0000 7f0e0b400640 1 --2- 172.21.15.16:0/2388334590 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0df80b6510 0x7f0df80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.493+0000 7f0e0b400640 1 -- 172.21.15.16:0/2388334590 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9055+0+0 (secure 0 0 0) 0x7f0e040e7b70 con 0x7f0e141047d0 2024-09-21T03:58:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.493+0000 7f0e0be00640 1 --2- 172.21.15.16:0/2388334590 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0df80b6510 0x7f0df80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.494+0000 7f0e0be00640 1 --2- 172.21.15.16:0/2388334590 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0df80b6510 0x7f0df80b8a20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009230 tx=0x7f0dfc031400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:47.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.495+0000 7f0e0b400640 1 -- 172.21.15.16:0/2388334590 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0e04056710 con 0x7f0e141047d0 2024-09-21T03:58:47.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:47.697+0000 7f0e18a00640 1 -- 172.21.15.16:0/2388334590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0e14103810 con 0x7f0e141047d0 2024-09-21T03:58:48.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:48.010+0000 7f0e0b400640 1 -- 172.21.15.16:0/2388334590 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v89) ==== 169+0+0 (secure 0 0 0) 0x7f0e040ae070 con 0x7f0e141047d0 2024-09-21T03:58:48.012 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-21T03:58:48.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:48.014+0000 7f0e18a00640 1 -- 172.21.15.16:0/2388334590 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0df80b6510 msgr2=0x7f0df80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:48.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:48.014+0000 7f0e18a00640 1 --2- 172.21.15.16:0/2388334590 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0df80b6510 0x7f0df80b8a20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009230 tx=0x7f0dfc031400 comp rx=0 tx=0).stop 2024-09-21T03:58:48.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:48.014+0000 7f0e18a00640 1 -- 172.21.15.16:0/2388334590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e141047d0 msgr2=0x7f0e14102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:48.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:48.014+0000 7f0e18a00640 1 --2- 172.21.15.16:0/2388334590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e141047d0 0x7f0e14102510 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400ea60 tx=0x7f0e0400ef20 comp rx=0 tx=0).stop 2024-09-21T03:58:48.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:48.015+0000 7f0e18a00640 1 -- 172.21.15.16:0/2388334590 shutdown_connections 2024-09-21T03:58:48.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:48.015+0000 7f0e18a00640 1 --2- 172.21.15.16:0/2388334590 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7f0df80b6510 0x7f0df80b8a20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:48.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:48.015+0000 7f0e18a00640 1 --2- 172.21.15.16:0/2388334590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e1410dc30 0x7f0e14102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:48.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:48.015+0000 7f0e18a00640 1 --2- 172.21.15.16:0/2388334590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e14105120 0x7f0e14102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:48.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:48.015+0000 7f0e18a00640 1 --2- 172.21.15.16:0/2388334590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e141047d0 0x7f0e14102510 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:48.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:48.015+0000 7f0e18a00640 1 -- 172.21.15.16:0/2388334590 >> 172.21.15.16:0/2388334590 conn(0x7f0e140fe2d0 msgr2=0x7f0e14106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:48.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:48.015+0000 7f0e18a00640 1 -- 172.21.15.16:0/2388334590 shutdown_connections 2024-09-21T03:58:48.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:48.015+0000 7f0e18a00640 1 -- 172.21.15.16:0/2388334590 wait complete. 2024-09-21T03:58:48.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:48 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2388334590' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:58:48.402 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:48 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2388334590' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:58:48.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:48 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2388334590' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:58:48.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 2b324840-77ca-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-21T03:58:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:58:49.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:49 smithi016 ceph-mon[30375]: pgmap v795: 97 pgs: 97 active+clean; 577 KiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:58:49.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:49 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/2388334590' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:58:49.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:49 smithi016 ceph-mon[30375]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:58:49.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:49 smithi134 ceph-mon[35031]: pgmap v795: 97 pgs: 97 active+clean; 577 KiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:58:49.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:49 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/2388334590' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:58:49.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:49 smithi134 ceph-mon[35031]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:58:49.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:49 smithi123 ceph-mon[34673]: pgmap v795: 97 pgs: 97 active+clean; 577 KiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:58:49.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:49 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/2388334590' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:58:49.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:49 smithi123 ceph-mon[34673]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:58:49.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.765+0000 7fa41d800640 1 -- 172.21.15.16:0/2384710178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa41810b6c0 msgr2=0x7fa41810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.765+0000 7fa41d800640 1 --2- 172.21.15.16:0/2384710178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa41810b6c0 0x7fa41810bad0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa400009900 tx=0x7fa40002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:58:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.765+0000 7fa41d800640 1 -- 172.21.15.16:0/2384710178 shutdown_connections 2024-09-21T03:58:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.765+0000 7fa41d800640 1 --2- 172.21.15.16:0/2384710178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa41810c010 0x7fa418112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.765+0000 7fa41d800640 1 --2- 172.21.15.16:0/2384710178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa41810b6c0 0x7fa41810bad0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.765+0000 7fa41d800640 1 --2- 172.21.15.16:0/2384710178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa418105d80 0x7fa418106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.765+0000 7fa41d800640 1 -- 172.21.15.16:0/2384710178 >> 172.21.15.16:0/2384710178 conn(0x7fa4181017d0 msgr2=0x7fa418103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.765+0000 7fa41d800640 1 -- 172.21.15.16:0/2384710178 shutdown_connections 2024-09-21T03:58:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.765+0000 7fa41d800640 1 -- 172.21.15.16:0/2384710178 wait complete. 2024-09-21T03:58:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.766+0000 7fa41d800640 1 Processor -- start 2024-09-21T03:58:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.767+0000 7fa41d800640 1 -- start start 2024-09-21T03:58:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.767+0000 7fa41d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa418105d80 0x7fa41819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.767+0000 7fa41d800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa41810b6c0 0x7fa41819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.767+0000 7fa41d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa41810c010 0x7fa4181a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.767+0000 7fa41d800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa418115d20 con 0x7fa418105d80 2024-09-21T03:58:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.767+0000 7fa41d800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa418115ba0 con 0x7fa41810c010 2024-09-21T03:58:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.767+0000 7fa41d800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa418115ea0 con 0x7fa41810b6c0 2024-09-21T03:58:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.767+0000 7fa416a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa418105d80 0x7fa41819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.767+0000 7fa40fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa41810b6c0 0x7fa41819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.767+0000 7fa416a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa418105d80 0x7fa41819c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42258/0 (socket says 172.21.15.16:42258) 2024-09-21T03:58:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.767+0000 7fa416a00640 1 -- 172.21.15.16:0/3934988185 learned_addr learned my addr 172.21.15.16:0/3934988185 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:58:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.768+0000 7fa417400640 1 --2- 172.21.15.16:0/3934988185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa41810c010 0x7fa4181a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.768+0000 7fa416a00640 1 -- 172.21.15.16:0/3934988185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa41810b6c0 msgr2=0x7fa41819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.768+0000 7fa416a00640 1 --2- 172.21.15.16:0/3934988185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa41810b6c0 0x7fa41819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.768+0000 7fa416a00640 1 -- 172.21.15.16:0/3934988185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa41810c010 msgr2=0x7fa4181a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.768+0000 7fa416a00640 1 --2- 172.21.15.16:0/3934988185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa41810c010 0x7fa4181a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.768+0000 7fa416a00640 1 -- 172.21.15.16:0/3934988185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4181a2620 con 0x7fa418105d80 2024-09-21T03:58:49.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.768+0000 7fa417400640 1 --2- 172.21.15.16:0/3934988185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa41810c010 0x7fa4181a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:58:49.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.768+0000 7fa416a00640 1 --2- 172.21.15.16:0/3934988185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa418105d80 0x7fa41819c6b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa40800e9f0 tx=0x7fa40800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:49.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.769+0000 7fa40f400640 1 -- 172.21.15.16:0/3934988185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40800cd80 con 0x7fa418105d80 2024-09-21T03:58:49.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.769+0000 7fa40f400640 1 -- 172.21.15.16:0/3934988185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4080108e0 con 0x7fa418105d80 2024-09-21T03:58:49.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.769+0000 7fa40f400640 1 -- 172.21.15.16:0/3934988185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408018980 con 0x7fa418105d80 2024-09-21T03:58:49.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.769+0000 7fa41d800640 1 -- 172.21.15.16:0/3934988185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4181a2910 con 0x7fa418105d80 2024-09-21T03:58:49.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.769+0000 7fa41d800640 1 -- 172.21.15.16:0/3934988185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4181a9070 con 0x7fa418105d80 2024-09-21T03:58:49.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.770+0000 7fa41d800640 1 -- 172.21.15.16:0/3934988185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa418107000 con 0x7fa418105d80 2024-09-21T03:58:49.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.772+0000 7fa40f400640 1 -- 172.21.15.16:0/3934988185 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa408010430 con 0x7fa418105d80 2024-09-21T03:58:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.774+0000 7fa40f400640 1 --2- 172.21.15.16:0/3934988185 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa3fc0b6590 0x7fa3fc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.774+0000 7fa40f400640 1 -- 172.21.15.16:0/3934988185 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8496+0+0 (secure 0 0 0) 0x7fa4080e7bf0 con 0x7fa418105d80 2024-09-21T03:58:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.774+0000 7fa40fe00640 1 --2- 172.21.15.16:0/3934988185 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa3fc0b6590 0x7fa3fc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.775+0000 7fa40fe00640 1 --2- 172.21.15.16:0/3934988185 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa3fc0b6590 0x7fa3fc0b8aa0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa40002a9e0 tx=0x7fa400005a20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.776+0000 7fa40f400640 1 -- 172.21.15.16:0/3934988185 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa4080ae340 con 0x7fa418105d80 2024-09-21T03:58:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:49.976+0000 7fa41d800640 1 -- 172.21.15.16:0/3934988185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa41810dde0 con 0x7fa418105d80 2024-09-21T03:58:50.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:50.019+0000 7fa40f400640 1 -- 172.21.15.16:0/3934988185 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v90) ==== 178+0+0 (secure 0 0 0) 0x7fa4080b2720 con 0x7fa418105d80 2024-09-21T03:58:50.021 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' removed 2024-09-21T03:58:50.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:50.021+0000 7fa41d800640 1 -- 172.21.15.16:0/3934988185 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa3fc0b6590 msgr2=0x7fa3fc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:50.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:50.021+0000 7fa41d800640 1 --2- 172.21.15.16:0/3934988185 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa3fc0b6590 0x7fa3fc0b8aa0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa40002a9e0 tx=0x7fa400005a20 comp rx=0 tx=0).stop 2024-09-21T03:58:50.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:50.021+0000 7fa41d800640 1 -- 172.21.15.16:0/3934988185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa418105d80 msgr2=0x7fa41819c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:50.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:50.021+0000 7fa41d800640 1 --2- 172.21.15.16:0/3934988185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa418105d80 0x7fa41819c6b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa40800e9f0 tx=0x7fa40800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:58:50.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:50.021+0000 7fa41d800640 1 -- 172.21.15.16:0/3934988185 shutdown_connections 2024-09-21T03:58:50.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:50.021+0000 7fa41d800640 1 --2- 172.21.15.16:0/3934988185 >> [v2:172.21.15.16:6800/3097741949,v1:172.21.15.16:6801/3097741949] conn(0x7fa3fc0b6590 0x7fa3fc0b8aa0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:50.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:50.021+0000 7fa41d800640 1 --2- 172.21.15.16:0/3934988185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa41810c010 0x7fa4181a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:50.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:50.021+0000 7fa41d800640 1 --2- 172.21.15.16:0/3934988185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa41810b6c0 0x7fa41819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:50.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:50.021+0000 7fa41d800640 1 --2- 172.21.15.16:0/3934988185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa418105d80 0x7fa41819c6b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:50.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:50.021+0000 7fa41d800640 1 -- 172.21.15.16:0/3934988185 >> 172.21.15.16:0/3934988185 conn(0x7fa4181017d0 msgr2=0x7fa418103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:50.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:50.021+0000 7fa41d800640 1 -- 172.21.15.16:0/3934988185 shutdown_connections 2024-09-21T03:58:50.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:58:50.021+0000 7fa41d800640 1 -- 172.21.15.16:0/3934988185 wait complete. 2024-09-21T03:58:50.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:50 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3934988185' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:58:50.406 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:50 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3934988185' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:58:50.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:50 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3934988185' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:58:51.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:58:51.179 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:58:51.280 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:51 smithi123 ceph-mon[34673]: pgmap v797: 33 pgs: 33 active+clean; 577 KiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:58:51.280 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:51 smithi123 ceph-mon[34673]: from='client.? 172.21.15.16:0/3934988185' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:58:51.280 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:51 smithi123 ceph-mon[34673]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T03:58:51.281 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:58:51.339 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:51 smithi134 ceph-mon[35031]: pgmap v797: 33 pgs: 33 active+clean; 577 KiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:58:51.339 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:51 smithi134 ceph-mon[35031]: from='client.? 172.21.15.16:0/3934988185' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:58:51.339 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:51 smithi134 ceph-mon[35031]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T03:58:51.348 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T03:58:51.348 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:58:51.405 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:58:51.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:51 smithi016 ceph-mon[30375]: pgmap v797: 33 pgs: 33 active+clean; 577 KiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:58:51.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:51 smithi016 ceph-mon[30375]: from='client.? 172.21.15.16:0/3934988185' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:58:51.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:51 smithi016 ceph-mon[30375]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T03:58:51.434 DEBUG:teuthology.orchestra.run.smithi134:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:58:51.465 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T03:58:51.466 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T03:58:51.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mon.a 2024-09-21T03:58:51.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:51 smithi016 systemd[1]: Stopping Ceph mon.a for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:58:52.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:51 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-a[30371]: 2024-09-21T03:58:51.948+0000 7fb911600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:58:52.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:51 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-a[30371]: 2024-09-21T03:58:51.948+0000 7fb911600640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T03:58:52.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:52 smithi016 podman[98562]: 2024-09-21 03:58:52.102592949 +0000 UTC m=+0.298958080 container died 01d63f4758cfefa6f0963cbf38a419697ee266258bf663a230fc8b1635436bb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-a, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:58:52.816 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:52 smithi016 podman[98562]: 2024-09-21 03:58:52.544302299 +0000 UTC m=+0.740667429 container cleanup 01d63f4758cfefa6f0963cbf38a419697ee266258bf663a230fc8b1635436bb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T03:58:52.816 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:52 smithi016 bash[98562]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-a 2024-09-21T03:58:53.079 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:58:52 smithi016 podman[98576]: 2024-09-21 03:58:52.814880779 +0000 UTC m=+0.699745215 container remove 01d63f4758cfefa6f0963cbf38a419697ee266258bf663a230fc8b1635436bb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T03:58:53.103 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T03:58:53.181 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:58:53.181 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T03:58:53.181 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T03:58:53.181 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mon.b 2024-09-21T03:58:53.592 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:53 smithi123 systemd[1]: Stopping Ceph mon.b for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:58:53.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:53 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-b[34669]: 2024-09-21T03:58:53.590+0000 7fa81dc8c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:58:53.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:53 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-b[34669]: 2024-09-21T03:58:53.590+0000 7fa81dc8c640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T03:58:53.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:53 smithi123 podman[66010]: 2024-09-21 03:58:53.631388974 +0000 UTC m=+0.194454321 container died 524ea85072327be3326a37857d673564373cf19039af9148b2f5aa59fe7a21c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T03:58:54.291 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:54 smithi123 podman[66010]: 2024-09-21 03:58:54.013298378 +0000 UTC m=+0.576363772 container cleanup 524ea85072327be3326a37857d673564373cf19039af9148b2f5aa59fe7a21c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T03:58:54.291 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 21 03:58:54 smithi123 bash[66010]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-b 2024-09-21T03:58:54.417 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T03:58:54.481 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:58:54.481 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T03:58:54.481 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T03:58:54.481 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mon.c 2024-09-21T03:58:54.820 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:54 smithi134 systemd[1]: Stopping Ceph mon.c for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:58:55.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:54 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-c[35027]: 2024-09-21T03:58:54.817+0000 7f493d800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:58:55.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:54 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-c[35027]: 2024-09-21T03:58:54.817+0000 7f493d800640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T03:58:55.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:54 smithi134 podman[67370]: 2024-09-21 03:58:54.919855546 +0000 UTC m=+0.203963586 container died c60a9108c7f0680ff6af03963c70d43f44da46af8294d8f748c59c7c66952ed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-21T03:58:55.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:55 smithi134 podman[67370]: 2024-09-21 03:58:55.318655526 +0000 UTC m=+0.602763566 container cleanup c60a9108c7f0680ff6af03963c70d43f44da46af8294d8f748c59c7c66952ed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:58:55.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Sep 21 03:58:55 smithi134 bash[67370]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mon-c 2024-09-21T03:58:55.696 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T03:58:55.746 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:58:55.746 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T03:58:55.746 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T03:58:55.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mgr.x 2024-09-21T03:58:56.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:58:55 smithi016 systemd[1]: Stopping Ceph mgr.x for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:58:56.629 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:58:56 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: 2024-09-21T03:58:56.205+0000 7efd45800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:58:56.630 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:58:56 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x[30600]: 2024-09-21T03:58:56.205+0000 7efd45800640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-21T03:58:56.630 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:58:56 smithi016 podman[98681]: 2024-09-21 03:58:56.297276884 +0000 UTC m=+0.249219475 container died 60b9b29bbfdb3e39283c364c4d4b0945b949580715779e9375dfe205519ee428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True) 2024-09-21T03:58:56.912 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:58:56 smithi016 podman[98681]: 2024-09-21 03:58:56.78562074 +0000 UTC m=+0.737563338 container cleanup 60b9b29bbfdb3e39283c364c4d4b0945b949580715779e9375dfe205519ee428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-21T03:58:56.913 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:58:56 smithi016 bash[98681]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x 2024-09-21T03:58:57.276 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:58:57 smithi016 podman[98693]: 2024-09-21 03:58:57.021369616 +0000 UTC m=+0.713912402 container remove 60b9b29bbfdb3e39283c364c4d4b0945b949580715779e9375dfe205519ee428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T03:58:57.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T03:58:57.358 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:58:57.359 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T03:58:57.359 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T03:58:57.359 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mgr.y 2024-09-21T03:58:57.656 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 21 03:58:57 smithi123 systemd[1]: Stopping Ceph mgr.y for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:58:58.030 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 21 03:58:57 smithi123 podman[66134]: 2024-09-21 03:58:57.706401997 +0000 UTC m=+0.157979491 container died 40fbbada280ecf136d60f6a810bed83bf2d928e061a058ec8b05011a4ee973aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-y, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:58:58.363 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 21 03:58:58 smithi123 podman[66134]: 2024-09-21 03:58:58.127282736 +0000 UTC m=+0.578860218 container cleanup 40fbbada280ecf136d60f6a810bed83bf2d928e061a058ec8b05011a4ee973aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-y, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:58:58.364 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 21 03:58:58 smithi123 bash[66134]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-y 2024-09-21T03:58:58.614 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 21 03:58:58 smithi123 podman[66146]: 2024-09-21 03:58:58.362310223 +0000 UTC m=+0.650720207 container remove 40fbbada280ecf136d60f6a810bed83bf2d928e061a058ec8b05011a4ee973aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True) 2024-09-21T03:58:58.614 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 21 03:58:58 smithi123 systemd[1]: ceph-2b324840-77ca-11ef-baf5-efdc52797490@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-21T03:58:58.618 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T03:58:58.654 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:58:58.654 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T03:58:58.655 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T03:58:58.655 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mgr.z 2024-09-21T03:58:58.996 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 21 03:58:58 smithi134 systemd[1]: Stopping Ceph mgr.z for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:58:59.277 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 21 03:58:58 smithi134 podman[67494]: 2024-09-21 03:58:58.994776294 +0000 UTC m=+0.141470610 container died 3cf07047e78696a6e98c49228c594927694375e481224d4a56c70de962fa0aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-21T03:58:59.602 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 21 03:58:59 smithi134 podman[67494]: 2024-09-21 03:58:59.37484682 +0000 UTC m=+0.521541122 container cleanup 3cf07047e78696a6e98c49228c594927694375e481224d4a56c70de962fa0aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-z, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:58:59.603 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Sep 21 03:58:59 smithi134 bash[67494]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mgr-z 2024-09-21T03:58:59.751 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T03:58:59.788 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:58:59.788 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T03:58:59.788 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T03:58:59.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.0 2024-09-21T03:59:00.163 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:58:59 smithi016 systemd[1]: Stopping Ceph osd.0 for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:00.662 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:59:00 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-0[45832]: 2024-09-21T03:59:00.218+0000 7f5f36a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:00.663 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:59:00 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-0[45832]: 2024-09-21T03:59:00.218+0000 7f5f36a00640 -1 osd.0 90 *** Got signal Terminated *** 2024-09-21T03:59:00.663 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:59:00 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-0[45832]: 2024-09-21T03:59:00.218+0000 7f5f36a00640 -1 osd.0 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:59:06.162 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:59:05 smithi016 podman[98800]: 2024-09-21 03:59:05.900821033 +0000 UTC m=+5.824786374 container died dbb8feffe1b041a6496cb46135ccbd66e19d8b15cdbc462cb282f69a0446c3e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:59:06.489 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:59:06 smithi016 podman[98800]: 2024-09-21 03:59:06.374955482 +0000 UTC m=+6.298920822 container cleanup dbb8feffe1b041a6496cb46135ccbd66e19d8b15cdbc462cb282f69a0446c3e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T03:59:06.489 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:59:06 smithi016 bash[98800]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-0 2024-09-21T03:59:06.788 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:59:06 smithi016 podman[98818]: 2024-09-21 03:59:06.643834507 +0000 UTC m=+0.730965094 container remove dbb8feffe1b041a6496cb46135ccbd66e19d8b15cdbc462cb282f69a0446c3e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T03:59:07.600 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:59:07 smithi016 podman[98914]: 2024-09-21 03:59:07.158127113 +0000 UTC m=+0.028726089 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:59:07.600 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:59:07 smithi016 podman[98914]: 2024-09-21 03:59:07.278413962 +0000 UTC m=+0.149012937 container create 2cfbc9d2e67a35fa88ce4aba14e48c76b7fba56dba2e11a5c71b34b372cbeb01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T03:59:07.600 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:59:07 smithi016 podman[98914]: 2024-09-21 03:59:07.598078224 +0000 UTC m=+0.468677223 container init 2cfbc9d2e67a35fa88ce4aba14e48c76b7fba56dba2e11a5c71b34b372cbeb01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-0-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T03:59:07.912 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:59:07 smithi016 podman[98914]: 2024-09-21 03:59:07.605259791 +0000 UTC m=+0.475858767 container start 2cfbc9d2e67a35fa88ce4aba14e48c76b7fba56dba2e11a5c71b34b372cbeb01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-0-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T03:59:07.913 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:59:07 smithi016 podman[98914]: 2024-09-21 03:59:07.73988658 +0000 UTC m=+0.610485567 container attach 2cfbc9d2e67a35fa88ce4aba14e48c76b7fba56dba2e11a5c71b34b372cbeb01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-0-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:59:08.413 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:59:08 smithi016 podman[98914]: 2024-09-21 03:59:08.134412157 +0000 UTC m=+1.005011132 container died 2cfbc9d2e67a35fa88ce4aba14e48c76b7fba56dba2e11a5c71b34b372cbeb01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T03:59:08.953 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T03:59:09.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:09.033 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T03:59:09.033 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T03:59:09.033 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.1 2024-09-21T03:59:09.156 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:59:09 smithi123 systemd[1]: Stopping Ceph osd.1 for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:09.656 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:59:09 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-1[38603]: 2024-09-21T03:59:09.331+0000 7ff98d7dc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:09.657 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:59:09 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-1[38603]: 2024-09-21T03:59:09.331+0000 7ff98d7dc640 -1 osd.1 90 *** Got signal Terminated *** 2024-09-21T03:59:09.657 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:59:09 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-1[38603]: 2024-09-21T03:59:09.331+0000 7ff98d7dc640 -1 osd.1 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:59:15.151 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:59:14 smithi123 podman[66270]: 2024-09-21 03:59:14.83317997 +0000 UTC m=+5.604153829 container died c0f75a2d2bba66c423b0451f75474765611d42bb5c115cd7487909494ae4e16a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T03:59:15.406 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:59:15 smithi123 podman[66270]: 2024-09-21 03:59:15.240344664 +0000 UTC m=+6.011318521 container cleanup c0f75a2d2bba66c423b0451f75474765611d42bb5c115cd7487909494ae4e16a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:59:15.407 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:59:15 smithi123 bash[66270]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-1 2024-09-21T03:59:15.902 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:59:15 smithi123 podman[66285]: 2024-09-21 03:59:15.492129942 +0000 UTC m=+0.662929306 container remove c0f75a2d2bba66c423b0451f75474765611d42bb5c115cd7487909494ae4e16a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-1, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:59:16.155 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:59:15 smithi123 podman[66374]: 2024-09-21 03:59:15.804117219 +0000 UTC m=+0.015995272 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:59:16.156 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:59:15 smithi123 podman[66374]: 2024-09-21 03:59:15.909579718 +0000 UTC m=+0.121457801 container create af0395640201ac828c31219a65eafd9f0b8a6789d8abece5ed313b1316c92d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:59:16.407 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:59:16 smithi123 podman[66374]: 2024-09-21 03:59:16.15413536 +0000 UTC m=+0.366013408 container init af0395640201ac828c31219a65eafd9f0b8a6789d8abece5ed313b1316c92d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T03:59:16.407 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:59:16 smithi123 podman[66374]: 2024-09-21 03:59:16.157577713 +0000 UTC m=+0.369455751 container start af0395640201ac828c31219a65eafd9f0b8a6789d8abece5ed313b1316c92d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:59:16.407 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:59:16 smithi123 podman[66374]: 2024-09-21 03:59:16.245838775 +0000 UTC m=+0.457716818 container attach af0395640201ac828c31219a65eafd9f0b8a6789d8abece5ed313b1316c92d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T03:59:16.407 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 21 03:59:16 smithi123 podman[66374]: 2024-09-21 03:59:16.369601665 +0000 UTC m=+0.581479703 container died af0395640201ac828c31219a65eafd9f0b8a6789d8abece5ed313b1316c92d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:59:17.017 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T03:59:17.057 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:17.057 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T03:59:17.057 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T03:59:17.057 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.2 2024-09-21T03:59:17.352 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:59:17 smithi134 systemd[1]: Stopping Ceph osd.2 for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:17.852 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:59:17 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-2[39432]: 2024-09-21T03:59:17.371+0000 7f7171970640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:17.852 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:59:17 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-2[39432]: 2024-09-21T03:59:17.371+0000 7f7171970640 -1 osd.2 90 *** Got signal Terminated *** 2024-09-21T03:59:17.852 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:59:17 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-2[39432]: 2024-09-21T03:59:17.371+0000 7f7171970640 -1 osd.2 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:59:23.133 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:59:22 smithi134 podman[67620]: 2024-09-21 03:59:22.852741285 +0000 UTC m=+5.606714370 container died a1bdb814fa5a8247ff5e0dc7842b2b5760b20051786b0ae99eade3e1fc6debf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T03:59:23.508 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:59:23 smithi134 podman[67620]: 2024-09-21 03:59:23.238835269 +0000 UTC m=+5.992808343 container cleanup a1bdb814fa5a8247ff5e0dc7842b2b5760b20051786b0ae99eade3e1fc6debf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T03:59:23.509 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:59:23 smithi134 bash[67620]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-2 2024-09-21T03:59:23.848 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:59:23 smithi134 podman[67636]: 2024-09-21 03:59:23.506918414 +0000 UTC m=+0.649502229 container remove a1bdb814fa5a8247ff5e0dc7842b2b5760b20051786b0ae99eade3e1fc6debf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:59:24.102 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:59:23 smithi134 podman[67727]: 2024-09-21 03:59:23.7485152 +0000 UTC m=+0.015417617 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:59:24.103 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:59:23 smithi134 podman[67727]: 2024-09-21 03:59:23.858009423 +0000 UTC m=+0.124911828 container create bdd300319c14ad6476582f875bc35df823f92e84b402dd3905b5c51215f03469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:59:24.103 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:59:24 smithi134 podman[67727]: 2024-09-21 03:59:24.085915693 +0000 UTC m=+0.352818097 container init bdd300319c14ad6476582f875bc35df823f92e84b402dd3905b5c51215f03469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-2-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T03:59:24.103 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:59:24 smithi134 podman[67727]: 2024-09-21 03:59:24.089328751 +0000 UTC m=+0.356231172 container start bdd300319c14ad6476582f875bc35df823f92e84b402dd3905b5c51215f03469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T03:59:24.603 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:59:24 smithi134 podman[67727]: 2024-09-21 03:59:24.186207986 +0000 UTC m=+0.453110398 container attach bdd300319c14ad6476582f875bc35df823f92e84b402dd3905b5c51215f03469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T03:59:24.603 INFO:journalctl@ceph.osd.2.smithi134.stdout:Sep 21 03:59:24 smithi134 podman[67727]: 2024-09-21 03:59:24.321434419 +0000 UTC m=+0.588336834 container died bdd300319c14ad6476582f875bc35df823f92e84b402dd3905b5c51215f03469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T03:59:24.928 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T03:59:24.968 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:24.968 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T03:59:24.968 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T03:59:24.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.3 2024-09-21T03:59:25.394 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:59:25 smithi016 systemd[1]: Stopping Ceph osd.3 for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:25.663 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:59:25 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-3[51972]: 2024-09-21T03:59:25.391+0000 7f48a0000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:25.663 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:59:25 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-3[51972]: 2024-09-21T03:59:25.391+0000 7f48a0000640 -1 osd.3 90 *** Got signal Terminated *** 2024-09-21T03:59:25.663 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:59:25 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-3[51972]: 2024-09-21T03:59:25.391+0000 7f48a0000640 -1 osd.3 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:59:27.656 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:27 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:27.328+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:28.163 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:59:27 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-3[51972]: 2024-09-21T03:59:27.903+0000 7f489ba00640 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:01.954113+0000 front 2024-09-21T03:59:01.954241+0000 (oldest deadline 2024-09-21T03:59:27.854243+0000) 2024-09-21T03:59:28.602 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:28 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:28.285+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:28.656 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:28 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:28.287+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:29.412 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:59:28 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-3[51972]: 2024-09-21T03:59:28.939+0000 7f489ba00640 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:01.954113+0000 front 2024-09-21T03:59:01.954241+0000 (oldest deadline 2024-09-21T03:59:27.854243+0000) 2024-09-21T03:59:29.602 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:29 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:29.256+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:29.656 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:29 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:29.268+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:30.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:29 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:29.693+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:30.412 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:59:29 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-3[51972]: 2024-09-21T03:59:29.956+0000 7f489ba00640 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:01.954113+0000 front 2024-09-21T03:59:01.954241+0000 (oldest deadline 2024-09-21T03:59:27.854243+0000) 2024-09-21T03:59:30.602 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:30 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:30.250+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:30.656 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:30 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:30.289+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:31.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:30 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:30.655+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:31.548 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:59:31 smithi016 podman[99015]: 2024-09-21 03:59:31.197096245 +0000 UTC m=+5.927819897 container died 2f0b90d7d69d86c76a9d908052ff5a202fe72683fd9f5e8cc412eaa14875b10b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-3, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T03:59:31.602 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:31 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:31.253+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:31.656 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:31 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:31.323+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:32.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:31 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:31.657+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:32.162 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:59:31 smithi016 podman[99015]: 2024-09-21 03:59:31.814739789 +0000 UTC m=+6.545463443 container cleanup 2f0b90d7d69d86c76a9d908052ff5a202fe72683fd9f5e8cc412eaa14875b10b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:59:32.163 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:59:31 smithi016 bash[99015]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-3 2024-09-21T03:59:32.163 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:59:32 smithi016 podman[99033]: 2024-09-21 03:59:32.016460801 +0000 UTC m=+0.803598420 container remove 2f0b90d7d69d86c76a9d908052ff5a202fe72683fd9f5e8cc412eaa14875b10b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:59:32.602 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:32 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:32.236+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:32.656 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:32 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:32.330+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:32.888 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:59:32 smithi016 podman[99120]: 2024-09-21 03:59:32.49500188 +0000 UTC m=+0.027245075 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:59:32.889 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:59:32 smithi016 podman[99120]: 2024-09-21 03:59:32.637056114 +0000 UTC m=+0.169299285 container create b9faedb3c58152269e263f4479de058718445226b2ab7b8fa729076b4d9af7f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:59:32.889 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:59:32 smithi016 podman[99120]: 2024-09-21 03:59:32.887167028 +0000 UTC m=+0.419410210 container init b9faedb3c58152269e263f4479de058718445226b2ab7b8fa729076b4d9af7f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-3-deactivate, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:59:33.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:32 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:32.647+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:33.163 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:59:32 smithi016 podman[99120]: 2024-09-21 03:59:32.894340988 +0000 UTC m=+0.426584170 container start b9faedb3c58152269e263f4479de058718445226b2ab7b8fa729076b4d9af7f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T03:59:33.163 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:59:33 smithi016 podman[99120]: 2024-09-21 03:59:33.003804532 +0000 UTC m=+0.536047712 container attach b9faedb3c58152269e263f4479de058718445226b2ab7b8fa729076b4d9af7f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-3-deactivate, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T03:59:33.525 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:59:33 smithi016 podman[99120]: 2024-09-21 03:59:33.164816777 +0000 UTC m=+0.697059961 container died b9faedb3c58152269e263f4479de058718445226b2ab7b8fa729076b4d9af7f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-21T03:59:33.602 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:33 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:33.220+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:33.656 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:33 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:33.375+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:33.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T03:59:33.938 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:33.938 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T03:59:33.938 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T03:59:33.938 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.4 2024-09-21T03:59:34.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:33 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:33.624+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:34.247 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:59:33 smithi123 systemd[1]: Stopping Ceph osd.4 for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:34.597 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:34 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:34.178+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:34.656 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:34 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:34.392+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:34.657 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:59:34 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-4[44742]: 2024-09-21T03:59:34.243+0000 7f21f59b4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:34.657 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:59:34 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-4[44742]: 2024-09-21T03:59:34.243+0000 7f21f59b4640 -1 osd.4 90 *** Got signal Terminated *** 2024-09-21T03:59:34.657 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:59:34 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-4[44742]: 2024-09-21T03:59:34.243+0000 7f21f59b4640 -1 osd.4 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:59:34.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:34 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:34.595+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:35.602 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:35 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:35.145+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:35.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:35 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:35.371+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:35.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:35 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:35.310+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:36.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:35 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:35.612+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:36.413 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:36 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:36.128+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:36.580 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:36 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:36.174+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:36.656 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:36 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:36.356+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:36.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:36 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:36.421+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:36.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:36 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:36.578+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:37.412 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:37 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:37.127+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:37.567 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:37 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:37.141+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:37.567 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:37 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:37.141+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.725359+0000 front 2024-09-21T03:59:13.725407+0000 (oldest deadline 2024-09-21T03:59:36.625267+0000) 2024-09-21T03:59:37.567 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:37 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:37.240+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:37.656 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:37 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:37.362+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:37.657 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:59:37 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-4[44742]: 2024-09-21T03:59:37.565+0000 7f21f1f9f640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:12.145460+0000 front 2024-09-21T03:59:12.145483+0000 (oldest deadline 2024-09-21T03:59:37.445580+0000) 2024-09-21T03:59:37.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:37 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:37.389+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:37.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:37 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:37.389+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:37.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:37 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:37.564+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:38.352 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:38 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:38.243+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:38.352 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:38 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:38.091+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:38.353 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:38 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:38.091+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.725359+0000 front 2024-09-21T03:59:13.725407+0000 (oldest deadline 2024-09-21T03:59:36.625267+0000) 2024-09-21T03:59:38.413 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:38 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:38.084+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:38.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:38 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:38.264+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:38.656 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:38 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:38.343+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:38.657 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:59:38 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-4[44742]: 2024-09-21T03:59:38.570+0000 7f21f1f9f640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:12.145460+0000 front 2024-09-21T03:59:12.145483+0000 (oldest deadline 2024-09-21T03:59:37.445580+0000) 2024-09-21T03:59:38.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:38 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:38.412+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:38.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:38 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:38.412+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:38.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:38 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:38.555+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:38.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:38 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:38.555+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:39.247 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:39 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:39.071+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:39.581 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:39 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:39.231+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:39.581 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:39 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:39.112+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:39.582 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:39 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:39.112+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.725359+0000 front 2024-09-21T03:59:13.725407+0000 (oldest deadline 2024-09-21T03:59:36.625267+0000) 2024-09-21T03:59:39.656 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:39 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:39.315+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:39.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:39 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:39.399+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:39.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:39 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:39.399+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:39.662 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:39 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:39.244+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:39.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:39 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:39.579+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:39.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:39 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:39.579+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:40.276 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:59:39 smithi123 podman[66490]: 2024-09-21 03:59:39.96546851 +0000 UTC m=+5.835525916 container died a81f6c650f687459771de2aca759abb116674b36a772a1905c5d66db1aed8459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T03:59:40.352 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:40 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:40.078+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:40.352 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:40 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:40.078+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.725359+0000 front 2024-09-21T03:59:13.725407+0000 (oldest deadline 2024-09-21T03:59:36.625267+0000) 2024-09-21T03:59:40.353 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:40 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:40.184+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:40.412 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:40 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:40.222+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:40.413 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:40 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:40.095+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:40.531 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:40 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:40.274+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:40.532 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:59:40 smithi123 podman[66490]: 2024-09-21 03:59:40.519410941 +0000 UTC m=+6.389468389 container cleanup a81f6c650f687459771de2aca759abb116674b36a772a1905c5d66db1aed8459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-4, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T03:59:40.532 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:59:40 smithi123 bash[66490]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-4 2024-09-21T03:59:40.532 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:40 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:40.362+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:40.532 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:40 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:40.362+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:40.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:40 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:40.579+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:40.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:40 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:40.579+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:41.156 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:59:40 smithi123 podman[66507]: 2024-09-21 03:59:40.837365524 +0000 UTC m=+0.874049659 container remove a81f6c650f687459771de2aca759abb116674b36a772a1905c5d66db1aed8459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T03:59:41.352 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:41 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:41.087+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:41.353 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:41 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:41.087+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.725359+0000 front 2024-09-21T03:59:13.725407+0000 (oldest deadline 2024-09-21T03:59:36.625267+0000) 2024-09-21T03:59:41.353 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:41 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:41.201+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:41.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:41 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:41.179+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:41.413 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:41 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:41.077+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:41.475 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:41 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:41.321+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:41.476 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:41 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:41.391+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:41.476 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:41 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:41.391+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:41.476 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:59:41 smithi123 podman[66596]: 2024-09-21 03:59:41.072206008 +0000 UTC m=+0.015573211 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:59:41.476 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:59:41 smithi123 podman[66596]: 2024-09-21 03:59:41.18827678 +0000 UTC m=+0.131643977 container create cb4482936162e5fdb6f281dae5a557f4d3debd3a753ec0c65e2258c22dbb6452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-21T03:59:41.476 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:59:41 smithi123 podman[66596]: 2024-09-21 03:59:41.399277919 +0000 UTC m=+0.342645111 container init cb4482936162e5fdb6f281dae5a557f4d3debd3a753ec0c65e2258c22dbb6452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:59:41.476 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:59:41 smithi123 podman[66596]: 2024-09-21 03:59:41.402588055 +0000 UTC m=+0.345955248 container start cb4482936162e5fdb6f281dae5a557f4d3debd3a753ec0c65e2258c22dbb6452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:59:41.476 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:59:41 smithi123 podman[66596]: 2024-09-21 03:59:41.474024228 +0000 UTC m=+0.417391431 container attach cb4482936162e5fdb6f281dae5a557f4d3debd3a753ec0c65e2258c22dbb6452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-4-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T03:59:41.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:41 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:41.570+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:41.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:41 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:41.570+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:41.870 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 21 03:59:41 smithi123 podman[66596]: 2024-09-21 03:59:41.588205332 +0000 UTC m=+0.531572529 container died cb4482936162e5fdb6f281dae5a557f4d3debd3a753ec0c65e2258c22dbb6452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T03:59:42.216 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T03:59:42.256 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:42.256 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T03:59:42.256 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T03:59:42.257 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.5 2024-09-21T03:59:42.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:42 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:42.146+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:42.413 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:42 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:42.107+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:42.526 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:42 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:42.216+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:42.526 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:42 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:42.123+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:42.527 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:42 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:42.123+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.725359+0000 front 2024-09-21T03:59:13.725407+0000 (oldest deadline 2024-09-21T03:59:36.625267+0000) 2024-09-21T03:59:42.527 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:42 smithi134 systemd[1]: Stopping Ceph osd.5 for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:42.656 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:42 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:42.303+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:42.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:42 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:42.402+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:42.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:42 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:42.402+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:42.854 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:42 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:42.524+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:42.854 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:42 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:42.524+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:42.854 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:42 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:42.525+0000 7fa6eda52640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:42.854 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:42 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:42.525+0000 7fa6eda52640 -1 osd.5 90 *** Got signal Terminated *** 2024-09-21T03:59:42.855 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:42 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:42.525+0000 7fa6eda52640 -1 osd.5 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:59:43.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:43 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:43.182+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:43.413 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:43 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:43.109+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:43.549 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:43 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:43.170+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:43.549 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:43 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:43.170+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.725359+0000 front 2024-09-21T03:59:13.725407+0000 (oldest deadline 2024-09-21T03:59:36.625267+0000) 2024-09-21T03:59:43.549 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:43 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:43.250+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:43.656 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:43 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:43.316+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:43.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:43 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:43.418+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:43.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:43 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:43.418+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:43.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:43 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:43.546+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:43.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:43 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:43.546+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:44.412 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:44 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:44.125+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:44.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:44 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:44.189+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:44.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:44 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:44.189+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:17.435198+0000 front 2024-09-21T03:59:17.435409+0000 (oldest deadline 2024-09-21T03:59:43.335251+0000) 2024-09-21T03:59:44.577 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:44 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:44.215+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:44.577 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:44 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:44.138+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:44.577 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:44 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:44.138+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.725359+0000 front 2024-09-21T03:59:13.725407+0000 (oldest deadline 2024-09-21T03:59:36.625267+0000) 2024-09-21T03:59:44.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:44 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:44.304+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:44.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:44 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:44.304+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T03:59:44.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:44 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:44.404+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:44.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:44 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:44.404+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:44.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:44 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:44.404+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T03:59:44.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:44 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:44.574+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:44.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:44 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:44.574+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:45.412 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:45 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:45.128+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:45.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:45 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:45.183+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:45.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:45 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:45.183+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:17.435198+0000 front 2024-09-21T03:59:17.435409+0000 (oldest deadline 2024-09-21T03:59:43.335251+0000) 2024-09-21T03:59:45.596 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:45 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:45.111+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:45.597 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:45 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:45.111+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.725359+0000 front 2024-09-21T03:59:13.725407+0000 (oldest deadline 2024-09-21T03:59:36.625267+0000) 2024-09-21T03:59:45.597 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:45 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:45.197+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:45.656 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:45 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:45.285+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:45.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:45 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:45.285+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T03:59:45.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:45 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:45.389+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:45.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:45 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:45.389+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:45.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:45 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:45.389+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T03:59:45.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:45 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:45.594+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:45.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:45 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:45.594+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:45.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:45 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:45.594+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T03:59:46.413 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:46 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:46.129+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:46.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:46 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:46.152+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:46.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:46 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:46.152+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:17.435198+0000 front 2024-09-21T03:59:17.435409+0000 (oldest deadline 2024-09-21T03:59:43.335251+0000) 2024-09-21T03:59:46.600 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:46 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:46.107+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:46.600 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:46 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:46.107+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.725359+0000 front 2024-09-21T03:59:13.725407+0000 (oldest deadline 2024-09-21T03:59:36.625267+0000) 2024-09-21T03:59:46.600 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:46 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:46.199+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:46.600 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:46 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:46.199+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T03:59:46.656 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:46 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:46.325+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:46.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:46 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:46.325+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T03:59:46.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:46 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:46.347+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:46.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:46 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:46.347+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:46.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:46 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:46.347+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T03:59:46.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:46 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:46.597+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:46.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:46 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:46.597+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:46.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:46 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:46.597+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T03:59:47.353 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:47 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:47.179+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:47.353 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:47 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:47.179+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T03:59:47.353 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:47 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:47.092+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.423450+0000 front 2024-09-21T03:59:04.423467+0000 (oldest deadline 2024-09-21T03:59:27.923378+0000) 2024-09-21T03:59:47.353 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:47 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:47.092+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.725359+0000 front 2024-09-21T03:59:13.725407+0000 (oldest deadline 2024-09-21T03:59:36.625267+0000) 2024-09-21T03:59:47.353 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:47 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5[45841]: 2024-09-21T03:59:47.092+0000 7fa6e6035640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.226619+0000 front 2024-09-21T03:59:21.226589+0000 (oldest deadline 2024-09-21T03:59:46.526786+0000) 2024-09-21T03:59:47.413 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:47 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:47.167+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:47.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:47 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:47.156+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:47.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:47 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:47.156+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:17.435198+0000 front 2024-09-21T03:59:17.435409+0000 (oldest deadline 2024-09-21T03:59:43.335251+0000) 2024-09-21T03:59:47.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:47 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:47.346+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:47.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:47 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:47.346+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T03:59:47.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:47 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:47.299+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:47.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:47 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:47.299+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:47.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:47 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:47.299+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T03:59:47.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:47 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:47.549+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:47.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:47 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:47.549+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:47.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:47 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:47.549+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T03:59:48.352 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:48 smithi134 podman[67840]: 2024-09-21 03:59:48.092221834 +0000 UTC m=+5.645147230 container died 1d1943ded8ac367f48cb0f0d190b855f99e2533a6b0c3dd28db70de2875f1834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:59:48.353 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:48 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:48.158+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:48.353 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:48 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:48.158+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T03:59:48.656 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:48 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:48.335+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:48.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:48 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:48.335+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T03:59:48.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:48 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:48.335+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:48.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:48 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:48.335+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:48.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:48 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:48.335+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T03:59:48.662 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:48 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:48.176+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:48.663 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:48 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:48.176+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:17.435198+0000 front 2024-09-21T03:59:17.435409+0000 (oldest deadline 2024-09-21T03:59:43.335251+0000) 2024-09-21T03:59:48.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:48 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:48.171+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:48.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:48 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:48.171+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T03:59:48.745 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:48 smithi134 podman[67840]: 2024-09-21 03:59:48.466947148 +0000 UTC m=+6.019872542 container cleanup 1d1943ded8ac367f48cb0f0d190b855f99e2533a6b0c3dd28db70de2875f1834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:59:48.745 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:48 smithi134 bash[67840]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5 2024-09-21T03:59:48.745 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:48 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:48.561+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:48.745 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:48 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:48.561+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:48.745 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:48 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:48.561+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T03:59:49.079 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:48 smithi134 podman[67855]: 2024-09-21 03:59:48.743252791 +0000 UTC m=+0.647718956 container remove 1d1943ded8ac367f48cb0f0d190b855f99e2533a6b0c3dd28db70de2875f1834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T03:59:49.348 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:49 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:49.180+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:49.348 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:49 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:49.180+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T03:59:49.349 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:49 smithi134 podman[67943]: 2024-09-21 03:59:49.077648553 +0000 UTC m=+0.111077937 container create eec61dd0d5944041e30231b58181f47a57cc6544188c607c7591de7a3b6dca16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-21T03:59:49.349 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:49 smithi134 podman[67943]: 2024-09-21 03:59:48.982696569 +0000 UTC m=+0.016125917 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:59:49.413 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:49 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:49.212+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:49.413 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:49 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:49.212+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T03:59:49.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:49 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:49.202+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:49.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:49 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:49.202+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:17.435198+0000 front 2024-09-21T03:59:17.435409+0000 (oldest deadline 2024-09-21T03:59:43.335251+0000) 2024-09-21T03:59:49.602 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:49 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:49.581+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:49.603 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:49 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:49.581+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:49.603 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:49 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:49.581+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T03:59:49.603 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:49 smithi134 podman[67943]: 2024-09-21 03:59:49.346993865 +0000 UTC m=+0.380423207 container init eec61dd0d5944041e30231b58181f47a57cc6544188c607c7591de7a3b6dca16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T03:59:49.603 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:49 smithi134 podman[67943]: 2024-09-21 03:59:49.350456225 +0000 UTC m=+0.383885566 container start eec61dd0d5944041e30231b58181f47a57cc6544188c607c7591de7a3b6dca16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T03:59:49.603 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:49 smithi134 podman[67943]: 2024-09-21 03:59:49.413641673 +0000 UTC m=+0.447071026 container attach eec61dd0d5944041e30231b58181f47a57cc6544188c607c7591de7a3b6dca16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T03:59:49.603 INFO:journalctl@ceph.osd.5.smithi134.stdout:Sep 21 03:59:49 smithi134 podman[67943]: 2024-09-21 03:59:49.542899005 +0000 UTC m=+0.576328347 container died eec61dd0d5944041e30231b58181f47a57cc6544188c607c7591de7a3b6dca16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-21T03:59:49.656 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:49 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:49.306+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:49.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:49 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:49.306+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T03:59:49.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:49 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:49.382+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:49.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:49 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:49.382+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:49.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:49 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:49.382+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T03:59:50.144 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T03:59:50.183 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:50.184 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T03:59:50.184 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T03:59:50.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.6 2024-09-21T03:59:50.579 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:50 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:50.147+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:50.579 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:50 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:50.147+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T03:59:50.617 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:50 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:50.252+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:50.617 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:50 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:50.252+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T03:59:50.617 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:50 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:50.234+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:50.617 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:50 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:50.234+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:17.435198+0000 front 2024-09-21T03:59:17.435409+0000 (oldest deadline 2024-09-21T03:59:43.335251+0000) 2024-09-21T03:59:50.617 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:50 smithi016 systemd[1]: Stopping Ceph osd.6 for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:50.618 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:50 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:50.614+0000 7f3fc6400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:50.618 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:50 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:50.614+0000 7f3fc6400640 -1 osd.6 90 *** Got signal Terminated *** 2024-09-21T03:59:50.618 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:50 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:50.614+0000 7f3fc6400640 -1 osd.6 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:59:50.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:50 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:50.270+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:50.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:50 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:50.270+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T03:59:50.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:50 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:50.389+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:50.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:50 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:50.389+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:50.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:50 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:50.389+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T03:59:50.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:50 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:50.576+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:50.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:50 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:50.576+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:50.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:50 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:50.576+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T03:59:51.596 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:51 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:51.123+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:51.597 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:51 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:51.123+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T03:59:51.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:51 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:51.246+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:51.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:51 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:51.246+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T03:59:51.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:51 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:51.391+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:51.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:51 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:51.391+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:51.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:51 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:51.391+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T03:59:51.662 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:51 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:51.200+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:51.663 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:51 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:51.200+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:17.435198+0000 front 2024-09-21T03:59:17.435409+0000 (oldest deadline 2024-09-21T03:59:43.335251+0000) 2024-09-21T03:59:51.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:51 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:51.268+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:51.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:51 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:51.268+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T03:59:51.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:51 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:51.594+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:51.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:51 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:51.594+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:51.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:51 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:51.594+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T03:59:52.553 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:52 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:52.151+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:52.553 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:52 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:52.151+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T03:59:52.553 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:52 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:52.151+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T03:59:52.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:52 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:52.269+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:52.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:52 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:52.269+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T03:59:52.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:52 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:52.439+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:52.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:52 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:52.439+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:52.658 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:52 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:52.439+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T03:59:52.658 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:52 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:52.439+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T03:59:52.663 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:52 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:52.200+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:52.663 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:52 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:52.200+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:17.435198+0000 front 2024-09-21T03:59:17.435409+0000 (oldest deadline 2024-09-21T03:59:43.335251+0000) 2024-09-21T03:59:52.663 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:52 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:52.200+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:28.036419+0000 front 2024-09-21T03:59:28.036520+0000 (oldest deadline 2024-09-21T03:59:51.536535+0000) 2024-09-21T03:59:52.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:52 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:52.301+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:52.664 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:52 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:52.301+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T03:59:52.664 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:52 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:52.301+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T03:59:52.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:52 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:52.551+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:52.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:52 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:52.551+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:52.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:52 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:52.551+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T03:59:53.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:53 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:53.155+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:53.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:53 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:53.155+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:17.435198+0000 front 2024-09-21T03:59:17.435409+0000 (oldest deadline 2024-09-21T03:59:43.335251+0000) 2024-09-21T03:59:53.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:53 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:53.155+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:28.036419+0000 front 2024-09-21T03:59:28.036520+0000 (oldest deadline 2024-09-21T03:59:51.536535+0000) 2024-09-21T03:59:53.413 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:53 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:53.350+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:53.413 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:53 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:53.350+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T03:59:53.413 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:53 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:53.350+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T03:59:53.509 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:53 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:53.102+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:53.509 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:53 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:53.102+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T03:59:53.509 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:53 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:53.102+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T03:59:53.656 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:53 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:53.298+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:53.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:53 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:53.298+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T03:59:53.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:53 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:53.488+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:53.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:53 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:53.488+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:53.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:53 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:53.488+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T03:59:53.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:53 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:53.488+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T03:59:53.778 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:53 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:53.507+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:53.778 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:53 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:53.507+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:53.778 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:53 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:53.507+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T03:59:54.397 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:54 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:54.126+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:54.397 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:54 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:54.126+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:17.435198+0000 front 2024-09-21T03:59:17.435409+0000 (oldest deadline 2024-09-21T03:59:43.335251+0000) 2024-09-21T03:59:54.397 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:54 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:54.126+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:28.036419+0000 front 2024-09-21T03:59:28.036520+0000 (oldest deadline 2024-09-21T03:59:51.536535+0000) 2024-09-21T03:59:54.558 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:54 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:54.125+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:54.558 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:54 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:54.125+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T03:59:54.558 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:54 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:54.125+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T03:59:54.656 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:54 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:54.290+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:54.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:54 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:54.290+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T03:59:54.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:54 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:54.449+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:54.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:54 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:54.449+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:54.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:54 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:54.449+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T03:59:54.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:54 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:54.449+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T03:59:54.662 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:54 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:54.395+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:54.662 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:54 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:54.395+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T03:59:54.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:54 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:54.395+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T03:59:54.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:54 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:54.555+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:54.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:54 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:54.555+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:54.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:54 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:54.555+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T03:59:55.348 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:55.079+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.534381+0000 front 2024-09-21T03:59:11.534230+0000 (oldest deadline 2024-09-21T03:59:37.434455+0000) 2024-09-21T03:59:55.348 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:55.079+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:17.435198+0000 front 2024-09-21T03:59:17.435409+0000 (oldest deadline 2024-09-21T03:59:43.335251+0000) 2024-09-21T03:59:55.348 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6[58574]: 2024-09-21T03:59:55.079+0000 7f3fbce00640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:28.036419+0000 front 2024-09-21T03:59:28.036520+0000 (oldest deadline 2024-09-21T03:59:51.536535+0000) 2024-09-21T03:59:55.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:55 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:55.133+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:55.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:55 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:55.133+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T03:59:55.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:55 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:55.133+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T03:59:55.656 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:55 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:55.261+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:55.656 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:55 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:55.261+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T03:59:55.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:55 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:55.422+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:55.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:55 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:55.422+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:55.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:55 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:55.422+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T03:59:55.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:55 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:55.422+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T03:59:55.662 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:55.346+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:55.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:55.346+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T03:59:55.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:55 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:55.346+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T03:59:55.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:55 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:55.599+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:55.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:55 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:55.599+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:55.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:55 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:55.599+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T03:59:56.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:56 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:56.127+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:56.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:56 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:56.127+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T03:59:56.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:56 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:56.127+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T03:59:56.644 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:56 smithi016 podman[99234]: 2024-09-21 03:59:56.306031838 +0000 UTC m=+5.819867987 container died 28e4fbbbbf6369ea9caf31710701f57a2efcde826d42ee7057df6f441b6ee856 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T03:59:56.644 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:56 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:56.338+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:56.644 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:56 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:56.338+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T03:59:56.644 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:56 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:56.338+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T03:59:56.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:56 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:56.214+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:56.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:56 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:56.214+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T03:59:56.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:56 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:56.214+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.090788+0000 front 2024-09-21T03:59:30.090660+0000 (oldest deadline 2024-09-21T03:59:55.390478+0000) 2024-09-21T03:59:56.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:56 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:56.400+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:56.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:56 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:56.400+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:56.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:56 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:56.400+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T03:59:56.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:56 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:56.400+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T03:59:57.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:56 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:56.641+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:57.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:56 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:56.641+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:57.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:56 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:56.641+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T03:59:57.336 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:56 smithi016 podman[99234]: 2024-09-21 03:59:56.936921003 +0000 UTC m=+6.450757191 container cleanup 28e4fbbbbf6369ea9caf31710701f57a2efcde826d42ee7057df6f441b6ee856 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-21T03:59:57.337 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:56 smithi016 bash[99234]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6 2024-09-21T03:59:57.337 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:57 smithi016 podman[99251]: 2024-09-21 03:59:57.155112984 +0000 UTC m=+0.838990158 container remove 28e4fbbbbf6369ea9caf31710701f57a2efcde826d42ee7057df6f441b6ee856 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T03:59:57.558 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:57 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:57.382+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:57.558 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:57 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:57.382+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:57.558 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:57 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:57.382+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T03:59:57.558 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:57 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:57.382+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T03:59:57.559 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:57 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:57.223+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:57.559 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:57 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:57.223+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T03:59:57.559 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:57 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:57.223+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.090788+0000 front 2024-09-21T03:59:30.090660+0000 (oldest deadline 2024-09-21T03:59:55.390478+0000) 2024-09-21T03:59:57.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:57 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:57.174+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:57.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:57 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:57.174+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T03:59:57.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:57 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:57.174+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T03:59:57.662 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:57 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:57.334+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:57.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:57 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:57.334+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T03:59:57.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:57 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:57.334+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T03:59:58.002 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:57 smithi016 podman[99338]: 2024-09-21 03:59:57.698135477 +0000 UTC m=+0.116363231 container create 9df321468ac16ade4362e20a53cda46ea3f356e6612c0f8d75dff6b84aff5c36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T03:59:58.002 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:57 smithi016 podman[99338]: 2024-09-21 03:59:57.608894793 +0000 UTC m=+0.027122548 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:59:58.002 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:58 smithi016 podman[99338]: 2024-09-21 03:59:58.000759343 +0000 UTC m=+0.418987097 container init 9df321468ac16ade4362e20a53cda46ea3f356e6612c0f8d75dff6b84aff5c36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T03:59:58.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:57 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:57.676+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:58.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:57 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:57.676+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:58.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:57 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:57.676+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T03:59:58.276 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:58 smithi016 podman[99338]: 2024-09-21 03:59:58.007719992 +0000 UTC m=+0.425947751 container start 9df321468ac16ade4362e20a53cda46ea3f356e6612c0f8d75dff6b84aff5c36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-09-21T03:59:58.276 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:58 smithi016 podman[99338]: 2024-09-21 03:59:58.108860397 +0000 UTC m=+0.527088165 container attach 9df321468ac16ade4362e20a53cda46ea3f356e6612c0f8d75dff6b84aff5c36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:59:58.573 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:58 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:58.217+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:58.573 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:58 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:58.217+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T03:59:58.574 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:58 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:58.217+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.090788+0000 front 2024-09-21T03:59:30.090660+0000 (oldest deadline 2024-09-21T03:59:55.390478+0000) 2024-09-21T03:59:58.574 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:58 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:58.376+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:58.574 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:58 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:58.376+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:58.574 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:58 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:58.376+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T03:59:58.574 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:58 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:58.376+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T03:59:58.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:58 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:58.140+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:58.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:58 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:58.140+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T03:59:58.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:58 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:58.140+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T03:59:58.626 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:59:58 smithi016 podman[99338]: 2024-09-21 03:59:58.275186727 +0000 UTC m=+0.693414481 container died 9df321468ac16ade4362e20a53cda46ea3f356e6612c0f8d75dff6b84aff5c36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T03:59:58.626 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:58 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:58.324+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:58.626 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:58 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:58.324+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T03:59:58.627 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:58 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:58.324+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T03:59:59.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T03:59:59.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:58 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:58.699+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T03:59:59.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:58 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:58.699+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T03:59:59.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:58 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:58.699+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T03:59:59.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:59:59.133 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T03:59:59.133 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T03:59:59.133 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.7 2024-09-21T03:59:59.407 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:59 smithi123 systemd[1]: Stopping Ceph osd.7 for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T03:59:59.407 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:59 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:59.239+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T03:59:59.407 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:59 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:59.239+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T03:59:59.407 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:59 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:59.239+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.090788+0000 front 2024-09-21T03:59:30.090660+0000 (oldest deadline 2024-09-21T03:59:55.390478+0000) 2024-09-21T03:59:59.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:59 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:59.183+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T03:59:59.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:59 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:59.183+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T03:59:59.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 03:59:59 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T03:59:59.183+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T03:59:59.662 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:59 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:59.369+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T03:59:59.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:59 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:59.369+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T03:59:59.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:59:59 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T03:59:59.369+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T03:59:59.820 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:59 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:59.437+0000 7f73b54f2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:59:59.820 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:59 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:59.437+0000 7f73b54f2640 -1 osd.7 90 *** Got signal Terminated *** 2024-09-21T03:59:59.820 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 03:59:59 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T03:59:59.437+0000 7f73b54f2640 -1 osd.7 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:59:59.820 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:59 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:59.415+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T03:59:59.820 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:59 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:59.415+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T03:59:59.821 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:59 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:59.415+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T03:59:59.821 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 03:59:59 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T03:59:59.415+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:00.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:59 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:59.726+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:00.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:59 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:59.726+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:00.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 03:59:59 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T03:59:59.726+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:00.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:00 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:00.209+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T04:00:00.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:00 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:00.209+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T04:00:00.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:00 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:00.209+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T04:00:00.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:00 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:00.244+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T04:00:00.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:00 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:00.244+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T04:00:00.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:00 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:00.244+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.090788+0000 front 2024-09-21T03:59:30.090660+0000 (oldest deadline 2024-09-21T03:59:55.390478+0000) 2024-09-21T04:00:00.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:00 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:00.398+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:00.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:00 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:00.398+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:00.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:00 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:00.398+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:00.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:00 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:00.398+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:00.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:00 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:00.334+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:00.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:00 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:00.334+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:00.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:00 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:00.334+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:00.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:00 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:00.334+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:01.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:00 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:00.765+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:01.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:00 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:00.765+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:01.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:00 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:00.765+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:01.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:01 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:01.186+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T04:00:01.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:01 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:01.186+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T04:00:01.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:01 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:01.186+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T04:00:01.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:01 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:01.186+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.014377+0000 front 2024-09-21T03:59:35.014334+0000 (oldest deadline 2024-09-21T04:00:00.814844+0000) 2024-09-21T04:00:01.656 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:01 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:01.230+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T04:00:01.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:01 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:01.230+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T04:00:01.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:01 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:01.230+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.090788+0000 front 2024-09-21T03:59:30.090660+0000 (oldest deadline 2024-09-21T03:59:55.390478+0000) 2024-09-21T04:00:01.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:01 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:01.408+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:01.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:01 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:01.408+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:01.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:01 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:01.408+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:01.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:01 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:01.408+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:01.662 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:01 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:01.360+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:01.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:01 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:01.360+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:01.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:01 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:01.360+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:01.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:01 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:01.360+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:02.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:01 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:01.769+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:02.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:01 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:01.769+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:02.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:01 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:01.769+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:02.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:02 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:02.147+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T04:00:02.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:02 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:02.147+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T04:00:02.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:02 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:02.147+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T04:00:02.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:02 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:02.147+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.014377+0000 front 2024-09-21T03:59:35.014334+0000 (oldest deadline 2024-09-21T04:00:00.814844+0000) 2024-09-21T04:00:02.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:02 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:02.202+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T04:00:02.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:02 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:02.202+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T04:00:02.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:02 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:02.202+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.090788+0000 front 2024-09-21T03:59:30.090660+0000 (oldest deadline 2024-09-21T03:59:55.390478+0000) 2024-09-21T04:00:02.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:02 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:02.202+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.390826+0000 front 2024-09-21T03:59:35.390794+0000 (oldest deadline 2024-09-21T04:00:01.290926+0000) 2024-09-21T04:00:02.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:02 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:02.413+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:02.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:02 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:02.413+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:02.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:02 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:02.413+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:02.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:02 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:02.413+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:02.658 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:02 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:02.413+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:02.662 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:02 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:02.398+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:02.662 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:02 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:02.398+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:02.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:02 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:02.398+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:02.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:02 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:02.398+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:03.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:02 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:02.720+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:03.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:02 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:02.720+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:03.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:02 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:02.720+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:03.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:03 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:03.105+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T04:00:03.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:03 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:03.105+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T04:00:03.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:03 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:03.105+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T04:00:03.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:03 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:03.105+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.014377+0000 front 2024-09-21T03:59:35.014334+0000 (oldest deadline 2024-09-21T04:00:00.814844+0000) 2024-09-21T04:00:03.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:03 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:03.227+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T04:00:03.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:03 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:03.227+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T04:00:03.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:03 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:03.227+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.090788+0000 front 2024-09-21T03:59:30.090660+0000 (oldest deadline 2024-09-21T03:59:55.390478+0000) 2024-09-21T04:00:03.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:03 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:03.227+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.390826+0000 front 2024-09-21T03:59:35.390794+0000 (oldest deadline 2024-09-21T04:00:01.290926+0000) 2024-09-21T04:00:03.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:03 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:03.392+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:03.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:03 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:03.392+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:03.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:03 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:03.392+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:03.658 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:03 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:03.392+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:03.658 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:03 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:03.392+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:03.662 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:03 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:03.402+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:03.662 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:03 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:03.402+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:03.662 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:03 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:03.402+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:03.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:03 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:03.402+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:04.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:03 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:03.704+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:04.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:03 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:03.704+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:04.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:03 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:03.704+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:04.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:04 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:04.114+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T04:00:04.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:04 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:04.114+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T04:00:04.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:04 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:04.114+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T04:00:04.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:04 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:04.114+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.014377+0000 front 2024-09-21T03:59:35.014334+0000 (oldest deadline 2024-09-21T04:00:00.814844+0000) 2024-09-21T04:00:04.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:04 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:04.244+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.585748+0000 front 2024-09-21T03:59:13.585775+0000 (oldest deadline 2024-09-21T03:59:35.285872+0000) 2024-09-21T04:00:04.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:04 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:04.244+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.989293+0000 front 2024-09-21T03:59:21.989100+0000 (oldest deadline 2024-09-21T03:59:44.289039+0000) 2024-09-21T04:00:04.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:04 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:04.244+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.090788+0000 front 2024-09-21T03:59:30.090660+0000 (oldest deadline 2024-09-21T03:59:55.390478+0000) 2024-09-21T04:00:04.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:04 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7[50917]: 2024-09-21T04:00:04.244+0000 7f73adad5640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.390826+0000 front 2024-09-21T03:59:35.390794+0000 (oldest deadline 2024-09-21T04:00:01.290926+0000) 2024-09-21T04:00:04.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:04 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:04.362+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:04.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:04 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:04.362+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:04.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:04 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:04.362+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:04.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:04 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:04.362+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:04.658 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:04 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:04.362+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:04.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:04 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:04.388+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:04.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:04 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:04.388+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:04.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:04 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:04.388+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:04.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:04 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:04.388+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:05.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:04 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:04.721+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:05.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:04 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:04.721+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:05.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:04 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:04.721+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:05.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:04 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:04.721+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:05.258 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:04 smithi123 podman[66710]: 2024-09-21 04:00:04.970546168 +0000 UTC m=+5.629515445 container died 611c8a7fc0e7e507ae7e3b1a25b28588ddba20040b08e7605565f1d373194139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T04:00:05.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:05 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:05.141+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T04:00:05.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:05 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:05.141+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T04:00:05.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:05 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:05.141+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T04:00:05.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:05 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:05.141+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.014377+0000 front 2024-09-21T03:59:35.014334+0000 (oldest deadline 2024-09-21T04:00:00.814844+0000) 2024-09-21T04:00:05.625 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:05 smithi123 podman[66710]: 2024-09-21 04:00:05.35527685 +0000 UTC m=+6.014246110 container cleanup 611c8a7fc0e7e507ae7e3b1a25b28588ddba20040b08e7605565f1d373194139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:00:05.625 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:05 smithi123 bash[66710]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7 2024-09-21T04:00:05.625 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:05 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:05.352+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:05.625 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:05 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:05.352+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:05.625 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:05 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:05.352+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:05.625 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:05 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:05.352+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:05.625 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:05 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:05.352+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:05.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:05 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:05.361+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:05.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:05 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:05.361+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:05.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:05 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:05.361+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:05.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:05 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:05.361+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:05.906 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:05 smithi123 podman[66730]: 2024-09-21 04:00:05.623274874 +0000 UTC m=+0.656133573 container remove 611c8a7fc0e7e507ae7e3b1a25b28588ddba20040b08e7605565f1d373194139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:00:06.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:05 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:05.721+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:06.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:05 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:05.721+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:06.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:05 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:05.721+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:06.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:05 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:05.721+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:06.211 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:05 smithi123 podman[66818]: 2024-09-21 04:00:05.830442138 +0000 UTC m=+0.015886073 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:00:06.211 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:05 smithi123 podman[66818]: 2024-09-21 04:00:05.957589325 +0000 UTC m=+0.143033259 container create e29ee15061ebadca1e4f808771d6313f93dcc68564604bfa55c5defa64de0348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:00:06.211 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:06 smithi123 podman[66818]: 2024-09-21 04:00:06.210082584 +0000 UTC m=+0.395526534 container init e29ee15061ebadca1e4f808771d6313f93dcc68564604bfa55c5defa64de0348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T04:00:06.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:06 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:06.179+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T04:00:06.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:06 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:06.179+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T04:00:06.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:06 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:06.179+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T04:00:06.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:06 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:06.179+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.014377+0000 front 2024-09-21T03:59:35.014334+0000 (oldest deadline 2024-09-21T04:00:00.814844+0000) 2024-09-21T04:00:06.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:06 smithi123 podman[66818]: 2024-09-21 04:00:06.213531789 +0000 UTC m=+0.398975727 container start e29ee15061ebadca1e4f808771d6313f93dcc68564604bfa55c5defa64de0348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T04:00:06.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:06 smithi123 podman[66818]: 2024-09-21 04:00:06.310065854 +0000 UTC m=+0.495509792 container attach e29ee15061ebadca1e4f808771d6313f93dcc68564604bfa55c5defa64de0348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T04:00:06.657 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 21 04:00:06 smithi123 podman[66818]: 2024-09-21 04:00:06.403498006 +0000 UTC m=+0.588941942 container died e29ee15061ebadca1e4f808771d6313f93dcc68564604bfa55c5defa64de0348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T04:00:06.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:06 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:06.376+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:06.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:06 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:06.376+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:06.658 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:06 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:06.376+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:06.658 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:06 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:06.376+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:06.658 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:06 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:06.376+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:06.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:06 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:06.361+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:06.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:06 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:06.361+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:06.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:06 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:06.361+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:06.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:06 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:06.361+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:07.096 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T04:00:07.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:06 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:06.768+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:07.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:06 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:06.768+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:07.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:06 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:06.768+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:07.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:06 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:06.768+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:07.140 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:00:07.140 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T04:00:07.140 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T04:00:07.141 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.8 2024-09-21T04:00:07.438 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:07 smithi134 systemd[1]: Stopping Ceph osd.8 for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T04:00:07.438 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:07 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:07.200+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T04:00:07.438 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:07 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:07.200+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T04:00:07.438 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:07 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:07.200+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T04:00:07.438 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:07 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:07.200+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.014377+0000 front 2024-09-21T03:59:35.014334+0000 (oldest deadline 2024-09-21T04:00:00.814844+0000) 2024-09-21T04:00:07.438 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:07 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:07.435+0000 7f59c5854640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:00:07.438 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:07 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:07.435+0000 7f59c5854640 -1 osd.8 90 *** Got signal Terminated *** 2024-09-21T04:00:07.439 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:07 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:07.435+0000 7f59c5854640 -1 osd.8 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:00:07.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:07 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:07.369+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:07.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:07 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:07.369+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:07.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:07 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:07.369+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:07.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:07 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:07.369+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:07.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:07 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:07.369+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:07.662 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:07 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:07.377+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:07.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:07 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:07.377+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:07.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:07 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:07.377+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:07.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:07 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:07.377+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:08.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:07 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:07.808+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:08.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:07 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:07.808+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:08.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:07 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:07.808+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:08.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:07 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:07.808+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:08.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:08 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:08.227+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T04:00:08.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:08 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:08.227+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T04:00:08.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:08 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:08.227+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T04:00:08.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:08 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:08.227+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.014377+0000 front 2024-09-21T03:59:35.014334+0000 (oldest deadline 2024-09-21T04:00:00.814844+0000) 2024-09-21T04:00:08.656 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:08 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:08.380+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:08.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:08 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:08.380+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:08.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:08 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:08.380+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:08.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:08 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:08.380+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:08.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:08 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:08.380+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:08.662 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:08 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:08.398+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:08.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:08 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:08.398+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:08.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:08 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:08.398+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:08.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:08 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:08.398+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:09.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:08 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:08.835+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:09.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:08 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:08.835+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:09.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:08 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:08.835+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:09.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:08 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:08.835+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:09.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:09 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:09.258+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T04:00:09.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:09 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:09.258+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T04:00:09.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:09 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:09.258+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T04:00:09.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:09 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:09.258+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.014377+0000 front 2024-09-21T03:59:35.014334+0000 (oldest deadline 2024-09-21T04:00:00.814844+0000) 2024-09-21T04:00:09.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:09 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:09.349+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:09.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:09 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:09.349+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:09.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:09 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:09.349+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:09.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:09 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:09.349+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:09.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:09 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:09.349+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:09.658 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:09 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:09.349+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:09.662 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:09 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:09.390+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:09.662 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:09 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:09.390+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:09.662 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:09 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:09.390+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:09.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:09 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:09.390+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:10.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:09 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:09.822+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:10.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:09 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:09.822+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:10.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:09 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:09.822+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:10.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:09 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:09.822+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:10.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:10 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:10.263+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T04:00:10.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:10 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:10.263+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T04:00:10.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:10 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:10.263+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T04:00:10.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:10 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:10.263+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.014377+0000 front 2024-09-21T03:59:35.014334+0000 (oldest deadline 2024-09-21T04:00:00.814844+0000) 2024-09-21T04:00:10.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:10 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:10.263+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.616354+0000 front 2024-09-21T03:59:46.616376+0000 (oldest deadline 2024-09-21T04:00:10.016909+0000) 2024-09-21T04:00:10.656 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:10 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:10.335+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:10.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:10 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:10.335+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:10.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:10 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:10.335+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:10.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:10 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:10.335+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:10.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:10 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:10.335+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:10.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:10 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:10.335+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:10.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:10 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:10.421+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:10.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:10 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:10.421+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:10.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:10 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:10.421+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:10.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:10 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:10.421+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:10.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:10 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:10.421+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:45.493612+0000 front 2024-09-21T03:59:45.493637+0000 (oldest deadline 2024-09-21T04:00:09.593267+0000) 2024-09-21T04:00:11.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:10 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:10.779+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:11.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:10 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:10.779+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:11.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:10 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:10.779+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:11.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:10 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:10.779+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:11.602 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:11 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:11.244+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T04:00:11.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:11 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:11.244+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T04:00:11.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:11 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:11.244+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T04:00:11.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:11 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:11.244+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.014377+0000 front 2024-09-21T03:59:35.014334+0000 (oldest deadline 2024-09-21T04:00:00.814844+0000) 2024-09-21T04:00:11.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:11 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:11.244+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.616354+0000 front 2024-09-21T03:59:46.616376+0000 (oldest deadline 2024-09-21T04:00:10.016909+0000) 2024-09-21T04:00:11.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:11 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:11.321+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:11.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:11 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:11.321+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:11.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:11 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:11.321+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:11.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:11 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:11.321+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:11.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:11 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:11.321+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:11.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:11 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:11.321+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:11.912 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:11 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:11.445+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:11.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:11 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:11.445+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:11.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:11 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:11.445+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:11.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:11 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:11.445+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:11.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:11 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:11.445+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:45.493612+0000 front 2024-09-21T03:59:45.493637+0000 (oldest deadline 2024-09-21T04:00:09.593267+0000) 2024-09-21T04:00:12.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:11 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:11.776+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:12.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:11 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:11.776+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:12.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:11 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:11.776+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:12.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:11 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:11.776+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:12.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:11 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:11.776+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:12.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:12 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:12.278+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.710621+0000 front 2024-09-21T03:59:11.710760+0000 (oldest deadline 2024-09-21T03:59:37.010672+0000) 2024-09-21T04:00:12.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:12 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:12.278+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:22.311743+0000 front 2024-09-21T03:59:22.311705+0000 (oldest deadline 2024-09-21T03:59:45.211968+0000) 2024-09-21T04:00:12.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:12 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:12.278+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.313078+0000 front 2024-09-21T03:59:29.313132+0000 (oldest deadline 2024-09-21T03:59:52.113352+0000) 2024-09-21T04:00:12.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:12 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:12.278+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.014377+0000 front 2024-09-21T03:59:35.014334+0000 (oldest deadline 2024-09-21T04:00:00.814844+0000) 2024-09-21T04:00:12.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:12 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8[52191]: 2024-09-21T04:00:12.278+0000 7f59c1e3f640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.616354+0000 front 2024-09-21T03:59:46.616376+0000 (oldest deadline 2024-09-21T04:00:10.016909+0000) 2024-09-21T04:00:12.656 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:12 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:12.321+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:12.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:12 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:12.321+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:12.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:12 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:12.321+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:12.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:12 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:12.321+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:12.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:12 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:12.321+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:12.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:12 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:12.321+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:12.912 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:12 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:12.482+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:12.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:12 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:12.482+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:12.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:12 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:12.482+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:12.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:12 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:12.482+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:12.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:12 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:12.482+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:45.493612+0000 front 2024-09-21T03:59:45.493637+0000 (oldest deadline 2024-09-21T04:00:09.593267+0000) 2024-09-21T04:00:13.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:12 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:12.782+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:13.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:12 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:12.782+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:13.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:12 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:12.782+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:13.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:12 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:12.782+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:13.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:12 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:12.782+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:13.392 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:13 smithi134 podman[68060]: 2024-09-21 04:00:13.118810783 +0000 UTC m=+5.774697771 container died 861b3500a6ae6e2f3dfaf486f18ad3e764126fd9199712b90e1a500ee48338fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T04:00:13.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:13 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:13.331+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:13.658 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:13 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:13.331+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:13.658 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:13 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:13.331+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:13.658 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:13 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:13.331+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:13.658 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:13 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:13.331+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:13.658 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:13 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:13.331+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:13.748 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:13 smithi134 podman[68060]: 2024-09-21 04:00:13.485782756 +0000 UTC m=+6.141669736 container cleanup 861b3500a6ae6e2f3dfaf486f18ad3e764126fd9199712b90e1a500ee48338fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T04:00:13.748 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:13 smithi134 bash[68060]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8 2024-09-21T04:00:13.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:13 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:13.483+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:13.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:13 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:13.483+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:13.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:13 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:13.483+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:13.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:13 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:13.483+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:13.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:13 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:13.483+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:45.493612+0000 front 2024-09-21T03:59:45.493637+0000 (oldest deadline 2024-09-21T04:00:09.593267+0000) 2024-09-21T04:00:14.023 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:13 smithi134 podman[68075]: 2024-09-21 04:00:13.745336336 +0000 UTC m=+0.628104738 container remove 861b3500a6ae6e2f3dfaf486f18ad3e764126fd9199712b90e1a500ee48338fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T04:00:14.023 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:13 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:13.753+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:14.023 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:13 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:13.753+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:14.023 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:13 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:13.753+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:14.023 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:13 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:13.753+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:14.023 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:13 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:13.753+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:14.333 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:14 smithi134 podman[68165]: 2024-09-21 04:00:14.020953913 +0000 UTC m=+0.082397200 container create 5c2f9178bbc30acb10341d17d7c2b47ddcb9da0b04046b7374cc2c98dcaf3129 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T04:00:14.333 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:14 smithi134 podman[68165]: 2024-09-21 04:00:13.953825138 +0000 UTC m=+0.015268425 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:00:14.333 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:14 smithi134 podman[68165]: 2024-09-21 04:00:14.256660136 +0000 UTC m=+0.318103423 container init 5c2f9178bbc30acb10341d17d7c2b47ddcb9da0b04046b7374cc2c98dcaf3129 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T04:00:14.333 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:14 smithi134 podman[68165]: 2024-09-21 04:00:14.259943659 +0000 UTC m=+0.321386949 container start 5c2f9178bbc30acb10341d17d7c2b47ddcb9da0b04046b7374cc2c98dcaf3129 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T04:00:14.333 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:14 smithi134 podman[68165]: 2024-09-21 04:00:14.331581955 +0000 UTC m=+0.393025250 container attach 5c2f9178bbc30acb10341d17d7c2b47ddcb9da0b04046b7374cc2c98dcaf3129 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8-deactivate, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:00:14.603 INFO:journalctl@ceph.osd.8.smithi134.stdout:Sep 21 04:00:14 smithi134 podman[68165]: 2024-09-21 04:00:14.454848502 +0000 UTC m=+0.516291799 container died 5c2f9178bbc30acb10341d17d7c2b47ddcb9da0b04046b7374cc2c98dcaf3129 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-8-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T04:00:14.656 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:14 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:14.344+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:14.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:14 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:14.344+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:14.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:14 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:14.344+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:14.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:14 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:14.344+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:14.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:14 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:14.344+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:14.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:14 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:14.344+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:14.912 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:14 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:14.454+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:14.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:14 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:14.454+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:14.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:14 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:14.454+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:14.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:14 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:14.454+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:14.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:14 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:14.454+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:45.493612+0000 front 2024-09-21T03:59:45.493637+0000 (oldest deadline 2024-09-21T04:00:09.593267+0000) 2024-09-21T04:00:15.053 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:14 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:14.714+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:15.054 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:14 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:14.714+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:15.054 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:14 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:14.714+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:15.054 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:14 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:14.714+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:15.054 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:14 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:14.714+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:15.069 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T04:00:15.110 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:00:15.110 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T04:00:15.110 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T04:00:15.110 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.9 2024-09-21T04:00:15.448 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:15 smithi016 systemd[1]: Stopping Ceph osd.9 for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T04:00:15.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:15 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:15.361+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:15.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:15 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:15.361+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:15.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:15 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:15.361+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:15.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:15 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:15.361+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:15.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:15 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:15.361+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:15.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:15 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:15.361+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:15.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:15 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:15.446+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:15.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:15 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:15.446+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:15.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:15 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:15.446+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:15.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:15 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:15.446+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:15.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:15 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:15.446+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:45.493612+0000 front 2024-09-21T03:59:45.493637+0000 (oldest deadline 2024-09-21T04:00:09.593267+0000) 2024-09-21T04:00:15.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:15 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:15.502+0000 7fe941c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:00:15.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:15 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:15.502+0000 7fe941c00640 -1 osd.9 90 *** Got signal Terminated *** 2024-09-21T04:00:15.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:15 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:15.502+0000 7fe941c00640 -1 osd.9 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:00:16.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:15 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:15.674+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:16.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:15 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:15.674+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:16.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:15 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:15.674+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:16.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:15 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:15.674+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:16.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:15 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:15.674+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:16.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:16 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:16.369+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:16.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:16 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:16.369+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:16.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:16 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:16.369+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:16.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:16 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:16.369+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:16.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:16 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:16.369+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:16.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:16 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:16.369+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:16.912 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:16 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:16.483+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:16.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:16 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:16.483+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:16.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:16 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:16.483+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:16.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:16 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:16.483+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:16.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:16 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:16.483+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:45.493612+0000 front 2024-09-21T03:59:45.493637+0000 (oldest deadline 2024-09-21T04:00:09.593267+0000) 2024-09-21T04:00:17.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:16 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:16.637+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:17.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:16 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:16.637+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:17.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:16 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:16.637+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:17.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:16 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:16.637+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:17.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:16 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:16.637+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:17.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:16 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:16.637+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:17.407 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:17 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:17.361+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:17.407 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:17 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:17.361+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:17.407 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:17 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:17.361+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:17.407 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:17 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:17.361+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:17.407 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:17 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:17.361+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:17.407 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:17 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:17.361+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:17.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:17 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:17.495+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:17.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:17 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:17.495+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:17.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:17 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:17.495+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:17.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:17 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:17.495+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:17.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:17 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:17.495+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:45.493612+0000 front 2024-09-21T03:59:45.493637+0000 (oldest deadline 2024-09-21T04:00:09.593267+0000) 2024-09-21T04:00:17.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:17 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:17.495+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:52.494191+0000 front 2024-09-21T03:59:52.494146+0000 (oldest deadline 2024-09-21T04:00:17.194107+0000) 2024-09-21T04:00:18.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:17 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:17.647+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:18.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:17 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:17.647+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:18.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:17 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:17.647+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:18.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:17 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:17.647+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:18.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:17 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:17.647+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:18.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:17 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:17.647+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:18.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:18 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:18.408+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:18.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:18 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:18.408+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:18.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:18 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:18.408+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:18.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:18 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:18.408+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:18.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:18 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:18.408+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:18.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:18 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:18.408+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:18.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:18 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:18.408+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:54.883374+0000 front 2024-09-21T03:59:54.883291+0000 (oldest deadline 2024-09-21T04:00:18.383020+0000) 2024-09-21T04:00:18.912 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:18 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:18.527+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:18.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:18 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:18.527+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:18.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:18 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:18.527+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:18.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:18 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:18.527+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:18.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:18 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:18.527+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:45.493612+0000 front 2024-09-21T03:59:45.493637+0000 (oldest deadline 2024-09-21T04:00:09.593267+0000) 2024-09-21T04:00:18.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:18 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:18.527+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:52.494191+0000 front 2024-09-21T03:59:52.494146+0000 (oldest deadline 2024-09-21T04:00:17.194107+0000) 2024-09-21T04:00:19.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:18 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:18.652+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:19.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:18 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:18.652+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:19.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:18 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:18.652+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:19.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:18 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:18.652+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:19.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:18 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:18.652+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:19.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:18 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:18.652+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:19.906 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:19 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:19.444+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:19.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:19 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:19.444+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:19.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:19 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:19.444+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:19.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:19 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:19.444+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:19.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:19 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:19.444+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:19.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:19 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:19.444+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:19.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:19 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:19.444+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:54.883374+0000 front 2024-09-21T03:59:54.883291+0000 (oldest deadline 2024-09-21T04:00:18.383020+0000) 2024-09-21T04:00:19.912 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:19 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:19.478+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:19.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:19 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:19.478+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:19.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:19 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:19.478+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:19.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:19 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:19.478+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:19.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:19 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:19.478+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:45.493612+0000 front 2024-09-21T03:59:45.493637+0000 (oldest deadline 2024-09-21T04:00:09.593267+0000) 2024-09-21T04:00:19.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:19 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:19.478+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:52.494191+0000 front 2024-09-21T03:59:52.494146+0000 (oldest deadline 2024-09-21T04:00:17.194107+0000) 2024-09-21T04:00:20.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:19 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:19.610+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:20.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:19 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:19.610+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:20.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:19 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:19.610+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:20.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:19 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:19.610+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:20.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:19 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:19.610+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:20.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:19 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:19.610+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:20.656 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:20 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:20.399+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:20.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:20 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:20.399+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:20.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:20 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:20.399+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:20.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:20 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:20.399+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:20.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:20 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:20.399+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:20.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:20 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:20.399+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:20.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:20 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:20.399+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:54.883374+0000 front 2024-09-21T03:59:54.883291+0000 (oldest deadline 2024-09-21T04:00:18.383020+0000) 2024-09-21T04:00:20.912 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:20 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:20.494+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:11.088129+0000 front 2024-09-21T03:59:11.088031+0000 (oldest deadline 2024-09-21T03:59:35.188042+0000) 2024-09-21T04:00:20.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:20 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:20.494+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.589847+0000 front 2024-09-21T03:59:21.589819+0000 (oldest deadline 2024-09-21T03:59:47.489736+0000) 2024-09-21T04:00:20.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:20 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:20.494+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:29.191222+0000 front 2024-09-21T03:59:29.190754+0000 (oldest deadline 2024-09-21T03:59:52.090697+0000) 2024-09-21T04:00:20.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:20 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:20.494+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:39.092287+0000 front 2024-09-21T03:59:39.092103+0000 (oldest deadline 2024-09-21T04:00:00.192104+0000) 2024-09-21T04:00:20.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:20 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:20.494+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:45.493612+0000 front 2024-09-21T03:59:45.493637+0000 (oldest deadline 2024-09-21T04:00:09.593267+0000) 2024-09-21T04:00:20.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:20 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9[65261]: 2024-09-21T04:00:20.494+0000 7fe93d600640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:52.494191+0000 front 2024-09-21T03:59:52.494146+0000 (oldest deadline 2024-09-21T04:00:17.194107+0000) 2024-09-21T04:00:21.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:20 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:20.640+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:21.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:20 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:20.640+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:21.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:20 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:20.640+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:21.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:20 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:20.640+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:21.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:20 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:20.640+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:21.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:20 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:20.640+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:21.583 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:21 smithi016 podman[99453]: 2024-09-21 04:00:21.293832168 +0000 UTC m=+5.895436174 container died 3d3215e10d7679c86aa48a1acb0bc88e7a0e96e9c03b4a6fd33e0b054c310bf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T04:00:21.656 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:21 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:21.367+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:21.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:21 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:21.367+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:21.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:21 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:21.367+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:21.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:21 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:21.367+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:21.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:21 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:21.367+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:21.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:21 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:21.367+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:21.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:21 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:21.367+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:54.883374+0000 front 2024-09-21T03:59:54.883291+0000 (oldest deadline 2024-09-21T04:00:18.383020+0000) 2024-09-21T04:00:21.835 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:21 smithi016 podman[99453]: 2024-09-21 04:00:21.807815752 +0000 UTC m=+6.409419757 container cleanup 3d3215e10d7679c86aa48a1acb0bc88e7a0e96e9c03b4a6fd33e0b054c310bf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T04:00:21.835 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:21 smithi016 bash[99453]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9 2024-09-21T04:00:22.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:21 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:21.631+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:22.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:21 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:21.631+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:22.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:21 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:21.631+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:22.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:21 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:21.631+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:22.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:21 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:21.631+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:22.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:21 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:21.631+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:22.412 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:21 smithi016 podman[99471]: 2024-09-21 04:00:21.98453296 +0000 UTC m=+0.655353003 container remove 3d3215e10d7679c86aa48a1acb0bc88e7a0e96e9c03b4a6fd33e0b054c310bf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T04:00:22.656 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:22 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:22.380+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:22.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:22 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:22.380+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:22.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:22 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:22.380+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:22.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:22 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:22.380+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:22.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:22 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:22.380+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:22.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:22 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:22.380+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:22.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:22 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:22.380+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:54.883374+0000 front 2024-09-21T03:59:54.883291+0000 (oldest deadline 2024-09-21T04:00:18.383020+0000) 2024-09-21T04:00:22.798 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:22 smithi016 podman[99555]: 2024-09-21 04:00:22.436464701 +0000 UTC m=+0.027551479 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:00:22.798 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:22 smithi016 podman[99555]: 2024-09-21 04:00:22.593642912 +0000 UTC m=+0.184729676 container create 904cd0a72b3fdd2526398d41582589f31c7bed9ecaa546639ae0d8265141184b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:00:22.798 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:22 smithi016 podman[99555]: 2024-09-21 04:00:22.796579389 +0000 UTC m=+0.387666167 container init 904cd0a72b3fdd2526398d41582589f31c7bed9ecaa546639ae0d8265141184b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T04:00:23.004 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:22 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:22.630+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:23.004 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:22 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:22.630+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:23.004 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:22 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:22.630+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:23.004 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:22 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:22.630+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:23.004 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:22 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:22.630+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:23.004 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:22 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:22.630+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:23.076 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:22 smithi016 podman[99555]: 2024-09-21 04:00:22.803290908 +0000 UTC m=+0.394377664 container start 904cd0a72b3fdd2526398d41582589f31c7bed9ecaa546639ae0d8265141184b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:00:23.076 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:22 smithi016 podman[99555]: 2024-09-21 04:00:22.862728598 +0000 UTC m=+0.453815365 container attach 904cd0a72b3fdd2526398d41582589f31c7bed9ecaa546639ae0d8265141184b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:00:23.412 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 04:00:23 smithi016 podman[99555]: 2024-09-21 04:00:23.075409772 +0000 UTC m=+0.666496540 container died 904cd0a72b3fdd2526398d41582589f31c7bed9ecaa546639ae0d8265141184b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-9-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T04:00:23.656 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:23 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:23.395+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:23.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:23 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:23.395+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:23.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:23 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:23.395+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:23.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:23 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:23.395+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:23.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:23 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:23.395+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:23.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:23 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:23.395+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:23.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:23 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:23.395+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:54.883374+0000 front 2024-09-21T03:59:54.883291+0000 (oldest deadline 2024-09-21T04:00:18.383020+0000) 2024-09-21T04:00:23.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T04:00:23.827 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:00:23.827 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T04:00:23.827 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T04:00:23.827 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.10 2024-09-21T04:00:24.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:23 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:23.627+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:24.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:23 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:23.627+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:24.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:23 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:23.627+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:24.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:23 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:23.627+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:24.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:23 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:23.627+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:24.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:23 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:23.627+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:24.126 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:23 smithi123 systemd[1]: Stopping Ceph osd.10 for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T04:00:24.390 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:24 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:24.123+0000 7f68dd13c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:00:24.390 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:24 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:24.123+0000 7f68dd13c640 -1 osd.10 90 *** Got signal Terminated *** 2024-09-21T04:00:24.390 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:24 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:24.123+0000 7f68dd13c640 -1 osd.10 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:00:24.390 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:24 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:24.386+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:24.390 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:24 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:24.386+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:24.390 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:24 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:24.386+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:24.390 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:24 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:24.386+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:24.390 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:24 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:24.386+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:24.390 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:24 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:24.386+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:24.390 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:24 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:24.386+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:54.883374+0000 front 2024-09-21T03:59:54.883291+0000 (oldest deadline 2024-09-21T04:00:18.383020+0000) 2024-09-21T04:00:25.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:24 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:24.633+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:25.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:24 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:24.633+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:25.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:24 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:24.633+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:25.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:24 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:24.633+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:25.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:24 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:24.633+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:25.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:24 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:24.633+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:25.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:25 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:25.585+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:25.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:25 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:25.585+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:25.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:25 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:25.585+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:25.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:25 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:25.585+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:25.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:25 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:25.585+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:25.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:25 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:25.585+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:25.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:25 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:25.429+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:25.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:25 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:25.429+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:25.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:25 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:25.429+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:25.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:25 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:25.429+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:25.908 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:25 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:25.429+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:25.908 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:25 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:25.429+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:25.908 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:25 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:25.429+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:54.883374+0000 front 2024-09-21T03:59:54.883291+0000 (oldest deadline 2024-09-21T04:00:18.383020+0000) 2024-09-21T04:00:26.602 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:26 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:26.556+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:26.603 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:26 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:26.556+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:26.603 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:26 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:26.556+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:26.603 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:26 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:26.556+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:26.603 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:26 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:26.556+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:26.603 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:26 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:26.556+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:26.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:26 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:26.414+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:26.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:26 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:26.414+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:26.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:26 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:26.414+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:26.907 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:26 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:26.414+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:26.908 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:26 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:26.414+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:26.908 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:26 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:26.414+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:26.908 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:26 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:26.414+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:54.883374+0000 front 2024-09-21T03:59:54.883291+0000 (oldest deadline 2024-09-21T04:00:18.383020+0000) 2024-09-21T04:00:27.656 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:27 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:27.373+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:27.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:27 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:27.373+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:27.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:27 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:27.373+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:27.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:27 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:27.373+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:27.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:27 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:27.373+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:27.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:27 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:27.373+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:27.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:27 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:27.373+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:54.883374+0000 front 2024-09-21T03:59:54.883291+0000 (oldest deadline 2024-09-21T04:00:18.383020+0000) 2024-09-21T04:00:27.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:27 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:27.595+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:27.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:27 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:27.595+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:27.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:27 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:27.595+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:27.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:27 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:27.595+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:27.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:27 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:27.595+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:27.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:27 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:27.595+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:28.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:28 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:28.382+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:04.474141+0000 front 2024-09-21T03:59:04.474125+0000 (oldest deadline 2024-09-21T03:59:26.774140+0000) 2024-09-21T04:00:28.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:28 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:28.382+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.775648+0000 front 2024-09-21T03:59:13.775738+0000 (oldest deadline 2024-09-21T03:59:36.675846+0000) 2024-09-21T04:00:28.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:28 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:28.382+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:16.676475+0000 front 2024-09-21T03:59:16.676570+0000 (oldest deadline 2024-09-21T03:59:43.676851+0000) 2024-09-21T04:00:28.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:28 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:28.382+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:59:30.078605+0000 front 2024-09-21T03:59:30.078455+0000 (oldest deadline 2024-09-21T03:59:51.778411+0000) 2024-09-21T04:00:28.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:28 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:28.382+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:36.379997+0000 front 2024-09-21T03:59:36.379949+0000 (oldest deadline 2024-09-21T04:00:01.580392+0000) 2024-09-21T04:00:28.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:28 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:28.382+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:47.481377+0000 front 2024-09-21T03:59:47.481509+0000 (oldest deadline 2024-09-21T04:00:09.181331+0000) 2024-09-21T04:00:28.657 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:28 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10[57150]: 2024-09-21T04:00:28.382+0000 7f68d9727640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:54.883374+0000 front 2024-09-21T03:59:54.883291+0000 (oldest deadline 2024-09-21T04:00:18.383020+0000) 2024-09-21T04:00:29.102 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:28 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:28.604+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:29.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:28 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:28.604+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:29.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:28 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:28.604+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:29.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:28 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:28.604+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:29.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:28 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:28.604+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:29.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:28 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:28.604+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:29.103 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:28 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:28.604+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-21T04:00:03.903799+0000 front 2024-09-21T04:00:03.903843+0000 (oldest deadline 2024-09-21T04:00:28.003673+0000) 2024-09-21T04:00:29.818 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:29 smithi123 podman[66920]: 2024-09-21 04:00:29.560303129 +0000 UTC m=+5.537733901 container died 13ba568f83f84db5f8b5318c35a2996e27a9e2b90aba5d9cd8359be0f280d58d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:00:29.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:29 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:29.555+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:29.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:29 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:29.555+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:29.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:29 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:29.555+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:29.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:29 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:29.555+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:29.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:29 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:29.555+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:29.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:29 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:29.555+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:29.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:29 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:29.555+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-21T04:00:03.903799+0000 front 2024-09-21T04:00:03.903843+0000 (oldest deadline 2024-09-21T04:00:28.003673+0000) 2024-09-21T04:00:30.156 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:29 smithi123 podman[66920]: 2024-09-21 04:00:29.906571539 +0000 UTC m=+5.884002314 container cleanup 13ba568f83f84db5f8b5318c35a2996e27a9e2b90aba5d9cd8359be0f280d58d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:00:30.157 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:29 smithi123 bash[66920]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10 2024-09-21T04:00:30.510 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:30 smithi123 podman[66934]: 2024-09-21 04:00:30.174417864 +0000 UTC m=+0.616154721 container remove 13ba568f83f84db5f8b5318c35a2996e27a9e2b90aba5d9cd8359be0f280d58d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T04:00:30.804 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:30 smithi123 podman[67028]: 2024-09-21 04:00:30.411009933 +0000 UTC m=+0.015912094 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:00:30.804 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:30 smithi123 podman[67028]: 2024-09-21 04:00:30.516947498 +0000 UTC m=+0.121849656 container create 6e1ce8a63a8253a727a527b5de90c2aa0bd66567262b94527b7becba9828539b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T04:00:30.804 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:30 smithi123 podman[67028]: 2024-09-21 04:00:30.719534731 +0000 UTC m=+0.324436882 container init 6e1ce8a63a8253a727a527b5de90c2aa0bd66567262b94527b7becba9828539b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T04:00:30.804 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:30 smithi123 podman[67028]: 2024-09-21 04:00:30.722716309 +0000 UTC m=+0.327618461 container start 6e1ce8a63a8253a727a527b5de90c2aa0bd66567262b94527b7becba9828539b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T04:00:30.804 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:30 smithi123 podman[67028]: 2024-09-21 04:00:30.802634642 +0000 UTC m=+0.407536806 container attach 6e1ce8a63a8253a727a527b5de90c2aa0bd66567262b94527b7becba9828539b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T04:00:30.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:30 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:30.521+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:30.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:30 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:30.521+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:30.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:30 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:30.521+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:30.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:30 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:30.521+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:30.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:30 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:30.521+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:30.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:30 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:30.521+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:30.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:30 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:30.521+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-21T04:00:03.903799+0000 front 2024-09-21T04:00:03.903843+0000 (oldest deadline 2024-09-21T04:00:28.003673+0000) 2024-09-21T04:00:31.157 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 21 04:00:30 smithi123 podman[67028]: 2024-09-21 04:00:30.912496412 +0000 UTC m=+0.517398575 container died 6e1ce8a63a8253a727a527b5de90c2aa0bd66567262b94527b7becba9828539b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T04:00:31.506 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T04:00:31.545 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:00:31.546 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T04:00:31.546 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T04:00:31.546 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.11 2024-09-21T04:00:31.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:31 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:31.494+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:31.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:31 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:31.494+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:31.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:31 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:31.494+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:31.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:31 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:31.494+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:31.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:31 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:31.494+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:31.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:31 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:31.494+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:31.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:31 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:31.494+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-21T04:00:03.903799+0000 front 2024-09-21T04:00:03.903843+0000 (oldest deadline 2024-09-21T04:00:28.003673+0000) 2024-09-21T04:00:31.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:31 smithi134 systemd[1]: Stopping Ceph osd.11 for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T04:00:32.352 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:31 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:31.853+0000 7f4bb498e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:00:32.352 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:31 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:31.853+0000 7f4bb498e640 -1 osd.11 90 *** Got signal Terminated *** 2024-09-21T04:00:32.352 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:31 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:31.853+0000 7f4bb498e640 -1 osd.11 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:00:32.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:32 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:32.471+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:32.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:32 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:32.471+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:32.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:32 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:32.471+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:32.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:32 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:32.471+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:32.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:32 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:32.471+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:32.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:32 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:32.471+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:32.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:32 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:32.471+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-21T04:00:03.903799+0000 front 2024-09-21T04:00:03.903843+0000 (oldest deadline 2024-09-21T04:00:28.003673+0000) 2024-09-21T04:00:33.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:33 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:33.426+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:33.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:33 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:33.426+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:33.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:33 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:33.426+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:33.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:33 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:33.426+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:33.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:33 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:33.426+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:33.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:33 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:33.426+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:33.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:33 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:33.426+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-21T04:00:03.903799+0000 front 2024-09-21T04:00:03.903843+0000 (oldest deadline 2024-09-21T04:00:28.003673+0000) 2024-09-21T04:00:34.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:34 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:34.417+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:34.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:34 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:34.417+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:34.854 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:34 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:34.417+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:34.854 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:34 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:34.417+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:34.854 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:34 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:34.417+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:34.854 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:34 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:34.417+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:34.854 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:34 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:34.417+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-21T04:00:03.903799+0000 front 2024-09-21T04:00:03.903843+0000 (oldest deadline 2024-09-21T04:00:28.003673+0000) 2024-09-21T04:00:34.854 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:34 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:34.417+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6820 osd.8 since back 2024-09-21T04:00:08.004102+0000 front 2024-09-21T04:00:08.004139+0000 (oldest deadline 2024-09-21T04:00:33.904100+0000) 2024-09-21T04:00:35.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:35 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:35.456+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:35.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:35 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:35.456+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:35.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:35 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:35.456+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:35.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:35 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:35.456+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:35.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:35 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:35.456+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:35.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:35 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:35.456+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:35.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:35 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:35.456+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-21T04:00:03.903799+0000 front 2024-09-21T04:00:03.903843+0000 (oldest deadline 2024-09-21T04:00:28.003673+0000) 2024-09-21T04:00:35.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:35 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:35.456+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6820 osd.8 since back 2024-09-21T04:00:08.004102+0000 front 2024-09-21T04:00:08.004139+0000 (oldest deadline 2024-09-21T04:00:33.904100+0000) 2024-09-21T04:00:36.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:36 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:36.461+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:59:03.295201+0000 front 2024-09-21T03:59:03.295312+0000 (oldest deadline 2024-09-21T03:59:29.195024+0000) 2024-09-21T04:00:36.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:36 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:36.461+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-21T03:59:13.896261+0000 front 2024-09-21T03:59:13.896361+0000 (oldest deadline 2024-09-21T03:59:38.496763+0000) 2024-09-21T04:00:36.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:36 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:36.461+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-09-21T03:59:21.298217+0000 front 2024-09-21T03:59:21.298154+0000 (oldest deadline 2024-09-21T03:59:44.798229+0000) 2024-09-21T04:00:36.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:36 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:36.461+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-21T03:59:35.200745+0000 front 2024-09-21T03:59:35.200652+0000 (oldest deadline 2024-09-21T04:00:04.001095+0000) 2024-09-21T04:00:36.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:36 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:36.461+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-09-21T03:59:46.301866+0000 front 2024-09-21T03:59:46.301978+0000 (oldest deadline 2024-09-21T04:00:11.601953+0000) 2024-09-21T04:00:36.853 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:36 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:36.461+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:59:51.602634+0000 front 2024-09-21T03:59:51.602667+0000 (oldest deadline 2024-09-21T04:00:16.302299+0000) 2024-09-21T04:00:36.854 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:36 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:36.461+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-21T04:00:03.903799+0000 front 2024-09-21T04:00:03.903843+0000 (oldest deadline 2024-09-21T04:00:28.003673+0000) 2024-09-21T04:00:36.854 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:36 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11[58587]: 2024-09-21T04:00:36.461+0000 7f4bacf71640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.134:6820 osd.8 since back 2024-09-21T04:00:08.004102+0000 front 2024-09-21T04:00:08.004139+0000 (oldest deadline 2024-09-21T04:00:33.904100+0000) 2024-09-21T04:00:37.651 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:37 smithi134 podman[68280]: 2024-09-21 04:00:37.379411184 +0000 UTC m=+5.629858077 container died 6390cfc5fa63629e39956d4ce10aa659c0ec8eb089bb5fe511a1458f546507df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:00:38.023 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:37 smithi134 podman[68280]: 2024-09-21 04:00:37.74501177 +0000 UTC m=+5.995458664 container cleanup 6390cfc5fa63629e39956d4ce10aa659c0ec8eb089bb5fe511a1458f546507df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-21T04:00:38.023 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:37 smithi134 bash[68280]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11 2024-09-21T04:00:38.348 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:38 smithi134 podman[68299]: 2024-09-21 04:00:38.021242993 +0000 UTC m=+0.638674801 container remove 6390cfc5fa63629e39956d4ce10aa659c0ec8eb089bb5fe511a1458f546507df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T04:00:38.601 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:38 smithi134 podman[68385]: 2024-09-21 04:00:38.347146195 +0000 UTC m=+0.102529442 container create daa324c07bccd71d87b4cb6b6025db00e1d62218652691b9239b07bba2e956f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T04:00:38.601 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:38 smithi134 podman[68385]: 2024-09-21 04:00:38.260683809 +0000 UTC m=+0.016067065 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:00:38.601 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:38 smithi134 podman[68385]: 2024-09-21 04:00:38.541409229 +0000 UTC m=+0.296792487 container init daa324c07bccd71d87b4cb6b6025db00e1d62218652691b9239b07bba2e956f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:00:38.601 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:38 smithi134 podman[68385]: 2024-09-21 04:00:38.544583202 +0000 UTC m=+0.299966455 container start daa324c07bccd71d87b4cb6b6025db00e1d62218652691b9239b07bba2e956f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T04:00:38.601 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:38 smithi134 podman[68385]: 2024-09-21 04:00:38.59947739 +0000 UTC m=+0.354860641 container attach daa324c07bccd71d87b4cb6b6025db00e1d62218652691b9239b07bba2e956f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11-deactivate, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:00:38.852 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:38 smithi134 podman[68385]: 2024-09-21 04:00:38.732005031 +0000 UTC m=+0.487388287 container died daa324c07bccd71d87b4cb6b6025db00e1d62218652691b9239b07bba2e956f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T04:00:39.332 INFO:journalctl@ceph.osd.11.smithi134.stdout:Sep 21 04:00:39 smithi134 podman[68408]: 2024-09-21 04:00:39.31125905 +0000 UTC m=+0.574013447 container remove daa324c07bccd71d87b4cb6b6025db00e1d62218652691b9239b07bba2e956f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-osd-11-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T04:00:39.338 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T04:00:39.377 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:00:39.377 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T04:00:39.378 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T04:00:39.378 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.a 2024-09-21T04:00:39.828 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 04:00:39 smithi016 systemd[1]: Stopping Ceph mds.a for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T04:00:40.142 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 04:00:39 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-a[71991]: 2024-09-21T04:00:39.904+0000 7fef22e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:00:40.142 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 04:00:39 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-a[71991]: 2024-09-21T04:00:39.904+0000 7fef22e00640 -1 mds.a *** got signal Terminated *** 2024-09-21T04:00:43.613 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 04:00:43 smithi016 podman[99656]: 2024-09-21 04:00:43.315029768 +0000 UTC m=+3.604214718 container died fc21c0a5e14e3283e3db6ab69deab2bb6a57c61614a84268dc600d646beea00c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-a, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T04:00:43.864 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 04:00:43 smithi016 podman[99656]: 2024-09-21 04:00:43.834860746 +0000 UTC m=+4.124045694 container cleanup fc21c0a5e14e3283e3db6ab69deab2bb6a57c61614a84268dc600d646beea00c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-a, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-21T04:00:43.864 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 04:00:43 smithi016 bash[99656]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-a 2024-09-21T04:00:44.163 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 04:00:44 smithi016 podman[99681]: 2024-09-21 04:00:44.053059049 +0000 UTC m=+0.727561138 container remove fc21c0a5e14e3283e3db6ab69deab2bb6a57c61614a84268dc600d646beea00c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-21T04:00:44.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T04:00:44.406 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:00:44.406 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T04:00:44.406 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T04:00:44.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.d 2024-09-21T04:00:44.663 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 04:00:44 smithi016 systemd[1]: Stopping Ceph mds.d for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T04:00:45.163 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 04:00:44 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-d[70887]: 2024-09-21T04:00:44.783+0000 7f14f3600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:00:45.163 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 04:00:44 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-d[70887]: 2024-09-21T04:00:44.783+0000 7f14f3600640 -1 mds.d *** got signal Terminated *** 2024-09-21T04:00:46.163 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 04:00:45 smithi016 podman[99788]: 2024-09-21 04:00:45.904752244 +0000 UTC m=+1.225376209 container died e609d21e660b31849c85d5df9259978a9820954d025c046410b5a6fadf8911e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-d, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T04:00:46.592 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 04:00:46 smithi016 podman[99788]: 2024-09-21 04:00:46.36437415 +0000 UTC m=+1.684998113 container cleanup e609d21e660b31849c85d5df9259978a9820954d025c046410b5a6fadf8911e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-d, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T04:00:46.592 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 04:00:46 smithi016 bash[99788]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-d 2024-09-21T04:00:46.858 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 04:00:46 smithi016 podman[99799]: 2024-09-21 04:00:46.590691538 +0000 UTC m=+0.675931492 container remove e609d21e660b31849c85d5df9259978a9820954d025c046410b5a6fadf8911e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-d, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T04:00:46.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T04:00:46.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:00:46.940 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T04:00:46.940 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T04:00:46.940 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.g 2024-09-21T04:00:47.162 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 04:00:47 smithi016 systemd[1]: Stopping Ceph mds.g for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T04:00:47.662 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 04:00:47 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-g[71616]: 2024-09-21T04:00:47.378+0000 7efc33800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:00:47.663 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 04:00:47 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-g[71616]: 2024-09-21T04:00:47.378+0000 7efc33800640 -1 mds.g *** got signal Terminated *** 2024-09-21T04:00:51.412 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 04:00:51 smithi016 podman[99903]: 2024-09-21 04:00:51.297002096 +0000 UTC m=+4.068645719 container died f21e48a25ecddee71dce7e861f357887d74a165e75328dd05b9341550c3c3928 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-g, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T04:00:51.913 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 04:00:51 smithi016 podman[99903]: 2024-09-21 04:00:51.726886723 +0000 UTC m=+4.498530346 container cleanup f21e48a25ecddee71dce7e861f357887d74a165e75328dd05b9341550c3c3928 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-g, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T04:00:51.913 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 04:00:51 smithi016 bash[99903]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-g 2024-09-21T04:00:52.207 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 04:00:51 smithi016 podman[99915]: 2024-09-21 04:00:51.938546409 +0000 UTC m=+0.627176940 container remove f21e48a25ecddee71dce7e861f357887d74a165e75328dd05b9341550c3c3928 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T04:00:52.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T04:00:52.304 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:00:52.304 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T04:00:52.305 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T04:00:52.305 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.j 2024-09-21T04:00:52.662 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 04:00:52 smithi016 systemd[1]: Stopping Ceph mds.j for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T04:00:53.163 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 04:00:52 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-j[70497]: 2024-09-21T04:00:52.700+0000 7f4bbbc00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:00:53.163 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 04:00:52 smithi016 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-j[70497]: 2024-09-21T04:00:52.700+0000 7f4bbbc00640 -1 mds.j *** got signal Terminated *** 2024-09-21T04:00:57.052 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 04:00:56 smithi016 podman[100024]: 2024-09-21 04:00:56.724479383 +0000 UTC m=+4.143217870 container died 89ba4424df0bd0a1df472bfaa724018dcb1e3aaec8824b56737ce8b8da4f9024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-j, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:00:57.412 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 04:00:57 smithi016 podman[100024]: 2024-09-21 04:00:57.164211418 +0000 UTC m=+4.582949904 container cleanup 89ba4424df0bd0a1df472bfaa724018dcb1e3aaec8824b56737ce8b8da4f9024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-j, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T04:00:57.413 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 04:00:57 smithi016 bash[100024]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-j 2024-09-21T04:00:57.719 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 04:00:57 smithi016 podman[100036]: 2024-09-21 04:00:57.456953809 +0000 UTC m=+0.701276417 container remove 89ba4424df0bd0a1df472bfaa724018dcb1e3aaec8824b56737ce8b8da4f9024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-j, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:00:57.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T04:00:57.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:00:57.798 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T04:00:57.798 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T04:00:57.798 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.b 2024-09-21T04:00:58.156 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 04:00:57 smithi123 systemd[1]: Stopping Ceph mds.b for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T04:00:58.656 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 04:00:58 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-b[62470]: 2024-09-21T04:00:58.197+0000 7fd746db0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:00:58.656 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 04:00:58 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-b[62470]: 2024-09-21T04:00:58.197+0000 7fd746db0640 -1 mds.b *** got signal Terminated *** 2024-09-21T04:01:02.483 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 04:01:02 smithi123 podman[67131]: 2024-09-21 04:01:02.211294911 +0000 UTC m=+4.220462324 container died f4db202a12ddd9b3b3f43013301c5f892718588fb1098b2bba39591061e73a6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True) 2024-09-21T04:01:02.841 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 04:01:02 smithi123 podman[67131]: 2024-09-21 04:01:02.572552014 +0000 UTC m=+4.581719426 container cleanup f4db202a12ddd9b3b3f43013301c5f892718588fb1098b2bba39591061e73a6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:01:02.842 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 21 04:01:02 smithi123 bash[67131]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-b 2024-09-21T04:01:03.010 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T04:01:03.047 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:01:03.047 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T04:01:03.047 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T04:01:03.047 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.e 2024-09-21T04:01:03.362 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 04:01:03 smithi123 systemd[1]: Stopping Ceph mds.e for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T04:01:03.656 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 04:01:03 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-e[61920]: 2024-09-21T04:01:03.359+0000 7f75380c8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:01:03.656 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 04:01:03 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-e[61920]: 2024-09-21T04:01:03.359+0000 7f75380c8640 -1 mds.e *** got signal Terminated *** 2024-09-21T04:01:07.407 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 04:01:07 smithi123 podman[67279]: 2024-09-21 04:01:07.079050017 +0000 UTC m=+3.815234932 container died 28797bd39437d78cce128c2a378877a73f02d158a982ef63d2819c00d4cd8cdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T04:01:07.720 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 04:01:07 smithi123 podman[67279]: 2024-09-21 04:01:07.509357363 +0000 UTC m=+4.245542264 container cleanup 28797bd39437d78cce128c2a378877a73f02d158a982ef63d2819c00d4cd8cdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:01:07.720 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 21 04:01:07 smithi123 bash[67279]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-e 2024-09-21T04:01:07.869 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T04:01:07.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:01:07.902 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T04:01:07.903 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T04:01:07.903 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.h 2024-09-21T04:01:08.240 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 04:01:07 smithi123 systemd[1]: Stopping Ceph mds.h for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T04:01:08.656 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 04:01:08 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-h[62190]: 2024-09-21T04:01:08.237+0000 7f52c7c52640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:01:08.657 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 04:01:08 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-h[62190]: 2024-09-21T04:01:08.237+0000 7f52c7c52640 -1 mds.h *** got signal Terminated *** 2024-09-21T04:01:11.406 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 04:01:11 smithi123 podman[67400]: 2024-09-21 04:01:11.082914047 +0000 UTC m=+2.965272075 container died 1059a271b1f20b5be8041698dfa6b9d1e8fe4df234050310c2437765e9325733 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-h, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:01:11.699 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 04:01:11 smithi123 podman[67400]: 2024-09-21 04:01:11.504881181 +0000 UTC m=+3.387239212 container cleanup 1059a271b1f20b5be8041698dfa6b9d1e8fe4df234050310c2437765e9325733 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-h, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T04:01:11.699 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 04:01:11 smithi123 bash[67400]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-h 2024-09-21T04:01:11.930 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T04:01:11.951 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 04:01:11 smithi123 podman[67412]: 2024-09-21 04:01:11.69764899 +0000 UTC m=+0.616991450 container remove 1059a271b1f20b5be8041698dfa6b9d1e8fe4df234050310c2437765e9325733 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-h, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T04:01:11.951 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 04:01:11 smithi123 systemd[1]: ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.h.service: Deactivated successfully. 2024-09-21T04:01:11.951 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 04:01:11 smithi123 systemd[1]: Stopped Ceph mds.h for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T04:01:11.951 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 21 04:01:11 smithi123 systemd[1]: ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.h.service: Consumed 1.936s CPU time. 2024-09-21T04:01:11.963 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:01:11.963 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T04:01:11.963 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T04:01:11.963 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.k 2024-09-21T04:01:12.285 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 21 04:01:12 smithi123 systemd[1]: Stopping Ceph mds.k for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T04:01:12.657 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 21 04:01:12 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-k[61641]: 2024-09-21T04:01:12.283+0000 7f3754b82640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:01:12.657 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 21 04:01:12 smithi123 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-k[61641]: 2024-09-21T04:01:12.283+0000 7f3754b82640 -1 mds.k *** got signal Terminated *** 2024-09-21T04:01:15.406 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 21 04:01:15 smithi123 podman[67523]: 2024-09-21 04:01:15.072201706 +0000 UTC m=+2.897020496 container died e7bc5d6c90f6b25fa725a6ef1ad04ae9f734ed4095c0d54b6332175f69cb6b22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-k, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T04:01:15.760 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 21 04:01:15 smithi123 podman[67523]: 2024-09-21 04:01:15.508173677 +0000 UTC m=+3.332992467 container cleanup e7bc5d6c90f6b25fa725a6ef1ad04ae9f734ed4095c0d54b6332175f69cb6b22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-k, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T04:01:15.761 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 21 04:01:15 smithi123 bash[67523]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-k 2024-09-21T04:01:15.885 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T04:01:15.923 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:01:15.924 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T04:01:15.924 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T04:01:15.924 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.c 2024-09-21T04:01:16.297 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 04:01:15 smithi134 systemd[1]: Stopping Ceph mds.c for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T04:01:16.602 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 04:01:16 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-c[63402]: 2024-09-21T04:01:16.294+0000 7fe431712640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:01:16.602 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 04:01:16 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-c[63402]: 2024-09-21T04:01:16.294+0000 7fe431712640 -1 mds.c *** got signal Terminated *** 2024-09-21T04:01:19.352 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 04:01:19 smithi134 podman[68514]: 2024-09-21 04:01:19.079258146 +0000 UTC m=+2.945714791 container died ca791862f45302e9e9e8c31973d43b844cdbc045b70a2b7014a965df94839e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-c, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T04:01:19.696 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 04:01:19 smithi134 podman[68514]: 2024-09-21 04:01:19.443644028 +0000 UTC m=+3.310100669 container cleanup ca791862f45302e9e9e8c31973d43b844cdbc045b70a2b7014a965df94839e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-c, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T04:01:19.696 INFO:journalctl@ceph.mds.c.smithi134.stdout:Sep 21 04:01:19 smithi134 bash[68514]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-c 2024-09-21T04:01:19.870 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T04:01:19.906 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:01:19.907 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T04:01:19.907 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T04:01:19.907 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.f 2024-09-21T04:01:20.352 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 04:01:19 smithi134 systemd[1]: Stopping Ceph mds.f for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T04:01:20.352 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 04:01:20 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-f[63132]: 2024-09-21T04:01:20.213+0000 7f2c20e63640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:01:20.352 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 04:01:20 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-f[63132]: 2024-09-21T04:01:20.213+0000 7f2c20e63640 -1 mds.f *** got signal Terminated *** 2024-09-21T04:01:23.352 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 04:01:23 smithi134 podman[68634]: 2024-09-21 04:01:23.072951797 +0000 UTC m=+2.951256584 container died e9f1b37174f336e3af118238a246320c7d84cd77b2a99fb7f93f1aa7840d6dd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-f, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T04:01:23.716 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 04:01:23 smithi134 podman[68634]: 2024-09-21 04:01:23.447242668 +0000 UTC m=+3.325547457 container cleanup e9f1b37174f336e3af118238a246320c7d84cd77b2a99fb7f93f1aa7840d6dd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T04:01:23.716 INFO:journalctl@ceph.mds.f.smithi134.stdout:Sep 21 04:01:23 smithi134 bash[68634]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-f 2024-09-21T04:01:23.882 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T04:01:23.915 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:01:23.915 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T04:01:23.915 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T04:01:23.916 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.i 2024-09-21T04:01:24.244 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 04:01:23 smithi134 systemd[1]: Stopping Ceph mds.i for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T04:01:24.602 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 04:01:24 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-i[63682]: 2024-09-21T04:01:24.242+0000 7fc277506640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:01:24.602 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 04:01:24 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-i[63682]: 2024-09-21T04:01:24.242+0000 7fc277506640 -1 mds.i *** got signal Terminated *** 2024-09-21T04:01:27.352 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 04:01:27 smithi134 podman[68755]: 2024-09-21 04:01:27.077538072 +0000 UTC m=+2.941666018 container died 5ec8b8f8c6014950e81fac8c07afcd9908f0b667a00cb3efb04bec2115d7cd8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-i, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T04:01:27.711 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 04:01:27 smithi134 podman[68755]: 2024-09-21 04:01:27.475618826 +0000 UTC m=+3.339746767 container cleanup 5ec8b8f8c6014950e81fac8c07afcd9908f0b667a00cb3efb04bec2115d7cd8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-i, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T04:01:27.711 INFO:journalctl@ceph.mds.i.smithi134.stdout:Sep 21 04:01:27 smithi134 bash[68755]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-i 2024-09-21T04:01:27.850 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T04:01:27.881 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:01:27.882 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T04:01:27.882 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T04:01:27.882 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.l 2024-09-21T04:01:28.352 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 04:01:27 smithi134 systemd[1]: Stopping Ceph mds.l for 2b324840-77ca-11ef-baf5-efdc52797490... 2024-09-21T04:01:28.353 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 04:01:28 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-l[62864]: 2024-09-21T04:01:28.187+0000 7fdb41fb1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:01:28.353 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 04:01:28 smithi134 ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-l[62864]: 2024-09-21T04:01:28.187+0000 7fdb41fb1640 -1 mds.l *** got signal Terminated *** 2024-09-21T04:01:31.352 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 04:01:31 smithi134 podman[68878]: 2024-09-21 04:01:31.076541569 +0000 UTC m=+2.980993292 container died d160eef181782272dd72429e062c2f647d1fee5de07b5e0d3294acbbed8ee520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-l, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:01:31.673 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 04:01:31 smithi134 podman[68878]: 2024-09-21 04:01:31.495419685 +0000 UTC m=+3.399871409 container cleanup d160eef181782272dd72429e062c2f647d1fee5de07b5e0d3294acbbed8ee520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-l, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True) 2024-09-21T04:01:31.673 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 04:01:31 smithi134 bash[68878]: ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-l 2024-09-21T04:01:31.913 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T04:01:31.934 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 04:01:31 smithi134 podman[68892]: 2024-09-21 04:01:31.671153294 +0000 UTC m=+0.590261051 container remove d160eef181782272dd72429e062c2f647d1fee5de07b5e0d3294acbbed8ee520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-2b324840-77ca-11ef-baf5-efdc52797490-mds-l, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T04:01:31.934 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 04:01:31 smithi134 systemd[1]: ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.l.service: Deactivated successfully. 2024-09-21T04:01:31.934 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 04:01:31 smithi134 systemd[1]: Stopped Ceph mds.l for 2b324840-77ca-11ef-baf5-efdc52797490. 2024-09-21T04:01:31.934 INFO:journalctl@ceph.mds.l.smithi134.stdout:Sep 21 04:01:31 smithi134 systemd[1]: ceph-2b324840-77ca-11ef-baf5-efdc52797490@mds.l.service: Consumed 2.127s CPU time. 2024-09-21T04:01:31.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:01:31.945 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T04:01:31.945 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2b324840-77ca-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T04:01:32.229 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T04:01:36.379 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2b324840-77ca-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T04:01:36.580 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T04:01:39.117 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2b324840-77ca-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T04:01:39.321 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T04:01:41.744 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:01:41.810 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:01:41.843 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:01:41.869 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T04:01:41.871 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913469/remote/smithi016/crash 2024-09-21T04:01:41.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/crash -- . 2024-09-21T04:01:41.953 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T04:01:41.953 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2024-09-21T04:01:41.956 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913469/remote/smithi123/crash 2024-09-21T04:01:41.957 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/crash -- . 2024-09-21T04:01:41.992 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T04:01:41.993 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now 2024-09-21T04:01:41.995 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913469/remote/smithi134/crash 2024-09-21T04:01:41.995 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/crash -- . 2024-09-21T04:01:42.032 INFO:teuthology.orchestra.run.smithi134.stderr:tar: /var/lib/ceph/2b324840-77ca-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T04:01:42.032 INFO:teuthology.orchestra.run.smithi134.stderr:tar: Error is not recoverable: exiting now 2024-09-21T04:01:42.033 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T04:01:42.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-21T04:01:42.156 INFO:tasks.cephadm:Compressing logs... 2024-09-21T04:01:42.156 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T04:01:42.160 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T04:01:42.163 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T04:01:42.187 INFO:teuthology.orchestra.run.smithi134.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T04:01:42.187 INFO:teuthology.orchestra.run.smithi134.stderr:: No such file or directory 2024-09-21T04:01:42.187 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T04:01:42.189 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T04:01:42.189 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T04:01:42.189 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T04:01:42.189 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T04:01:42.189 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T04:01:42.189 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.audit.log: /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.i.log: /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mon.c.log: /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log: /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T04:01:42.190 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.log 2024-09-21T04:01:42.190 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T04:01:42.190 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T04:01:42.190 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.l.log: -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T04:01:42.192 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T04:01:42.192 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T04:01:42.192 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T04:01:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-21T04:01:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T04:01:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T04:01:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T04:01:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T04:01:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T04:01:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T04:01:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T04:01:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T04:01:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T04:01:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T04:01:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.log 2024-09-21T04:01:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T04:01:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T04:01:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T04:01:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T04:01:42.219 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2024-09-21T04:01:42.219 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-21T04:01:42.219 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T04:01:42.220 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T04:01:42.221 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T04:01:42.222 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T04:01:42.223 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T04:01:42.223 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T04:01:42.224 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T04:01:42.225 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T04:01:42.226 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T04:01:42.227 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.log 2024-09-21T04:01:42.227 INFO:teuthology.orchestra.run.smithi016.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-21T04:01:42.227 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T04:01:42.228 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T04:01:42.229 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T04:01:42.230 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T04:01:42.230 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T04:01:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T04:01:42.232 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.e.log: /var/log/ceph/cephadm.log: 82.0% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T04:01:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.8% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T04:01:42.237 INFO:teuthology.orchestra.run.smithi016.stderr: 94.9% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T04:01:42.251 INFO:teuthology.orchestra.run.smithi123.stderr: 94.6% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T04:01:42.261 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.f.log: /var/log/ceph/cephadm.log: 81.9% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T04:01:42.271 INFO:teuthology.orchestra.run.smithi123.stderr: 87.3% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T04:01:42.287 INFO:teuthology.orchestra.run.smithi123.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T04:01:42.303 INFO:teuthology.orchestra.run.smithi123.stderr: 92.6% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T04:01:42.432 INFO:teuthology.orchestra.run.smithi123.stderr: 90.3% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T04:01:42.454 INFO:teuthology.orchestra.run.smithi134.stderr: 93.2% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T04:01:42.477 INFO:teuthology.orchestra.run.smithi134.stderr: 91.0% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T04:01:42.478 INFO:teuthology.orchestra.run.smithi016.stderr: 94.4% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T04:01:42.482 INFO:teuthology.orchestra.run.smithi123.stderr: 92.7% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T04:01:42.502 INFO:teuthology.orchestra.run.smithi016.stderr: 82.1% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T04:01:42.509 INFO:teuthology.orchestra.run.smithi134.stderr: 87.3% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T04:01:42.511 INFO:teuthology.orchestra.run.smithi016.stderr: 94.4% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T04:01:42.529 INFO:teuthology.orchestra.run.smithi134.stderr: 94.6% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T04:01:42.530 INFO:teuthology.orchestra.run.smithi134.stderr: 90.3% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T04:01:42.531 INFO:teuthology.orchestra.run.smithi016.stderr: 90.1% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T04:01:42.552 INFO:teuthology.orchestra.run.smithi134.stderr: 92.8% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T04:01:42.573 INFO:teuthology.orchestra.run.smithi134.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T04:01:42.593 INFO:teuthology.orchestra.run.smithi016.stderr: 90.9% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T04:01:42.692 INFO:teuthology.orchestra.run.smithi123.stderr: 93.0% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T04:01:42.719 INFO:teuthology.orchestra.run.smithi123.stderr: 90.9% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T04:01:43.014 INFO:teuthology.orchestra.run.smithi123.stderr: 91.7% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T04:01:43.135 INFO:teuthology.orchestra.run.smithi016.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T04:01:43.159 INFO:teuthology.orchestra.run.smithi134.stderr: 93.6% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T04:01:43.176 INFO:teuthology.orchestra.run.smithi134.stderr: 93.1% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T04:01:43.753 INFO:teuthology.orchestra.run.smithi134.stderr: 91.7% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T04:01:46.428 INFO:teuthology.orchestra.run.smithi016.stderr: 89.9% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T04:01:47.383 INFO:teuthology.orchestra.run.smithi123.stderr: 93.3% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T04:01:50.458 INFO:teuthology.orchestra.run.smithi016.stderr: 90.4% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T04:02:19.406 INFO:teuthology.orchestra.run.smithi123.stderr: 93.1% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T04:02:58.673 INFO:teuthology.orchestra.run.smithi123.stderr: 93.1% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T04:03:14.855 INFO:teuthology.orchestra.run.smithi123.stderr: 93.1% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T04:03:15.230 INFO:teuthology.orchestra.run.smithi134.stderr: 93.1% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T04:03:16.585 INFO:teuthology.orchestra.run.smithi134.stderr: 93.1% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T04:03:26.323 INFO:teuthology.orchestra.run.smithi134.stderr: 93.1% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T04:03:27.490 INFO:teuthology.orchestra.run.smithi123.stderr: 93.1% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T04:03:27.492 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-21T04:03:27.493 INFO:teuthology.orchestra.run.smithi123.stderr:real 1m45.314s 2024-09-21T04:03:27.493 INFO:teuthology.orchestra.run.smithi123.stderr:user 3m41.957s 2024-09-21T04:03:27.493 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m7.858s 2024-09-21T04:03:42.041 INFO:teuthology.orchestra.run.smithi134.stderr: 93.0% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T04:03:42.043 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-21T04:03:42.043 INFO:teuthology.orchestra.run.smithi134.stderr:real 1m59.866s 2024-09-21T04:03:42.043 INFO:teuthology.orchestra.run.smithi134.stderr:user 3m46.785s 2024-09-21T04:03:42.043 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m8.161s 2024-09-21T04:07:52.898 INFO:teuthology.orchestra.run.smithi016.stderr: 93.0% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T04:08:29.153 INFO:teuthology.orchestra.run.smithi016.stderr: 93.0% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T04:09:01.431 INFO:teuthology.orchestra.run.smithi016.stderr: 93.0% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T04:09:25.656 INFO:teuthology.orchestra.run.smithi016.stderr: 92.9% -- replaced with /var/log/ceph/2b324840-77ca-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T04:09:25.659 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-21T04:09:25.659 INFO:teuthology.orchestra.run.smithi016.stderr:real 7m43.470s 2024-09-21T04:09:25.659 INFO:teuthology.orchestra.run.smithi016.stderr:user 4m58.681s 2024-09-21T04:09:25.659 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m36.350s 2024-09-21T04:09:25.660 INFO:tasks.cephadm:Archiving logs... 2024-09-21T04:09:25.661 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913469/remote/smithi016/log 2024-09-21T04:09:25.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T04:10:08.413 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913469/remote/smithi123/log 2024-09-21T04:10:08.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T04:10:43.377 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913469/remote/smithi134/log 2024-09-21T04:10:43.378 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T04:11:19.417 INFO:tasks.cephadm:Removing cluster... 2024-09-21T04:11:19.418 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2b324840-77ca-11ef-baf5-efdc52797490 --force 2024-09-21T04:11:19.734 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T04:11:21.208 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2b324840-77ca-11ef-baf5-efdc52797490 --force 2024-09-21T04:11:21.411 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T04:11:21.941 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2b324840-77ca-11ef-baf5-efdc52797490 --force 2024-09-21T04:11:22.144 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: 2b324840-77ca-11ef-baf5-efdc52797490 2024-09-21T04:11:22.668 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T04:11:22.669 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T04:11:22.720 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T04:11:22.738 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T04:11:22.754 INFO:tasks.cephadm:Teardown complete 2024-09-21T04:11:22.754 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T04:11:22.764 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T04:11:22.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T04:11:22.768 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T04:11:22.780 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T04:11:22.906 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T04:11:22.917 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T04:11:22.917 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T04:11:22.921 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T04:11:22.924 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T04:11:22.937 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-09-21T04:11:22.938 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-09-21T04:11:22.948 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T04:11:22.948 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-09-21T04:11:22.948 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv01.front.sepia.ceph.com 3 6 377 42 -528us[ -490us] +/- 51ms 2024-09-21T04:11:22.949 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 +1167us[+1204us] +/- 60ms 2024-09-21T04:11:22.949 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 -491us[ -491us] +/- 65ms 2024-09-21T04:11:22.949 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:11:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-21T04:11:22.955 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T04:11:22.955 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-09-21T04:11:22.955 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv01.front.sepia.ceph.com 3 6 377 42 -554us[ -528us] +/- 51ms 2024-09-21T04:11:22.955 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 +1193us[+1219us] +/- 60ms 2024-09-21T04:11:22.955 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 -505us[ -479us] +/- 65ms 2024-09-21T04:11:22.955 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:11:23.064 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T04:11:23.064 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-21T04:11:23.064 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv01.front.sepia.ceph.com 3 7 377 41 -589us[ -627us] +/- 51ms 2024-09-21T04:11:23.064 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv02.front.sepia.ceph.com 3 8 377 243 +1125us[+1051us] +/- 57ms 2024-09-21T04:11:23.064 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 -559us[ -559us] +/- 65ms 2024-09-21T04:11:23.064 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:11:23.066 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T04:11:23.078 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T04:11:23.078 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T04:11:23.088 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T04:11:23.305 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T04:11:23.389 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T04:11:23.487 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T04:11:23.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:11:23.549 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T04:11:23.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:11:23.580 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T04:11:23.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:11:23.608 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T04:11:23.618 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T04:11:23.664 INFO:teuthology.task.internal:Duration was 2724.520660 seconds 2024-09-21T04:11:23.664 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T04:11:23.674 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T04:11:23.674 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T04:11:23.678 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T04:11:23.681 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T04:11:23.754 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T04:11:23.770 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T04:11:23.775 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T04:11:24.212 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T04:11:24.213 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-09-21T04:11:24.213 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T04:11:24.268 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-09-21T04:11:24.269 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T04:11:24.295 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-09-21T04:11:24.295 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T04:11:24.334 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T04:11:24.335 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T04:11:24.339 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T04:11:24.342 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T04:11:24.402 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T04:11:24.402 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T04:11:24.406 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T04:11:24.417 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T04:11:25.266 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T04:11:25.277 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T04:11:25.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T04:11:25.334 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T04:11:25.361 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T04:11:25.387 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T04:11:25.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:11:25.401 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:11:25.404 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:11:25.432 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-09-21T04:11:25.451 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-09-21T04:11:25.491 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-09-21T04:11:25.528 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:11:25.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:11:25.561 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:11:25.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:11:25.576 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:11:25.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:11:25.591 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T04:11:25.602 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T04:11:25.603 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913469/remote/smithi016 2024-09-21T04:11:25.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T04:11:25.687 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913469/remote/smithi123 2024-09-21T04:11:25.687 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T04:11:25.725 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913469/remote/smithi134 2024-09-21T04:11:25.725 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T04:11:25.763 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T04:11:25.763 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T04:11:25.767 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T04:11:25.769 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T04:11:25.811 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T04:11:25.831 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T04:11:25.831 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T04:11:25.867 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T04:11:25.868 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T04:11:25.872 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T04:11:25.874 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T04:11:25.887 INFO:teuthology.orchestra.run.smithi123.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 04:11 /home/ubuntu/cephtest 2024-09-21T04:11:25.889 INFO:teuthology.orchestra.run.smithi134.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 04:11 /home/ubuntu/cephtest 2024-09-21T04:11:25.903 INFO:teuthology.orchestra.run.smithi016.stdout: 393956 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 04:11 /home/ubuntu/cephtest 2024-09-21T04:11:25.905 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T04:11:25.916 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T04:11:25.955 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iogen}} duration: 2724.5206604003906 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T04:11:25.955 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T04:11:26.049 INFO:teuthology.run:pass